PlaNet: Classifying Planets with Neural Networks

After completing Jeremy Howard’s Deep Learning course, I wanted to put my skills to the test on something fun and interesting, so I set out to train a neural network that classified planets. I’m happy with the end result (and its cheeky name): plaNet.

I wanted to classify major solar system planets based on salient features. The issue with this approach is that there isn’t very much data to train a neural network on. I scraped AstroBin for amateur photos of planets, but I found that most of them simply looked like smudges, and the outer planets were either unrecognizable or missing entirely.

Some of the unaugmented training data used for Jupiter, mostly from NASA.

To get around these issues, I based my approach on two methods: data augmentation on my small dataset, and fine-tuning an existing neural network. Data augmentation is simple in Keras, so I dramatically increased my dataset size simply by applying transformations to my initial images. I fine-tuned my network on VGG’s ImageNet convolutional layers (a classic approach to transfer learning). I dropped out the last fully-connected layer, which was trained to classify everyday objects, and kept the convolutional layers. These layers are great for identifying features — edges, shapes, and patterns — that could still be found in my images of planets. At this point, I pre-calculated the output of the convolutional layer on the initial and augmented datasets in order to easily combine them into one feature set, then I was able to train with a relatively solid test accuracy (~90%). I used a high dropout rate in order to avoid overfitting to my small training dataset, and it seems to have worked.

I want to highlight the simplicity of this approach. Because we’re simply fine-tuning a pre-trained neural network, we can access what is essentially the state of the art in deep learning with just a few lines of code and a small amount of computing time and power (compared to training an entire network from scratch). My work was mostly in preparing the datasets and fine-tuning different parameters until I was happy with the results. If you haven’t already, I encourage you to take a look at the course online. Many thanks to Jeremy Howard for giving me a practical approach to something I’ve only had theoretical backing for so far.

Observations from the New York Scientific Data Summit

Deep learning impresses and disappoints

Multiple talks discussed results from deep learning techniques, especially convolutional neural networks, and the effectiveness of the methods varied wildly. Some experiments yielded only 50% classification accuracy, which doesn’t ultimately seem helpful or effective at all. I’m unsure whether other techniques were attempted or considered, but it’s clear that deep learning isn’t the most effective approach for every single problem. It’s a shiny new hammer that makes every problem look like a nail. Libraries like TensorFlow make it more accessible, but there is still a visible gap between those who can implement it and those who can implement it effectively.

Re-inventing the wheel

A few groups demonstrated tools that were developed in-house that already have excellent open source alternatives. I’m not sure whether they were unaware of the existing libraries or just wanted something more finely-tuned for their own purposes, but it seems that a lot of scientific time is spent coming up with solutions for problems that are already solved. Regardless, there were plenty of examples of people who did use open source libraries effectively, so the progress there is something to be proud of.

Sagan Exoplanet Workshop Day 1

Morning

Eric Feigelson (Penn State) discusses Statistics and the Astronomical Enterprise, and why statistics are so essential to the discovery and study of Exoplanets. The history and development of statistics in relation to astronomy as well as the present state of the field are covered. Key examples of essential statistical applications in astronomy and astrophysics are discussed, as well as an outlook on potential future developments and applications. Practical computing implementations with R are discussed.

Jessi Cisewski (Yale) goes over Bayesian Methods. She covers the basics of Bayesian analysis (Bayes’ Theorem, prior and posterior distributions, and inference with posteriors). Examples of different analyses using different models and distributions are given. Classical/Frequentist approaches are contrasted with the Bayesian approach, and best practices are covered.

Xavier Dumusque (Universite de Geneva) and Nikole Lewis (STScI) introduce the hands-on sessions of the week. Xavier discussed differentiating Radial Velocity signals from planetary signals, and Nikole discussed detecting exoplanets in upcoming JWST data and analyzing their spectra.

Afternoon

David Kipping (Columbia) gives A Beginner’s Guide to Monte Carlo Markov Chain (MCMC) Analysis. MCMC examples are covered as a way to find a posterior distribution. Metropolis and Metropolis-Hastings algorithms are discussed, and situations where other algorithms may be more or less effective are presented.

POPs Session I

Slides here.

  • Ines Juvan (Space Research Institute Graz) – PyTranSpot – A tool for combined transit and stellar spot light-curve modeling
  • Anthony Gai (Univ. at Albany) – Bayesian Model Testing of Models for Ellipsoidal Variation on Stars Due to Hot Jupiters
  • Emiliano Jofre (OAC-CONICET) – Searching for planets in southern stars via transit timing variations
  • Sean McCloat (Univ. of North Dakota) – Follow-up Observations of Recently Discovered Hot Jupiters
  • Romina Petrucci (OAC-CONICET) – A search for orbital decay in southern transiting planets
  • Luke Bouma (MIT) – What should we do next with the Transiting Exoplanet Survey Satellite?
  • Akshata Krishnamurthy (MIT) – A precision optical test bench to measure the absolute quantum efficiency of the Transiting Exoplanet Survey Satellite CCD detectors

David Kipping (Columbia) teaches us about using Bayesian Priors for Transits and RVs. Selection, implementation, strengths and weaknesses, and ideal use cases for different types of priors are discussed.

POPs Session II

Slides here.

  • Allen Davis (Yale) – Assessing the information content of spectra using PCA
  • Matteo Pinamonti (Univ. of Trieste) – Searching for planetary signals in Doppler time series: a performance evaluation of tools for periodogram analysis
  • Keara Wright (Univ. of Florida) – Stellar Parameters for FGK MARVELS Targets
  • Richard Hall (Univ. of Cambridge) – Measuring the Effective Pixel Positions of the HARPS3 CCD
  • Sarah Millholland (UCSC) – A Search for Non-Transiting Hot Jupiters with Transiting Super-Earth Companions
  • Tarun Kumar (Thapar Univ.) – Radial Velocity Curves of Polytropic Models of Stars of Polytropic Index N=1.5
  • Fei Dai (MIT) – The K2-ESPRINT Collaboration

Eric Feigelson (Penn State) covers Statistical Approaches to Exoplanetary Science. The talk focuses on time series analysis. Parametric and nonparametric methods are shown for time domain and frequency domain  problems. Code examples and other potentially useful methods are given.