Wheat is the most important Ethiopian crop, and rust one of its greatest antagonists. There is a need for cheap and scalable rust monitoring in the developing world, but existing methods employ costly data collection techniques. We introduce a scalable, accurate, and inexpensive method for tracking outbreaks with publicly available remote sensing data. Our approach improves existing techniques in two ways. First, we forgo the spectral features employed by the remote sensing community in favor of automatically learned features generated by Convolutional and Long Short-Term Memory Networks. Second, we aggregate data into larger geospatial regions. We evaluate our approach on nine years of agricultural outcomes, show that it outperforms competing techniques, and demonstrate its predictive foresight. This is a promising new direction in crop disease monitoring, one that has the potential to grow more powerful with time.