GOES-R Video Generator
This is a simple video-generator leveraging the wonders of free online stuff. It uses the GOES-R red, blue, and veggie bands to generate a near-realtime, true-color video of the past few days of weather.
With the most recent GOES satellites, NOAA has been posting near-realtime imagery to this Google Cloud bucket for public use.
Google also offers a very small VPS as a free demo, which I used to set the project up. See it live at bluemarble.nitk.in!
This code should be fairly easy to use - you'll need to modify paths to remove my username, but everything works!
The project consists of three largely independent components:
- Image processing
- Video generation
- HTTP server Cron acts as a job supervisor to hold everything together. (Again, quick and dirty).
GOES16.sci.py implements the image processing. When started, it goes into an infinite loop:
- Poll the Cloud Bucket API
- Check that all images are time-synchronized
- Check if images have changed
- If so, process into a JPG file
- If not, wait 5 minutes and try again.
Image processing is a fairly simple pipeline. For each layer (red, blue, veggie):
- Download the layer
- Convert from netCDF4 to netCDF3
- netCDF4 is newer, but the Python bindings for it are 5x slower than the netCDF3 libraries.
- nccopy is used for the conversion
- Scale to a managable size
- layers start as 10848 pixels square, which swamps the VPS
- Ensure all values are positive
- Negatives show up sometimes and mess up gamma correction
- Apply gamma correction
- Fancy word for taking x^0.55 for each pixel value
- Convert from numpy array to a PIL image
- Saves space and prepares for final formatting
Finally, the images are combined
- Create a green channel using red, blue, and veggie
- Stack the layers to make a composite
- Scale down to a thumbnail size for video-ing
The video generation is as dumb as a rock.
Cron periodically calls a makefile;
the makefile checks whether the videos need regeneration, and they're rebuilt if so.
Refer to the
crontab sample for cron rules. The makefile
video.mk allows cron to
call the script every few minutes without undue overhead.
videogen.py looks at all the files in the output directory, finds the newest
(up to a configured limit) and compiles them into a video with ffmpeg.
The length of the video and output filename are configured programatically by the FRAMES
year all do about what you'd expect.
The final component of the whole setup is a simple Python HTTP server. You could use Apache (or Nginx, or anything else), but on a disk-constrained server, I chose to use what was installed.
It just hosts
index.html and the videos.
The HTML file allows a user to select between the different generated videos, and automatically refreshes the videos to keep them near realtime.
Cron provides the glue for this project. The rules included periodically run the video generator,
and will automatically restart both the HTTP server and the image processor if either die.