GOES-R Video Generator
This is a simple video-generator leveraging the wonders of free online stuff. It uses the GOES-R red, blue, and veggie bands to generate a near-realtime, true-color video of the past few days of weather.
With the most recent GOES satellites, NOAA has been posting near-realtime imagery to this Google Cloud bucket for public use.
Google also offers a very small VPS as a free demo, which I used to set the project up. See it live at bluemarble.nitk.in!
This code should be fairly easy to use - you'll need to modify paths to remove my username, but everything works!
The project consists of three largely independent components:
- Image processing
- Video generation
- HTTP server Cron acts as a job supervisor to hold everything together. (Again, quick and dirty).
GOES16.sci.py implements the image processing. When started, it goes into an infinite loop:
- Poll the Cloud Bucket API
- Check that all images are time-synchronized
- Check if images have changed
- If so, process into a JPG file
- If not, wait 5 minutes and try again.
Image processing is a fairly simple pipeline. For each layer (red, blue, veggie):
- Download the layer
- Convert from netCDF4 to netCDF3
- netCDF4 is newer, but the Python bindings for it are 5x slower than the netCDF3 libraries.
- nccopy is used for the conversion
- Scale to a managable size
- layers start as 10848 pixels square, which swamps the VPS
- Ensure all values are positive
- Negatives show up sometimes and mess up gamma correction
- Apply gamma correction
- Fancy word for taking x^0.55 for each pixel value
- Convert from numpy array to a PIL image
- Saves space and prepares for final formatting
Finally, the images are combined
- Create a green channel using red, blue, and veggie
- Stack the layers to make a composite
- Scale down to a thumbnail size for video-ing
The video generation is as dumb as a rock.
cron periodically calls the video generator
day/twoday/week/month/year as an argument.
The generator uses
ffmpeg to build a video in
/tmp; it errors out if the video
exists. (This prevents the generator from stepping on itself under high system loads)
Once generated, the video is moved from
DEST. Moves are atomic, which
ensures the file in
DEST is always a valid video.
The length of the video and output filename are configured programatically by the FRAMES
year all do about what you'd expect.
year find the image nearest noon for each day. They
create a video with one frame per day, rather than six frames per hour.
The final component of the whole setup is a simple Python HTTP server. You could use Apache (or Nginx, or anything else), but on a disk-constrained server, I chose to use what was installed.
Compared to the standard Python server, it provides better support for
It just hosts
static.html and the videos.
The HTML file allows a user to select between the different generated videos, and automatically refreshes the videos to keep them near realtime.
Cron provides the glue for this project. The rules included periodically run the video generator,
and will automatically restart both the HTTP server and the image processor if either die.