PyVoyager automatically creates and stabilizes Voyager flyby movies - the eventual goal is to produce a single movie with titles and audio as automatically as possible, with each planet and target having a separate segment.
The most challenging part will be reconstructing the geometry and assembling the mosaics automatically, as the camera pointing information available is not very accurate - it's generally to within 100 or so pixels of an 800x800 pixel image. This is due to limitations of the Voyager spacecraft pointing systems.
You can read more about the Planetary Data System (PDS) which hosts the archives here.
There are a total of 70k+ images in the Voyager archives, so there is a lot to explore!
- Example Movies
- Centering Images
- Aligning Composites
- More details
These movies are still in early stages, so pardon any jitters and mini 'volcanoes' (leftover from removal of reseau marks).
Voyager 2 Io approach v0.43
Voyager 1 Jupiter flyby (no moons), mostly false color (3mins) v0.47 (link to YouTube video)
There's a Trello board to track issues and progress.
I'm in the process of moving the system from Windows to Linux so it can use ISIS, which also requires switching from PDS archives to EDR archives - some of the README documentation is still geared towards the older system - the new one is in transition.
Contributors are very welcome - take a look at the Trello board and see if there's anything you'd like to work on!
Images where the target fits completely in the frame are centered using blob detection, Hough circle detection, and ECC Maximization. The expected target radius is calculated through SPICE data, from which the spacecraft and target position can be determined - this is used to help limit the Hough circle search, and then to draw a disc with the expected target size to which the image is aligned using ECC Maximization. The Hough circle detection is only accurate to a few pixels, so the ECC Maximization is needed for the final stabilization.
Here are a couple of images showing the result of the centering/stabilization - the yellow circle is the expected target size:
Centering is turned off at closest approach by determining when the target size is over a threshold (e.g. when the diameter is over 80% of the image width).
Composite channels for closeup images are aligned using feature detection and matching, with RANSAC to eliminate outliers from a least-squares fit model for the translation (which amounts to the translation tx, ty between images being an average of the feature movements).
In more detail, 'interesting' features are detected using ORB in one image, and matched with their corresponding point in another image. This is done for dozens-hundreds of interest points - they are each described with a feature vector, also obtained by ORB, then matched up with their corresponding point by a brute-force search. The RANSAC algorithm is used to throw out outliers, which would otherwise throw off the determined average translation.
If this approach fails to find a good translation (due to lack of enough corresponding points, for instance), it will fall back on ECC Maximization to try to align the images.
Here is an image showing what the feature-matching process looks like, and the resulting combined image (with enhanced contrast).
Voyager consists of a command line interface to a pipeline of Python programs with the following steps (some in progress):
- Download - download archives from PDS EDR (raw) archives
- Unzip - decompress archive volumes to IMQ files
- Import - import IMQ files to ISIS cube files, attach SPICE geometry data with
- Adjust - rotate 180 degrees, calibrate images
- Flatfield - subtract good flatfields (dark images)
- Dereseau - remove reseau marks cleanly (set to null)
- Denoise - identify/eliminate noise where possible (set to null)
- Inpaint - fill in missing information with pixels from prior frame or average of surrounding pixels - be careful with reseau marks on limbs of target
- Center - center and stabilize images where entire target is visible
- Map - project image to cylindrical map using SPICE information, fit there with ISIS jigsaw to refine pointing information
- Colorize - colorize images by pulling missing channels from the map
- Crop - crop and zoom frames, e.g. volcanoes on Io
- Annotate - add caption information, point out features, etc.
- Movies - combine images into movies, add music
For Windows, set up a Linux virtual machine
- Install VirtualBox
- Create a VM - set disk space at least 20GB, Memory at least 1GB
- Install a 64-bit Linux distro on it (ISIS is only 64-bit), e.g. Ubuntu or Xubuntu
- Install the VirtualBox Guest Additions (for higher screen resolutions and clipboard support)
Starting from Ubuntu 16.04
# set a location for applications, e.g. ~/Apps mkdir ~/Apps export APPS=~/Apps # install PyVoyager # (if using a virtual machine, can install on Windows instead so can access the image files from there also) cd $APPS git clone https://github.com/bburns/PyVoyager.git # install CSPICE (C language version of SPICE) # see https://naif.jpl.nasa.gov/naif/toolkit_C_PC_Linux_GCC_64bit.html cd $APPS wget http://naif.jpl.nasa.gov/pub/naif/toolkit//C/PC_Linux_GCC_64bit/packages/cspice.tar.Z wget http://naif.jpl.nasa.gov/pub/naif/toolkit//C/PC_Linux_GCC_64bit/packages/importCSpice.csh /bin/csh -f importCSpice.csh rm cspice.tar.Z rm importCSpice.csh ## install Java, for the ISIS installer #sudo apt install default-jre #sudo add-apt-repository ppa:webupd8team/java #sudo apt update #sudo apt install oracle-java8-installer # install ISIS cd $APPS wget https://isis.astrogeology.usgs.gov/documents/InstallGuide/assets/isisInstall.sh chmod +x isisInstall.sh mkdir Isis ./isisInstall.sh -n -d $APPS/Isis # add to .profile: export APPS=~/Apps export PYVOYAGER=$APPS/PyVoyager export SPICEROOT=$APPS/cspice export ISISROOT=$APPS/Isis/isis . $ISISROOT/scripts/isis3Startup.sh source ~/.profile # build camrotate #. or just include the binary - is it static? # get some libraries for building ISIS programs $ sudo apt install libxerces-c-dev $ sudo apt install libsuperlu-dev # change a line in $ISISROOT/make/config.linux-x86_64 as I couldn't get it to # recognize superlu4 as superlu4.3. superlu4.3 isn't available yet as a package - # it would require compiling it from source, which I didn't want to get into. # not sure if any ISIS programs need the 4.3 version. from SUPERLULIB = -lsuperlu_4.3 -lblas -lgfortran to SUPERLULIB = -lsuperlu -lblas -lgfortran # comment out a couple of lines in $ISISROOT/inc/SpecialPixel.h to turn off # some unused variable warnings - couldn't get pragma diagnostic to work line 101 // const double ValidMinimum = IVALID_MIN8.d; line 162 // const int IVALID_MAX4 = (*((const int *) &VALID_MAX4)); # make the program cd $PYVOYAGER/src/camrotate . setpaths.sh make #. add camrotate to PATH # get Voyager SPICE kernels locally pushd $ISIS3DATA rsync -avz --partial --progress --delete isisdist.wr.usgs.gov::isis3data/data/voyager1 . rsync -avz --partial --progress --delete isisdist.wr.usgs.gov::isis3data/data/voyager2 . popd # get some different Voyager 1 and Jupiter SPICE SPK kernels #. could just add these to git mkdir ~/PyVoyager/kernels/spk pushd ~/PyVoyager/kernels/spk wget ftp://naif.jpl.nasa.gov/pub/naif/generic_kernels/spk/satellites/a_old_versions/jup100.bsp wget ftp://naif.jpl.nasa.gov/pub/naif/VOYAGER/kernels/spk/Voyager_1.a54206u_V0.2_merged.bsp popd # get some libraries for ISIS (don't need if using earlier Ubuntu, e.g. 12.04) ## libblas3gf wget http://mirrors.kernel.org/ubuntu/pool/main/b/blas/libblas3gf_1.2.20110419-2ubuntu1_amd64.deb sudo dpkg -i libblas3gf_1.2.20110419-2ubuntu1_amd64.deb ## libjpeg62 sudo apt install libjpeg62 ## libvpx wget http://ftp.us.debian.org/debian/pool/main/libv/libvpx/libvpx1_1.3.0-3_amd64.deb sudo dpkg -i libvpx1_1.3.0-3_amd64.deb # install OpenCV version 3 ## (Ubuntu package is version 2) ## sudo apt install libopencv-dev sudo apt install build-essential sudo apt install cmake git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev sudo apt install python-dev python-numpy libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libjasper-dev libdc1394-22-dev cd $APPS wget https://github.com/Itseez/opencv/archive/3.1.0.zip unzip 3.1.0.zip rm 3.1.0.zip cd opencv-3.1.0 mkdir release cd release cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local .. cd .. make sudo make install ## release is >2gb, so remove it rmdir release ## can keep opencv-3.1.0 around though, as contains source code cd ~ # get some Python libraries # Python 2.7 is included with Ubuntu ## pip sudo apt install python-pip sudo pip install --upgrade pip ## numpy, scipy, matplotlib sudo apt install python-numpy python-scipy python-matplotlib ## cv2 (OpenCV Python interface) sudo apt install python-opencv ## SpiceyPy (SPICE Python interface) sudo pip install spiceypy ## miscellaneous sudo pip install tabulate sudo pip install more_itertools sudo pip install python-dateutil # get some other commands ## make a beep sound sudo apt install beep ## convenient way to run batch commands sudo apt install parallel ## image viewer, e.g. for jpegs sudo apt install eog
vg will show the available commands:
PyVoyager commands vg download - download volume(s) vg unzip - unzip volume(s) vg convert - convert IMGs to PNGs vg adjust - adjust images (rotate and enhance) vg denoise - remove noise from images vg center - center images vg inpaint - fill in missing pixels where possible vg composite - create color images vg target - copy images into target subfolders vg clips - create bw or color clips vg movies - create movies from clips vg list - show status of local datasets vg test center - run centering tests vg test denoise - run denoising tests where most commands can be followed by <filter> and <options>, where <filter> = [<volnums>] [<imageIds>] [<targetpath>] (all are anded together) <volnums> = 5101..5120 Voyager 1 Jupiter 6101..6121 Voyager 1 Saturn 5201..5214 Voyager 2 Jupiter 6201..6215 Voyager 2 Saturn 7201..7207 Voyager 2 Uranus 8201..8210 Voyager 2 Neptune (ranges and wildcards like 5101-5104 or 51* are ok) <imageIds> = imageId or range, like C1234567, C1234567-C1234569 <targetpath> = [<system>]/[<spacecraft>]/[<target>]/[<camera>] <system> = Jupiter|Saturn|Uranus|Neptune <spacecraft> = Voyager1|Voyager2 <target> = Jupiter|Io|Europa|, etc. <camera> = Narrow|Wide <options> = -y overwrite existing volume data e.g. vg clips 8205 //triton
Most commands will fill in any missing intermediate steps, so for example, to download, denoise, center, infill, composite, mosaic, and annotate all the Uranus images (which might take a while - there are 7 volumes of 1-3GB each), enter
> vg annotate 7*
Or you can be more explicit and run them individually, as follows (and note, many steps are optional, like denoise, infill, mosaic, annotate - though might need to tweak the code to turn off the automatic running of previous step) -
Download a tarfile volume, e.g. volume 5101 - the first dataset, Jupiter approach
> vg download 5101
Unzip the tarfile
> vg unzip 5101
Convert the IMG files to PNGs with img2png
> vg convert 5101
Adjust the contrast levels and rotate the images
> vg adjust 5101
Remove noise where possible
> vg denoise 5101
Center the images on the main body in the images
> vg center 5101
Colorize the images
> vg composite 5101
Annotate the images
> vg annotate 5101
Then you can make short movies of all the downloaded datasets, organized by planet/spacecraft/target/camera (this step must be performed in an Admin console, because it uses
mklink to make symbolic links, which require elevated privileges)
> vg clips [targetpath]
> vg clips //triton/narrow
to generate the narrow angle Triton flyby movies, or
> vg clips
to generate all available movies.
Then these clips can be assembled into movies (one per system and then one overall movie, as specified in db/movies.csv) with
> vg movies
vg list [volnums] command to keep track of what stages different volumes are at, e.g.:
Volume Download Unzip Convert Adjust Center Composite -------- ---------- ------- -------- -------- -------- ----------- 5101 x x x 5102 x x x 5103 x x x 5201 x x x x x x 6201 x x 7201 x x 7202 x x
All configuration settings are stored in
config.py - the goal is for the same set of parameters to work across all datasets as much as possible.
The data for each step is put into the following folders in the
step01_download step02_unzip step03_import
There are 87 PDS volumes for all the Voyager images, each ~1-3GB, as described here.
Each image comes in 4 formats - RAW, CLEANED, CALIB, and GEOMED.
- RAW images are the least processed images available - they're 800x800 pixels, and include the reseau marks (the grid of dots) used for calibration.
- CLEANED images have had the reseau marks removed, but leave noticeable artifacts that look like volcanoes on the limbs of planets.
- CALIB images have had dark images subtracted from the CLEANED images, and
- GEOMED are the CALIB images geometrically corrected and projected to 1000x1000 pixels.
Ideally the RAW images would be used with a better reseau removal algorithm, but for now the CALIB images are used.
After downloading the tar.gz files, unzipping them, extracting the PNGs, adjusting and denoising them, the CALIB images are centered based on blob detection, Hough circle detection, the expected target radius, and ECC maximization for stabilization. See the section on centering below for more details.
The expected radius of the target is determined in advance by the
vg init positions command, which uses SPICE position data, target position, target size, and camera FOV to determine size of target in image, which is stored in
db/positions.csv (included in the distribution). This helps with the Hough circle detection, and also to stabilize the image.
The PDS volumes come with index files for all the images they contain, which have been compiled into one smaller file using
vg init files. The resulting file (
db/files.csv, included with the distribution) looks like this:
fileid,volume,phase,craft,target,time,instrument,filter,note C1541422,5104,Jupiter,Voyager1,Jupiter,1979-02-01T00:37:04,Narrow,Blue,3 COLOR ROTATION MOVIE C1541424,5104,Jupiter,Voyager1,Jupiter,1979-02-01T00:38:40,Narrow,Orange,3 COLOR ROTATION MOVIE C1541426,5104,Jupiter,Voyager1,Jupiter,1979-02-01T00:40:16,Narrow,Green,3 COLOR ROTATION MOVIE ...
though different targets and camera records can be also interleaved with others.
One issue is that some images have more than one target in them (e.g. Jupiter with Io) - in the PDS index these images are listed with just one target. For now, you can change which target the image gets sorted under by editing the
db/targets.csv file - in the future it could be enhanced to also split the image into two records so each target can be included in the appropriate movie. One retargeting that is performed in advance is from rings to their planet - otherwise the rings would show up in separate movies.
The master list of files (
db/files.csv) has been compiled into a list of composite frames to build using the
vg init composites command, based on repeating groups of filters for the different targets and cameras. The resulting file (
db/composites.csv) looks like this:
compositeId,centerId,volume,filter C1541422,C1541422,5104,Blue C1541422,C1541424,5104,Orange C1541422,C1541426,5104,Green
This file is used by the
vg composite <volume> command to generate the color frames.
The clips are generated with the
vg clips [targetpath] command, which links all the images into target subfolders (arranged by planet/spacecraft/target/camera), numbering them sequentially, and running ffmpeg to generate an mp4 clip for each. The target size is also used to control the speed of the movie, slowing down when the target is closer, but the framerate can also be controlled via the
vg movies command then concatenates all available clips into movies, using the order specified in
Some centering test images are included in the
test/center folder, and their correct center values in
test/testCenterFiles.csv. You can run the tests on them with
vg test center. The goal is to include some easy targets and lots of edge cases to test the centering/stabilizing routines.
Denoising test images are located in
test/denoise - you can run the tests with
vg test denoise - check the results in the same denoise folder.
This software is released under the MIT license - see LICENSE.md.