Synda can download files from the Earth System Grid Federation (ESGF) archive in an easy way, based on a list of facets (variables, experiments, ensemble members, etc..). The program will evolve together with the ESGF archive backend functionalities.
The user defines one or many templates. Each of them have a list of facets (variables, frequencies, experiments, ensemble, model..). Using these templates, the program explore the ESGF archive and dowload all the corresponding available files. The program may be run regularly to download the possible new files. Typically each template is associated with an analysis (cfmip template, downscaling template and so on). Create as many template_name.txt as you want in the 'selection' folder. Selection file examples are given in the 'sample' sub-folder.
User guide | Admin guide | Upgrade guide | Howto & Tutorial
Wiki | Slides | Faq |Template sample | License
Search datasets
synda search FACET..
Install a dataset
synda install DATASET
Start the daemon
synda daemon start
Once downloaded, files are available in this location
$HOME/sdt/data
This program is a command-line alternative to the ESGF web front-end.
- Simple data installation using an apt-get like command
- Parallel downloads
- Incremental process (download only what's new)
- Download history stored in a db
- Transfer priority
- Linux
- Mac
Stable: 3.0
Testing: 3.1
Python 2.6+
As root, install the following system packages:
Centos
yum install gcc python python-pip python-devel openssl-devel sqlite sqlite-devel libxslt-devel libxml2-devel zlib-devel libffi-devel
Debian
apt-get install gcc python python-pip python-dev libssl-dev sqlite3 libsqlite-dev libxslt-dev libxml2-dev libz-dev libffi-dev
Then install the application (as simple user or root):
wget --no-check-certificate https://raw.githubusercontent.com/Prodiguer/synda/master/sdc/install.sh
chmod +x ./install.sh
./install.sh
Add lines below in your shell configuration file (e.g. '.bashrc')
export ST_HOME=$HOME/sdt
export PATH=$ST_HOME/bin:$PATH
Then edit $ST_HOME/conf/sdt.conf to set openid and password (ESGF credential).
Note: to download file from ESGF, you need to create an openID account on one ESGF identity provider website (e.g. PCMDI, BADC or DKRZ) and subscribe to CMIP5-RESEARCH role.
sdipsl AT ipsl DOT jussieu DOT fr
synda has incorporated code from several sources. Users have contributed patches and suggestions over time. This work has been undertaken by IPSL and as been funded by IPSL, IS-ENES and France-Grilles.