Coriolis data processing chain for Argo floats Containerized
graph TD
subgraph Docker[Docker]
Argo-decoder[DCK Argo.x.y.z]
end
subgraph Volumes[Volumes]
subgraph Data[DATA]
Message[message]
Config[json]
Output[netcdf]
end
Runtime[RUNTIME]
subgraph Ref[REF]
Gebco[GEBCO]
Greylist[Greylist]
end
end
Docker -- Read only --> Runtime
Docker -- Read / Write --> Data
Docker -- Read only --> Ref
Flux :
https://data-argo.ifremer.fr/ar_greylist.txt
Volumes :
/mnt/runtime
: Matlab runtime environment/mnt/data/output
: Output files directory/mnt/data/rsync
: Input files directory/mnt/data/config
: external configurations directory/mnt/ref/gebco.nc
: GEBCO file
- Define next variables to configure the decoder on your environment
DECODER_IMAGE=<decoder image path>
DECODER_IMAGE_TAG=<decoder image tag>
DECODER_RUNTIME_DIR=<path to runtime directory>
DECODER_DATA_INPUT=<path to input directory>
DECODER_DATA_CONF=<path to conf directory>
DECODER_DATA_OUTPUT=<path to output directory>
DECODER_REF_GEBCO=<path to gebco file> # Optionnal
DECODER_REF_GREYLIST=<path to gebco file> # Optionnal
USER_ID=<uid volumes owner>
GROUP_ID=<gid volumes owner>
- Run the following script as an example to decode a single float.
FLOAT_WMO=6902892
rm -rf $DATA_OUTPUT/iridium/*$FLOAT_WMO
rm -rf $DATA_OUTPUT/nc/$FLOAT_WMO
docker run -it --rm \
--name "argo-decoder-container" \
--user $USER_ID:$GROUP_ID \
--group-add gbatch \
-v $DECODER_RUNTIME_DIR:/mnt/runtime:ro \
-v $DECODER_DATA_INPUT:/mnt/data/rsync:rw \
-v $DECODER_DATA_CONF:/mnt/data/config:ro \
-v $DECODER_DATA_OUTPUT:/mnt/data/output:rw \
$DECODER_IMAGE:$DECODER_IMAGE_TAG /mnt/runtime 'rsynclog' 'all' 'configfile' '/app/config/_argo_decoder_conf_ir_sbd.json' 'configfile' '/app/config/_argo_decoder_conf_ir_sbd_rem.json' 'xmlreport' 'co041404_'$(date +"%Y%m%dT%H%M%SZ")'_'$FLOAT_WMO'.xml' 'floatwmo' ''$FLOAT_WMO'' 'PROCESS_REMAINING_BUFFERS' '1'
- Use the following command to build the Docker image.
docker build -t decode-argo:develop .
- Run see run section to run the image.
This demonstration will run the Coriolis-data-processing-chain-for-Argo-floats based on a Dockerised Matlab Runtime on two Argo floats :
- Arvor 6903014 : https://fleetmonitoring.euro-argo.eu/float/6903014
- Arvor Deep 6903014 : https://fleetmonitoring.euro-argo.eu/float/6902892
-
Lunix operating system Required
-
Folowing procedure to Install Docker Engine according to your operating system, or use next example using generic script (not recommended for production environments)
# Install docker with generic script curl -fsSL https://get.docker.com/ | sudo sh # add your user to docker group sudo usermod -aG docker $USER # Check that the Docker Engine installation is successful by running the hello-world image sudo docker run hello-world
-
Install Git (Optionnal)
sudo apt update sudo apt install git
-
Upload the project with demonstration dataset
- Option 1 : Using Git
cd path-to-working-directory git clone -b feature/workshop https://github.com/euroargodev/Coriolis-data-processing-chain-for-Argo-floats.git cd Coriolis-data-processing-chain-for-Argo-floats
- Option 2 : Manual download
cd path-to-working-directory wget https://github.com/euroargodev/Coriolis-data-processing-chain-for-Argo-floats/archive/refs/heads/feature/workshop.zip unzip workshop.zip -d ./Coriolis-data-processing-chain-for-Argo-floats cd Coriolis-data-processing-chain-for-Argo-floats
-
Edit environment variables
.env
file with your favorite text editor to setup your configuration, or use the next commands :# or at least these commands tu setup your user sed -i "s/REPLACE_BY_USER_ID/$UID/g" .env sed -i "s/REPLACE_BY_GROUP_ID/$(id -g $UID)/g" .env
-
Run decoder demo with matlab runtime thanks to docker compose
docker compose --profile matlab up
-
Check next directory to see decoder outputs :
./decArgo_demo/output