Ingest Core API

Core Ingest Service for submitting experimental data to the DCP.

The API exposes a number of REST resources for representing experiment data:

• /projects
• /biomaterials
• /protocols
• /processes
• /files

See this document for an overview of how experimental data is structured together.

An example of the process of making a submission is in docs/primary-submission-walkthrough.md.

Tests

./gradlew verify

You will need a mongo database running on localhost with default ports to run the tests

Running locally

You can run ingest-core on your local system by using the local Spring profile. You would need to have a local mongodb and rabbitmq running, which you can run using docker-compose.

export SPRING_PROFILES_ACTIVE=local
./gradlew bootRun

populating the local DB

You could populate the local DB with prod data using the following script:

mkdir -p ~/dev/ait/data/mongodb/
cd ~/dev/ait/data/mongodb/
latest_backup=$(aws s3 ls s3://ingest-db-backup/prod/ | awk '{print $4}' | sort | tail -n 1)
aws s3 cp "s3://ingest-db-backup/prod/${latest_backup}" ${latest_backup}
tar -xzvf $latest_backup
backup_dir=$(echo "$latest_backup" | sed "s/\.tar\.gz//g")
mongorestore --drop "./data/db/dump/${backup_dir}"

./run_local.sh

Populating the local docker db

mkdir -p ~/dev/ait/data/mongodb/
cd ~/dev/ait/data/mongodb/
latest_backup=$(aws s3 ls s3://ingest-db-backup/prod/ | awk '{print $4}' | sort | tail -n 1)
aws s3 cp "s3://ingest-db-backup/prod/${latest_backup}" ${latest_backup}
mongo_container_name=ingest-core-mongo-1
docker cp $latest_backup $mongo_container_name:/$latest_backup
docker exec -i $mongo_container_name tar -xzvf $latest_backup
backup_dir=$(echo "$latest_backup" | sed "s/\.tar\.gz//g")
docker exec -i $mongo_container_name /usr/bin/mongorestore /data/db/dump/$backup_dir --drop

Privacy

Usage of the Ingest Core API require limited processing of personal data. For more information, please read our Privacy Policy.