Metadata and data product catalogue.
The catalog API is a Node.js REST server application written in Typescript.
Install Node.js (Note: Ubuntu requires the nodejs-legacy
package to create a node
symlink
https://nodejs.org/en/download/package-manager/#debian-and-ubuntu-based-linux-distributions)
apt install nodejs-legacy
Install Typescript and Yarn
npm i -g typescript
npm i -g yarn
Install Sphinx for documentation
pip install sphinx sphinx-autobuild
For linting:
pip install restructuredtext-lint
Install the reStructuredText extension for vs code by searching for:
restructuredtext publisher:"LeXtudio"
Further configuration for the plugin can be found here: https://github.com/vscode-restructuredtext/vscode-restructuredtext/blob/master/docs/sphinx.md
Database access parameters such as server location and authentication are provided through environment variables. A selection of scripts is available in /scripts folder.
Save a copy outside of source control and edit.
Either fill in a .env file with the appropriate Environment values or add the apporpriate variables to your system environment, all required variables are listed in .env.example
You're good to go.
cd ./app
yarn
yarn dev
This will run the API server. Don't forget the tests!
yarn test
-
Install postgres package.
-
Install postgis package.
-
su to the postgres user
sudo su - postgres
-
create the database
psql -c "CREATE DATABASE catalog;" psql -d catalog -f ./dev/database/setup-scripts/database.sql psql -d catalog -f ./dev/database/setup-scripts/collection.sql psql -d catalog -f ./dev/database/setup-scripts/product.sql psql -d catalog -f ./dev/database/setup-scripts/product_view.sql
-
create the user with a password (CHANGE THE PASSWORD BELOW)
psql -d catalog -c "CREATE USER catalog WITH ENCRYPTED PASSWORD 'password';" psql -d catalog -c "GRANT connect ON DATABASE catalog TO catalog;" psql -d catalog -c "GRANT usage ON SCHEMA public TO catalog;" psql -d catalog -c "GRANT SELECT, INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA public TO catalog;"
The Catalog API has documentation generator. Docs follow this template: https://docs.google.com/document/d/1HSQ3Fe77hnthw8hizqvXJU-qGEPHavMkctvCCadkVbY/edit?pli=1#
to build docs, run
yarn docs
Table generator is handy for building complex tables. http://www.tablesgenerator.com/text_tables
To build the docker container simply run:
yarn docker
while in the app directory and this should build an image called jncc/catalog.
This image is currently being hosted at docker hub under our JNCC account if you want to pull a particular verison (1.0.0 - 1.0.4 currently) or just the latest run docker pull jncc/catalog:latest
.
If you need to run the container locally for testing you can run with the following command: NB - Run from the project root
docker run --name catalog_api -p 9001:8081 -d --env-file .env jncc/catalog
where the --env-file .env
parameter points to a .env with all the configuration required as in the .env.example
file.
The NODE_ENV
environment varaible must not be set to developement
when its being supplied to the docker container however as this is an option only for local development outside the docker container
This will run a container with the port exposed at http://localhost:9001
.
Postgres will by default only listen to the localhost (127.0.0.1) and probably hasn't been configured to do anything else on a development machine. However docker vm's run on there own network and postgres must be configured to interact with this.
Get the docker network :
ifconfig docker0 | grep inet
This will give an output as follows:
inet 172.17.0.1 netmask 255.255.0.0 broadcast 172.17.255.255
inet6 fe80::42:54ff:feab:e808 prefixlen 64 scopeid 0x20<link>
In this case Postgres needs to listen on 172.17.0.1 and accept authenticated connections from the 172.17.0.1/16 ip address range before it will work with docker.
Launch a database instance with an API:
cd dev
docker-compose build
Use up
and down
to run the images.
docker-compose up
An API should be running on http://localhost:6080/alive
Load the example data in:
docker exec -it dev_database_1 /bin/bash -c "/setup/setup.sh"
To clean up, remove the .temp
dir.
There is a .vscode/launch.json configuration called Mocha Tests
for debugging the tests. Not used much yet.