SILO API

SILO (Seed Images Location Operation) is a REST API provided by the Seed team for discovering Seed images. The API allows users to scan one or more repositories for seed images and then search the resulting images and their manifests by keywords. Frameworks such as Scale can use these endpoints to assist users in finding algorithm images and creating jobs out of them.

Install

To build/install the SILO webapp on linux:

Install go, docker and httpd packages
yum install go
yum install docker
yum install httpd
go get github.com/ngageoint/seed-silo
~/go/src/github.com/ngageoint/seed-silo/install-silo.sh

Or even better, run SILO from a seed-silo container:

docker run -p 9000:9000 docker.io/geoint/seed-silo:1.2.1

SILO runs on port 9000 so that port needs to be opened to the docker host. It could be changed to port 80 if desired with the flag '-p 80:9000'. The command above is all that is needed, but if customization is desired, the following environment variables can be passed in:

Table 1. Environment Variables

Name	Description
SILO_ADMIN	Specifies the username of the default admin. If not specified, this defaults to admin.
SILO_ADMIN_PASSWORD	Specifies the password of the default admin. If not specified, the default can be found in the codebase.
DATABASE_URL	Specifies the connection parameters for an existing database to use. This database should have a silo schema. If such a schema does not exist, it will be created. The environment variable follows the syntax outlined at dj-database-url project. In short, it follows the form: postgis://user:password@host[:port]/name
SILO_LITE_PATH	Specifies the path to use for a SQLite database if a postgres url is not specified via DATABASE_URL. If neither it nor DATABASE_URL is set the path /usr/silo/seed-silo.db is used. A database will be created inside the container if needed or a database can be mounted into the container at the path for persistence between runs.

Usage

Registry

Registries can be added, deleted and scanned. A registry consists of a name, url, organization (optional), username (optional), and password (optional).

Get Registry

Retrieves a registry

URL	/registries/{id}
Method	GET
URL Params	id = integer
Data Params	None
Success Response	Code: 200 Content: {"ID":1,"Name":"dockerhub","Url":"https://hub.docker.com","Org":"geointseed","Username":"","Password":""}
Error Response	Code: 400 Bad Request Content: { error : "Invalid ID" } Code: 404 File not found Content: { error : "No registry found with that ID" }
Sample Call	curl -X "GET" http://localhost:9000/registries/1

Add Registry

Adds a registry to the list of registries to be scanned. An error will be returned and the registry won’t be added if the daemon is unable to connect to the registry.

URL	/registries/add
Method	POST
URL Params	None
Data Params	{"name":"localhost", "url":"https://localhost:5000", "org":"", "username":"testuser", "password": "testpassword"}
Success Response	Code: 201 Content: {}
Error Response	Code: 400 Bad Request Content: { error : "Unable to connect to registry" } Code: 401 Unauthorized Content: { error : "Invalid authorization token" } or { error : "Missing authorization token" } Code: 403 Forbidden Content: { error : "User does not have permission to perform this action" } Code: 422 Unprocessable Entity Content: { error : "Error unmarshalling json. " }
Sample Call	curl -H "Authorization: Token <token>" -H "Content-Type: application/json" -d '{"name":"localhost", "url":"https://localhost:5000", "org":"", "username":"testuser", "password": "testpassword"}' http://localhost:9000/registries/add

Delete Registry

Removes a registry from the list of registries along with all images associated with that registry.

URL	/registries/delete/{id}
Method	DELETE
URL Params	id = integer
Data Params	None
Success Response	Code: 200 Content: { }
Error Response	Code: 400 Bad Request Content: { error : "Invalid ID" } Code: 401 Unauthorized Content: { error : "Invalid authorization token" } or { error : "Missing authorization token" } Code: 403 Forbidden Content: { error : "User does not have permission to perform this action" }
Sample Call	curl -H "Authorization: Token <token>" -X "DELETE" http://localhost:9000/registries/delete/1

Scan Registries

Removes all existing image entries, scans all registries for seed images and adds them to the database. Requires admin authorization token

URL	/registries/scan
Method	GET
URL Params	None
Data Params	None
Success Response	Code: 202 Content: { } Code: 202 Content: {"message":"Scanning Registries"}
Error Response	Code: 401 Unauthorized Content: { error : "Invalid authorization token" } or { error : "Missing authorization token" } Code: 403 Forbidden Content: { error : "User does not have permission to perform this action" }
Sample Call	curl -H "Authorization: Token <token>" "https://localhost:9000/registries/scan"

Scan Registry

Removes all existing image entries, scans all registries for seed images and adds them to the database. Requires admin authorization token

URL	/registries/{id}/scan
Method	GET
URL Params	id = integer
Data Params	None
Success Response	Code: 202 Content: { } Code: 202 Content: {"message":"Scanning Registries"}
Error Response	Code: 401 Unauthorized Content: { error : "Invalid authorization token" } or { error : "Missing authorization token" } Code: 403 Forbidden Content: { error : "User does not have permission to perform this action" }
Sample Call	curl -H "Authorization: Token <token>" "https://localhost:9000/registries/1/scan"

List Registries

Retrieves all of the registries that have been successfully added

URL	/registries
Method	GET
URL Params	None
Data Params	None
Success Response	Code: 200 Content: [ { "ID": 1, "Name": "localhost", "Url": "https://localhost:5000", "Org": "", "Username": "", "Password": "" } ]
Error Response	None
Sample Call	curl "https://localhost:9000/registries"

Image

Images are added/removed by scanning registries. An image consists of a name, registry, organization (optional), and the Seed manifest.

List Images

Retrieves all of the Seed images that have been scanned from registries

URL	/images
Method	GET
URL Params	None
Data Params	None
Success Response	Code: 200 Content: [ { "ID": 1, "RegistryId": 1, "Name": "my-job-0.1.0-seed:0.1.0", "Registry": "docker.io", "Org": "geointseed", "JobName": "my-job", "Title": "My first job", "Maintainer": "John Doe", "Email": "jdoe@example.com", "MaintOrg": "E-corp", "Description": "Reads an HDF5 file and outputs two TIFF images, a CSV and manifest containing cell_count", "JobVersion": "0.1.0", "PackageVersion": "0.1.0" }, { "ID": 2, "RegistryId": 1, "Name": "extractor-0.1.0-seed:0.1.0", "Registry": "docker.io", "Org": "geointseed", "JobName": "extractor", "Title": "Extractor", "Maintainer": "John Tobe", "Email": "jtobe@example.com", "MaintOrg": "", "Description": "Read’s a zip file and extracts the contents", "JobVersion": "0.1.0", "PackageVersion": "0.1.0" }, ]
Error Response	None
Sample Call	curl "https://localhost:9000/images"

Search Images

Searches the Seed images that have been scanned from registries and returns images matching the given query. Images are returned if the name, organization or manifest strings match the given query.

URL	/images/search/{query}
Method	GET
URL Params	query = string
Data Params	None
Success Response	Code: 200 Content: [ { "ID": 1, "RegistryId": 1, "Name": "my-job-0.1.0-seed:0.1.0", "Registry": "docker.io", "Org": "geointseed", "JobName": "my-job", "Title": "My first job", "Maintainer": "John Doe", "Email": "jdoe@example.com", "MaintOrg": "E-corp", "Description": "Reads an HDF5 file and outputs two TIFF images, a CSV and manifest containing cell_count", "JobVersion": "0.1.0", "PackageVersion": "0.1.0" }, { "ID": 2, "RegistryId": 1, "Name": "extractor-0.1.0-seed:0.1.0", "Registry": "docker.io", "Org": "geointseed", "JobName": "extractor", "Title": "Extractor", "Maintainer": "John Tobe", "Email": "jtobe@example.com", "MaintOrg": "", "Description": "Read’s a zip file and extracts the contents", "JobVersion": "0.1.0", "PackageVersion": "0.1.0" }, ]
Error Response	None
Sample Call	curl "https://localhost:9000/images/search/test"

Get Image

Retrieves an image

URL	/images/{id}
Method	GET
URL Params	id = integer
Data Params	None
Success Response	Code: 200 Content: { "ID": 1, "RegistryId": 1, "JobId": 1, "JobVersionId": 1, "FullName": "my-job-0.1.0-seed:0.1.0", "ShortName": "my-job", "Title": "My first job", "Maintainer": "John Doe", "Email": "jdoe@example.com", "MaintOrg": "E-corp", "JobVersion": "0.1.0", "PackageVersion": "0.1.0", "Description": "Reads an HDF5 file and outputs two TIFF images, a CSV and manifest containing cell_count", "Registry": "docker.io", "Org": "geointseed", "Manifest": "{\"seedVersion\":\"0.1.0\",\"job\":{\"name\":\"my-job\",…​}}" <full seed json> sample manifest }
Error Response	Code: 400 Bad Request Content: { error : "Invalid ID" } Code: 404 File not found Content: { error : "No image found with that ID" }
Sample Call	curl -X "GET" http://localhost:9000/images/1

Image Manifest

Returns the Seed manifest json for the given image id.

URL	/images/{id}/manifest
Method	GET
URL Params	id = integer
Data Params	None
Success Response	Code: 200 Content: sample manifest
Error Response	Code: 400 Bad Request Content: { error : "Invalid ID" } Code: 404 File not found Content: { error : "No image found with that ID" }
Sample Call	curl "https://localhost:9000/images/1/manifest"

Job

Jobs are groups of images with the same job name. A job has a name, title, maintainer, email, organization, description, latest job version and latest package version. It also has a list of images and job versions.

List Jobs

Retrieves all of the Jobs that have been scanned from registries

URL	/jobs
Method	GET
URL Params	None
Data Params	None
Success Response	Code: 200 Content: [ { "ID": 1, "Name": "my-job", "LatestJobVersion": "0.1.0", "LatestPackageVersion": "0.1.0", "Title": "My first job", "Maintainer": "John Doe", "Email": "jdoe@example.com", "MaintOrg": "E-corp", "Description": "Reads an HDF5 file and outputs two TIFF images, a CSV and manifest containing cell_count", "ImageIDs": [0, 1, 2], "JobVersions": [{JobVersion struct}, {JobVersion struct}…​] }, { "ID": 2, "Name": "another-job", "LatestJobVersion": "1.0.0", "LatestPackageVersion": "1.0.0", "Title": "My second job", "Maintainer": "John Doe", "Email": "jdoe@example.com", "MaintOrg": "E-corp", "Description": "blah blah blah", "ImageIDs": [3], "JobVersions": [{JobVersion struct}] }, ]
Error Response	None
Sample Call	curl "https://localhost:9000/jobs"

Search Jobs

Searches the Seed images that have been scanned from registries and returns jobs for the images matching the given query. Images are returned if the name, organization or seed manifest match the given query. Images/job versions that are irrelevant to the query are omitted from the ImageIDs and JobVersions structures.

URL	/jobs/search/{query}
Method	GET
URL Params	query = string
Data Params	None
Success Response	Code: 200 Content: [ { "ID": 1, "Name": "my-job", "LatestJobVersion": "0.1.0", "LatestPackageVersion": "0.1.0", "Title": "My first job", "Maintainer": "John Doe", "Email": "jdoe@example.com", "MaintOrg": "E-corp", "Description": "Reads an HDF5 file and outputs two TIFF images, a CSV and manifest containing cell_count", "ImageIDs": [0, 2], "JobVersions": [{JobVersion struct}, {JobVersion struct}…​] }, { "ID": 2, "Name": "another-job", "LatestJobVersion": "1.0.0", "LatestPackageVersion": "1.0.0", "Title": "My second job", "Maintainer": "John Doe", "Email": "jdoe@example.com", "MaintOrg": "E-corp", "Description": "blah blah blah", "ImageIDs": [3], "JobVersions": [{JobVersion struct}] }, ]
Error Response	None
Sample Call	curl "https://localhost:9000/jobs/search/test"

Get Job

Retrieves a job

URL	/jobs/{id}
Method	GET
URL Params	id = integer
Data Params	None
Success Response	Code: 200 Content: { "ID": 1, "Name": "my-job", "LatestJobVersion": "0.1.0", "LatestPackageVersion": "0.1.0", "Title": "My first job", "Maintainer": "John Doe", "Email": "jdoe@example.com", "MaintOrg": "E-corp", "Description": "Reads an HDF5 file and outputs two TIFF images, a CSV and manifest containing cell_count", "ImageIDs": [0, 2], "JobVersions": [{JobVersion struct}, {JobVersion struct}…​] }
Error Response	Code: 400 Bad Request Content: { error : "Invalid ID" } Code: 404 File not found Content: { error : "No job found with that ID" }
Sample Call	curl -X "GET" http://localhost:9000/jobs/1

Job Version

Job Versions are groups of images with the same job name and the same job version. A job version has a job name, job id, job version, latest package version and a list of images.

List Job Versions

Retrieves all of the Job Versions that have been scanned from registries

URL	/job-versions
Method	GET
URL Params	None
Data Params	None
Success Response	Code: 200 Content: [ { "ID": 1, "JobName": "my-job", "JobId": "1", "JobVersion": "0.1.0", "LatestPackageVersion": "0.1.0", "Images": [{Image struct}, {Image struct}…​] }, { "ID": 2, "JobName": "my-job", "JobId": "1", "JobVersion": "0.2.0", "LatestPackageVersion": "0.2.0", "Images": [{Image struct}] }, ]
Error Response	None
Sample Call	curl "https://localhost:9000/job-versions"

Get Job Versions

Returns the job versions for a specific job

URL	/jobs/{id}/job-versions
Method	GET
URL Params	id = int
Data Params	None
Success Response	Code: 200 Content: [ { "ID": 1, "JobName": "my-job", "JobId": "1", "JobVersion": "0.1.0", "LatestPackageVersion": "0.1.0", "Images": [{Image struct}, {Image struct}…​] }, { "ID": 2, "JobName": "my-job", "JobId": "1", "JobVersion": "0.2.0", "LatestPackageVersion": "0.2.0", "Images": [{Image struct}] }, ]
Error Response	Code: 400 Bad Request Content: { error : "Invalid ID" } Code: 404 File not found Content: { error : "No job found with that ID" }
Sample Call	curl "https://localhost:9000/jobs/1/job-versions"

Get Job Version

Retrieves a job version

URL	/job-versions/{id}
Method	GET
URL Params	id = integer
Data Params	None
Success Response	Code: 200 Content: { "ID": 1, "JobName": "my-job", "JobId": "1", "JobVersion": "0.1.0", "LatestPackageVersion": "0.1.0", "Images": [{Image struct}, {Image struct}…​] }
Error Response	Code: 400 Bad Request Content: { error : "Invalid ID" } Code: 404 File not found Content: { error : "No job version found with that ID" }
Sample Call	curl -X "GET" http://localhost:9000/jobs/1

User

Users can be added, deleted, listed and used to login. A user consists of a username, password, and a role.

Get User

Retrieves a user

URL	/users/{id}
Method	GET
URL Params	id = integer
Data Params	None
Success Response	Code: 200 Content: {"ID":1,"username":"admin","role":"admin"}
Error Response	Code: 400 Bad Request Content: { error : "Invalid ID" } Code: 404 File not found Content: { error : "No user found with that ID" }
Sample Call	curl -X "GET" http://localhost:9000/user/1

Add User

Adds a user to the system. Requires a valid token from an admin user.

URL	/users/add
Method	POST
URL Params	None
Data Params	{"username":"admin", "password": "hunter17", "role": "admin"}
Success Response	Code: 201 Content: {"username":"admin", "password": "hunter17", "role": "admin"}
Error Response	Code: 401 Unauthorized Content: { error : "Invalid authorization token" } or { error : "Missing authorization token" } Code: 403 Forbidden Content: { error : "User does not have permission to perform this action" } Code: 422 Unprocessable Entity Content: { error : "Error unmarshalling json. " }
Sample Call	curl -H "Content-Type: application/json" -d '{"username":"admin", "password": "hunter17", "role": "admin"}' -H "Authorization: Token <token>" http://localhost:9000/users/add

Delete User

Removes a user from the system. Requires a valid token from an admin user.

URL	/users/delete/{id}
Method	DELETE
URL Params	id = integer
Data Params	None
Success Response	Code: 200 Content: { }
Error Response	Code: 400 Bad Request Content: { error : "Invalid ID" } Code: 401 Unauthorized Content: { error : "Invalid authorization token" } or { error : "Missing authorization token" } Code: 403 Forbidden Content: { error : "User does not have permission to perform this action" }
Sample Call	curl -X "DELETE" -H "Authorization: Token <token>" http://localhost:9000/users/delete/1

List Users

Retrieves all of the users in the system

URL	/users
Method	GET
URL Params	None
Data Params	None
Success Response	Code: 200 Content: [ { "ID": 1, "username": "admin", "role": "admin" }, { "ID": 2, "username": "user", "role": "user" } ]
Error Response	None
Sample Call	curl "https://localhost:9000/users"

Login

Authenticates a user and returns a token to be used in subsequent api calls

URL	/login
Method	GET
URL Params	None
Data Params	{"username":"admin", "password": "password"}
Success Response	Code: 200 Content: {"token":"<token>"}
Error Response	Code: 401 Unauthorized Content: { error : "Invalid login" } Code: 422 Unprocessable Entity Content: { error : "Error unmarshalling json. " }
Sample Call	curl -H "Content-Type: application/json" -d '{"username":"admin", "password": "password"}' "http://localhost:9000/login"