Skip to content
Marketplace service REST JSON API server for registration and distribution of containerized applications.
Branch: master
Clone or download
Latest commit 940a6df Apr 2, 2019

README.md

HSP Marketplace Server

The Health Service Platform Marketplace Server is a REST JSON API reference implementation for publication, discovery and management of published service container images. It is assumed to be a relying party to an externally preconfigured OpenID Connect Identity Provider SSO system according to the OAuth 2 specification. The simple API does not contain a UI other than for account management. A post-authentication dashboard URL must instead be injected at runtime. The underlying internal domain model is represented as a normalized relational (PostgeSQL) schema. The Marketplace Server auto-forward-migrates its own schema and includes all the tools you need to establish default data for your deployment. For details, see:

https://healthservices.atlassian.net/wiki/display/PE/Platform+Engineering

Conceptual Overview

The system allows for initial web-based login via a configurable set OpenID Connect providers: part of the OAuth2 family of protocols. After, both a browser-based sessions is established, as well as a JWT that may be used to access the API for a time-limited period. As this is a model-driven system, the documentation is generated based on the model. For what the API actually does, see:

  • High-Level List of REST Routes - This is a generated dump.
  • Interactive API Tutorial. Note: You'll need Paw for OS X to open this, and need to replace the JWT and server instance configuration with your own details to run it.
  • Schema Diagram - This is a normalized .svg showing the physical database model, with additional OR/M-level annotations. It's useful in understanding how resources relate behind the API.
  • Database DSL - Generated database schema in ActiveRecord format.

The API

Pre-existing Security Resources

These endpoints would be pre-populated with "real" user data in an implementation prior to any actual use case. Most REST verbs are supported, used in the standard path'ing practice of "/<plural_noun>", "/<plural_noun>/:id", and utilization of sub-resources to represent model compositions and aggregations, when appropriate.

/users # Generic type for all known users.
/users/:id/identities # Actual authentication-related details specific to the IAM system.
/users/:id/roles # Roles granted to the User.
/groups	# An aggregation of Users.
/groups/:id/members # Establishes a given User's membership in a Group.
/groups/:id/roles # Roles granted to the Group; transitively to all Members.
/roles # Defines a granular set of permissions.

Developer Quick Start (OSX with Homebrew)

If you don't already have Postgres running locally:

brew install postgresql

Create a "hsp-marketplace-server" Postgres user using the dev/test credentials in config/database.yml, and assigned them full rights to manage schemas. As with most Ruby projects, use RVM to manage your local Ruby versions. Install RVM and:

rvm install 2.4.1
rvm use 2.4.1

Then,

bundle install # to install all server-side library dependencies.

The HSP Marketplace Server application is designed in 12factor style. Thus, the following environment variables are required to be set to support cookie-based CDN authorization grants. Set these in your ~/.bash_profile (or similar) and reload your terminal.

  • export MARKETPLACE_PASSWORD_SALT="some_unique_string" # Used for database password salting.
  • export MARKETPLACE_SECRET_KEY_BASE="some_unique_string" # Used for cryptographic signing of user sessions.
  • export MARKETPLACE_DATABASE_URL="postgres://marketplace:password@db.example.com:5432/marketplace_production " # Only used in "production" mode!
  • export MARKETPLACE_DATABASE_URL_TEST="postgres://marketplace:password@db.example.com:5432/marketplace_test " # Only used in "test" mode!
  • export MARKETPLACE_REDIS_URL="redis://localhost:6379"

The following additional environment variables are optional, but potentially useful in a production context. Note that the database connection pool is adjusted automatically based on these values. If in doubt, do NOT set these.

  • export MARKETPLACE_SERVER_PROCESSES=8 # To override the number of pre-forked workers.
  • export MARKETPLACE_SERVER_THREAD=8 # To override the number of threads per process.

Now,

rake db:create # to create empty marketplace_development and marketplace_test databases in Postgres
rake db:migrate # to apply all database migrations, in order, transactionally
rake db:seed # loads a simple set of starter data
rake test # to run all regression tests and generate a code coverage report. everything should pass!

You're now ready to run the application.

rails s # to run the server in development mode in the foreground.

To automatically re-run regression tests on detected code changes, open another terminal window and run

guard # hit <enter> to manually re-run all tests to run if a change isn't detected

Deployment

Deployment is done exclusively with Docker, though "raw" deployment using Passenger and all another common methods, including Heroku, are supported as well.

Building a Container

To build your current version:

docker build -t p3000/hsp-marketplace-server:latest .

Running a Container

You need a recent PostgreSQL database set up with an empty schema ready for the Marketplace to use. The schema will be automatically forward-migrated to the most current state every time the container starts.

When running the container, all environment variables defined in the above section must be set using -e FOO="bar" options to docker. The foreground form of the command is:

docker run -it --rm -p 3000:3000 --name hsp-marketplace-server \
	-e "MARKETPLACE_UI_URL=http://localhost:9000" \
	-e "MARKETPLACE_TITLE=My Marketplace" \
	-e "MARKETPLACE_SECRET_KEY_BASE=development_only" \
	-e "MARKETPLACE_DATABASE_URL=postgresql://marketplace:password@192.168.1.115:5432/marketplace_development" \
	p3000/hsp-marketplace-server:latest

...or to run in the background:

docker run -d -p 3000:3000 --name hsp-marketplace-server \
	-e "MARKETPLACE_UI_URL=http://localhost:9000" \
	-e "MARKETPLACE_TITLE=My Marketplace" \
	-e "MARKETPLACE_SECRET_KEY_BASE=development_only" \
	-e "MARKETPLACE_DATABASE_URL=postgresql://marketplace:password@192.168.1.103:5432/marketplace_development" \
	p3000/hsp-marketplace-server:latest

Once your server appears to be running, check your Postgres database to make sure tables have been created. First, run the including data seeding script: (You may keep any existing Makretplace containers running.)

docker run -it --rm -p 3000:3000 --name hsp-marketplace-server \
	-e "MARKETPLACE_UI_URL=http://localhost:9000" \
	-e "MARKETPLACE_TITLE=My Marketplace" \
	-e "MARKETPLACE_SECRET_KEY_BASE=development_only" \
	-e "MARKETPLACE_DATABASE_URL=postgresql://marketplace:password@192.168.1.115:5432/marketplace_development" \
	p3000/hsp-marketplace-server:latest \
	rake db:seed

This should be fast, and the script will exit once completed. Finally, you'll want to add a default data set and establish yourself as the initial administrator. Since the Marketplace requires an OpenID Connect identity provider (IDP) and does not authenticate on its own, the initial IDP setup takes a few additional steps. Once the first IDP is established, however, all future IDPs may be managed by the Marketplace UI or another client.

The seed data includes a default configuration for Google as an example. To use it, go to the Google Developers Console -> Credentials and create an "OAutuh Client ID" for a "web application". Note the client ID and client secret, which are specific to your installation. Now, start an interactive container into Marketplace Console mode to update the IDP configuration.

docker run -it --rm -p 3000:3000 --name hsp-marketplace-server \
	-e "MARKETPLACE_UI_URL=http://localhost:9000" \
	-e "MARKETPLACE_TITLE=My Marketplace" \
	-e "MARKETPLACE_SECRET_KEY_BASE=development_only" \
	-e "MARKETPLACE_DATABASE_URL=postgresql://marketplace:password@192.168.1.115:5432/marketplace_development" \
	p3000/hsp-marketplace-server:latest \
	rails console

idp = IdentityProvider.where(name: 'Google').first
idp.client_id = 'your_client_id'
idp.client_secret = 'your_client_secret'
idp.save!

Load a Marketplace UI in your browser and test your configuration. You should be able to log in using your existing Google account, which will trigger the Marketplace to create a local account for you as well. Finally, make yourself a full administrator at the previous contaier console by assigning your account to the adminstrator role.

me = idp.identities.first.user
administrator_role = Role.where(name: 'Administrator').first
Appointment.create(entity: me, role: administrator_role)

You're done! You may now use the Marketplace UI for all further admistratration. You should probably back up your database, too. ;)

Regression Testing a Container

The container includes a regression test suite to ensure proper operation. Running in test mode is slightly different, as to not inadvertently affect your production database(s). The application server process must also be told to run in 'test' mode.

docker run -it -m="512MB" \
	-e "RAILS_ENV=test" \
	-e "MARKETPLACE_DATABASE_URL_TEST=postgresql://hsp_marketplace:password@192.168.1.103:5432/marketplace_test" \
	... \
	p3000/hsp-marketplace-server:latest \
	rake test

License

Apache 2.0

You can’t perform that action at this time.