This is a sample project I've been using to learn API development in the node ecosystem. Technologies include:
- MySql for storing data
- sqlite3 for storing data used by automated integration tests
- Knex.js querybuilder used by Objection
- Objection.js bare-bones ORM for Node.js
- express node http server
- jest / supertest for automated tests
- postman for manual integration tests
- swagger-jsdoc and swagger-ui-express to generate and display OpenAPI 3 API specification
- Docker to stand up the five services needed for authentication and running the API
- OAuth/OIDC for authentication
- jose for JWT validation
- Balsamiq for documentation drawings
The API is intended to back a simplified "scheduling" application where events can be placed on a calendar and people can be assigned to them. The entities are:
- event type: This is an activity that can be scheduled on the calendar multiple times (i.e. "lunch" can be scheduled on multiple days).
- event: An event type that has been scheduled (i.e."lunch" on Nov. 30, 2021 from 11:30 AM to 12:30 PM). This date/time values are qualified by a timezone, which is constrained by a list of timezones.
- person: A person who can be assigned to an event.
- assignment: The association of a person to an event.
The API was very simple until I decided to implement realistic token authentication. To do this, I needed to implement an authenticating proxy backed by a local federating OAuth/OIDC provider. It has the features I need to test token authentication in the API layer, but the login sequence between the proxy and the provider doesn't really meet my expectations, so it might be rewritten again using https://github.com/panva/node-oidc-provider and https://github.com/http-party/node-http-proxy, or perhaps Keycloak. In a real production environment, the OAuth/OIDC provider would be a third-party solution, either from the cloud provider or from a vendor like Okta.
MySql database names and passwords are mounted into the Docker instances as volumes. The mounted files must be created locally, since they are not checked in. Each secret is contained in its own file. Files are in screaming snake case with no extension. You can choose any legal username and password. There are two secrets directories: one for the API and one for Dex.
-
./config/MYSQL_USERNAME
-
./config/MYSQL_PASSWORD
-
./config/MYSQL_ROOT_PASSWORD
-
./proxy/config/MYSQL_USERNAME
-
./proxy/config/MYSQL_PASSWORD
-
./proxy/config/MYSQL_ROOT_PASSWORD
To leave the committed configuration files unchanged, the application must be addressed as:
https://lednicky.localhost
If you want to skip configuring your hosts file, you can update the paths in \proxy\oauth2-proxy\oauth2-proxy.config.yaml and \proxy\dex\dex.config.yaml.
Some of the scripts in package.json can run the API independently of the docker-compose environment. These include "devstart", "migrate", and "seed", which allow you to point the application at any MySQL 8 database. To do this, you'll need to provide a dotenv file that exposes the secret files as environment variables. Do this by creating a file at /api/.env It should include the following:
MYSQL_HOST_FILE='../config/MYSQL_HOST'
MYSQL_DBNAME_FILE='../config/MYSQL_DBNAME'
MYSQL_USERNAME_FILE='../config/MYSQL_USERNAME'
MYSQL_PASSWORD_FILE='../config/MYSQL_PASSWORD'
OAUTH_ISSUER='https://lednicky.localhost/dex'
OAUTH_AUDIENCE='lednicky.localhost'
Add this to your hosts file:
127.0.0.1 lednicky.localhost
127.0.0.1 dex.lednicky.localhost
For instructions on how to do this on Windows 10, see https://superuser.com/a/1120395
Create a self-signed wildcard certificate for *lednicky.localhost
NOTE: Take special care to make sure this is a wildcard certificate, since it is used for both https://lednicky.localhost and https://dex.lednicky.localhost
For instructions on how to do this on Windows 10, see https://stackoverflow.com/questions/21397809/create-a-trusted-self-signed-ssl-cert-for-localhost-for-use-with-express-node
Create the following file and put the public key in it: ./proxy/config/lednicky.localhost.crt
Create the following file and put the decrypted private key in it: ./proxy/config/lednicky.localhost.decrypted.key
Both the proxy and dex will pick up these files and configure themselves correctly.
This is easy to get wrong.
The proxy and the postman tests have been configured to ignore invalid SSL certificates to eliminate some of the problems associated with this configuration.
To build and stand up the API (on port 3333) and the proxy (on port 443)
Make sure Docker Desktop is running first and then execute this command:
docker-compose up -d --build
The MySql container reports itself as ready before the database can receive connections. Dex frequently errors out on startup with an error about being unable to access the database. You can restart the dex container by itself after the environment is running with the following command: docker-compose restart dex.
The swagger documentation from the API layer does not require authentication, so you should be able to navigate directly to:
https://lednicky.localhost/api/api-doc/ https://lednicky.localhost/api/api-doc-ui/
After you've authenticated, you can call this API diagnostic endpoint to see the JWT:
https://lednicky.localhost/api/inspectRequest/headers
You won't be able to log into Dex locally until there are users in the Dex database. To do that, log into the dex-sql database docker container (named "dex-sql"), run MySql, then execute the script to insert the test users.
Open the shell:
docker exec -it dex-sql sh
From the shell command prompt, login as root to MySql
# mysql -u root -p
When prompted, enter the password you created for the root user of the dex database.
From the resulting command prompt, you can paste the contents of /proxy/test-users/test-users.sql
That file creates test users as follows.
+------------------+------------------+---------+
| email | username | user_id |
+------------------+------------------+---------+
| ann@dot.com | ann.abbot | 1 |
| bob@dot.com | bob.boebert | 2 |
| carol@dot.com | carol.cruz | 3 |
| dan@dot.com | dan.dinkle | 4 |
| erin@dot.com | erin.emmerlin | 5 |
| federico@dot.com | federico.fuentes | 6 |
+------------------+------------------+---------+
password is the first name in lowercase
Get out of MySql by typing "quit", then get out of the container by typing "exit"
See more at api/db/migrations/migrations.notes.md
The API won't work at all until the schema has been created. It won't do anything useful until it has been seeded with data. Do that by logging into the API container (named "api") and running knex migrations and seeds.
Open the shell:
docker exec -it api sh
From the prompt, run the migrations, which set up the tables in the database:
npx knex migrate:latest
From the prompt, run the seeds, which populates the tables with test data:
npx knex seed:run
For a description of the test data, see api/db/seeds/seeds.notes.md
Run jest tests with the following command from /api:
yarn test
These are mostly integration tests targeting the service layer.
For details, see api/tests/services/services.tests.md;
Postman test exports can be found at api/tests/postman
For details, see tests/postman/postman.tests.md;