This project was created initially for research purposes, reviewing how various other apps organize their Node/Express/Typescript repos and trying to pick the best parts of each. It is now a reference / starter API project any startup could use to start off on the right foot with key functionality needed for future scaling and support.
- Authentication (register, login, verify email, lost password, login as (assume identity))
- Authorization (role-based access control with attribute filtering)
- Features flags (flags, multi-variant flags, user segments, environments, goal tracking)
- Event logging (logs activity stream of user events, search and filter log records)
- Graph relations (automatically builds graph from activity stream events)
- Performance monitoring (logging process time per request to identify bottlenecks)
- Security (token deny list, helmet, more to come...)
- Email using Sendgrid transactional email
- Docker Compose (rapid setup including database admin tools)
- Database integrations (Postgres, Redis, Elasticsearch)
- Containers (optionally build app in Docker container with multi-stage build)
- Continuous Integration (CI config builds, lints, tests on push with CircleCI)
- Flexible architecture (Dao layer allows adding future "views", EventEmitter allows CQRS-ES)
- Strongly-typed codebase (written in Typescript)
- Interactive API documentation using Swagger UI (Open API spec)
- Automated testing (including integration tests for API routes)
- Jest unit and integration tests (
npm test
) - Locust load testing (see
locustfile.py
)
- Jest unit and integration tests (
- User groups support (authorization based on membership roles)
- User invite support
- Internationalization
- Additional view layers (i.e. GraphQL)
- Workspaces
- clone the repo
npm install
- Setup temp environment configs (TEST only)
- RUN in CLI from project root
./setenv.test.sh
- RUN in CLI from project root
- Make note of generated files and change to your preferences
- IMPORTANT: when deploying app, don't use the
.env
file, simply set vars in your CI provider or container manager
- IMPORTANT: when deploying app, don't use the
docker-compose up
- This will spin up Postgres, PGAdmin, Elasticsearch, Kibana, and Redis
- IMPORTANT: run
./setup_es.sh
to create index mapping templates for Elasticsearch after startup - To stop them, and remove local volumes:
docker-compose down -v
- Run tests (will load up test data in tables)
npm run test
- Start up app in developer mode (will watch and recompile for changes)
npm run dev
- Open browser tab to Swagger UI Explorer to explore API
- Open browser tab to Postgres Admin for Postgres Admin
- click on "Servers" and then "Object > Create > Server"
- "General > Name" the connection "Test Server"
- click on "Connection" tab:
- Host:
postgres
(network exposed by docker-compose) - Password:
admin
(or whatever you set in ENV vars)
- Host:
- click on "Save"
- traverse "Servers > Test Server > Databases > auth_example > Schemas > public"
- Open browser tab to Kibana for Elasticsearch Admin
- You will have to know a little about ES if you choose to use it
This app includes automated tests using Supertest and Jest to test routes, etc.
npm test
ornpm run coverage
- NOTE: the Docker database must be running (see Step 5 above)
- SEE:
__tests__
folders in application for test source code
Every DAO method should emit
an event in an activity stream format as shown. For max flexibility,
like to disable writes and make the architecture CQRS, you can create a new handler in utils/activity.helper.ts
.
You can follow along the commit history relating to the issues (closed) and see how, but a general idea is:
- add a new folder (i.e. category) in the
src/services/
folder - if CRUD feature with database, copy
src/services/role/*
and find/replace to match new names - edit
src/config/data.test.ts
and add necessary test data and permissions for CRUD - if new providers, add
src/config/{provider}.ts
and make connections fromprocess.env.{vars here}
- be sure to add test vars to
setenv.test.sh
,src/utils/validation.helper.ts
,.env
- be sure to add test vars to
- edit
src/config/openapi.json
and add routes to documentation (if REST implementation) - if non-CRUD feature, add a new
src/config/event.ts
listener and be sure toemit
within Dao methods- this is key to support CQRS-ES architecture and scale to PubSub or message bus in future as needed
See the /config/data.test.ts
file to see how permissions, roles, and users were added to the database
that fulfill the requirements below. The /util/{type}.helper.ts
files abstract the specific module implementation
as much as possible so we could change out solutions in future without modifying the code base.
- As a
guest
, I want to be able toregister
orlogin
, so that I can access features within the app - As a
guest
, I want to confirm my validemail
address, so that I can gain access to the application - As a
guest
, I want to be able to submit myemail
credentials, so I can stilllogin
if my password is lost
- As a
user
, I want to be able tosearch
by city name, so I can view geo data about the city - As a
user
, I want to be able to viewusers
(without age or password), so I know users of the system - As a
user
, I want to be able to edit my ownuser
record, so I can keep my information current - As a
user
, I want to be able to view a list ofroles
(without permissions) so I know what roles are available - As a
user
, I want to be able tologout
, so that my authentication session cannot be used by others - As a
user
, I want to be informed if attempts to gain access to my account occur, so I can help prevent unauthorized access - As a
user
, I want to be able to disable one or more devices (tokens), so I can prevent unauthorized access - As a
user
, I want the app to respond quickly, so I don't have to wait for information
- As an
admin
, I want to be able tosearch
by city name, so I can view geo data about the city - As an
admin
, I want to be able to viewusers
(with age but no password), so I know users of the system and their age - As an
admin
, I want to be able to create anyuser
record, so I can manage users in the system - As an
admin
, I want to be able to edit anyuser
record, so I can manage users in the system - As an
admin
, I want to be able to delete anyuser
record, so I can manage users and keep the system current - As an
admin
, I want to be able to viewroles
(with permissions), so I know roles of the system and their permissions - As an
admin
, I want to be able to create anyrole
record, so I can manage roles in the system - As an
admin
, I want to be able to edit anyrole
record, so I can manage roles in the system - As an
admin
, I want to be able to delete anyrole
record, so I can manage roles and keep the system current - As an
admin
, I want to be able to viewroles
for any user, so I know users of the system and their roles - As an
admin
, I want to be able to add anyrole
record to any user, so I can manage users and their roles in the system - As an
admin
, I want to be able to remove anyrole
record from any user, so I can manage users and keep the system current - As an
admin
, I want to be able to viewpermissions
for any role, so I know permissions of the system and their permissions - As an
admin
, I want to be able to add anypermission
record to any role, so I can manage permissions in the system - As an
admin
, I want to be able to remove anypermission
record from any role, so I can manage permissions and keep the system current - As an
admin
, I want to be able to deny any user or user device token, so I can manage user and device access
- As a
sysadmin
, I want to be able to automatically check app health, so I can automate scaling and recovery - As a
sysadmin
, I want the app to log all events, so that I can optionally add alerts if acceptable thresholds are exceeded - As a
sysadmin
, I want to be able to deny protected access to any user or individual token, so I can prevent unauthorized access - As a
sysadmin
, I want to be able to disable client features that may have issues, so I can maintain app stability - As a
sysadmin
, I want to identify actions a support user performs on behalf of users, so I can audit and explain data
- As a
support user
, I want to be able to initiate a lost password request for a user, so I can take care of them in real time - As a
support user
, I want to be able to assume a users identity without password, so I can use the app as they would and help identify issues - As a
support user
, I want to be able to search for and view all users, so I can take care of them in real time - As a
support user
, I want to be able to edit a limited set of user data, so I can take care of them in real time
- As a
tech support
, I want to view and filter user activity logs in real time, so I can troubleshoot issues before escalation - As a
tech support
, I want to be able to edit a limited set of user data, so I can test different configurations during troubleshooting
- As a
product owner
, I want an API that supports various authorization levels, so we can support future revenue and feature models - As a
product owner
, I want all features of the app automatically tested using TDD, so we can keep customers happy with stability - As a
product owner
, I want to allow external authentication providers (IdP), so we can offload effort or meet compliance guidelines - As a
product owner
, I want to test new features on a subset of users or geographies, so we can measure impact, refine, or revert as needed - As a
product owner
, I want to be able to track usage of toggle/flag features, so we can fine-tune before global deployment (or omit) - As a
product owner
, I want to ignore metrics performed by support users on behalf of users, so we can accurately measure metrics
- As an
architect
, I want to centralize events/activity stream, so that I can easily add stream pipeline, queue, or bus to implment CQRS-ES - As an
architect
, I want the app to be 'stateless' with remote DB, so that I can easily scale to meet growth requirements - As an
architect
, I want the app to be layered, so it's extensible with minimal duplicate code and able to change providers - As an
architect
, I want to app to be able to run in containers, so it is isolated and can easily scale to meet growth requirements - As an
architect
, I want to be able to change password hash solutions, so we can stay current as security standards evolve - As an
architect
, I want to be able to plug in security middleware, so we can stay current as security standards evolve - As an
architect
, I want to log performance metrics, so I can eliminate bottlenecks or calculate resource needs for scaling
- As a
developer
, I want to be able to toggle/flag new functionality, so we can safely build/deploy and test out new features
- Methodology: 12-factor
- Language: Typescript
- Framework: Express
- Documentation: Swagger UI
- Documentation: OpenAPI
- Config: Dotenv
- Logging: Winston
- Security: Helmet
- Authentication: JWT
- Authorization: AccessControl
- Validation: Class Validator
- Database: TypeORM
- Database: PostgreSQL
- Database: Redis
- Testing: Jest
- Testing: Supertest
- Testing: Docker Compose
- Testing: Postgres Admin
- Testing: Redis CLI
- Formatting: MomentJS
- Formatting: Google Phonelib
As I wanted to piece together RBAC/ABAC using popular stack choices, I found several good examples online. I'd like to give credit and thanks to the following for their hard work and excellent articles and examples.
- Node Best Practices
- Microsoft Typescript Node Starter
- Article by Alex Permyakov
- Article by Marcin Wanago
- Jest with Typescript
As this is just a research project, I don't plan on maintaining LTS but if any suggestions on improving the app, please write Issue or PR and I'll consider. Thanks!