Skip to content

Latest commit

 

History

History
370 lines (278 loc) · 18.4 KB

README.md

File metadata and controls

370 lines (278 loc) · 18.4 KB
Raw

master commit release npm version npm Sonarcloud Status Known Vulnerabilities MadeWithVueJs.com shield

About

TL;DR ExpressJS, VueJS cookbook, with evergreen recipes and templates (CRUD, CI/CD, QA, Testing, Cloud container deployment, Web Components, ES Modules, etc.) to develop applications faster, while reducing the need for rewrite or refactoring due to changes in dependencies.

Latest Version 0.6.10 - Released 2022 January 20 1030 +8GMT

  • upgrade to react-router-dom 6, update list of useful VS Code plugins

Ask for help and recommend improvements here

Considerations for this project are similar to favv. The difference between them are:

  • this repo is more of a cookbook and recipes are constantly being improved and updated
  • favv backend is written in Python, they can be used to improve on each other

We do not use Typescript in most folders because TS people can understand JS easily, but JS-only people need to pickup TS and its ecosystem. However, we use Typescript for ReactJS and NestJS applications

Features

Folder Description Features
@es-labs/esm resuable ES module codes Available as npm package also
@es-labs/node reusable common JS codes Available as npm package also
docker-devenv Docker containers
supporting local development		 Mongodb, Mysql, Keycloak(SAML/OIDC, etc IDP), Kafka, Hashicorp Vault, Redis
docs Documentation
To constantly improved/updated		 - Main documentation
- Secrets documentation
- Deployment notes
- Kafka docs and code
- TCP Server docs and code
js-node/expressjs Base ExpressJS application
(REST API, Websocket, etc)		 - CORS, proxy middleware, helmet, error handling, logging, OpenAPI
- Objection ORM (removed), Knex, MongoDb, Relational DB data example, migration, seed, GraphQL, Redis
- Webpush & FCM push notification, Sendgrid email, Nexmo SMS, Telegram
- AgendaJS message queue
- Unit Test & Integration Test
js-node/expressjs/apps/app-template Custom application (app-template)
built on Base ExpressJS application		 - [config] app configs
- [controllers]
- [models]
- [openapi] OpenAPI yamls
- [routes] API routes (also websocket handlers)
- [services] services to startup/shutdown
- [tables] config tables for generic table crud (t4t)
- [tests] folder for tests
- [graphql-schema.js] application GraphQL codes
js-node/expressjs/public/demo-express Frontend to test backend features - GraphQL, File uploads, Signed URL file upload to GCP Storage, websockets, SSE, webworkers (frontend demo)
- JWT using RSA, JWT refresh token, token in HttpOnly cookies, GA OTP, role, Passport SAML, OIDC
- Github OAuth2 login (setup - https://www.sohamkamani.com/blog/javascript/2018-06-24-oauth-with-node-js)
- Fido & Webauthn
js-node/* Other Backend applications - TCP server (event/stream)
- Scalable Websocket with Redis Pubsub
- Serial server
- openapi-file-joiner combine OpenAPI files
js-node/wip Work In Progress - Kafka consumer/producer
- cron / long-running process example
js-node/nest-admin https://www.udemy.com/course/react-nest-admin example application (e-commerce) backend using NestJS, for use with js-web/react-admin
Nest Microservice Reference https://github.com/Denrox/nestjs-microservices-example
https://www.merixstudio.com/blog/microservice-nestjs/
js-node/nest-cqrs TBD Implement CQRS https://docs.nestjs.com/recipes/cqrs
js-web/solid SolidJS SPA WIP
js-web/vue-nobundler Vue 3 SPA no bundler + Bulma - signed uploads, recaptcha
- Web component table, form & CRUD backend (files to note)
js-node/expressjs/apps/app-template/tables/sample custom app table configurations
js-node/expressjs/router/t4t.jshandle backend CRUD API
@es-labs/esm/t4t-fe.jsfrontend operations to interact with t4t.js
@es-labs/esm/t4t-validate.jsvalidation used by both front and backend
@es-labs/esm/bwc-tableused to display table
@es-labs/esm/bwc-t4t-form.jsform generated from table configurations
js-web/vue-nobundler/views/ui1.jsautcomplete, combobox, file upload example
js-web/vue-nobundler/views/ui2.jstable example
js-web/vue-nobundler/views/ui3.jsform example (with connection to backend)
js-web/vue-nobundler/views/ui4.jstable and form example (with connection to backend)
js-web/vue-vite Vue 3 SPA using vite + Ant Design - Leaflet Map, AntV Charts, PWA, Websockets, rxJS
- JWT refresh token, 2FA GA OTP, OIDC, SAML, Github OAuth
- Web Components (Webcam, Signature)
- Cypress Testing
js-web/react-admin https://www.udemy.com/course/react-nest-admin example application (e-commerce) frontend using ReactJS, for use with js-node/nest-admin
example-webpack
(Deprecated & removed)
last updated version		 Vue 2 SPA using webpack + Vuetify - Graphql, REST, VueCrudX, i18n, rxJS
.github/workflows Github Actions (CI/CD) - Manually Trigger
- On-push Trigger (WIP)
AMP Website removed -

QUICK START - ON YOUR LOCAL MACHINE

Requirements

  • Node 16+ LTS
  • Npm 8+ (using workspaces)
  • For Windows, integrate bash shell to cmd shell (when installing git), or use git-bash
  • Docker

Optional VS Code Plugins

NOTE Useful plugins if using VS Code:

  • Essentials
    • Docker
    • Live Server
    • REST Client
    • SSH FS
    • vscode-database-client
  • Recommended
    • SonarLint (requires java)
    • GitLens
  • JS Language Specific
    • es6-string-html
    • ESLint
    • Prettier (disabled)
    • Vetur -> Volar (for VueJS)
  • NOT NEEDED
    • thunder client (REST API)
    • mongodb
    • sqlite

Download

# clone repo and install backend
git clone https://github.com/ais-one/cookbook.git
cd cookbook

Installing & Updating Dependencies

Install dependencies for all workspaces!

Note

  • when doing npm i, it will always install latest version matching your package
  • sometimes you need to rebuild, delete all node_modules folders and the package-lock.json file in the root
# https://github.com/npm/cli/issues/708
# https://github.com/npm/cli/issues/2032
npm i --legacy-peer-deps # use this for now as there can be peer dependencies issues

Update dependencies for all workspaces!

# step 1
npm outdated # use this to check for outdated dependencies

# step 2
# manually change the versions in the package.json files

# step 3
npm i --legacy-peer-deps

# use npm ls to check on actual versions installed
npm ls

Install for single workspace

npm i @vscode/sqlite3 --workspace=@es-labs/node

npm i lorem-ipsum --workspace=@es-labs/node

ExpressJS Backend Setup & Run - development environment

npm run app --workspace=js-node/expressjs -- development

NOTES

  • MongoDB examples needs MongoDB to work. To resolve, chose one of the methods to install MongoDB in docs/backend/mongodb/install.md
  • The js-node/expressjs/apps/app-template/config/secret/*.env,js files are not present. So there maybe some console log errors (but it is ok to ignore) and websockets will not work. Quick start is still usable. Use the README.md to fill up

Run migration & app

# create and seed relational db on SQLite, (delete the dev.sqlite file each time before you run this)
# command: npm run knex -- <development / uat / production> <custom app name> <seed / migrate>
npm run knex --workspace=js-node/expressjs -- development app-template migrate
npm run knex --workspace=js-node/expressjs -- development app-template seed

# create and seed MongoDB requires MongoDB - you can skip this but MongoDB examples will not work
# command: npm run mongo -- <development / uat / production> <custom app name> <seed / update>
npm run mongo --workspace=js-node/expressjs -- development app-template seed

# run the backend
# command: npm run app --workspace=js-node/expressjs -- <development / uat / production> <custom app name, default = app-template>

# app name implied (implied as app-template if not in env)
npm run app --workspace=js-node/expressjs -- development

# or app name specified
npm run app --workspace=js-node/expressjs -- development app-template

# to include eslint checks
npm run app:lint --workspace=js-node/expressjs -- development app-template

Visit the following URLs

Note: to generate api docs, visit js-node/openapi-file-joiner and follow readme file, also look at the config properties OPENAPI_PATH and OPENAPI_VALIDATOR in js-node/expressjs/apps/app-template/config/common.env.js.

No bundler frontend

See js-web/vue-nobundler. Served from http://127.0.0.1:3000/native/index.html

Testing

  • To run unit & integration test on /api/categories endpoint. E2E testing is Work In Progress
  • TO TEST EVERYTHING PLEASE change describe.only(...) to describe(...) in the test scripts in js-node/expressjs/apps/app-template/tests
# run in development only
npm run test --workspace=js-node/expressjs

Long Running Processes

For long running processes such as tcp server (event mode, streaming mode), serial server, kafka producer, consumer, cron-triggered process, etc.

See js-node/README.md

Vite SPA Setup & Run - development environment

  1. See js-web/vue-vite/README.md. To setup the configuration files. End-to-end testing example using cypress is here also.

  2. Run the following

npm run dev --workspace=js-web/vue-vite
  1. Visit
  1. See js-web/vue-vite/README.md for more information on the vue-vite package

Why No SSR or SSG

  • potential slow rendering by server app, added complexity in code, rehydration errors, added complexity in server
  • nuxt/nuxt#8102
  • prefer static sites and lazy loaded SPA for now

SAML, OIDC, OAuth

Refer to link below on how to try out...

  • Keycloak README.md
  • Refer also to the following files
  • You can test out on the js-web/vue-vite Signin UI,
    • Use the following username / password credentials
      • for simple Login...
        • just clock on Login button, credentials test / test is already prefilled
        • then click on OTP button, the OTP 111111 is already prefilled
      • for SAML and OIDC... test-user / password
        • redirect to a keycloak IDP
      • for OAuth (Github), you need to configure your github settings (not recommended for starting out)
        • redirect to github login
      • for Mock user login, just click on Mock button

Fido2

Refer to following files for SPA sample (uses fido2-lib in backend)

Configuration

The js-node/expressjs/apps/app-template/config folder contains the config information.

You can override the configurations using <NODE_ENV>.env.js files, e.g. development.env.js or uat.env.js in the folder

NestJS and ReactJS E-commerce Application Example

NestJS Example

npm run start:dev --workspace=js-node/nest-admin # create the SQlite database called db.sqlite (in actual should be created using migration), shutdown server once it is fully up and running
npm run seed --workspace=js-node/nest-admin # seed the data
npm run start:dev --workspace=js-node/nest-admin # start the server again
  • Navigate to http://127.0.0.1:3000
  • You can use test.http file in the nest-admin folder file with VS Code REST Client plugin to test enpoints

ReactJS Example

npm run start --workspace=js-web/react-admin

Project Strcuture

+- .github/ : github related CICD and automations
+- .husky : git hooks
+- @es-labs/
|  +- esm/ : [shared es modules]
|     +- package.json
|  +- node/ : [shared cjs modules]
|     +- auth/ : authentication
|     +- comms/ : messaging
|     +- services/ : db, mq, etc.
|     +- config.default.js: defaults
|     +- config.js: config loader
|     +- package.json
|     +- traps.js
+- docker-devenv/ : docker for development environment
+- docs/ : documentation
+- js-node/ : nodejs applications (kafka, cron triggered, long running)
|  +- expressjs/ : express backend - See [js-node/expressjs/README.md](js-node/expressjs/README.md) for project structure
|  +- nest-admin/
|  +- openapi-file-joiner/ : pre-process utility to combine openapi yaml files for use in openapi related packages
|  +- serialserver/
|  +- tcpserver/
|  +- wip/ : projects in progress
|  +- worker-threads/ : demo on use of worker threads
+- js-web
|  +- react-admin/
|  +- solid/
|  +- vue-nobundler/ : frontend (Vue3 no bundler) - See [js-web/vue-nobundler/README.md](js-web/vue-nobundler/README.md) for Project Structure
|  +- vue-vite/: frontend (Vue3 rollup) - See [js-web/vue-vite/README.md](js-web/vue-nobundler/README.md) for Project Structure
+- .editorconfig
+- .gitignore
+- BACKLOG.md
+- CHANGELOG.md
+- LICENCE
+- package.json
+- README.md

CI/CD & Cloud Deployment

Cloud Services

The following Google Cloud Platform (GCP) services are used

  • Container Registry
  • Cloud Run
  • Cloud Storage
  • Mongo Atlas (hosted on GCP)

Refer to doc/deployment/home.md for documentation on deployments

Deployment Using Github Actions

  • .github/workflows/manual-gcp-expressjs.yml (Manually deploy js-node/expressjs to GCP CloudRun)
    • selectable inputs
      • environment (uat for now, development does not deploy anything)
      • service (default = app-template)
      • branch
  • .github/workflows/manual-gcp-vue-vite.yml (Manually deploy js-web/vue-vite to GCP Cloud Storage)
    • selectable inputs
      • environment (uat for now, development does not deploy anything)
      • csutom_app (to be implemented)
      • branch
  • .github/workflows/manual-gh-pages.yml (Manually deploy js-web/vue-vite to Github Pages)
    • selectable inputs
      • environment (uat for now, development does not deploy anything)
      • csutom_app (to be implemented)
      • branch

NOTE config/secret contents will not be in repo for CI/CD (so you can get errors), those should be put in VAULT

Current secrets

  • GCP_PROJECT_ID
  • GCP_SA_KEY
  • VAULT_uat, passed in as VAULT
# do not merge
VAULT="unused"

# connect to a hashicorp vault and obtain secrets to merge
VAULT={ url, token } # base64 encoded

 # pass in secrets, this way is insecure and not a good way to send in secrets
VAULT={ secrets: { ... all your secrets here } } # base64 encoded

VERSION CHANGE NOTES

  • v0.6+ Improve organization, graceful exit, logging, project rename, add more nodejs applications, repo name vue-crud-x changed to cookbook
  • v0.5+ Improve organization and authentication, add new features
  • v0.4+ Improve folders and structure organization, handle CI/CD better
  • v0.3+ Reorganize folders and structure, for ease of developing and maintaining multiple applications.
  • v0.2+ uses Vuetify 2. Due to breaking changes from Vuetify 1 to 2, CRUD component code was refactored to be more UI framework agnostic (reduce dependencies!), easier to use, improving code quality, documentation and supprting article - VueJS+ExpressJS CRUD & Cookbook
  • v0.1 and Vuetify 1 will be supported under the v1 branch. You can refer to the v1 Legacy Article (For Historical Reference)