Skip to content
minimalist journaling
Branch: develop
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
app
bin
browser
deploy
logs
migrations
wwwroot
.dockerignore
.esformatter
.eslintignore
.eslintrc.json
.gitignore
.nvmrc
README.md
Vagrantfile
config.default.js
config.local.sample.js
knexfile.js
license.txt
package-lock.json
package.json

README.md

mjournal

Minimalist journal aiming to be one journal for all of your technical projects. Geared toward sparse and clean UI and categorization through labels.

Build Status js-standard-style

(grid background image stolen from http://bulletjournal.com)

How to Setup for Local Development

  • install the prerequisites
    • node and npm (nvm recommended)
      • see .nvmrc for correct version of node
    • curl (version included with OS X is fine)
    • git (homebrew recommended on OS X)
    • GNU coreutils and findutils (homebrew recommended on OS X)
  • Get access to some docker host
    • docker for Mac recommended
  • Get access to a PostgreSQL 9.4 database
  • Adjust local configuration
    • cp config.local.example.js config.local.js and override anything from config.default.js you need such as db.user, db.password, etc
    • you local configuration should not get checked into the git repo and is thus ignored in the mjournal .gitignore

Sample docker-compose.json file

Here is a sample of a docker-compose.json file you can use for local development running postgres under docker.

{
  "services": {
    "postgres": {
      "container_name": "postgres",
      "environment": {
        "POSTGRES_PASSWORD": "password"
      },
      "image": "postgres",
      "ports": ["127.0.0.1:5432:5432"],
      "volumes": ["~/docker-volume/postgres:/host"]
    }
  },
  "version": "2"
}

How to Run the Tests

  • npm test

How to Run Code Lint

  • npm run lint

How to run the app server directly

  • node .
  • OR node-dev .
  • OR ./bin/start-dev.sh
    • this one captures logs as well. RTFS for details.

How to Run with docker-compose

  • ./bin/start-docker-compose.sh

This should get the data volume, postgres container, and node container created, linked, and running. You should be able to access the app at http://127.0.0.1:9090.

This is suitable for local tests that the app works under docker. It does differ from production by a few details (TLS, etc), so it does not constitute a suitable stage or pre-production environment.

Docker Setup Details

We use the public docker registry to host our images. This vastly simplies things over trying to run our own registries.

Containers

  • mjournal_data
    • Data-only container storing the PostgreSQL database data files
  • mjournal_db
    • Runs the PostgreSQL relational database (docker image name "postgres")
    • Holds all mjournal application and session data by using the mjournal_data data volume
  • mjournal
    • The mjournal node/express web application
    • Linked to mjournal_db for access to the database
    • exposes port 9090 for the HTTP API and web application
    • configuration (secrets, etc) comes from a configuration file at /var/local/mjournal/config.js which is mounted in the container at /etc/mjournal/config.js

Deployed Host Setup and OS Integration

When deployed on stage and production, I use an Ubuntu 14.04 x64 droplet on digital ocean, but that OS on any hosting platform should work fine. The only manual setup from a fresh droplet is ssh access:

  • create your OS user (as root on droplet: adduser plyons)
  • allow sudo with password (as root on droplet: adduser plyons sudo)
  • Configure passwordless ssh (from your local machine: ssh-copy-id new-droplet-ip)

We integrate with the host OS (Ubuntu x64) on stage and production for the following functionality

  • application configuration is specified in a CommonJS module config.js file
    • This is mounted into the mjournal container at /etc/mjournal and mapped to /var/local/mjournal on the host OS
  • A cron job to run daily database backups
    • Configured in /etc/cron.daily/backup-mjournal-db
  • nginx reverse proxy
    • Handles TLS and serving static files
    • TLS certs from letsencrypt managed by certbot-auto

Further Notes

  • Backups live at /var/local/mjournal_db_backups on the docker host
    • more details on backup/restore below
  • all released docker images are tagged with semver
    • docker images are also tagged by environment for "production" and "stage"

How to Prepare a Release

  • get new changes in the develop branch committed and ready to go
  • ensure your working directory is in a clean git state
  • Make sure to pull down changes in github/develop. There normally be won't any, but for the rare pull request.
  • run ./bin/release-candidate.sh <major|minor|patch>
    • This will pull origin/master into develop
    • This will do an npm version to increment the version and tag the commit
  • Move on to the build and test instructions below

How to Build and Deploy Code

  • do a docker build
    • ./bin/build-docker.sh
  • If that succeeds, note the container ID it prints out such as Successfully built a2452ff73a95
  • tag and deploy that for testing on stage
    • ./bin/deploy-stage.sh <container_id_from_above>
  • If this is the first deploy to this host, the script will stop after the empty configuration file has been created
    • ssh in and edit the configuration file /var/local/mjournal/config.js with per-deployment values for exports.MJ_PG_PASSWORD. exports.MJ_PG_ADMIN_PASSWORD and exports.MJ_SESSION_SECRET
    • run the deploy script a second time
  • Test in a browser
    • open "https://stage-mj.peterlyons.com"
  • If the app is working, tag for prod and release
    • ./bin/tag-production.sh <image-id-from-docker-build>
    • ./bin/release.sh
  • Deploy to production
    • ./bin/deploy-production.sh

Daily Rotating Backup System

  • Backups are taken daily and monthly by way of a cron.daily job on the docker host
  • The cron job is installed when the docker host is prepared via ./bin/deploy.sh <domain>
  • The details can be found in deploy/backup-db.tpl.sh but TL;DR it's basically pg_dumpall
  • Files live as bzip-compressed SQL files at /var/local/mjournal_db_backups on the docker host with obvious timestamp file names
  • Stale backups are pruned automatically. We retain 1 month of dailies and 3 months of monthlies

How to Restore from Backup

  • Yes, this has actually been tested on stage ;-p
  • run ./bin/render-template.js ./deploy/restore-db.tpl.sh | ssh <docker_host> tee /tmp/restore.sh
  • ssh to the docker host
  • run sudo bash /tmp/restore.sh /var/local/mjournal_db_backups/<FILE_TO_RESTORE_FROM>
    • sudo is necessary because of how postgresql requires filesystem permissions to be locked down
    • the restore file should be a .sql.bz2 file

License

MIT

You can’t perform that action at this time.