whatgotdone
Architecture
Overview
What Got Done has a simple architecture consisting of the following parts:
- What Got Done Backend: A Go HTTP service (running on AppEngine) that handles all HTTP requests, datastore requests, and user authentication (via UserKit).
- What Got Done Frontend: A Vue2 app that renders pages in the user's browser.
- Cloud Firestore: What Got Done's storage provider.
- UserKit: A third-party service that manages What Got Done's user authentication.
Page Rendering Flow
What Got Done uses a somewhat unusual system for rendering pages. The Go backend first pre-renders the page server-side to populate tags related to SEO or social media that need to be set server-side. The Vue2 frontend renders the remainder of the page client-side. To avoid conflicts between the two systems' template syntax, Go uses [[, ]] delimiters, while Vue uses {{, }} delimiters.
The Go backend handles all of What Got Done's /api/* routes. These routes are What Got Done's RESTful interface between the frontend and the backend. These routes never send HTML, and instead only send JSON back and forth.
User authentication
What Got Done uses UserKit for user authentication. For user signup, user login, and password reset, What Got Done loads the UserKit UI widgets in JavaScript. On the backend, the auth package is responsible for translating UserKit auth tokens into What Got Done usernames.
Datastore
What Got Done uses Google Cloud Firestore for data storage.
Only the What Got Done backend can access the Firestore database. Specifically, the datastore package manages all interactions with Firestore.
E2E tests
What Got Done's end-to-end tests use Cypress and follow the testing pattern defined in the article End-to-End Testing Web Apps: The Painless Way. The testing architecture consists of two Docker containers (see docker-compose.yml):
- What Got Done container
- Cypress container
The Cypress container runs a browser to exercise What Got Done's critical functionality. It uses an independent environment and credentials from the production app so that nothing in the E2E tests affect state on production UserKit or Google Cloud Platform.
To run the E2E tests yourself, see the section below.
Contributing to What Got Done
Interested in contributing code or bug reports to What Got Done? That's great! Check our Contibutor Guidelines for more details.
QuickStart
To run What Got Done in a Docker container, run
docker-compose upWhat Got Done will be running at http://localhost:3001.
Dev-mode authentication uses UserKit dummy mode. You can log in with any username using the password password.
Development notes
0. Pre-requisites
1. Start a Firestore emulator
Run the following command to start a Google Cloud Firestore Emulator in a Docker container:
export GOOGLE_CLOUD_PROJECT="dummy-local-gcp-project"
docker run \
--detach \
--env "FIRESTORE_PROJECT_ID=${GOOGLE_CLOUD_PROJECT}" \
--env "PORT=8080" \
--publish 8080:8080 \
--name firestore-emulator \
mtlynch/firestore-emulator2. Build the frontend
To build the Vue frontend for What Got Done, run the following command:
pushd frontend && \
npm install && \
npm run build -- --mode development && \
popd3. Run the backend
To run the Go backend server, run the following command:
export USERKIT_SECRET="dummy.dummy"
export FIRESTORE_EMULATOR_HOST="localhost:8080"
mkdir bin
go build --tags 'dev' -o ./bin/main backend/main.go && \
./bin/mainWhat Got Done is now running on http://localhost:3001.
Dev-mode authentication uses UserKit dummy mode. You can log in with any username using the password password.
Optional: Run frontend with hot reloading
If you're making changes to the Vue code, you'll probably want to run the standard Vue HTTP server with hot reloading. Keep the backend running, and in a separate shell session, run the following command:
cd frontend && \
npm run serveA hot-reloading Vue server will run on port http://localhost:8085. It will communicate with the What Got Done backend at port 3001.
Quirks of the dev environment
Because the production What Got Done server runs both the frontend and the backend on a single port, there are a few hacks to make a development version work:
- CORS is enabled in dev mode so that the frontend can make CORS requests to the backend from a different HTTP port.
- CSRF protection is disabled in dev mode because the Vue dev server doesn't know how to render the
<meta name="csrf-token" />tag. - Page titles don't work properly in dev mode because the Vue dev server doesn't know how to render the
<title>tag. - The Content Security Policy header in dev mode needs
unsafe-evalandunsafe-inline, whereas we disable this in production.
Optional: Run backend unit tests
Unit tests run in normal Golang fashion:
go test ./...Optional: Run integration tests
Integration tests run all components together using a local Firestore emulator as the datastore and UserKit dummy mode as authentication:
dev-scripts/run-integration-testsOptional: Run E2E tests
The E2E tests are a subset of the integration tests, but they use a real GCP Firestore instance instead of a local emulator.
To run the end to end tests, you'll need to create a dedicated GCP project. Specify the GCP project in e2e/docker-compose.yml under GOOGLE_CLOUD_PROJECT.
Then, run the E2E tests as follows:
dev-scripts/run-e2e-testsIntegration tests vs. E2E tests
Integration tests have the advantage of running faster because they have fewer remote dependencies. It's also possible for third-party developers to run them because they require no secrets (this also allows CI to run them on third-party pull requests).
E2E tests are more likely to catch real-world bugs because their configuration more closely matches production infrastructure.
Optional: Enable public analytics from Google Analytics
What Got Done supports pulling metrics from Google Analytics into the page content. To enable this:
- Enable the Google Analytics Reporting API in your Google Cloud Platform project.
- Create a service account in Google Cloud Platform console for your What Got Done project.
- Assign the service account no permissions/roles, but save its private key as JSON.
- Click "Create Key" to create a private key and save it in JSON format as
google-analytics-service-account.jsonin the What Got Done root directory.
- In Google Analytics, open Admin > View > View User Management and add the email address of the service account you just created (it will have an email like
[name]@[project ID].iam.gserviceaccount.com.- Grant the user only "Read & Analyze" permissions.
- In Google Analytics, open Admin > View > View Settings
- Save the View ID as an environment variable like
export GOOGLE_ANALYTICS_VIEW_ID=12345789