Overview

This is my Computer Engineering senior thesis. I got my degree in 2022 from the University of São Paulo.

It is a language learning platform. Users can sign in as either instructors or students. Instructors will author language learning activities and students will complete them and get written feedback from their instructors.

Watch this video for a demonstration of how it works, or check out the application itself.

Architecture

The app is designed as a client-server architecture with microsservices.

The frontend is in Next.js and deployed to Vercel.

The backend consists of two Node.js REST APIs, one that takes care of authentication and authorization and another one that does all the rest of the domain logic.

All of the backend is in AWS. Both applications are Dockerized and deployed by ECS (with Fargate). They communicate via SQS, and persist their data on a single (for cost purposes) instance of RDS running Postgre. S3 is used to store user profile pictures.

There is a bastion host inside the VPC which I can use to see the contents of the databases.

There are two CICD pipelines, one which builds the Docker images and deploys the backend applications to a staging environment after pushes to develop and another one which deploys those containers to a production environment after pushes to main. Github actions run tasks which check for linting, formatting and run automated tests.

All of AWS infra is provisioned using Terraform (see the IaaC in /infra).

Authentication and Authorization

The auth API takes care of all basic authentication funcionalities, such as sign up, sign in, change of passwords, verification of accounts, and change of profile pictures.

When a user signs in, the auth app returns a JWT token with the user's current token version (which starts at 0 on sign up). The token is stored on the browser and sent on all requests to the domain app, which will respond to requests only if the token is valid and its token version is the user's most current one.

When a user signs out, the auth app will increment the token version and send a message via the queue to the domain app, so that it persists the most current token version for that user. That allows me to invalidate and reject requests from all current sessions of a user.

Internationalization

Frontend

The app is (almost) fully internationalized.

On the frontend, there is a select which sets the language in a global state. This state is then used on every page/component that has strings to select which entry in a big Translations object. Having pulled the entry with all of the labels of the chosen language, the correct label is then used. Something like this:

  const { language } = useLanguage();
  Translations[language][labels.activities])

Whenever this global value is changed, the axios instance changes its X-Accept-Language header and sends the new language setting to the backend, to intl it also.

Backend

On the backend, all error messages as well as e-mails are sent are set to the preferred language informed in the request.

Whenever a use-case needs to send an e-mail, the REST Controller adapter parses the preferred language header and passes it on to the use-case code, which then passes the language value to the e-mail service.

To intl errors, I've used an Express middleware treats error messages as labels, or as keys to a Translations object, and responds with the corresponding value. The correct translation object is chosen according to the header. Check out the platform code to see this middleware.

Frontend

Atomic Design

It uses some version of atomic design.

Check out in packages/web-app/atomic the components that have been considered atoms, molecules and organisms.

Outside the atomic folder, a components folder holds complex components that belong to the whole application, like the navigation bar and other components that are in it. Perhaps those would be atomic's organisms, but I found that they didn't really belong there.

Modules and Pages

Next.js needs a pages directory so it can set up the application's routing. However, all .jsx files in that folder are treated as pages. That makes it hard to keep components which belong exclusively to some page or set of pages, but are not pages themselves.

So, this project has a modules folder, which holds all pages and subcomponents related to one domain area. The routes in the pages folder, then, import te pages to the correct route. For example, the activities module defines all pages related to activities, like do, listing and new, as well as the components related exclusively to activities.

API Calls

Backend

Clean Architecture

The code design closely follows the Clean Architecture.

Platform Code

Platform code is common code, used by both backend applications. It consists of APIs created to hide complexity from product teams (who build the actual use cases of the microsservices), like SDKs for external services as well as adapters for HTTP servers and DI Containers.

Here is all of the Clean Architecture's Framework layer code.

Infrastructure

AWS and Terraform

Development Experience

Monorepo

All applications and code dependencies exist on this repository, under src/packages.

• auth_web_api is the backend authentication Node.js API.
• common-core contains business rules which are shared by all three apps.
• common-platform contains the backend platform code, as described above.
• common-utils contains code related to errors, testing utils and internationalization, used by all three apps.
• web_api is the backend domain Node.JS API.
• web_app is the frontend Next.js API.

Developing Locally

In order to setup the entire infrastructure locally, these are the steps:

• Copy .env.example into a new .env file, and fill in the appropriate values. You will need AWS credentials in order to access S3 and SQS, and a SendGrid account to send e-mails.
• Run docker-compose up --build on the root of the project to build the necessary images locally and start the application servers and database instance. Change the entrypoint of the services to npm run dev to leverage ts-node-dev.
• Run npm run dev on packages/web-app to run the Next.js server.

Pre-production and Production Environments

There is a staging and a production environment. Both the front-end in Vercel and the AWS infrastructure listen for changes in the Github repo, on branches develop and main, to update the staging and production environments, respectively.

These environments do not exist on the IaaC on the main branch for cost-saving reasons (extra ALB outside free tier), but do at the tag staging_and_prod.

CICD

The project is setup to follow some version of GitLab Workflow.

All feature branches come out of develop, and are merged into it after Code Review. Merging into develop sets out pre-production CICD:

• CI. When there are pushes to develop, Github Actions check formatting, linting and run unit and integration test scripts on the frontend as well as on both backend applications.
• CD. If all CI checks pass, CD starts (see Infra TODO). The staging front-end application is updated on Vercel whenever the commit has changes to packages/web-app or packages/common-core, and backend applications staging applications are redeployed via AWS' staging-code-pipeline (see Infra TODO), which builds the images and updates ECS' task definitions to use the new containers.

After manual QA and e2e tests (not implemented), develop is merged into main by the Tech Lead, which automatically sets out a production deploy via another CD pipeline. Whenever there are changes to main, Vercel redeploy the production front-end app and main-code-pipeline will redeploy the backend apps using the images built on the staging CICD.

TODO

Project Setup

• lint backend code

Infrastructure

• AWS Distro and OTeL for observability.

CICD

• Improve CICD to support building based on paths on AWS CD. Currently, both applications' CD will trigger whenever there is any change anywhere in the monorepo.
• CD actually starts at the same time as CI. Have CI greenlight CD instead, by running CI on pushes to feature branches and CD on pushes to develop and main.

Frontend

The front-end needs some serious refactoring and reconsidering of code design. And automated tests.

• Refactor API calls, separate handlers from hooks.
• Refactor instruction components, have different ones for creating activities, doing activities and displaying done activities.
• Refactor useEffects into custom hooks
• Rethink approach to internationalization of interface.
• Provide loader components for pages, to avoid weird page transitions.

UX and Funcionalities

• Start and end time of videos in minutes and seconds. Option for whole video.
• Ability to change order of instruction options by drag and drop. Redesign instructions modal
• Mixed activity types: video + text
• Field to add source to text activities
• See what's up with pagination
• Separate instructions into gist and detail, and have some content for instructors on activity design.
• Allow users to filer activities by instruction type
• Notify instructors when their students make output
• Allow for audio output.
• Have true/false be a different kind of answer, to disallow both being correct, and not make people type in "True" and "False"
• Allow editing of activities (figure out business rules around existing student output)
• A better dashboard