Climate Connect

The client and server code for the web platform on https://climateconnect.earth.

Table of Contents

1. Development setup
   1.1. One-click setup using VSCode Dev Containers
   1.2. Manual Setup
   1.3. Project Dependencies
   1.3.1. Backend
   1.3.2. Frontend
   1.4. Deploy
2. Documentation

Development setup

Climate Connect depends on PostgreSQL/PostGIS and Redis.

We use Python/Django for our backend and Next.js for the frontend.

Note: we use Python 3, so for all instructions we assume python means python3.

First, clone the GitHub repository

git clone https://github.com/climateconnect/climateconnect

One-click setup using VSCode Dev Containers

VSCode Dev Containers allow deploying a development environment including using Docker independent of the host operating system or installed programs.

Make sure you have the Remote Development Extension Pack VSCode extension installed.

1. Open the repo in VS Code

2. Start Docker (if it's not running)

3. Run the command "Reopen in Container"

   

4. Wait for it to be done setting up the Dev Container and running the setup scripts. This will take 1-5 minutes the first time and 10 seconds after that. It should look something like this:

5. Start the frontend dev server using cd frontend && yarn dev

6. Start the backend server using cd backend && make start. If you get a "Django is not installed" error run Ctrl+Shift+P (or Cmd+Shift+P) "Python: Select interpreter" -> Recommended so it uses the Python venv. Then reopen your vscode terminal to apply the change. which python3 should show backend/.venv/bin/python now.

7. Open http://localhost:3000

You can get a Redis REPL using redis-cli -h redis and a PostgreSQL REPL using psql.

Manual Setup

If you can't or don't want to use VS Code dev containers, follow the steps below.

Postgres

1. Create a local Postgres database with your own username and password. E.g., createdb climateconnect-dev.
2. Install PostGIS on your local machine.
3. Create the PostGIS extension within that database: run CREATE EXTENSION postgis;.

You will connect to this for your local backend project.

• Create a new superuser
• Alter your new user's password
• Create a new database

Supply these values to your local backend/.backend_env.

Docker

We use Docker to run the local Redis server. See the Docker install docs if you don't have it.

Make sure to install docker-ce, docker-ce-cli, containerd.io, and docker-compose.

Project Dependencies

Run ./install_deps.sh to install the JavaScript dependencies and the Python dependencies in a virtualenv.

Backend

First Time Setup

1. Go to backend directory: cd backend
2. Make sure pdm is installed: https://pdm.fming.dev/latest/#recommended-installation-method
3. Run make install to install all backend libraries.
4. Create .backend_env to set environment variables.
   • You can use the script ./initial_dev_setup.sh as inspiration.
   • You can find up-to-date sample env variables in backend/local-env-setup.md.
   • For the Django SECRET_KEY, run openssl rand -base64 32 to create a 32 char random secret.
5. Run make migrate to run Django migrations.
   • Note: This command is used for when you first start, or whenever you are adding or updating database models.
6. Create a superuser using python manage.py createsuperuser
   • You can then access your admin panel via <API_URL>/admin/

Continual Development

1. Ensure Docker is running and then run sudo docker-compose up. This will start a Redis server on Docker.
2. Ensure the Postgres server is running.
3. Run server using make start.
4. Run Celery using celery -A climateconnect_main worker -l INFO

Creating and Removing Test Data

If test data is needed, run

python manage.py create_test_data --number_of_rows 4

If you need to wipe your local database and start over, run

sudo -u postgres psql  # note this might differ slightly in name based on your postgres setup

And then at the psql prompt,

postgres-# \list

to show available databases. Once you've identified the Climate Connect database name (e.g. we'll call it $DATABASE_NAME), you can,

postgres-# \connect $DATABASE_NAME
$DATABASE_NAME-# \dt
$DATABASE_NAME-# DROP SCHEMA public CASCADE;
$DATABASE_NAME-# CREATE SCHEMA public; # recreate schema
CREATE EXTENSION postgis; # readd postgis
ALTER SCHEMA public OWNER TO $YOUR_SUPER_USER; # make sure your superuser is the owner of the table
$DATABASE_NAME-# \q

Then run

make migrate

to update your migrations, and

python manage.py createsuperuser

to re-create a super user to be used in the Django admin panel.

Testing and Code Health

For unit tests, to run the test suite use:

make test

Or a specific test file or test class:

python manage.py test <file_path> or <file_path + class_name>

For linting, we use ruff. Lint with

make lint

For formatting, we use Black. Format a specific file with

make format path_to_file

or a directory with

make format directory

More configuration for Black can be found in the pyproject.toml file.

Frontend

1. cd frontend.
2. yarn to download all npm packages.
3. Add a .env file for frontend environment variables. You can find variables you need to set in /frontend/next.config.js/.

For local development, use the following for .env:

API_HOST="localhost"
API_URL="http://127.0.0.1:8000"
BASE_URL_HOST="localhost"
SOCKET_URL="ws://localhost"
ENVIRONMENT="development"
CUSTOM_HUB_URLS="example"
LOCATION_HUBS=valu1,value2,value3

Note: This is for people who are using newer version of node (v17.0.1) or have new apple M1 devices. Before running yarn dev, please run this command export NODE_OPTIONS=--openssl-legacy-provider. You can save this in your ~/.zshrc file as well.

And finally yarn dev to spin up the Next.js app! Check out our frontend (FE) code style guidelines wiki to follow codebase best practices and contribute to a healthy and maintainable codebase.

Testing and Code Health

For unit tests, we use Jest. Write test files with .test.js and execute them directly with

yarn jest path/to/testfile.test.js

For linting, we use eslint. Watch files to fix lint issues with

yarn lint:watch

For formatting, we use prettier. Format with

yarn format

See npm scripts in package.json for more.

You can analyze size of the (frontend) bundle with

yarn analyze-bundle

Deploy

Currently the project is utilizing credits to deploy onto Azure. That deployment utilizes the two scripts in the root of the project, start_backend.sh and start_frontend.sh.

Frontend

Option 1: GitHub Actions

1. Use GitHub actions to push to a server. A deploy file can be found in .github/workflows

Option 2: Manually

1. cd frontend
2. yarn --production
3. yarn build
4. node server.js OR next start

Backend

1. Make sure your ENVIRONMENT env variable is set to production when deploying to the production server
2. Follow steps 1-5 of the "Getting started locally - backend" (above in this file)

Documentation

Architecture
Domain Entities
Environent Variables