Skip to content

IN‐CORE Developer Setup

Jump to bottom
Chen Wang edited this page Sep 21, 2026 · 1 revision

This guide covers getting a local development environment running, and running the tests, for every repository in the IN-CORE platform. It is the mechanical companion to CONTRIBUTING.md, which covers the process side (GitFlow, branch naming, PR review, CLA).

Each repository section has the same five parts:

  1. Prerequisites: runtimes and services you need before you start
  2. Setup: from a clean clone
  3. Run it: how to start the component locally
  4. Test it: how to run the test suite
  5. After a dependency change: what to regenerate, and what to commit

Contents

Getting the source

This repository is a superproject that tracks several of the component repositories as git submodules.

git clone --recurse-submodules https://github.com/IN-CORE/IN-CORE.git

If you already cloned without --recurse-submodules:

git submodule update --init --recursive

12 repositories are tracked here as submodules: incore-auth, incore-docs, incore-helm, incore-lab, incore-services, incore-studio, incore-ui, plotting-service, pyincore, pyincore-data, pyincore-incubator, pyincore-viz.

maestro-service is part of the platform but is not a submodule of this repository, and must be cloned separately:

git clone https://github.com/IN-CORE/maestro-service.git

All repositories use GitFlow: branch from develop, PR back into develop. The exception is this superproject, which currently has only main. Only admins can make contributions directly to the superproject. See also CONTRIBUTING.md.

Shared prerequisites

You will not need all of these. Install what the repositories you are working on ask for.

Tool Used by Notes
conda or micromamba all Python repos conda is the officially supported installation path for the pyincore family. CI uses micromamba. mamba is a drop-in faster replacement for conda if you prefer it.
Python 3.11+ pyincore, pyincore-viz, pyincore-data pyincore sets python_requires=">=3.11"; CI tests 3.11, 3.12, 3.13.
Python 3.9 pyincore-incubator, maestro-service Older pins; see Known problems.
GDAL C library pyincore, pyincore-viz, pyincore-incubator Only needed for the (unsupported) pip install path. On Ubuntu: gdal-bin, libgdal-dev. Conda pulls this in for you.
Ipopt pyincore Required by the CGE analyses. Comes from the ipopt>=3.11 conda dependency.
pre-commit pyincore, pyincore-viz, pyincore-data, pyincore-incubator brew install pre-commit or pip install pre-commit.
JDK 11 incore-services CI uses Zulu 11. The Gradle build sets source/target compatibility to 11.
Node 18+ incore-studio engines requires node >= 16; CI builds on 18.
Node 12–14 incore-ui CI builds on 12 and 14 only; see Known problems.
Poetry maestro-service pip install poetry
Docker every repo Every component ships a Dockerfile; several are easiest to run this way.
MongoDB incore-services, incore-auth
PostgreSQL maestro-service, incore-services (GeoServer-backed dataset publishing)
Helm 3 + a Kubernetes cluster incore-helm

IN-CORE account and tokens

Most of the Python repositories talk to the IN-CORE web services, and their tests authenticate against the dev deployment.

  1. Create an IN-CORE account: https://tools.in-core.org/ (sign-up link can also be found in CONTRIBUTING.md).
  2. pyincore caches tokens under ~/.incore/. IncoreClient creates this directory on first use and writes one token file per service URL.
  3. pyincore's test suite does not use the cache. tests/conftest.py reads tests/pyincore/.incorepw, a two-line file containing a JWT and the key to decode it, and uses the credentials inside to log into INCORE_API_DEV_URL. In CI this file is written from the PYTEST_USER_TOKEN secret. Ask in the #in-core-dev-team Slack channel for the equivalent for local runs. Do not commit this file.

pyincore-data

Utilities for preparing data for IN-CORE analyses.

1. Prerequisites

conda, Python 3.9, pre-commit.

2. Setup

Note the file name: this repository uses environment.yaml, not environment.yml:

cd pyincore-data
conda env create -f environment.yaml
conda activate pyincore_data
pip install -e .
pre-commit install

3. Run it

Library only.

4. Test it

These are real pytest tests (def test_* with @pytest.fixture). CI (.github/workflows/pytest.yaml) runs only:

python -m pytest --cache-clear -s tests/pyincore_data/test_pyincore_data.py

CI also writes tests/pyincore_data/.incorepw from PYTEST_USER_TOKEN, but the suite above does not use an IncoreClient (census/NSI helpers), and this repository has no conftest.py that reads the file.

Other files under tests/pyincore_data/ that CI does not run:

  • test_nsi.py: real pytest (def test_*); run locally with python -m pytest tests/pyincore_data/test_nsi.py if you want it.
  • test.py: not pytest. It is a local runnable script (if __name__ == "__main__"), gitignored via *test.py, and is not collected by pytest.

5. After a dependency change

setup.py reads requirements.txt directly (install_requires=[line.strip() for line in open("requirements.txt")]), so requirements.txt and environment.yaml must be kept in sync by hand. This repository has no requirements.sh. Update both, then:

conda env update -f environment.yaml --prune
conda activate pyincore_data
pip install -e .

Commit both files with your change.

Clone this wiki locally