Skip to content

Developer's Instructions

Jump to bottom
Piotr Szajowski edited this page Mar 10, 2023 · 10 revisions

Introduction

In this document we describe how to set up a development environment for TRAC D.A.P.

The code of included in this repository covers TRAC Engine written in Java and an API wrapper (TRAC Runtime) written in Python, therefore both Java and Python environments are needed to cover all elements.

The recommended software for development of the code from this repository includes:

Please, note, that we will not cover installation of above mentioned packages in this document. For installation instructions search the websites mentioned by each of the packages.

Configuration of the local environment

To start working on development of TRAC D.A.P. you need to:

  • install the software mentioned above in the introduction
  • in the working directory, using command line (cmd.exe in Windows or bash in Linux):
  • clone the repository from GitHub.com: git clone https://github.com/finos/tracdap.git
  • create a venv environment: python -m venv tracvenv
  • activate the venv environment: tracvenv\Scripts\activate.bat (in Windows) or tracvenv/Scripts/activate (in Linux)
  • install required packages in the venv environment: pip install -r tracdap\tracdap-runtime\python\requirements.txt (in Windows) or pip install -r tracdap/tracdap-runtime/python/requirements.txt (in Linux)
  • open IntelliJ IDE, open tracdap folder as a new project and follow the steps:
  • use dev/ide/copy_settings.bat (in Windows) or dev/ide/copy_settings.sh (in Linux) to copy run configurations to .idea folder, which make them available in the run configurations menu of the IDE
  • set up SDK for tracdap-runtime module:
  • open Project Structure window (click tracdap-runtime > Python folder in project tree and select File > Project Structure or press F4)
  • in the Project Structure window make sure, that tracdap-runtime-py module is marked on the list of Modules
  • in the Dependencies tab, in Module SDK drop down menu select "Add SDK" and then "Python SDK..."
  • in the Add Python Interpreter window select "Existing environment" and provide the path to Python interpreter in the venv environment, for instance: C:\Users\user_name\tracvenv\Scripts\python.exe
  • set up default tenant name and description:
  • open "Tools: Deploy Meta DB" run configuration for editing
  • in the run parameters field add: --task add_tenant ACME_CORP "ACME Corp"
  • save the run configuration and start it

Running tests locally

With the configuration described above you will be able to run unit tests for any of the TRAC Engine service. To do it, it is enough to start the application using one of the predefined run configurations: "Unit tests: -gateway", "Unit tests: -lib-common", and so on.

To run any example or test for the runtime (Python code), it is necessary to perform the following steps:

  • build the Java code
  • start "Build runtime (PY): Codegen" run configuration
  • start "Build runtime (PY): Dist" run configuration
  • find the name of the .whl file in tracdap-runtime/python/build/dist
  • use that file name (with its path) to install the TRAC module in the venv environment - in the command line with activated venv environment run: pip install file_path\file_name.whl, for instance: pip install tracdap-runtime/python/build/dist/tracdap_runtime-0.5.25+dev15.g5f9964b-py3-none-any.whl

After performing the steps listed above it will be possible to start run configurations for the runtime ("Runtime(PY): Examples" and "Runtime(PY): Unit Tests"). It is also required for E2E testing.

Note: if you modified anything, what may influence the Python code, for instance one of .proto files defining messages for one of the services, it is necessary to clean build folders (tracdap-runtime/python/build and build), build the Java code and perform the steps mentioned above again.

E2E testing

In order to perform E2E test first it is necessary to make sure, that all requirements for running tests locally are fulfilled (see above).

To start the E2E test it is necessary to create a new run configuration (for time being it is not stored in the repository and it is needed to be created manually). In order to create it:

  • copy any of "Unit tests: ..." run configurations into a new one
  • modify the following fields:
  • in the class path field select tracdap.tracdap-svc-orch.test module
  • in the Tags field type: int-e2e
  • in the environment variables field define the variable TRAC_EXEC_DIR and assign the path to the parrent folder of the project there, for example: TRAC_EXEC_DIR=C:\Users\user-name\myworkingfolder

Afterwards it should be enough to select the newly created run configuration and start it to perform E2E testing.

Note: By default the configuration of git is not set up, therefore the test for cloning a model code from a repository fails. This is normal for time being.

Code structure - high level

  • dev - general configuration and tools
  • dist - deployment scripts and template configuration
  • doc - documentation
  • examples - example applications using TRAC Engine API (in JavaScript and Python)
  • gradle - configuration of Gradle
  • tracdap-api - API definition in Google's Protocol Buffers format
  • tracdap-libs - the source code of several basic modules for TRAC Engine
  • tracdap-plugins - the source code of several plugins (for storage, config, etc.)
  • tracdap-runtime - the source code of the runtime (in Python)
  • tracdap-services - the source code for TRAC Engine services
  • tracdap-tools - tools for deploying MetaDB and the secret tool

If you are unsure where to look for the Java code you need to modify, start searching tracdap-api, tracdap-libs and tracdap-services.

Clone this wiki locally