Better Testing by sg-s · Pull Request #1 · deeporiginbio/do-dd-client

sg-s · 2024-04-10T19:25:09Z

changes

ability to run tests using mocking or on a live instance
tests run on github actions
good coverage (see below)
added a codeowners file

coverage

90% on _api

Makefile

.github/workflows/main.yml

pyproject.toml

README.md

jonrkarr

Looks good. I noted a few minor things, mostly for the documentation. This can be tackled in this or the next PR.

Two more things

Should we call the repo something more general like client or python-client?
We can remove icon.svg. Its not relevant in the context of this standalone repo.

sg-s · 2024-04-11T12:06:36Z

@jonrkarr thank you! yes, renaming this repo at this point is easy because nothing depends on it

## Summary When running against the **local platform** (e.g. E2E from platform-e2e with `E2E_DEEPORIGIN_ENV=local`), the SDK should call the real gateway on **port 6010** instead of the mock server. Unit tests (pytest with `--env local`) continue to use the **mock on port 4931** so they do not depend on the platform. This PR aligns the SDK with that behaviour: default local API is 6010; tests override to 4931 via conftest; and when `DO_AUTH_TOKEN` / `DO_ORG_KEY` are set (e.g. by platform-e2e pybridge), the SDK uses them for local so E2E can use a real Keycloak token. ## Changes - **`src/utils/constants.py`** - `API_ENDPOINT["local"]` set to `http://127.0.0.1:6010` (platform gateway). Previously pointed at the mock port; the mock is now only used when explicitly configured (e.g. in tests). - **`conftest.py`** - For `env == "local"`, monkeypatch `constants.API_ENDPOINT["local"]` to `http://127.0.0.1:4931` so unit tests hit the mock server. - In `finally`, restore the default `http://127.0.0.1:6010`. - **`src/platform/client.py`** - In `_resolve_token_and_org_key`, for `env == "local"`: if `DO_AUTH_TOKEN` is set (e.g. from pybridge), use it and use `DO_ORG_KEY` when set; otherwise keep the existing behaviour (auto-generated token and `org_key="deeporigin"`). - Allows E2E to inject a real Keycloak token for local runs while unit tests (no `DO_AUTH_TOKEN`) keep using the fake token against the mock. ## Rationale: 6010 (platform) vs 4931 (mock) - **6010** — Gateway (and rest of platform) when running the stack locally. E2E specs that use the SDK with `E2E_DEEPORIGIN_ENV=local` expect the SDK to talk to this gateway. - **4931** — Mock server used only by **unit tests** (pytest) so they can run without a real platform. By defaulting local to 6010 and overriding to 4931 only in conftest, we avoid adding a new env var (e.g. `DEEPORIGIN_BASE_UR`) and keep a single source of truth in `constants.py` with a test-only override. ## Companion WIP work (platform repo) Platform-e2e pybridge was updated to obtain a real Keycloak token when `targetEnv === 'local'` and inject `DO_AUTH_TOKEN` and `DO_ORG_KEY` into the Python process, so the SDK uses the real token against the local gateway. That change is in the platform repo (e.g. `local-auth.ts`, pybridge `injectLocalAuthIfNeeded`). ## How to verify - **Unit tests (mock):** `uv run pytest --env local -x` Expect requests to go to 4931 (conftest monkeypatch). No `DO_AUTH_TOKEN` needed. - **E2E against local platform:** From platform repo, with gateway on 6010 and Keycloak configured: `E2E_DEEPORIGIN_ENV=local E2E_DO_DD_CLIENT_LOCAL_PATH=/path/to/do-dd-client pnpm test:e2e:lv4 -- apps/platform-e2e/src/specs/do-client/...` Pybridge injects `DO_AUTH_TOKEN`; SDK uses it and talks to 6010. ### Test execution: Note: the communication between client SDK and platform local is happening, however, the functions/tools are not registered as part of platform-setup seeder by default, that's why it returned 404 (FUNCTION.NOT_FOUND) Execution flow: ```mermaid flowchart LR Playwright["Playwright Spec"] --> Pybridge["Pybridge executeJob"] Pybridge --> |"DO_AUTH_TOKEN, DO_ORG_KEY, spawn"| SDK["Client SDK (Python)"] SDK --> |"HTTP :6010 Bearer token"| Platform["Platform Gateway :6010"] ``` ```E2E_DEEPORIGIN_ENV=local pnpm test:e2e:lv4 -- apps/platform-e2e/src/specs/do-client/functions.spec.ts > @deeporigin/source@1.0.0 test:e2e:lv4 /Users/tiago/projects/platform > nx e2e-lvl4 platform-e2e --output=test-results -- apps/platform-e2e/src/specs/do-client/functions.spec.ts > nx run platform-e2e:e2e-lvl4 apps/platform-e2e/src/specs/do-client/functions.spec.ts npm warn Unknown env config "_biosim-ai-registry". This will stop working in the next major version of npm. { message: '[repo] repoUrl', repoUrl: 'https://github.com/deeporiginbio/do-dd-client', branch: 'main' } { message: '[repo] Using local do-dd-client path', localPath: '/Users/tiago/projects/do-dd-client' } Running 1 test using 1 worker { message: '[repo] repoUrl', repoUrl: 'https://github.com/deeporiginbio/do-dd-client', branch: 'main' } { message: '[repo] Using local do-dd-client path', localPath: '/Users/tiago/projects/do-dd-client' } ✘ 1 [e2e] › src/specs/do-client/functions.spec.ts:102:8 › deeporigin python client functions testing › @lv3-reg: execute property_prediction (3.8s) { message: '[repo] Using local do-dd-client', repoPath: '/Users/tiago/projects/do-dd-client' } Python 3.12 is already installed { message: '[venv] Skipping make install (already set up)' } { message: '[venv] Python', version: 'Python 3.13.11' } { message: '[venv] Verifying imports: deeporigin, yaml' } Python 3.12 is already installed { message: '[venv] Skipping make install (already set up)' } { message: '[venv] Python', version: 'Python 3.13.11' } { message: '[venv] Verifying imports: deeporigin, yaml' } { message: '[executeJob] Starting job', jobId: '97d2fe91-843b-4588-ae69-c9ea9ae935f6', tool: 'property_prediction' } { message: '[local-auth] Fetching Keycloak token', url: 'http://localhost:5090/auth/realms/deeporigin/protocol/openid-connect/token' } { message: '[pybridge] Injected local Keycloak token and org key' } { message: '[executeJob] Delayed start', startAfterMs: 200 } { message: '[process] Spawned', pythonPath: '/Users/tiago/projects/do-dd-client/.venv/bin/python', cmd: 'src/libs/pybridge/tool-runners/property_prediction.py --ligandIdentifier serotonin --useCache False', pid: 90431 } { message: '[process] Exited', cmd: 'src/libs/pybridge/tool-runners/property_prediction.py --ligandIdentifier serotonin --useCache False', pid: 90431, code: 1, signal: null } { "exitCode": 1, "stdout": "{\"error\": \"\\u2554\\u2550 Request to platform API failed. \\u2550\\u2557\\nA POST request to the platform API failed (HTTP 404). Error message: {'errors': [{'id': 'c211162a-5218-49d7-bf51-96313fb3bc00', 'status': 404, 'code': 'FUNCTION.NOT_FOUND', 'title': 'Function not found deeporigin.mol-props-cyp:latest', 'detail': 'Function not found deeporigin.mol-props-cyp:latest'}]} Validation errors:\\n[\\n {\\n \\\"id\\\": \\\"c211162a-5218-49d7-bf51-96313fb3bc00\\\",\\n \\\"status\\\": 404,\\n \\\"code\\\": \\\"FUNCTION.NOT_FOUND\\\",\\n \\\"title\\\": \\\"Function not found deeporigin.mol-props-cyp:latest\\\",\\n \\\"detail\\\": \\\"Function not found deeporigin.mol-props-cyp:latest\\\"\\n }\\n] Curl command to reproduce the request saved to: /Users/tiago/.deeporigin/ecb156cb-c40f-42b8-9545-80444ba1ccbb.txt\\nPlease contact support at https://help.deeporigin.com/ and provide this text file.\"}\n", "stderr": "", "meta": { "jobId": "97d2fe91-843b-4588-ae69-c9ea9ae935f6" } } 1) [e2e] › src/specs/do-client/functions.spec.ts:102:8 › deeporigin python client functions testing › @lv3-reg: execute property_prediction Error: expect(received).toBe(expected) // Object.is equality Expected: 0 Received: 1 103 | const result = await runFunction('property_prediction'); 104 | console.log(JSON.stringify(result, null, 2)); > 105 | expect(result.exitCode).toBe(0); | ^ 106 | 107 | const outJSON = JSON.parse(result.stdout); 108 | expect(outJSON).toEqual( at /Users/tiago/projects/platform/apps/platform-e2e/src/specs/do-client/functions.spec.ts:105:29 attachment #1: trace (application/zip) ───────────────────────────────────────────────────────── ../../dist/.playwright/apps/platform-e2e/test-output/specs-do-client-functions--7599e-execute-property-prediction-e2e/trace.zip Usage: pnpm exec playwright show-trace ../../dist/.playwright/apps/platform-e2e/test-output/specs-do-client-functions--7599e-execute-property-prediction-e2e/trace.zip ──────────────────────────────────────────────────────────────────────────────────────────────── 1 failed [e2e] › src/specs/do-client/functions.spec.ts:102:8 › deeporigin python client functions testing › @lv3-reg: execute property_prediction ``` --------- Signed-off-by: Tiago Silva <tiago.goes2009@gmail.com>

sg-s added 3 commits April 10, 2024 14:05

fix(testing): proof-of-concept tests work

3be0806

fix(managed-data): fixed bugs in dataframe export

319fd48

feat(testing): support for running tests on live deployment

092efa7

sg-s self-assigned this Apr 10, 2024

sg-s added 2 commits April 10, 2024 21:26

feat(testing): full support for mocking

8631be9

fix(testing): wrote readme

088830f

sg-s requested a review from jonrkarr April 11, 2024 01:36

sg-s added 6 commits April 10, 2024 21:39

fix(testing): moved codeowners to correct location

7a1b758

feat(testing): github action

c34a7a7

fix(testing): formatted files

260faee

fix(testing): added a fake api_endpoint

ab36445

fix(testing): added API tokens file

2341672

fix(testing): should work now

9f02c1c

sg-s marked this pull request as ready for review April 11, 2024 01:58