Skip to content

feat: add label for self-hosted runner in neug-test#199

Merged
lnfjpt merged 4 commits into
alibaba:mainfrom
lnfjpt:add_self_hosted_label
Apr 10, 2026
Merged

feat: add label for self-hosted runner in neug-test#199
lnfjpt merged 4 commits into
alibaba:mainfrom
lnfjpt:add_self_hosted_label

Conversation

@lnfjpt
Copy link
Copy Markdown
Collaborator

@lnfjpt lnfjpt commented Apr 10, 2026

Fixes #201

@lnfjpt lnfjpt requested a review from zhanglei1949 April 10, 2026 02:29
Copy link
Copy Markdown
Contributor

@greptile-apps greptile-apps Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your free trial has ended. If you'd like to continue receiving code reviews, you can add a payment method here.

@qodo-code-review
Copy link
Copy Markdown

ⓘ You are approaching your monthly quota for Qodo. Upgrade your plan

Review Summary by Qodo

Add daily label to self-hosted runners and LDBC benchmark workflow

✨ Enhancement

Grey Divider

Walkthroughs

Description
• Add daily label to self-hosted runners across multiple workflows
• Create new LDBC SNB Benchmark workflow for scheduled performance testing
• Standardize runner labels for build and test jobs
Diagram
flowchart LR
  A["Workflows"] --> B["Add daily label"]
  A --> C["New LDBC Benchmark"]
  B --> D["build-extensions.yml"]
  B --> E["neug-extension-test.yml"]
  B --> F["neug-test.yml"]
  B --> G["wheels-common.yml"]
  C --> H["ldbc-benchmark.yml"]
  H --> I["Scheduled Testing"]
  I --> J["Load Dataset"]
  I --> K["Run Benchmark"]
Loading

Grey Divider

File Changes

1. .github/workflows/build-extensions.yml ⚙️ Configuration changes +2/-2

Add daily label to build extension jobs

• Add daily label to build_linux_x86_64 job runner configuration
• Add daily label to build_linux_arm64 job runner configuration

.github/workflows/build-extensions.yml


2. .github/workflows/ldbc-benchmark.yml ✨ Enhancement +67/-0

Add LDBC SNB Benchmark scheduled workflow

• Create new workflow for LDBC SNB Benchmark testing
• Schedule runs on Tuesdays and Saturdays at UTC 19:00
• Build NeuG Python package with HTTP server and mimalloc enabled
• Load LDBC dataset and run interactive benchmark with result archival
• Implement server startup verification with timeout handling

.github/workflows/ldbc-benchmark.yml


3. .github/workflows/neug-extension-test.yml ⚙️ Configuration changes +3/-3

Add daily label to extension test jobs

• Add daily label to extension_tests_default job runner
• Add daily label to extension_tests_wheel_linux_x86_64 job runner
• Add daily label to extension_tests_wheel_linux_arm64 job runner

.github/workflows/neug-extension-test.yml


View more (2)
4. .github/workflows/neug-test.yml ⚙️ Configuration changes +1/-1

Add daily label to build and test job

• Add daily label to build_and_test job runner configuration

.github/workflows/neug-test.yml


5. .github/workflows/wheels-common.yml ⚙️ Configuration changes +2/-2

Add daily label to wheel build jobs

• Add daily label to build_wheels_linux_x86_64 job runner
• Add daily label to build_wheels_linux_arm64 job runner

.github/workflows/wheels-common.yml


Grey Divider

Qodo Logo

@qodo-code-review
Copy link
Copy Markdown

qodo-code-review Bot commented Apr 10, 2026

Code Review by Qodo

🐞 Bugs (2)   📘 Rule violations (0)   📎 Requirement gaps (0)   🎨 UX Issues (0)
🐞\ ☼ Reliability (2)

Grey Divider


Action required

1. Workflow references missing scripts🐞
Description
.github/workflows/ldbc-benchmark.yml invokes python3 benchmark/interactive/load_ldbc.py and
serve_ldbc.py, but this repository’s LDBC interactive benchmark implementation is under
examples/ldbc_interactive_benchmark/, so the job will fail when it tries to execute those
non-existent paths.
Code

.github/workflows/ldbc-benchmark.yml[R33-46]

+      - name: Load LDBC dataset
+        run: |
+          export PYTHONPATH=${{ github.workspace }}/tools/python_bind:${{ github.workspace }}/tools/python_bind/build/lib:$PYTHONPATH
+          export DATA_PATH=/data/benchmark/social_network-sf100-CsvComposite-StringDateFormatter
+          start=$(date +%s)
+          python3 benchmark/interactive/load_ldbc.py
+          end=$(date +%s)
+          echo "::notice::Load LDBC dataset took $((end - start)) seconds"
+
+      - name: Start NeuG server and run benchmark
+        run: |
+          export PYTHONPATH=${{ github.workspace }}/tools/python_bind:${{ github.workspace }}/tools/python_bind/build/lib:$PYTHONPATH
+          nohup python3 benchmark/interactive/serve_ldbc.py > serve.log 2>&1 &
+          echo $! > serve.pid
Evidence
The workflow explicitly runs benchmark/interactive/load_ldbc.py and
benchmark/interactive/serve_ldbc.py, while the repo documents and provides
examples/ldbc_interactive_benchmark/run_benchmark.py as the LDBC SNB Interactive benchmark
entrypoint.

.github/workflows/ldbc-benchmark.yml[33-46]
examples/ldbc_interactive_benchmark/README.md[37-45]
examples/ldbc_interactive_benchmark/run_benchmark.py[2-16]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`ldbc-benchmark.yml` calls `benchmark/interactive/load_ldbc.py` and `benchmark/interactive/serve_ldbc.py`, but the repo’s LDBC benchmark scripts live under `examples/ldbc_interactive_benchmark/`. This makes the workflow fail when it reaches those steps.
### Issue Context
There is an existing, documented benchmark script `examples/ldbc_interactive_benchmark/run_benchmark.py`.
### Fix Focus Areas
- .github/workflows/ldbc-benchmark.yml[33-67]
- examples/ldbc_interactive_benchmark/README.md[37-45]
- examples/ldbc_interactive_benchmark/run_benchmark.py[2-16]
### Proposed fix
Update the workflow to run the existing script(s) under `examples/ldbc_interactive_benchmark/` (or add the missing `benchmark/interactive/*.py` files if that path is intended), and adjust environment variables/arguments accordingly (e.g., pass `--data-dir`, `--db-path`, output dir).

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


2. Build without requirements install 🐞
Description
The new ldbc-benchmark.yml runs make build for tools/python_bind without installing its Python
requirements first, so the build/runtime can fail due to missing dependencies (e.g., Flask is in
requirements.txt while make build does not install requirements).
Code

.github/workflows/ldbc-benchmark.yml[R26-32]

+      - name: Build NeuG Python package
+        working-directory: tools/python_bind
+        run: |
+          export BUILD_HTTP_SERVER=ON
+          export WITH_MIMALLOC=ON
+          make build
+
Evidence
make build is just an alias for setup.py build_ext and does not install dependencies; dependency
installation is a separate make requirements target and includes packages like Flask. Other CI
workflows install requirements before building, indicating that’s the expected invariant.

.github/workflows/ldbc-benchmark.yml[26-32]
tools/python_bind/Makefile[7-25]
tools/python_bind/requirements.txt[1-11]
.github/workflows/benchmark.yml[54-69]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`ldbc-benchmark.yml` runs `make build` in `tools/python_bind` without first installing `requirements.txt`/`requirements_dev.txt`. Since `make build` doesn’t install dependencies, the job may fail depending on runner state.
### Issue Context
The repo’s `tools/python_bind/Makefile` separates `requirements` from `build`, and requirements include runtime deps like `Flask`.
### Fix Focus Areas
- .github/workflows/ldbc-benchmark.yml[26-32]
- tools/python_bind/Makefile[7-25]
- tools/python_bind/requirements.txt[1-11]
### Proposed fix
In the "Build NeuG Python package" step, run `make requirements` (or explicitly `python3 -m pip install -r requirements.txt -r requirements_dev.txt`) before `make build`.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools



Remediation recommended

3. Orphaned server process 🐞
Description
ldbc-benchmark.yml starts the NeuG server with nohup ... & on a self-hosted runner but never
stops it, which can leave orphan processes and cause future runs to fail (e.g., port conflicts) or
slowly exhaust runner resources.
Code

.github/workflows/ldbc-benchmark.yml[R42-67]

+      - name: Start NeuG server and run benchmark
+        run: |
+          export PYTHONPATH=${{ github.workspace }}/tools/python_bind:${{ github.workspace }}/tools/python_bind/build/lib:$PYTHONPATH
+          nohup python3 benchmark/interactive/serve_ldbc.py > serve.log 2>&1 &
+          echo $! > serve.pid
+          for i in $(seq 1 2700); do
+            if grep -q "Brpc server started on : 0.0.0.0:57381" serve.log 2>/dev/null; then
+              echo "::notice::NeuG server started (PID $(cat serve.pid))"
+              break
+            fi
+            if ! kill -0 $(cat serve.pid) 2>/dev/null; then
+              echo "::error::NeuG server process exited unexpectedly"
+              cat serve.log
+              exit 1
+            fi
+            sleep 1
+          done
+          if ! grep -q "Brpc server started on : 0.0.0.0:57381" serve.log 2>/dev/null; then
+            echo "::error::NeuG server did not start within 2700 seconds"
+            cat serve.log
+            exit 1
+          fi
+          RESULT_DIR="/data/benchmark-result/$(date +%Y%m%d-%H%M%S)-${GITHUB_SHA:0:7}"
+          mkdir -p "$RESULT_DIR"
+          /data/benchmark/flex_ldbc_snb/driver/neug/driver/benchmark.sh /data/benchmark/flex_ldbc_snb/driver/neug/driver/benchmark-sf100.properties 2>&1 | tee "$RESULT_DIR/benchmark.log"
+          echo "::notice::Benchmark results saved to $RESULT_DIR"
Evidence
The job runs on a self-hosted runner and backgrounds a server process while only recording its PID;
there is no later step to kill that PID or a trap to ensure cleanup on success/failure.

.github/workflows/ldbc-benchmark.yml[14-17]
.github/workflows/ldbc-benchmark.yml[42-67]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The workflow backgrounds a long-running server (`nohup ... &`) but never terminates it, leaving orphan processes on a persistent self-hosted runner.
### Issue Context
A PID is written to `serve.pid`, but it’s never used for cleanup.
### Fix Focus Areas
- .github/workflows/ldbc-benchmark.yml[42-67]
### Proposed fix
Add cleanup in the same step via `trap` (e.g., `trap 'kill $(cat serve.pid) 2>/dev/null || true' EXIT`) or add a final "Stop server" step guarded with `if: always()` that kills the PID and prints logs when failures occur.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

ⓘ The new review experience is currently in Beta. Learn more

Grey Divider

Qodo Logo

Comment thread .github/workflows/ldbc-benchmark.yml Outdated
Comment thread .github/workflows/ldbc-benchmark.yml Outdated
Comment on lines +26 to +32
- name: Build NeuG Python package
working-directory: tools/python_bind
run: |
export BUILD_HTTP_SERVER=ON
export WITH_MIMALLOC=ON
make build

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

2. Build without requirements install 🐞 Bug ☼ Reliability

The new ldbc-benchmark.yml runs make build for tools/python_bind without installing its Python
requirements first, so the build/runtime can fail due to missing dependencies (e.g., Flask is in
requirements.txt while make build does not install requirements).
Agent Prompt
### Issue description
`ldbc-benchmark.yml` runs `make build` in `tools/python_bind` without first installing `requirements.txt`/`requirements_dev.txt`. Since `make build` doesn’t install dependencies, the job may fail depending on runner state.

### Issue Context
The repo’s `tools/python_bind/Makefile` separates `requirements` from `build`, and requirements include runtime deps like `Flask`.

### Fix Focus Areas
- .github/workflows/ldbc-benchmark.yml[26-32]
- tools/python_bind/Makefile[7-25]
- tools/python_bind/requirements.txt[1-11]

### Proposed fix
In the "Build NeuG Python package" step, run `make requirements` (or explicitly `python3 -m pip install -r requirements.txt -r requirements_dev.txt`) before `make build`.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Copy link
Copy Markdown
Member

@zhanglei1949 zhanglei1949 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@lnfjpt lnfjpt merged commit 6306c59 into alibaba:main Apr 10, 2026
16 checks passed
@lnfjpt lnfjpt mentioned this pull request May 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: Add label for self-hosted runner

2 participants