Add Dev Container, Benchmarking & SBOM workflows#67841
Add Dev Container, Benchmarking & SBOM workflows#67841c6zks4gssn-droid wants to merge 3 commits intoopenclaw:mainfrom
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5b1c92880e
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| node-version: 22 | ||
| cache: npm | ||
|
|
||
| - run: npm ci |
There was a problem hiding this comment.
This repository is a pnpm workspace (it ships pnpm-lock.yaml and no package-lock.json), so npm ci fails immediately with EUSAGE and the benchmark job cannot run its intended setup. As written, every benchmark run on main/PRs will fail before collecting meaningful data unless a lockfile for npm is added, which is inconsistent with the repo’s package-manager contract.
Useful? React with 👍 / 👎.
| - name: Run benchmarks | ||
| run: | | ||
| hyperfine --warmup 2 --export-json bench-results.json \ | ||
| 'node -e "require(\"./dist/cli.js\")" --help' \ |
There was a problem hiding this comment.
The command node -e "require(\"./dist/cli.js\")" --help is parsed as Node’s own --help, so Node prints its built-in help and exits before running the require(...) expression. That means this benchmark measures Node help startup time rather than OpenClaw CLI behavior, so regressions in the actual CLI path won’t be detected.
Useful? React with 👍 / 👎.
|
|
||
| - uses: anchore/sbom-action@v0 | ||
| with: | ||
| image: node:22 |
There was a problem hiding this comment.
Setting image: node:22 makes the SBOM workflow describe the upstream Node container image rather than this repository’s released package contents, so the uploaded sbom.spdx.json does not represent OpenClaw’s own dependency/material list. This undermines the release transparency goal because consumers receive an SBOM for a different artifact.
Useful? React with 👍 / 👎.
Greptile SummaryThis PR adds a dev container config, a performance benchmarking workflow, and an SBOM generation workflow. All three files have P1 issues that will prevent them from working correctly as written.
Confidence Score: 2/5Not safe to merge — all three new files have blocking defects that prevent them from functioning. Five P1 issues across all three changed files: wrong package manager (npm vs pnpm) causing install failure, missing build step and wrong CLI entry point in the benchmark, an SBOM that scans the wrong target entirely, and a missing permissions grant that will cause the release attachment to 403. All three files need attention: Prompt To Fix All With AIThis is a comment left during a code review.
Path: .github/workflows/benchmark.yml
Line: 18-20
Comment:
**Wrong package manager — workflow will fail to install dependencies**
The repo uses `pnpm` with a `pnpm-lock.yaml` lockfile; there is no `package-lock.json`. Running `npm ci` will exit with `npm error The \`npm ci\` command can only install with an existing package-lock.json` before any benchmark runs. The `cache: npm` setting is similarly incorrect.
```suggestion
cache: pnpm
- uses: pnpm/action-setup@v4
with:
run_install: false
- run: pnpm install --frozen-lockfile
```
How can I resolve this? If you propose a fix, please make it concise.
---
This is a comment left during a code review.
Path: .github/workflows/benchmark.yml
Line: 27-31
Comment:
**Benchmark command references a non-existent file and uses the wrong entry point**
`dist/cli.js` is never built (there is no `pnpm build` step above), so hyperfine will immediately error on a missing file. The repo's actual CLI entry point is `openclaw.mjs` — the CI smoke test and the `bin` field in `package.json` both confirm this. Using `require()` will also fail at runtime because the project is ESM. The `|| true` then silences all of this, so the benchmark appears green regardless of outcome.
A corrected block would be:
```yaml
- name: Build dist
run: pnpm build
- name: Run benchmarks
run: |
hyperfine --warmup 2 --export-json bench-results.json \
'node openclaw.mjs --help'
```
How can I resolve this? If you propose a fix, please make it concise.
---
This is a comment left during a code review.
Path: .github/workflows/sbom.yml
Line: 12-17
Comment:
**SBOM scans the Node.js runtime image, not the project's own dependencies**
Passing `image: node:22` to `anchore/sbom-action` runs Syft against the upstream `node:22` Docker image, producing a bill of materials for the Node.js runtime itself rather than for OpenClaw's npm dependency tree. The resulting SBOM is unrelated to this release.
To generate an SBOM for the project's npm packages, omit `image` and let Syft scan the checked-out source directory:
```yaml
- uses: anchore/sbom-action@v0
with:
path: .
format: spdx-json
output-file: sbom.spdx.json
```
How can I resolve this? If you propose a fix, please make it concise.
---
This is a comment left during a code review.
Path: .github/workflows/sbom.yml
Line: 7-8
Comment:
**Missing `contents: write` permission — release attachment will fail**
The `softprops/action-gh-release@v2` step needs write access to modify the release assets. Without an explicit `permissions` block the job inherits the repository default (`contents: read`), so the upload step will fail with a 403.
```suggestion
jobs:
sbom:
runs-on: ubuntu-latest
permissions:
contents: write
```
How can I resolve this? If you propose a fix, please make it concise.
---
This is a comment left during a code review.
Path: .devcontainer/devcontainer.json
Line: 4
Comment:
**`postCreateCommand` uses `npm install` — will produce wrong lockfile**
The repo is managed with `pnpm` (`pnpm-lock.yaml`); running `npm install` will create a stale `package-lock.json` in the container, ignore the existing lockfile, and likely fail to wire up the pnpm workspace correctly. The `typescript-node` base image does not include pnpm by default.
```suggestion
"postCreateCommand": "npm install -g pnpm && pnpm install",
```
How can I resolve this? If you propose a fix, please make it concise.
---
This is a comment left during a code review.
Path: .devcontainer/devcontainer.json
Line: 10-13
Comment:
**Wrong linter/formatter extensions suggested**
The repo uses Oxlint and Oxfmt (not ESLint or Prettier) as its lint and format toolchain. `dbaeumer.vscode-eslint` and `esbenp.prettier-vscode` won't wire up to the project's `pnpm check` / `pnpm format` commands and may surface confusing false positives. Consider replacing them with the Oxc extension (`oxc.oxc-vscode`) if one is available, or removing them.
How can I resolve this? If you propose a fix, please make it concise.Reviews (1): Last reviewed commit: "Add SBOM generation workflow using ancho..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| cache: npm | ||
|
|
||
| - run: npm ci |
There was a problem hiding this comment.
The repo uses pnpm with a pnpm-lock.yaml lockfile; there is no package-lock.json. Running npm ci will exit with npm error The \npm ci` command can only install with an existing package-lock.jsonbefore any benchmark runs. Thecache: npm` setting is similarly incorrect.
| cache: npm | |
| - run: npm ci | |
| cache: pnpm | |
| - uses: pnpm/action-setup@v4 | |
| with: | |
| run_install: false | |
| - run: pnpm install --frozen-lockfile |
Prompt To Fix With AI
This is a comment left during a code review.
Path: .github/workflows/benchmark.yml
Line: 18-20
Comment:
**Wrong package manager — workflow will fail to install dependencies**
The repo uses `pnpm` with a `pnpm-lock.yaml` lockfile; there is no `package-lock.json`. Running `npm ci` will exit with `npm error The \`npm ci\` command can only install with an existing package-lock.json` before any benchmark runs. The `cache: npm` setting is similarly incorrect.
```suggestion
cache: pnpm
- uses: pnpm/action-setup@v4
with:
run_install: false
- run: pnpm install --frozen-lockfile
```
How can I resolve this? If you propose a fix, please make it concise.| - name: Run benchmarks | ||
| run: | | ||
| hyperfine --warmup 2 --export-json bench-results.json \ | ||
| 'node -e "require(\"./dist/cli.js\")" --help' \ | ||
| || true |
There was a problem hiding this comment.
dist/cli.js is never built (there is no pnpm build step above), so hyperfine will immediately error on a missing file. The repo's actual CLI entry point is openclaw.mjs — the CI smoke test and the bin field in package.json both confirm this. Using require() will also fail at runtime because the project is ESM. The || true then silences all of this, so the benchmark appears green regardless of outcome.
A corrected block would be:
- name: Build dist
run: pnpm build
- name: Run benchmarks
run: |
hyperfine --warmup 2 --export-json bench-results.json \
'node openclaw.mjs --help'Prompt To Fix With AI
This is a comment left during a code review.
Path: .github/workflows/benchmark.yml
Line: 27-31
Comment:
**Benchmark command references a non-existent file and uses the wrong entry point**
`dist/cli.js` is never built (there is no `pnpm build` step above), so hyperfine will immediately error on a missing file. The repo's actual CLI entry point is `openclaw.mjs` — the CI smoke test and the `bin` field in `package.json` both confirm this. Using `require()` will also fail at runtime because the project is ESM. The `|| true` then silences all of this, so the benchmark appears green regardless of outcome.
A corrected block would be:
```yaml
- name: Build dist
run: pnpm build
- name: Run benchmarks
run: |
hyperfine --warmup 2 --export-json bench-results.json \
'node openclaw.mjs --help'
```
How can I resolve this? If you propose a fix, please make it concise.|
|
||
| - uses: anchore/sbom-action@v0 | ||
| with: | ||
| image: node:22 | ||
| format: spdx-json | ||
| output-file: sbom.spdx.json |
There was a problem hiding this comment.
Passing image: node:22 to anchore/sbom-action runs Syft against the upstream node:22 Docker image, producing a bill of materials for the Node.js runtime itself rather than for OpenClaw's npm dependency tree. The resulting SBOM is unrelated to this release.
To generate an SBOM for the project's npm packages, omit image and let Syft scan the checked-out source directory:
- uses: anchore/sbom-action@v0
with:
path: .
format: spdx-json
output-file: sbom.spdx.jsonPrompt To Fix With AI
This is a comment left during a code review.
Path: .github/workflows/sbom.yml
Line: 12-17
Comment:
**SBOM scans the Node.js runtime image, not the project's own dependencies**
Passing `image: node:22` to `anchore/sbom-action` runs Syft against the upstream `node:22` Docker image, producing a bill of materials for the Node.js runtime itself rather than for OpenClaw's npm dependency tree. The resulting SBOM is unrelated to this release.
To generate an SBOM for the project's npm packages, omit `image` and let Syft scan the checked-out source directory:
```yaml
- uses: anchore/sbom-action@v0
with:
path: .
format: spdx-json
output-file: sbom.spdx.json
```
How can I resolve this? If you propose a fix, please make it concise.| jobs: | ||
| sbom: |
There was a problem hiding this comment.
contents: write permission — release attachment will fail
The softprops/action-gh-release@v2 step needs write access to modify the release assets. Without an explicit permissions block the job inherits the repository default (contents: read), so the upload step will fail with a 403.
| jobs: | |
| sbom: | |
| jobs: | |
| sbom: | |
| runs-on: ubuntu-latest | |
| permissions: | |
| contents: write |
Prompt To Fix With AI
This is a comment left during a code review.
Path: .github/workflows/sbom.yml
Line: 7-8
Comment:
**Missing `contents: write` permission — release attachment will fail**
The `softprops/action-gh-release@v2` step needs write access to modify the release assets. Without an explicit `permissions` block the job inherits the repository default (`contents: read`), so the upload step will fail with a 403.
```suggestion
jobs:
sbom:
runs-on: ubuntu-latest
permissions:
contents: write
```
How can I resolve this? If you propose a fix, please make it concise.| { | ||
| "name": "OpenClaw Dev", | ||
| "image": "mcr.microsoft.com/devcontainers/typescript-node:22", | ||
| "features": { |
There was a problem hiding this comment.
postCreateCommand uses npm install — will produce wrong lockfile
The repo is managed with pnpm (pnpm-lock.yaml); running npm install will create a stale package-lock.json in the container, ignore the existing lockfile, and likely fail to wire up the pnpm workspace correctly. The typescript-node base image does not include pnpm by default.
| "features": { | |
| "postCreateCommand": "npm install -g pnpm && pnpm install", |
Prompt To Fix With AI
This is a comment left during a code review.
Path: .devcontainer/devcontainer.json
Line: 4
Comment:
**`postCreateCommand` uses `npm install` — will produce wrong lockfile**
The repo is managed with `pnpm` (`pnpm-lock.yaml`); running `npm install` will create a stale `package-lock.json` in the container, ignore the existing lockfile, and likely fail to wire up the pnpm workspace correctly. The `typescript-node` base image does not include pnpm by default.
```suggestion
"postCreateCommand": "npm install -g pnpm && pnpm install",
```
How can I resolve this? If you propose a fix, please make it concise.| "extensions": [ | ||
| "dbaeumer.vscode-eslint", | ||
| "esbenp.prettier-vscode", | ||
| "ms-vscode.vscode-typescript-next" |
There was a problem hiding this comment.
The repo uses Oxlint and Oxfmt (not ESLint or Prettier) as its lint and format toolchain. dbaeumer.vscode-eslint and esbenp.prettier-vscode won't wire up to the project's pnpm check / pnpm format commands and may surface confusing false positives. Consider replacing them with the Oxc extension (oxc.oxc-vscode) if one is available, or removing them.
Prompt To Fix With AI
This is a comment left during a code review.
Path: .devcontainer/devcontainer.json
Line: 10-13
Comment:
**Wrong linter/formatter extensions suggested**
The repo uses Oxlint and Oxfmt (not ESLint or Prettier) as its lint and format toolchain. `dbaeumer.vscode-eslint` and `esbenp.prettier-vscode` won't wire up to the project's `pnpm check` / `pnpm format` commands and may surface confusing false positives. Consider replacing them with the Oxc extension (`oxc.oxc-vscode`) if one is available, or removing them.
How can I resolve this? If you propose a fix, please make it concise.1 participant
Improvements
This PR adds 3 missing project infrastructure features identified by fork-doctor analysis:
1. Dev Container (
.devcontainer/devcontainer.json)2. Performance Benchmarking (
.github/workflows/benchmark.yml)3. SBOM Generation (
.github/workflows/sbom.yml)Score: 10/13 → 13/13 ✅
Tested with fork-doctor analyze.