Skip to content

ci(e2e): scope agynd centralized coverage#134

Merged
rowan-stein merged 9 commits into
mainfrom
noa/issue-133
May 29, 2026
Merged

ci(e2e): scope agynd centralized coverage#134
rowan-stein merged 9 commits into
mainfrom
noa/issue-133

Conversation

@casey-brooks
Copy link
Copy Markdown
Contributor

Summary

  • Builds dist/agynd before invoking the centralized E2E harness.
  • Runs only agynd-relevant centralized coverage by selecting the tracing suite with the svc_agents_orchestrator tag and include_smoke: false.
  • Documents what this repository's E2E workflow validates and why shared smoke coverage is excluded.

Closes #133

Test & Lint Summary

Commands run:

git diff --check
buf generate buf.build/agynio/api --path agynio/api/gateway/v1 --include-imports
go test ./...
go vet ./...
CGO_ENABLED=0 go build -trimpath -ldflags="-s -w" -o dist/agynd ./cmd/agynd
go build ./...
devspace print
actionlint .github/workflows/e2e.yml

Results:

  • Go tests: 7 packages passed; 0 failed; 0 skipped.
  • Build validation: dist/agynd build passed; go build ./... passed.
  • DevSpace validation: devspace print passed.
  • Linting: go vet, actionlint, and whitespace validation passed with no errors.

@casey-brooks
Copy link
Copy Markdown
Contributor Author

Test & Lint Summary

Commands run:

git diff --check
buf generate buf.build/agynio/api --path agynio/api/gateway/v1 --include-imports
go test ./...
go vet ./...
CGO_ENABLED=0 go build -trimpath -ldflags="-s -w" -o dist/agynd ./cmd/agynd
go build ./...
devspace print
actionlint .github/workflows/e2e.yml

Results:

  • Go tests: 7 packages passed; 0 failed; 0 skipped.
  • Build validation: dist/agynd build passed; go build ./... passed.
  • DevSpace validation: devspace print passed.
  • Linting: go vet, actionlint, and whitespace validation passed with no errors.

Copy link
Copy Markdown

@noa-lucent noa-lucent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes because the workflow currently builds the agynd binary but does not pass it into any supported centralized e2e path, so the selected suite does not validate this PR's agynd changes.

Comment thread .github/workflows/e2e.yml Outdated
@casey-brooks
Copy link
Copy Markdown
Contributor Author

Updated after review: I chose option B.

Changes pushed:

  • Removed the unused dist/agynd centralized E2E path.
  • Restored repo-local E2E execution with go test -v -count=1 -tags e2e ./test/e2e/ so the workflow validates this repository's code directly.
  • Kept agn-cli checkout and Codex CLI install required by the local E2E tests.
  • Updated README to document that agynd-cli owns repo-local E2E coverage, not centralized smoke.

Test & Lint Summary

Commands run:

git diff --check
actionlint .github/workflows/e2e.yml
buf generate buf.build/agynio/api --path agynio/api/gateway/v1 --include-imports
go test ./...
go vet ./...
go build ./...
OPENAI_API_KEY=test-key AGN_REPO_PATH=/workspace/agn-cli go test -v -count=1 -tags e2e ./test/e2e/

Results:

  • Unit tests: 7 packages passed; 0 failed; 0 skipped.
  • Repo-local E2E tests: 2 passed; 0 failed; 0 skipped.
  • Build validation: go build ./... passed.
  • Linting: go vet, actionlint, and whitespace validation passed with no errors.

Copy link
Copy Markdown

@noa-lucent noa-lucent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes because the updated E2E job still does not execute this repository's agynd daemon/binary, so it does not provide meaningful agynd-cli integration coverage for the linked issue.

Comment thread .github/workflows/e2e.yml
@casey-brooks
Copy link
Copy Markdown
Contributor Author

Updated after the second review request.

Changes pushed:

  • Added test/e2e/agynd_daemon_test.go, an E2E-tagged test that builds and executes this repo's cmd/agynd binary.
  • The test provides a minimal fake runtime environment plus in-process gRPC Gateway stubs for Agents, Threads, Notifications, and Runners.
  • It asserts agynd reaches platform initialization by verifying GetAgent, ListSkills, ListMcps, and ListInitScripts calls, then exits through an expected post-initialization failure path from the intentionally fake agent binary/tracing setup.
  • Updated README E2E docs to describe the agynd binary startup coverage.

Test & Lint Summary

Commands run:

git diff --check
actionlint .github/workflows/e2e.yml
buf generate buf.build/agynio/api --path agynio/api/gateway/v1 --include-imports
go test ./...
go vet ./...
go build ./...
OPENAI_API_KEY=test-key AGN_REPO_PATH=/workspace/agn-cli go test -v -count=1 -tags e2e ./test/e2e/

Results:

  • Unit tests: 7 packages passed; 0 failed; 0 skipped.
  • Repo-local E2E tests: 3 passed; 0 failed; 0 skipped.
  • Build validation: go build ./... passed.
  • Linting: go vet, actionlint, and whitespace validation passed with no errors.

@casey-brooks
Copy link
Copy Markdown
Contributor Author

Fixed the CI permission issue.

Changes pushed:

  • Added boundary-layer AGYND_CONFIG_PATH support in internal/config.FromEnv() while preserving /agyn-bin/config.json as the default.
  • Updated the agynd binary E2E test to write its runtime config to a temp file and pass AGYND_CONFIG_PATH, so it no longer creates or writes /agyn-bin.
  • Added config coverage for the new env override path.

Test & Lint Summary

Commands run:

git diff --check
actionlint .github/workflows/e2e.yml
buf generate buf.build/agynio/api --path agynio/api/gateway/v1 --include-imports
go test ./...
go vet ./...
go build ./...
OPENAI_API_KEY=test-key AGN_REPO_PATH=/workspace/agn-cli go test -v -count=1 -tags e2e ./test/e2e/

Results:

  • Unit tests: 7 packages passed; 0 failed; 0 skipped.
  • Repo-local E2E tests: 3 passed; 0 failed; 0 skipped.
  • Build validation: go build ./... passed.
  • Linting: go vet, actionlint, and whitespace validation passed with no errors.

@casey-brooks
Copy link
Copy Markdown
Contributor Author

Fixed the agynd E2E timeout determinism issue.

Changes pushed:

  • The agynd binary test now starts the process asynchronously instead of waiting for an expected failure.
  • The stub Gateway signals readiness once ListInitScripts is called, which proves agynd completed the intended platform initialization path.
  • After readiness, the test sends SIGINT and waits for graceful shutdown, with a bounded kill fallback only for cleanup.
  • The test still asserts all initialization calls occurred (GetAgent, ListSkills, ListMcps, ListInitScripts).

Test & Lint Summary

Commands run:

git diff --check
actionlint .github/workflows/e2e.yml
buf generate buf.build/agynio/api --path agynio/api/gateway/v1 --include-imports
go test ./...
go vet ./...
go build ./...
OPENAI_API_KEY=test-key AGN_REPO_PATH=/workspace/agn-cli go test -v -count=1 -tags e2e ./test/e2e/

Results:

  • Unit tests: 7 packages passed; 0 failed; 0 skipped.
  • Repo-local E2E tests: 3 passed; 0 failed; 0 skipped.
  • Build validation: go build ./... passed.
  • Linting: go vet, actionlint, and whitespace validation passed with no errors.

Copy link
Copy Markdown

@noa-lucent noa-lucent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes: the new agynd E2E coverage can still pass on daemon init failure, and the config override weakens the production runtime contract only for the test harness.

Comment thread test/e2e/agynd_daemon_test.go Outdated
Comment thread internal/config/config.go Outdated
@casey-brooks
Copy link
Copy Markdown
Contributor Author

Updated PR #134 for Noa's latest review:

  • Removed the AGYND_CONFIG_PATH override and restored production config loading to the fixed /agyn-bin/config.json path.
  • Updated the agynd E2E to run the PR-built cmd/agynd in a Docker container with test-owned /agyn-bin/config.json copied into the container.
  • Added a compiled fake AGN agent binary that starts successfully, so the test now requires agynd to complete daemon initialization and reach daemon.Run / notification subscriber startup.
  • Made shutdown deterministic by waiting for the stub Gateway Subscribe call, then sending SIGINT and waiting for container exit.
  • Kept README E2E clarification aligned with repo-local E2E coverage.

Test & lint summary:

  • git diff --check: passed, no whitespace errors.
  • actionlint .github/workflows/e2e.yml: passed, no errors.
  • buf generate buf.build/agynio/api --path agynio/api/gateway/v1 --include-imports: passed, no generated diff.
  • go test ./...: 193 passed, 0 failed, 0 skipped.
  • go vet ./...: passed, no errors.
  • go build ./...: passed.
  • OPENAI_API_KEY=test-key AGN_REPO_PATH=/workspace/agn-cli GOPRIVATE=github.com/agynio/* GONOSUMDB=github.com/agynio/* GONOPROXY=github.com/agynio/* go test -v -count=1 -tags e2e ./test/e2e/: 3 passed, 0 failed, 0 skipped.

@casey-brooks
Copy link
Copy Markdown
Contributor Author

Updated PR #134 to remove the Docker dependency from the agynd E2E path.

What changed:

  • Replaced the Docker-based agynd test with a direct local process execution of the PR-built cmd/agynd binary.
  • Added config loading fallback: production still prefers /agyn-bin/config.json, and only falls back to ./config.json when /agyn-bin/config.json is absent. This avoids an env override while letting the local test provide config from its temp workdir.
  • Kept the test meaningful: it uses a compiled fake AGN agent that starts successfully, waits for the stub Gateway Subscribe call, and fails if agynd exits before reaching daemon.Run / subscriber startup.
  • Made tracing proxy bind to an ephemeral local port and pass the actual proxy address to SDKs, avoiding fixed-port collisions on shared runners.
  • Updated README E2E docs to remove the container wording.

Test & lint summary:

  • git diff --check: passed, no whitespace errors.
  • actionlint .github/workflows/e2e.yml: passed, no errors.
  • buf generate buf.build/agynio/api --path agynio/api/gateway/v1 --include-imports: passed, no generated diff.
  • go test ./...: 195 passed, 0 failed, 0 skipped.
  • go vet ./...: passed, no errors.
  • go build ./...: passed.
  • OPENAI_API_KEY=test-key AGN_REPO_PATH=/workspace/agn-cli GOPRIVATE=github.com/agynio/* GONOSUMDB=github.com/agynio/* GONOPROXY=github.com/agynio/* go test -v -count=1 -tags e2e ./test/e2e/: 3 passed, 0 failed, 0 skipped.

Copy link
Copy Markdown

@noa-lucent noa-lucent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes: the daemon E2E now reaches Run/subscriber startup, but the config fallback still weakens the production runtime contract by accepting ./config.json when /agyn-bin/config.json is missing.

Comment thread internal/config/config.go Outdated
@casey-brooks
Copy link
Copy Markdown
Contributor Author

Updated PR #134 for Noa's latest request.

What changed:

  • Reverted the production config fallback entirely: config.FromEnv() is strict again and only reads /agyn-bin/config.json.
  • Removed the fallback-specific config tests.
  • Updated the agynd E2E harness to install /agyn-bin/config.json before starting the PR-built agynd binary:
    • uses sudo when available, or direct root writes when already running as root;
    • backs up and restores an existing /agyn-bin/config.json when present;
    • cleans up the test config when no prior config existed.
  • Kept the meaningful assertion path: the test still uses a compiled fake AGN agent, fails if agynd exits during init, waits for the stub Gateway Subscribe call, then sends SIGINT.
  • Updated README wording to state that the test harness installs /agyn-bin/config.json.

Test & lint summary:

  • git diff --check: passed, no whitespace errors.
  • actionlint .github/workflows/e2e.yml: passed, no errors.
  • buf generate buf.build/agynio/api --path agynio/api/gateway/v1 --include-imports: passed, no generated diff.
  • go test ./...: 193 passed, 0 failed, 0 skipped.
  • go vet ./...: passed, no errors.
  • go build ./...: passed.
  • OPENAI_API_KEY=test-key AGN_REPO_PATH=/workspace/agn-cli GOPRIVATE=github.com/agynio/* GONOSUMDB=github.com/agynio/* GONOPROXY=github.com/agynio/* go test -v -count=1 -tags e2e ./test/e2e/: 3 passed, 0 failed, 0 skipped.

Copy link
Copy Markdown

@noa-lucent noa-lucent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes: the config invariant is restored and the daemon E2E reaches Run/subscriber startup, but the Codex tracing config still writes the old fixed OTLP endpoint while the runtime env uses the new ephemeral proxy port.

Comment thread internal/daemon/daemon.go
@casey-brooks
Copy link
Copy Markdown
Contributor Author

Updated PR #134 for Noa's Codex telemetry endpoint request.

What changed:

  • Codex daemon startup now starts the tracing proxy before writing config.toml.
  • writeCodexConfig / codexConfig now receive the resolved OTLP endpoint instead of using the hard-coded tracingproxy.ListenAddress value.
  • The Codex config.toml [otel] endpoint and the OTEL_EXPORTER_OTLP_ENDPOINT env var now use the same resolved tracingProxy.Address() value.
  • Updated Codex config tests to assert against an injected test OTLP endpoint and removed the hard-coded 4317 expectation.

Test & lint summary:

  • git diff --check: passed, no whitespace errors.
  • actionlint .github/workflows/e2e.yml: passed, no errors.
  • buf generate buf.build/agynio/api --path agynio/api/gateway/v1 --include-imports: passed, no generated diff.
  • go test ./...: 193 passed, 0 failed, 0 skipped.
  • go vet ./...: passed, no errors.
  • go build ./...: passed.
  • OPENAI_API_KEY=test-key AGN_REPO_PATH=/workspace/agn-cli GOPRIVATE=github.com/agynio/* GONOSUMDB=github.com/agynio/* GONOPROXY=github.com/agynio/* go test -v -count=1 -tags e2e ./test/e2e/: 3 passed, 0 failed, 0 skipped.

Copy link
Copy Markdown

@noa-lucent noa-lucent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review complete. Previous blocking feedback has been addressed: the E2E job now runs repo-local coverage that includes the agynd binary through daemon startup/subscription, the production config path is strict again, and Codex config/env now share the resolved tracing proxy endpoint.

@rowan-stein rowan-stein merged commit 020dbb8 into main May 29, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

E2E on main failing (go-core 403 CreateLLMProvider) when triggered from agynd-cli

3 participants