Conversation
skudasov
changed the title
Contributor
|
✅ No conflicts with other open PRs targeting |
Tofel
reviewed
Mar 6, 2026
| testlistgenerator -o compatibility_test_list.json -p runlog -r TestRunLogBasic -f './smoke/runlog_test.go' -e reth -d "${{ needs.get-latest-available-images.outputs.reth_images }}" -t "evm-implementation-compatibility-test" -n "ubuntu-latest" | ||
| testlistgenerator -o compatibility_test_list.json -p log_poller -r TestLogPollerFewFiltersFixedDepth -f './smoke/log_poller_test.go' -e reth -d "${{ needs.get-latest-available-images.outputs.reth_images }}" -t "evm-implementation-compatibility-test" -n "ubuntu-latest" | ||
| testlistgenerator -o compatibility_test_list.json -p ocr -r TestOCRBasic -f './smoke/ocr_test.go' -e reth -d "${{ needs.get-latest-available-images.outputs.reth_images }}" -t "evm-implementation-compatibility-test" -n "ubuntu-latest" | ||
| testlistgenerator -o compatibility_test_list.json -p ocr2 -r '^TestOCRv2Basic/plugins$' -f './smoke/ocr2_test.go' -e reth -d "${{ needs.get-latest-available-images.outputs.reth_images }}" -t "evm-implementation-compatibility-test" -n "ubuntu-latest" |
Contributor
There was a problem hiding this comment.
shouldn't we replace them with devenv versions? Still waiting for a response from @davidcauchi whether these tests are still used/required.
Tofel
reviewed
Mar 6, 2026
Tofel
reviewed
Mar 6, 2026
skudasov
changed the title
erikburt
previously approved these changes
Mar 6, 2026
Tofel
reviewed
Mar 9, 2026
Contributor
There was a problem hiding this comment.
Pull request overview
Risk Rating: HIGH — large-scale migration/removal of integration tests plus CI/workflow changes and devenv product config changes.
Migrates JD + finality-violation coverage into the devenv system test harness, while removing legacy Cron/Flux/OCR/OCR2/Runlog smoke tests and the old tracing setup, and updating CI to stop referencing the removed suites.
Changes:
- Remove legacy
integration-tests/smokesuites (Cron/Flux/Runlog/OCR/OCR2/JD) and related CI wiring. - Add
devenv/tests/featurescoverage for multi-job-distributor and head-tracker finality-violation detection, plus a scheduled workflow to run them. - Remove old
.github/tracing/*local/CI configs and update docs to point atdevenvworkflows.
Scrupulous human review recommended (high impact areas):
.github/workflows/*and.github/e2e-tests.ymlchanges that alter what CI runs (risk of unintentionally dropping coverage).devenv/tests/features/reorg_finality_violation_test.gopolling/assertion behavior (flakiness risk).devenv/products/ocr2/configuration.gofeature toggles (MultiFeedsManagers) affecting default node config.
Suggested reviewers (per CODEOWNERS):
@smartcontractkit/devex-tooling(owns/devenv/,/devenv/products/ocr2/)@smartcontractkit/devex-cicd(owns/.github/**)@smartcontractkit/core(global + integration tests / CI impact)
Reviewed changes
Copilot reviewed 31 out of 32 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| integration-tests/smoke/runlog_test.go | Removed legacy Runlog smoke test. |
| integration-tests/smoke/ocr_test.go | Removed legacy OCRv1 smoke tests. |
| integration-tests/smoke/ocr2_test.go | Removed legacy OCRv2 smoke tests (incl. plugin assertions). |
| integration-tests/smoke/job_distributor_test.go | Removed legacy JD smoke test (migrated to devenv features). |
| integration-tests/smoke/flux_test.go | Removed legacy Flux smoke test. |
| integration-tests/smoke/cron_test.go | Removed legacy Cron smoke tests. |
| integration-tests/experiments/gas_test.go | Removed experimental gas test. |
| integration-tests/docker/README.md | Updated coverage-report path example to placeholders. |
| integration-tests/README.md | Marked integration-tests as deprecated in favor of devenv. |
| devenv/tests/ocr2/soak_test.go | Simplified context usage to t.Context(). |
| devenv/tests/ocr2/smoke_test.go | Simplified context usage to t.Context(). |
| devenv/tests/ocr2/chaos_test.go | Simplified context usage to t.Context(). |
| devenv/tests/logpoller/test_helpers.go | Removed unused pipe-reading helper and imports. |
| devenv/tests/features/reorg_finality_violation_test.go | Added devenv feature test for head-tracker finality violation on reorg. |
| devenv/tests/features/jd_test.go | Added devenv feature test for registering multiple job distributors. |
| devenv/tests/README.md | Added devenv tests quickstart documentation. |
| devenv/products/ocr2/configuration.go | Enabled MultiFeedsManagers in generated node config for OCR2 devenv product. |
| devenv/go.mod | Bumped CTF framework; added indirect deps for new gql client usage. |
| devenv/go.sum | Updated sums for new/bumped dependencies. |
| devenv/env-cl-rebuild.toml | Increased nodeset size and added per-node build specs for rebuild flow. |
| devenv/cmd/cl/completion.go | Updated cl up completion entry for rebuild flow (OCR2 + rebuild). |
| devenv/README.md | Updated “custom CL image” instructions to include OCR2 product config. |
| .github/workflows/devenv-cl-features.yml | Added scheduled workflow to run the new devenv feature tests. |
| .github/workflows/client-compatibility-tests.yml | Removed legacy smoke suites from compatibility matrix generation; updated example run target. |
| .github/tracing/tempo.yaml | Removed old tracing local stack config. |
| .github/tracing/replay.sh | Removed old trace replay helper. |
| .github/tracing/otel-collector-dev.yaml | Removed old OTEL collector dev config. |
| .github/tracing/otel-collector-ci.yaml | Removed old OTEL collector CI config. |
| .github/tracing/local-smoke-docker-compose.yaml | Removed old tracing docker-compose stack. |
| .github/tracing/grafana-datasources.yaml | Removed old Grafana datasource config for tracing stack. |
| .github/tracing/README.md | Updated tracing docs to use devenv-based flow. |
| .github/e2e-tests.yml | Removed large legacy OCR* runner matrix entries; added pointers to devenv workflows. |
erikburt
previously approved these changes
Mar 9, 2026
Tofel
previously approved these changes
Mar 9, 2026
jmank88
previously approved these changes
Mar 9, 2026
|
jmank88
approved these changes
Mar 9, 2026
Tofel
approved these changes
Mar 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request makes significant updates to the end-to-end (E2E) test runner matrix and the local distributed tracing setup. The main changes include moving all OCR-related smoke, soak, and chaos tests to dedicated workflow files, removing or replacing several legacy and less-used test configurations, and cleaning up the local tracing infrastructure by removing obsolete files and updating documentation for the new development environment workflow.
E2E Test Matrix Updates:
OCR Test Migration:
.github/e2e-tests.ymland are now managed in separate workflow files (devenv-ocr2-soak.ymlanddevenv-ocr2-chaos.yml). This centralizes and simplifies the maintenance of OCR-related tests. [1] [2]Other Test Matrix Cleanups:
runlog_test.go,cron_test.go,flux_test.go, andjob_distributor_test.go, streamlining the test matrix. [1] [2]upgrade_version_test.goto ensure upgrade paths are explicitly tested. [1] [2]Distributed Tracing Infrastructure:
local-smoke-docker-compose.yaml, old Grafana/Tempo/OTEL configs, and the replay script, reflecting a move to the new development environment-based workflow. [1] [2] [3] [4] [5] [6]devenvandjustcommands for running and testing distributed tracing locally, aligning documentation with current best practices.These changes improve test management, reduce maintenance overhead, and modernize the developer experience for distributed tracing.
References:
[1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13]