Support running static tests on gh-hosted runners #1668

isegall-da · 2025-07-23T18:14:01Z

On main, uploads to a GCS bucket (publicly readable) the latest nix cache
On PRs from fork, runs static checks on GH-hosted runners without approval, using the nix cache from GCS
Other caches don't work ATM, so static tests take about 20 minutes, but I decided to merge as-is for now, and improve later.

See a test execution here, which is the closest I could get before merging this (the merged job runs on: pull_request_target, so will be available only after merging)

Part of #1662

Pull Request Checklist

Cluster Testing

If a cluster test is required, comment /cluster_test on this PR to request it, and ping someone with access to the DA-internal system to approve it.
If a hard-migration test is required (from the latest release), comment /hdm_test on this PR to request it, and ping someone with access to the DA-internal system to approve it.

PR Guidelines

Include any change that might be observable by our partners or affect their deployment in the release notes.
Specify fixed issues with Fixes #n, and mention issues worked on using #n
Include a screenshot for frontend-related PRs - see README or use your favorite screenshot tool

Merge Guidelines

Make the git commit message look sensible when squash-merging on GitHub (most likely: just copy your PR description).

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

isegall-da · 2025-08-19T20:03:51Z

.github/actions/nix/setup_nix/action.yml

Argh, GH is not smart enough to catch the file rename + changes. The new parts are mostly in lines 195 and on.

Ran locally with -M40; matched the rename successfully.

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

.github/actions/nix/setup_nix/action.yml

Co-authored-by: Stephen Compall <stephen.compall@digitalasset.com> Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

…1668)" This reverts commit 13bcefe. Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

…1966) This reverts commit 13bcefe. Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

…1966) This reverts commit 5815742. Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

@martinflorian-da

* Tweak fluent bit configuration (#1940) [static] Based on CILR experience - fix severity parsing - truncate long log messages because otherise stack driver gets angry - make time parsing more lenient - make fluent bit parse its own logs better Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org> Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org> * Reduce multi-validator deployment parallelism to 2 (#1938) Signed-off-by: Julien Tinguely <julien.tinguely@digitalasset.com> * Bump Canton for KMS resilience fix (#1941) Fixes DACH-NY/canton-network-internal#1337 [ci] Signed-off-by: Martin Florian <martin.florian@digitalasset.com> * Refactor some form components in sv ui (#1936) - Make form errors a re-usable form component - Make EffectiveField a re-usable field component - Upgrade tanstack-form Signed-off-by: fayi-da <fayimora.femibalogun@digitalasset.com> * Docs: Clarifications around validator DR (#1937) Inspired by questions on Slack: https://daholdings.slack.com/archives/C08AP9QR7K4/p1755245551957219?thread_ts=1753278207.186399&cid=C08AP9QR7K4 [static] Signed-off-by: Martin Florian <martin.florian@digitalasset.com> * Fix tag prefix in stackdriver export (#1944) [static] Don't ask me why fluentbit has mutually incompatible defaults between different filters and outputs … Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org> Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org> * query to aggregate traffic purchases over a time period (#1926) * fork part of total supply query * aggregate .amuletPaid to a separate value * use a bracketed start time as well --------- Signed-off-by: Stephen Compall <stephen.compall@digitalasset.com> * Implement DeleteCorruptAcsSnapshotTrigger (#1096) * Implement DeleteCorruptAcsSnapshotTrigger Signed-off-by: Robert Autenrieth <robert.autenrieth@digitalasset.com> * [static] increase multi validators parallelism to 5 (#1949) Signed-off-by: Julien Tinguely <julien.tinguely@digitalasset.com> * Write how-to docs for token standard usage (#1872) --------- Signed-off-by: Oriol Muñoz <oriol.munoz@digitalasset.com> * Reduce gcp logging components (#1951) I think I accidentally turned on too much when I tried to disable workloads in favor of our own fluentbit. [static] Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org> Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org> * WalletSurviveCantonRestartIntegrationTest: bump wait on participant init (#1952) Fixes DACH-NY/cn-test-failures#5417 The participant did come up eventually and AFAICT the validator app would have continued init if we hadn't stopped that. [static] Signed-off-by: Martin Florian <martin.florian@digitalasset.com> * Bump cometbft mempool and cache size (#1953) fixes #1934 [ci] I honestly don't have a great reason for choosing these specific values. Doubling seems as good as anything else 🤷 See https://github.com/DACH-NY/canton-network-node/pull/17821/files for an earlier change we made in the same direction. Note that I didn't bump the TTL because I don't see a compelling reason why that helps with anything. Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org> Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org> * [static] Add istio rate limits to pulumi (#1798) Signed-off-by: Nicu Reut <nicu.reut@digitalasset.com> * Implement Amulet Rules Proposal Form in new SV UI (#1945) --------- Signed-off-by: fayi-da <fayimora.femibalogun@digitalasset.com> * Fix fluentbit log truncation (#1959) [static] I should not be allowed to write lua Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org> Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org> * [static] include rate of sequencer events processed in the participant dashboard (#1960) Signed-off-by: Nicu Reut <nicu.reut@digitalasset.com> * move pulumi npm packages into lfdt namespace (#1848) * don't alert a Slack channel unless explicitly set in .envrc.vars (#1913) * don't alert a Slack channel unless explicitly set in .envrc.vars The default for alerting was #team-canton-network-internal-alerts. Now that default is removed; only long-running, production and near-production clusters like dev/test/main should now alert. * also don't default SLACK_ALERT_NOTIFICATION_CHANNEL - suggested by @martinflorian-da; thanks * fail if SLACK_ALERT_NOTIFICATION_CHANNEL defined but not FULL_NAME --------- Signed-off-by: Stephen Compall <stephen.compall@digitalasset.com> * Support running static tests on gh-hosted runners (#1668) Signed-off-by: Itai Segall <itai.segall@digitalasset.com> Co-authored-by: Stephen Compall <stephen.compall@digitalasset.com> * Revert "Support running static tests on gh-hosted runners (#1668)" (#1966) This reverts commit 13bcefe. Signed-off-by: Itai Segall <itai.segall@digitalasset.com> * Make pulumi stack parallelism configurable (#1967) * Make pulumi stack parallelism configurable [static] Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org> * fmt [static] Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org> --------- Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org> Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org> * [static] Make the cluster node pools sizes configurable (#1957) Signed-off-by: Nicu Reut <nicu.reut@digitalasset.com> * Try to fix grafana alert expansion (#1970) [static] We still get tons of spam from logger=ngalert.state.manager rule_uid=ady2ks9ehbw1sb org_id=1 t=2025-08-20T07:37:44.687289759Z level=error msg="Error in expanding template" error="failed to expand template '{{- $labels := .Labels -}}{{- $values := .Values -}}{{- $value := .Value -}}{{- if (gt $values.runs.Value 2) -}}\ncritical\n{{- else -}}\nwarning\n{{- end -}}': error executing template __alert_Busy task-based automation: template: __alert_Busy task-based automation:1:84: executing \"__alert_Busy task-based automation\" at <gt $values.runs.Value 2>: error calling gt: incompatible types for comparison" and for the other one. My current theory is: go templates seem to distinguish integers and floats. And we have one missing null check. Would be too easy if it actually told you the mismatching types … Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org> Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org> * [ci] More lenient scan rate limit test (#1971) Signed-off-by: Nicu Reut <nicu.reut@digitalasset.com> * Match package name on template filter (#1955) --------- Signed-off-by: Oriol Muñoz <oriol.munoz@digitalasset.com> * Document routing of the JSON API (#1973) [static] Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org> Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org> * Synchronize on scan processing lock archival (#1969) [ci] fixes DACH-NY/cn-test-failures#5415 Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org> Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org> * Add config rendering helper function and enhance splice-participant helm template (#1939) Fixes #1915 Signed-off-by: timpel-fcs <tim.pelzer@finoa.io> * Remove migrate-istio (#1977) Deletes code, must be good. More seriously this was added 7 months ago so we can pretty confidently assume everything is migrated by now. [static] Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org> Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org> * mention BFT success requirement in validator onboarding doc (#1979) We explain the tradeoffs already where we document how to do it, so not going to repeat all that, just linking to it. Onboarding real production nodes shouldn't do this anyway. Reifies this comment <global-synchronizer-foundation/docs#8 (reply in thread)> from @martinflorian-da. Signed-off-by: Stephen Compall <stephen.compall@digitalasset.com> * shorter output/timeout/portability in validator onboarding test scriptlets (#1982) - overall max-time for curl calls - don't try to jq 4xx responses, just fail - jq portability Adapted from this comment <global-synchronizer-foundation/docs#8 (comment)> from @stas-sbi. * grpcurl output has quotes --------- Signed-off-by: Stephen Compall <stephen.compall@digitalasset.com> * Support running static tests on gh-hosted runners (#1978) Signed-off-by: Itai Segall <itai.segall@digitalasset.com> * Make workflow ids of import updates consistent (#1981) Signed-off-by: Robert Autenrieth <robert.autenrieth@digitalasset.com> * Further clarify safe ways of bypassing the party limit (#1984) [static] Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org> Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org> * Remove todo artifacts (#1986) With the new static test job, CI on forks now fails as it conflicts between that job and the main job. Rather than trying to make it conditional or rename it to avoid the conflict, this just removes the step. Noone has used this for years afaik. [static] Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org> Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org> * Mention existing transfer preapproval proposal (#1987) [static] Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org> Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org> * vagrant: Restart nix-daemon after mounting cache (#1985) - Makes the initial boot more predictable. - Allows recovering after deleting the cache file without re-creating the VM. To recover run `vagrant up --provision`. Signed-off-by: Stanislav German-Evtushenko <ginermail@gmail.com> * Filter pr_cluster_test for pull requests (#1988) [static] Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org> Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org> * [static] Update release notes for 0.4.12 (#1989) Signed-off-by: Nicu Reut <nicu.reut@digitalasset.com> * stop triggering ciupgrade tests (#1983) Signed-off-by: Itai Segall <itai.segall@digitalasset.com> * Upgrade Canton to 3.3.0-snapshot.20250821.16057.0.v3719b9e9 (#1994) [ci] Includes the fix for the initial topology validator that is blocking sv runbook reonboarding on cilr atm. Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org> Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org> * [ci] Update VERSION to 0.4.13 (#1995) Signed-off-by: Nicu Reut <nicu.reut@digitalasset.com> * run BigQuery integration test daily (#1873) * add run scheduled for 2:17am CET, allow manual run * flexible version selection * log service account email when setting up BQ test --------- Signed-off-by: Stephen Compall <stephen.compall@digitalasset.com> * Add missing CO_TransferPreapprovalSend case in UserWalletTxLogParser (#2006) Signed-off-by: Oriol Muñoz <oriol.munoz@digitalasset.com> --------- Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org> Signed-off-by: Julien Tinguely <julien.tinguely@digitalasset.com> Signed-off-by: Martin Florian <martin.florian@digitalasset.com> Signed-off-by: fayi-da <fayimora.femibalogun@digitalasset.com> Signed-off-by: Stephen Compall <stephen.compall@digitalasset.com> Signed-off-by: Robert Autenrieth <robert.autenrieth@digitalasset.com> Signed-off-by: Oriol Muñoz <oriol.munoz@digitalasset.com> Signed-off-by: Nicu Reut <nicu.reut@digitalasset.com> Signed-off-by: Itai Segall <itai.segall@digitalasset.com> Signed-off-by: timpel-fcs <tim.pelzer@finoa.io> Signed-off-by: Stanislav German-Evtushenko <ginermail@gmail.com> Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org> Co-authored-by: Julien Tinguely <julien.tinguely@digitalasset.com> Co-authored-by: Martin Florian <martin.florian@digitalasset.com> Co-authored-by: fayi-da <112705750+fayi-da@users.noreply.github.com> Co-authored-by: Stephen Compall <stephen.compall@digitalasset.com> Co-authored-by: Robert Autenrieth <31539813+rautenrieth-da@users.noreply.github.com> Co-authored-by: Oriol Muñoz <oriol.munoz@digitalasset.com> Co-authored-by: Nicu Reut <nicu.reut@digitalasset.com> Co-authored-by: Itai Segall <itai.segall@digitalasset.com> Co-authored-by: Tim <tim.pelzer@finoa.io> Co-authored-by: Stanislav German-Evtushenko <ginermail@gmail.com>

) Signed-off-by: Itai Segall <itai.segall@digitalasset.com> Co-authored-by: Stephen Compall <stephen.compall@digitalasset.com> Signed-off-by: hrischuk-da <curtis.hrischuk@digitalasset.com>

…r-labs#1668)" (hyperledger-labs#1966) This reverts commit 13bcefe. Signed-off-by: Itai Segall <itai.segall@digitalasset.com> Signed-off-by: hrischuk-da <curtis.hrischuk@digitalasset.com>

isegall-da added 30 commits July 23, 2025 18:12

[static] prelim test

64e35f7

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

[static] prelim test

7eac6fa

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

[static] default container

8699c2a

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

[static] .

46eacf9

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

[static] .

c9dcba2

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

[static] .

17377b2

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

[static] .

7371d9a

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

[static] .

4d51d1f

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

[static] .

074f422

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

[static] .

8c04471

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

[static] .

22ab153

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

[static] .

5a03401

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

[static] .

66ba974

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

[static] pull cache from gcp

270f716

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

[static] does an empty container work?

b93b9ef

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

[static] wget

ff4b48b

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

[static] .

e99d55b

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

[static] .

94bd4a4

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

[static] argh!

ccb82e7

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

[static] double argh!

7b4636f

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

[static] oss_only

b6066b0

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

[static] compute cache key

930c27d

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

[static] fix cache key

894f427

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

[static] .

652a3c7

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

[static] refactor

f126552

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

[static] whoops

b6594c3

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

[static] .

6639bf4

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

[static] .

8044cad

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

[static] cleanup

a6fa6b9

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

[static] .

38980e5

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

isegall-da added 7 commits August 5, 2025 23:22

[static] test finding latest

ad131b6

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

[static] test not finding latest by name

7e55f69

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

[static] enable cross-os caching

b86a506

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

[static] bump cache version, to force save

d438138

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

Merge remote-tracking branch 'origin/main' into isegall/fork-runners

5c535a4

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

[static] revert enableCrossOsArchive

e626cbb

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

[ci] cleanup

368ec79

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

isegall-da force-pushed the isegall/fork-runners branch from db2d70f to 368ec79 Compare August 19, 2025 16:59

isegall-da added 2 commits August 19, 2025 18:37

[static] also on fork base

17d5b9b

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

[ci] docs

33cc2af

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

isegall-da changed the title ~~WIP: static tests on gh-hosted runners~~ Support running static tests on gh-hosted runners Aug 19, 2025

isegall-da added 4 commits August 19, 2025 19:07

[static] fix damlDarLock checks

9cba845

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

[ci] better

eb94c99

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

[ci] cleanup

7bfe458

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

[ci] more cleanup

cc37fc3

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

isegall-da requested review from nicu-da and stephencompall-DA August 19, 2025 20:00

isegall-da marked this pull request as ready for review August 19, 2025 20:02

isegall-da commented Aug 19, 2025

View reviewed changes

[ci] missed one

15fef7b

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

stephencompall-DA approved these changes Aug 19, 2025

View reviewed changes

.github/actions/nix/setup_nix/action.yml Outdated Show resolved Hide resolved

[static] Update .github/actions/nix/setup_nix/action.yml

926330c

Co-authored-by: Stephen Compall <stephen.compall@digitalasset.com> Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

isegall-da merged commit 13bcefe into main Aug 19, 2025
39 checks passed

isegall-da deleted the isegall/fork-runners branch August 19, 2025 22:40

isegall-da added a commit that referenced this pull request Aug 19, 2025

[static] Revert "Support running static tests on gh-hosted runners (#…

4dca48b

…1668)" This reverts commit 13bcefe. Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

isegall-da added a commit that referenced this pull request Aug 19, 2025

Revert "Support running static tests on gh-hosted runners (#1668)" (#…

5815742

…1966) This reverts commit 13bcefe. Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

isegall-da added a commit that referenced this pull request Aug 20, 2025

Reapply "Support running static tests on gh-hosted runners (#1668)" (#…

184bd97

…1966) This reverts commit 5815742. Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Support running static tests on gh-hosted runners #1668

Support running static tests on gh-hosted runners #1668

isegall-da commented Jul 23, 2025 •

edited

Loading

Uh oh!

isegall-da Aug 19, 2025

Uh oh!

stephencompall-DA Aug 19, 2025

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Support running static tests on gh-hosted runners #1668

Support running static tests on gh-hosted runners #1668

Conversation

isegall-da commented Jul 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Checklist

Cluster Testing

PR Guidelines

Merge Guidelines

Uh oh!

isegall-da Aug 19, 2025

Choose a reason for hiding this comment

Uh oh!

stephencompall-DA Aug 19, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

isegall-da commented Jul 23, 2025 •

edited

Loading