Skip to content

Conversation

@stephencompall-DA
Copy link
Contributor

@stephencompall-DA stephencompall-DA commented Aug 7, 2025

Fixes #1730. Also logs the SA's email in the BQ test for permission debugging.

Successful run.

Needed these grants in IAM on the runner's SA:

- condition:
    description: Limit access to the temporary datasets created by ScanTotalSupplyBigQueryIntegrationTest.
    expression: resource.name.startsWith("projects/da-cn-splice/datasets/scantotalsupply_test_dataset_")
    title: BQ Integration Test datasets only
  members:
  - serviceAccount:splice-kms-integration-test@da-cn-splice.iam.gserviceaccount.com
  role: roles/bigquery.dataEditor
- members:
  - serviceAccount:splice-kms-integration-test@da-cn-splice.iam.gserviceaccount.com
  role: roles/bigquery.jobUser

(Jobs aren't dataset-scoped.)

Pull Request Checklist

Cluster Testing

  • If a cluster test is required, comment /cluster_test on this PR to request it, and ping someone with access to the DA-internal system to approve it.
  • If a hard-migration test is required (from the latest release), comment /hdm_test on this PR to request it, and ping someone with access to the DA-internal system to approve it.

PR Guidelines

  • Include any change that might be observable by our partners or affect their deployment in the release notes.
  • Specify fixed issues with Fixes #n, and mention issues worked on using #n
  • Include a screenshot for frontend-related PRs - see README or use your favorite screenshot tool

Merge Guidelines

  • Make the git commit message look sensible when squash-merging on GitHub (most likely: just copy your PR description).

Signed-off-by: Stephen Compall <stephen.compall@digitalasset.com>
@stephencompall-DA stephencompall-DA self-assigned this Aug 7, 2025
Signed-off-by: Stephen Compall <stephen.compall@digitalasset.com>
@stephencompall-DA stephencompall-DA force-pushed the s11/1730-ci-bq-it branch 2 times, most recently from 0153e65 to 8d52d9a Compare August 7, 2025 21:32
@stephencompall-DA
Copy link
Contributor Author

Considering switching to scratchnet or an SA grant/turning on BQ on splice gcp

[info] ***
[info]   "code": 403,
[info]   "errors": [
[info]     ***
[info]       "domain": "global",
[info]       "message": "Access Denied: Project da-cn-splice: User does not have bigquery.datasets.create permission in project da-cn-splice.",
[info]       "reason": "accessDenied"
[info]     ***
[info]   ],
[info]   "message": "Access Denied: Project da-cn-splice: User does not have bigquery.datasets.create permission in project da-cn-splice.",
[info]   "status": "PERMISSION_DENIED"
[info] ***

@stephencompall-DA stephencompall-DA marked this pull request as ready for review August 20, 2025 19:29
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR sets up daily BigQuery integration testing with service account permission debugging. It creates a scheduled GitHub Actions workflow to run BigQuery tests Monday through Friday and adds logging to help debug service account permissions.

  • Added a new GitHub Actions workflow for daily BigQuery integration tests with manual trigger support
  • Enhanced BigQuery test logging to display the service account email for permission debugging

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
.github/workflows/build.scala_test_bigquery.yml New workflow file for scheduled daily BigQuery testing
ScanTotalSupplyBigQueryIntegrationTest.scala Added service account email logging for debugging permissions

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Signed-off-by: Stephen Compall <stephen.compall@digitalasset.com>
Copy link
Contributor

@isegall-da isegall-da left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice thank you.
Nit: how hard is it to have a dedicated SA instead of reusing the kms one? (I don't remember exactly where that's defined)

@stephencompall-DA
Copy link
Contributor Author

Nit: how hard is it to have a dedicated SA instead of reusing the kms one? (I don't remember exactly where that's defined)

Moved to #1999 tech debt.

@stephencompall-DA stephencompall-DA merged commit 332e06a into main Aug 21, 2025
56 checks passed
@stephencompall-DA stephencompall-DA deleted the s11/1730-ci-bq-it branch August 21, 2025 18:09
moritzkiefer-da added a commit that referenced this pull request Aug 25, 2025
* Tweak fluent bit configuration (#1940)

[static]

Based on CILR experience

- fix severity parsing
- truncate long log messages because otherise stack driver gets angry
- make time parsing more lenient
- make fluent bit parse its own logs better

Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* Reduce multi-validator deployment parallelism to 2 (#1938)



Signed-off-by: Julien Tinguely <julien.tinguely@digitalasset.com>

* Bump Canton for KMS resilience fix (#1941)

Fixes DACH-NY/canton-network-internal#1337

[ci]

Signed-off-by: Martin Florian <martin.florian@digitalasset.com>

* Refactor some form components in sv ui (#1936)

- Make form errors a re-usable form component
- Make EffectiveField a re-usable field component
- Upgrade tanstack-form

Signed-off-by: fayi-da <fayimora.femibalogun@digitalasset.com>

* Docs: Clarifications around validator DR (#1937)

Inspired by questions on Slack: https://daholdings.slack.com/archives/C08AP9QR7K4/p1755245551957219?thread_ts=1753278207.186399&cid=C08AP9QR7K4

[static]

Signed-off-by: Martin Florian <martin.florian@digitalasset.com>

* Fix tag prefix in stackdriver export (#1944)

[static]

Don't ask me why fluentbit has mutually incompatible defaults between
different filters and outputs …

Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* query to aggregate traffic purchases over a time period (#1926)

* fork part of total supply query

* aggregate .amuletPaid to a separate value

* use a bracketed start time as well

---------

Signed-off-by: Stephen Compall <stephen.compall@digitalasset.com>

* Implement DeleteCorruptAcsSnapshotTrigger (#1096)

* Implement DeleteCorruptAcsSnapshotTrigger

Signed-off-by: Robert Autenrieth <robert.autenrieth@digitalasset.com>

* [static] increase multi validators parallelism to 5 (#1949)

Signed-off-by: Julien Tinguely <julien.tinguely@digitalasset.com>

* Write how-to docs for token standard usage (#1872)


---------

Signed-off-by: Oriol Muñoz <oriol.munoz@digitalasset.com>

* Reduce gcp logging components (#1951)

I think I accidentally turned on too much when I tried to disable
workloads in favor of our own fluentbit.

[static]

Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* WalletSurviveCantonRestartIntegrationTest: bump wait on participant init (#1952)

Fixes DACH-NY/cn-test-failures#5417

The participant did come up eventually and AFAICT the validator app would have continued init if we hadn't stopped that.

[static]

Signed-off-by: Martin Florian <martin.florian@digitalasset.com>

* Bump cometbft mempool and cache size (#1953)

fixes #1934

[ci]

I honestly don't have a great reason for choosing these specific
values. Doubling seems as good as anything else 🤷

See https://github.com/DACH-NY/canton-network-node/pull/17821/files
for an earlier change we made in the same direction.

Note that I didn't bump the TTL because I don't see a compelling
reason why that helps with anything.

Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* [static] Add istio rate limits to pulumi (#1798)

Signed-off-by: Nicu Reut <nicu.reut@digitalasset.com>

* Implement Amulet Rules Proposal Form in new SV UI (#1945)


---------

Signed-off-by: fayi-da <fayimora.femibalogun@digitalasset.com>

* Fix fluentbit log truncation (#1959)

[static]

I should not be allowed to write lua

Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* [static] include rate of sequencer events processed in the participant dashboard (#1960)

Signed-off-by: Nicu Reut <nicu.reut@digitalasset.com>

* move pulumi npm packages into lfdt namespace  (#1848)

* don't alert a Slack channel unless explicitly set in .envrc.vars (#1913)

* don't alert a Slack channel unless explicitly set in .envrc.vars

The default for alerting was #team-canton-network-internal-alerts. Now
that default is removed; only long-running, production and
near-production clusters like dev/test/main should now alert.

* also don't default SLACK_ALERT_NOTIFICATION_CHANNEL

- suggested by @martinflorian-da; thanks

* fail if SLACK_ALERT_NOTIFICATION_CHANNEL defined but not FULL_NAME

---------

Signed-off-by: Stephen Compall <stephen.compall@digitalasset.com>

* Support running static tests on gh-hosted runners (#1668)


Signed-off-by: Itai Segall <itai.segall@digitalasset.com>
Co-authored-by: Stephen Compall <stephen.compall@digitalasset.com>

* Revert "Support running static tests on gh-hosted runners (#1668)" (#1966)

This reverts commit 13bcefe.

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

* Make pulumi stack parallelism configurable (#1967)

* Make pulumi stack parallelism configurable

[static]

Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* fmt

[static]

Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

---------

Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* [static] Make the cluster node pools sizes configurable (#1957)


Signed-off-by: Nicu Reut <nicu.reut@digitalasset.com>

* Try to fix grafana alert expansion (#1970)

[static]

We still get tons of spam from

logger=ngalert.state.manager rule_uid=ady2ks9ehbw1sb org_id=1
t=2025-08-20T07:37:44.687289759Z level=error msg="Error in expanding
template" error="failed to expand template '{{- $labels := .Labels
-}}{{- $values := .Values -}}{{- $value := .Value -}}{{- if (gt
$values.runs.Value 2) -}}\ncritical\n{{- else -}}\nwarning\n{{- end
-}}': error executing template __alert_Busy task-based automation:
template: __alert_Busy task-based automation:1:84: executing
\"__alert_Busy task-based automation\" at <gt $values.runs.Value 2>:
error calling gt: incompatible types for comparison"

and for the other one.

My current theory is: go templates seem to distinguish integers and
floats. And we have one missing null check. Would be too easy if it
actually told you the mismatching types …

Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* [ci] More lenient scan rate limit test (#1971)

Signed-off-by: Nicu Reut <nicu.reut@digitalasset.com>

* Match package name on template filter (#1955)


---------

Signed-off-by: Oriol Muñoz <oriol.munoz@digitalasset.com>

* Document routing of the JSON API (#1973)

[static]

Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* Synchronize on scan processing lock archival (#1969)

[ci]

fixes DACH-NY/cn-test-failures#5415

Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* Add config rendering helper function and enhance splice-participant helm template (#1939)

Fixes #1915

Signed-off-by: timpel-fcs <tim.pelzer@finoa.io>

* Remove migrate-istio (#1977)

Deletes code, must be good.

More seriously this was added 7 months ago so we can pretty
confidently assume everything is migrated by now.

[static]

Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* mention BFT success requirement in validator onboarding doc (#1979)

We explain the tradeoffs already where we document how to do it, so not
going to repeat all that, just linking to it. Onboarding real production
nodes shouldn't do this anyway.

Reifies this comment
<global-synchronizer-foundation/docs#8 (reply in thread)>
from @martinflorian-da.

Signed-off-by: Stephen Compall <stephen.compall@digitalasset.com>

* shorter output/timeout/portability in validator onboarding test scriptlets (#1982)

- overall max-time for curl calls
- don't try to jq 4xx responses, just fail
- jq portability

Adapted from this comment
<global-synchronizer-foundation/docs#8 (comment)>
from @stas-sbi.

* grpcurl output has quotes

---------

Signed-off-by: Stephen Compall <stephen.compall@digitalasset.com>

* Support running static tests on gh-hosted runners (#1978)


Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

* Make workflow ids of import updates consistent (#1981)


Signed-off-by: Robert Autenrieth <robert.autenrieth@digitalasset.com>

* Further clarify safe ways of bypassing the party limit (#1984)

[static]

Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* Remove todo artifacts (#1986)

With the new static test job, CI on forks now fails as it conflicts
between that job and the main job.

Rather than trying to make it conditional or rename it to avoid the conflict, this just
removes the step. Noone has used this for years afaik.

[static]

Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* Mention existing transfer preapproval proposal (#1987)

[static]

Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* vagrant: Restart nix-daemon after mounting cache (#1985)

- Makes the initial boot more predictable.
- Allows recovering after deleting the cache file without re-creating
  the VM. To recover run `vagrant up --provision`.

Signed-off-by: Stanislav German-Evtushenko <ginermail@gmail.com>

* Filter pr_cluster_test for pull requests (#1988)

[static]

Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* [static] Update release notes for 0.4.12 (#1989)

Signed-off-by: Nicu Reut <nicu.reut@digitalasset.com>

* stop triggering ciupgrade tests (#1983)

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

* Upgrade Canton to 3.3.0-snapshot.20250821.16057.0.v3719b9e9 (#1994)

[ci]

Includes the fix for the initial topology validator that is blocking
sv runbook reonboarding on cilr atm.

Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* [ci] Update VERSION to 0.4.13 (#1995)

Signed-off-by: Nicu Reut <nicu.reut@digitalasset.com>

* run BigQuery integration test daily (#1873)

* add run scheduled for 2:17am CET, allow manual run

* flexible version selection

* log service account email when setting up BQ test

---------

Signed-off-by: Stephen Compall <stephen.compall@digitalasset.com>

* Add missing CO_TransferPreapprovalSend case in UserWalletTxLogParser (#2006)



Signed-off-by: Oriol Muñoz <oriol.munoz@digitalasset.com>

---------

Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Signed-off-by: Julien Tinguely <julien.tinguely@digitalasset.com>
Signed-off-by: Martin Florian <martin.florian@digitalasset.com>
Signed-off-by: fayi-da <fayimora.femibalogun@digitalasset.com>
Signed-off-by: Stephen Compall <stephen.compall@digitalasset.com>
Signed-off-by: Robert Autenrieth <robert.autenrieth@digitalasset.com>
Signed-off-by: Oriol Muñoz <oriol.munoz@digitalasset.com>
Signed-off-by: Nicu Reut <nicu.reut@digitalasset.com>
Signed-off-by: Itai Segall <itai.segall@digitalasset.com>
Signed-off-by: timpel-fcs <tim.pelzer@finoa.io>
Signed-off-by: Stanislav German-Evtushenko <ginermail@gmail.com>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Julien Tinguely <julien.tinguely@digitalasset.com>
Co-authored-by: Martin Florian <martin.florian@digitalasset.com>
Co-authored-by: fayi-da <112705750+fayi-da@users.noreply.github.com>
Co-authored-by: Stephen Compall <stephen.compall@digitalasset.com>
Co-authored-by: Robert Autenrieth <31539813+rautenrieth-da@users.noreply.github.com>
Co-authored-by: Oriol Muñoz <oriol.munoz@digitalasset.com>
Co-authored-by: Nicu Reut <nicu.reut@digitalasset.com>
Co-authored-by: Itai Segall <itai.segall@digitalasset.com>
Co-authored-by: Tim <tim.pelzer@finoa.io>
Co-authored-by: Stanislav German-Evtushenko <ginermail@gmail.com>
hrischuk-da pushed a commit to hrischuk-da/splice that referenced this pull request Aug 29, 2025
* add run scheduled for 2:17am CET, allow manual run

* flexible version selection

* log service account email when setting up BQ test

---------

Signed-off-by: Stephen Compall <stephen.compall@digitalasset.com>
Signed-off-by: hrischuk-da <curtis.hrischuk@digitalasset.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

run BigQuery integration test daily

3 participants