Skip to content

Reduce gcp logging components #1951

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Aug 19, 2025
Merged

Reduce gcp logging components #1951

merged 1 commit into from
Aug 19, 2025

Conversation

@moritzkiefer-da
Copy link
Contributor

I think I accidentally turned on too much when I tried to disable workloads in favor of our own fluentbit.

Compare here to devnet

image

[static]

Pull Request Checklist

Cluster Testing

  • If a cluster test is required, comment /cluster_test on this PR to request it, and ping someone with access to the DA-internal system to approve it.
  • If a hard-migration test is required (from the latest release), comment /hdm_test on this PR to request it, and ping someone with access to the DA-internal system to approve it.

PR Guidelines

  • Include any change that might be observable by our partners or affect their deployment in the release notes.
  • Specify fixed issues with Fixes #n, and mention issues worked on using #n
  • Include a screenshot for frontend-related PRs - see README or use your favorite screenshot tool

Merge Guidelines

  • Make the git commit message look sensible when squash-merging on GitHub (most likely: just copy your PR description).

@cocreature
Reduce gcp logging components
c23975b 
I think I accidentally turned on too much when I tried to disable
workloads in favor of our own fluentbit.

[static]

Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
@moritzkiefer-da moritzkiefer-da enabled auto-merge (squash) August 19, 2025 09:13
@moritzkiefer-da moritzkiefer-da merged commit 77b6708 into main Aug 19, 2025
40 checks passed
@moritzkiefer-da moritzkiefer-da deleted the cocreature/google-logs branch August 19, 2025 09:20
moritzkiefer-da added a commit that referenced this pull request Aug 25, 2025
@moritzkiefer-da @cocreature
@julientinguely-da @martinflorian-da @fayi-da @stephencompall-DA @rautenrieth-da @OriolMunoz-da @nicu-da @isegall-da @timpel-fcs @giner
Merge remote-tracking branch 'origin/main' into canton-3.4 (#2008)
49e9fe0 
* Tweak fluent bit configuration (#1940)

[static]

Based on CILR experience

- fix severity parsing
- truncate long log messages because otherise stack driver gets angry
- make time parsing more lenient
- make fluent bit parse its own logs better

Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* Reduce multi-validator deployment parallelism to 2 (#1938)



Signed-off-by: Julien Tinguely <julien.tinguely@digitalasset.com>

* Bump Canton for KMS resilience fix (#1941)

Fixes DACH-NY/canton-network-internal#1337

[ci]

Signed-off-by: Martin Florian <martin.florian@digitalasset.com>

* Refactor some form components in sv ui (#1936)

- Make form errors a re-usable form component
- Make EffectiveField a re-usable field component
- Upgrade tanstack-form

Signed-off-by: fayi-da <fayimora.femibalogun@digitalasset.com>

* Docs: Clarifications around validator DR (#1937)

Inspired by questions on Slack: https://daholdings.slack.com/archives/C08AP9QR7K4/p1755245551957219?thread_ts=1753278207.186399&cid=C08AP9QR7K4

[static]

Signed-off-by: Martin Florian <martin.florian@digitalasset.com>

* Fix tag prefix in stackdriver export (#1944)

[static]

Don't ask me why fluentbit has mutually incompatible defaults between
different filters and outputs …

Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* query to aggregate traffic purchases over a time period (#1926)

* fork part of total supply query

* aggregate .amuletPaid to a separate value

* use a bracketed start time as well

---------

Signed-off-by: Stephen Compall <stephen.compall@digitalasset.com>

* Implement DeleteCorruptAcsSnapshotTrigger (#1096)

* Implement DeleteCorruptAcsSnapshotTrigger

Signed-off-by: Robert Autenrieth <robert.autenrieth@digitalasset.com>

* [static] increase multi validators parallelism to 5 (#1949)

Signed-off-by: Julien Tinguely <julien.tinguely@digitalasset.com>

* Write how-to docs for token standard usage (#1872)


---------

Signed-off-by: Oriol Muñoz <oriol.munoz@digitalasset.com>

* Reduce gcp logging components (#1951)

I think I accidentally turned on too much when I tried to disable
workloads in favor of our own fluentbit.

[static]

Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* WalletSurviveCantonRestartIntegrationTest: bump wait on participant init (#1952)

Fixes DACH-NY/cn-test-failures#5417

The participant did come up eventually and AFAICT the validator app would have continued init if we hadn't stopped that.

[static]

Signed-off-by: Martin Florian <martin.florian@digitalasset.com>

* Bump cometbft mempool and cache size (#1953)

fixes #1934

[ci]

I honestly don't have a great reason for choosing these specific
values. Doubling seems as good as anything else 🤷

See https://github.com/DACH-NY/canton-network-node/pull/17821/files
for an earlier change we made in the same direction.

Note that I didn't bump the TTL because I don't see a compelling
reason why that helps with anything.

Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* [static] Add istio rate limits to pulumi (#1798)

Signed-off-by: Nicu Reut <nicu.reut@digitalasset.com>

* Implement Amulet Rules Proposal Form in new SV UI (#1945)


---------

Signed-off-by: fayi-da <fayimora.femibalogun@digitalasset.com>

* Fix fluentbit log truncation (#1959)

[static]

I should not be allowed to write lua

Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* [static] include rate of sequencer events processed in the participant dashboard (#1960)

Signed-off-by: Nicu Reut <nicu.reut@digitalasset.com>

* move pulumi npm packages into lfdt namespace  (#1848)

* don't alert a Slack channel unless explicitly set in .envrc.vars (#1913)

* don't alert a Slack channel unless explicitly set in .envrc.vars

The default for alerting was #team-canton-network-internal-alerts. Now
that default is removed; only long-running, production and
near-production clusters like dev/test/main should now alert.

* also don't default SLACK_ALERT_NOTIFICATION_CHANNEL

- suggested by @martinflorian-da; thanks

* fail if SLACK_ALERT_NOTIFICATION_CHANNEL defined but not FULL_NAME

---------

Signed-off-by: Stephen Compall <stephen.compall@digitalasset.com>

* Support running static tests on gh-hosted runners (#1668)


Signed-off-by: Itai Segall <itai.segall@digitalasset.com>
Co-authored-by: Stephen Compall <stephen.compall@digitalasset.com>

* Revert "Support running static tests on gh-hosted runners (#1668)" (#1966)

This reverts commit 13bcefe.

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

* Make pulumi stack parallelism configurable (#1967)

* Make pulumi stack parallelism configurable

[static]

Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* fmt

[static]

Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

---------

Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* [static] Make the cluster node pools sizes configurable (#1957)


Signed-off-by: Nicu Reut <nicu.reut@digitalasset.com>

* Try to fix grafana alert expansion (#1970)

[static]

We still get tons of spam from

logger=ngalert.state.manager rule_uid=ady2ks9ehbw1sb org_id=1
t=2025-08-20T07:37:44.687289759Z level=error msg="Error in expanding
template" error="failed to expand template '{{- $labels := .Labels
-}}{{- $values := .Values -}}{{- $value := .Value -}}{{- if (gt
$values.runs.Value 2) -}}\ncritical\n{{- else -}}\nwarning\n{{- end
-}}': error executing template __alert_Busy task-based automation:
template: __alert_Busy task-based automation:1:84: executing
\"__alert_Busy task-based automation\" at <gt $values.runs.Value 2>:
error calling gt: incompatible types for comparison"

and for the other one.

My current theory is: go templates seem to distinguish integers and
floats. And we have one missing null check. Would be too easy if it
actually told you the mismatching types …

Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* [ci] More lenient scan rate limit test (#1971)

Signed-off-by: Nicu Reut <nicu.reut@digitalasset.com>

* Match package name on template filter (#1955)


---------

Signed-off-by: Oriol Muñoz <oriol.munoz@digitalasset.com>

* Document routing of the JSON API (#1973)

[static]

Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* Synchronize on scan processing lock archival (#1969)

[ci]

fixes DACH-NY/cn-test-failures#5415

Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* Add config rendering helper function and enhance splice-participant helm template (#1939)

Fixes #1915

Signed-off-by: timpel-fcs <tim.pelzer@finoa.io>

* Remove migrate-istio (#1977)

Deletes code, must be good.

More seriously this was added 7 months ago so we can pretty
confidently assume everything is migrated by now.

[static]

Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* mention BFT success requirement in validator onboarding doc (#1979)

We explain the tradeoffs already where we document how to do it, so not
going to repeat all that, just linking to it. Onboarding real production
nodes shouldn't do this anyway.

Reifies this comment
<global-synchronizer-foundation/docs#8 (reply in thread)>
from @martinflorian-da.

Signed-off-by: Stephen Compall <stephen.compall@digitalasset.com>

* shorter output/timeout/portability in validator onboarding test scriptlets (#1982)

- overall max-time for curl calls
- don't try to jq 4xx responses, just fail
- jq portability

Adapted from this comment
<global-synchronizer-foundation/docs#8 (comment)>
from @stas-sbi.

* grpcurl output has quotes

---------

Signed-off-by: Stephen Compall <stephen.compall@digitalasset.com>

* Support running static tests on gh-hosted runners (#1978)


Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

* Make workflow ids of import updates consistent (#1981)


Signed-off-by: Robert Autenrieth <robert.autenrieth@digitalasset.com>

* Further clarify safe ways of bypassing the party limit (#1984)

[static]

Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* Remove todo artifacts (#1986)

With the new static test job, CI on forks now fails as it conflicts
between that job and the main job.

Rather than trying to make it conditional or rename it to avoid the conflict, this just
removes the step. Noone has used this for years afaik.

[static]

Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* Mention existing transfer preapproval proposal (#1987)

[static]

Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* vagrant: Restart nix-daemon after mounting cache (#1985)

- Makes the initial boot more predictable.
- Allows recovering after deleting the cache file without re-creating
  the VM. To recover run `vagrant up --provision`.

Signed-off-by: Stanislav German-Evtushenko <ginermail@gmail.com>

* Filter pr_cluster_test for pull requests (#1988)

[static]

Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* [static] Update release notes for 0.4.12 (#1989)

Signed-off-by: Nicu Reut <nicu.reut@digitalasset.com>

* stop triggering ciupgrade tests (#1983)

Signed-off-by: Itai Segall <itai.segall@digitalasset.com>

* Upgrade Canton to 3.3.0-snapshot.20250821.16057.0.v3719b9e9 (#1994)

[ci]

Includes the fix for the initial topology validator that is blocking
sv runbook reonboarding on cilr atm.

Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* [ci] Update VERSION to 0.4.13 (#1995)

Signed-off-by: Nicu Reut <nicu.reut@digitalasset.com>

* run BigQuery integration test daily (#1873)

* add run scheduled for 2:17am CET, allow manual run

* flexible version selection

* log service account email when setting up BQ test

---------

Signed-off-by: Stephen Compall <stephen.compall@digitalasset.com>

* Add missing CO_TransferPreapprovalSend case in UserWalletTxLogParser (#2006)



Signed-off-by: Oriol Muñoz <oriol.munoz@digitalasset.com>

---------

Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Signed-off-by: Julien Tinguely <julien.tinguely@digitalasset.com>
Signed-off-by: Martin Florian <martin.florian@digitalasset.com>
Signed-off-by: fayi-da <fayimora.femibalogun@digitalasset.com>
Signed-off-by: Stephen Compall <stephen.compall@digitalasset.com>
Signed-off-by: Robert Autenrieth <robert.autenrieth@digitalasset.com>
Signed-off-by: Oriol Muñoz <oriol.munoz@digitalasset.com>
Signed-off-by: Nicu Reut <nicu.reut@digitalasset.com>
Signed-off-by: Itai Segall <itai.segall@digitalasset.com>
Signed-off-by: timpel-fcs <tim.pelzer@finoa.io>
Signed-off-by: Stanislav German-Evtushenko <ginermail@gmail.com>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Julien Tinguely <julien.tinguely@digitalasset.com>
Co-authored-by: Martin Florian <martin.florian@digitalasset.com>
Co-authored-by: fayi-da <112705750+fayi-da@users.noreply.github.com>
Co-authored-by: Stephen Compall <stephen.compall@digitalasset.com>
Co-authored-by: Robert Autenrieth <31539813+rautenrieth-da@users.noreply.github.com>
Co-authored-by: Oriol Muñoz <oriol.munoz@digitalasset.com>
Co-authored-by: Nicu Reut <nicu.reut@digitalasset.com>
Co-authored-by: Itai Segall <itai.segall@digitalasset.com>
Co-authored-by: Tim <tim.pelzer@finoa.io>
Co-authored-by: Stanislav German-Evtushenko <ginermail@gmail.com>
hrischuk-da pushed a commit to hrischuk-da/splice that referenced this pull request Aug 29, 2025
@moritzkiefer-da @cocreature @hrischuk-da
Reduce gcp logging components (hyperledger-labs#1951)
41744f7 
I think I accidentally turned on too much when I tried to disable
workloads in favor of our own fluentbit.

[static]

Signed-off-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Signed-off-by: hrischuk-da <curtis.hrischuk@digitalasset.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@nicu-da nicu-da nicu-da approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@moritzkiefer-da @nicu-da @cocreature