Phase 28: harden hosted-GCE deploy (SDK dep fix + infra IAM/monitoring/budgets)#98
Merged
Conversation
s3fs and gcsfs each hard-pin fsspec==<their-own-release> on differing cadences, so the open '>=2024.0' floor made pip explode the s3fs x gcsfs x fsspec exact-pin combinatorics and fail with ResolutionImpossible -- the [satellite] extra (and every deploy image that installs it) never built. Pin the triple exactly to the latest co-released, py3.12-installable version (2026.6.0).
Fold in fixes the first live full apply surfaced: - deletion_protection=false on the 6 Cloud Run service/job resources so TF can replace a tainted/failed initial revision. - runtime_agent_iam.tf: grant the 3 Cloud Run service agents AR reader (cross-project image pull) + the weather-backfill runtime SA batch.agentReporter + logging.logWriter + AR reader (Batch VM needs these; the first pilot failed 'no VM has agent reporting' without them). - monitoring.tf: r2_partition_age -> DISTRIBUTION + bucket_options; data_freshness aligner -> ALIGN_PERCENTILE_99; per-project email channels so satellite/ingest alerts use same-project channels. - budgets.tf: email-only until the Cloud Billing budget system SA is Console-provisioned (Pub/Sub budget-notify + publisher IAM deferred). - enable_public_invoker gate (default false): allUsers is rejected by the org Domain-Restricted-Sharing policy until an org-admin adds a project-scoped allowAll exception on mr-serving. - .gcloudignore for the Cloud Build image-seed upload.
|
Parity ticket gate: PASSED See |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hardens the Phase 28 hosted-GCE deploy after executing the first live full apply of the 90-resource platform layer (all 4 Cloud Run services/jobs live, 1-station backfill pilot SUCCEEDED). Folds in every fix the live apply surfaced so a clean
tofu applyfrom scratch reproduces a working, green platform.SDK dependency fix (blocks CI image builds)
packages/weather[satellite]extra pinneds3fs/gcsfsopen-ended, but each hard-pinsfsspec==<own-release>on differing cadences → pipResolutionImpossible; the images had never built. Pinnedfsspec==s3fs==gcsfs==2026.6.0(latest co-released, py3.12-installable triple).Infra hardening (
infra/)deletion_protection=falseon the 6 Cloud Run service/job resources — lets TF replace a tainted/failed initial revision.runtime_agent_iam.tf(new) — grants the 3 Cloud Run service agentsartifactregistry.reader(cross-project image pull) and theweather-backfillruntime SAbatch.agentReporter+logging.logWriter+ AR reader (the first pilot failed "no VM has agent reporting" without these). Folds the manual grants into Terraform.monitoring.tf—r2_partition_age→DISTRIBUTION/DELTA+bucket_options(value_extractorrequires a distribution);data_freshnessaligner →ALIGN_PERCENTILE_99; per-project email channels so the satellite/ingest alert policies reference same-project channels.budgets.tf— email-only tripwires until the Cloud Billing budget system SA is Console-provisioned (Pub/Sub budget-notify + its publisher IAM deferred, documented inline).enable_public_invokergate (default false) —allUsersinvoker is rejected by the org Domain-Restricted-Sharing policy; flip true after an org-admin adds a project-scopedallowAllexception onmr-serving..gcloudignore— trims the Cloud Build image-seed source upload.Verified
tofu apply→Apply complete! 15 added, 2 changed, 0 destroyed, 0 errors (state 115 → 130).Operator follow-ups (not code)
iam.allowedPolicyMemberDomainsallowAll exception onmr-serving, then setenable_public_invoker=true+ re-apply.billing-budget-alerts@system, then restore the Pub/Sub budget-notify +billing_budget_publisher.