Skip to content

SRE-739: Fix backend Deploy concurrency β€” dropped builds and out-of-order mutable tags#8801

Merged
TimDiekmann merged 25 commits into
mainfrom
t/sre-739-fix-backend-deploy-concurrency-dropped-builds-and-out-of
Jun 2, 2026
Merged

SRE-739: Fix backend Deploy concurrency β€” dropped builds and out-of-order mutable tags#8801
TimDiekmann merged 25 commits into
mainfrom
t/sre-739-fix-backend-deploy-concurrency-dropped-builds-and-out-of

Conversation

@TimDiekmann
Copy link
Copy Markdown
Member

🌟 What is the purpose of this PR?

The backend Deploy workflow was intermittently cancelling Docker builds on main (e.g. Build kratos (arm64) β†’ "The operation was canceled"). Root cause: the shared …-publish concurrency group dropped pending main-push runs (GitHub keeps only one pending run per group). Combined with the per-commit turbo affected change detection and the promote walk-back, a dropped run for a service-changing commit could let promote ship a stale image to production with no error.

This reworks the workflow's concurrency, manifest assembly, mutable-tag handling, and build cache so that no run is dropped and mutable tags only ever move forward.

πŸ”— Related links

🚫 Blocked by

  • none

πŸ” What does this change?

  • Concurrency: the workflow-level group is now github.run_id (unique per run β†’ never evicted); serialization moved to job-level queue: max per resource. Builds run unserialized (their pushes are immutable), so the slow Rust builds no longer block each other. Only the mutable tags/rollouts serialize β€” :staging in a dedicated stage job, :latest in manifest, ECS in deploy (which runs needs: stage, so the redeploy pulls the freshly-advanced :staging rather than the previous image).
  • by-digest manifests: GHCR images are pushed by digest; the multi-arch manifest is assembled from those exact digests instead of reading back the moving :latest-<arch> tags. Removes the "Frankenstein manifest" race (arm64 from one commit + amd64 from another) and drops the per-arch GHCR tags.
  • Mutable-tag guard (tag-if-newer action): :staging / :latest advance onto a commit only if it is a strict git-descendant of the currently-tagged commit (resolved via the org.opencontainers.image.revision label). Closes the "queue: max FIFO not guaranteed" gap and protects a manual :staging rollback against older/parallel runs.
  • Build cache β†’ GHCR registry: moved off the GHA cache backend (type=gha,mode=max was 504-ing on the large Rust builds) to a dedicated private …/<service>/cache:<arch> package. Written on push/workflow_dispatch (trusted events) only; PR/merge_group read it but never write it.
  • Retry (nick-fields/retry, repo convention): wraps the imagetools create calls (manifest assemble + guard) against transient GHCR 5xx.

Pre-Merge Checklist πŸš€

🚒 Has this modified a publishable library?

This PR:

  • does not modify any publishable blocks or libraries, or modifications do not need publishing

πŸ“œ Does this require a change to the docs?

The changes in this PR:

  • are internal and do not require a docs change

πŸ•ΈοΈ Does this require a change to the Turbo Graph?

The changes in this PR:

  • do not affect the execution graph

⚠️ Known issues

  • The is_main path is not exercised by CI before merge. ECR push (:sha/:run/:staging), the :staging guard, the stage job, and the ECS deploy job all require is_main, so they are skipped on pull_request (PUSH=false) and workflow_dispatch (non-main). They run for the first time on the first main push β€” that run should be watched. The riskiest unverified assumption is imagetools create against ECR for :staging (ECR's image-index media-type quirk); the fallback is aws ecr put-image.
  • The out-of-order guard is logically correct but not empirically tested with two overlapping runs of different commits (the :latest guard only ran single-run).
  • First main run: existing :staging/:latest images have no revision label yet β†’ the guard advances them once unconditionally (by design).

🐾 Next steps

  • Rollback pin: the guard only moves :staging forward, so a manual rollback isn't durable on its own (the next merge overtakes it). Needs a separate pin/freeze mechanism if "freeze staging" is ever required.
  • Move :staging tagging into internal-infra via workflow_call (keep the :latest guard in this OSS repo β€” it can't go closed-source).

πŸ›‘ What tests cover this?

  • No automated tests (this is CI workflow config). Validated by a workflow_dispatch run on this branch: all 12 builds parallel, GHCR by-digest + registry cache (no 504), 5 multi-arch manifests assembled from digests, :latest guard. A separate throwaway probe confirmed job-level queue: max serializes without dropping runs. The is_main path is covered only by the first main run after merge.

❓ How to test this?

  1. gh workflow run deploy.yml --ref <this branch> β†’ exercises the GHCR path (build-parallel, by-digest, cache, manifest, :latest guard). stage/deploy are skipped (non-main).
  2. On merge, watch the first main run for: ECR push, the stage job (:staging guard), and the ECS redeploy β€” these cannot be exercised beforehand.

πŸ“Ή Demo

n/a β€” CI / infra change.

@vercel
Copy link
Copy Markdown

vercel Bot commented Jun 1, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
hash Ready Ready Preview, Comment Jun 1, 2026 9:51pm
hashdotdesign-tokens Ready Ready Preview, Comment Jun 1, 2026 9:51pm
petrinaut Ready Ready Preview Jun 1, 2026 9:51pm

@cursor
Copy link
Copy Markdown

cursor Bot commented Jun 1, 2026

PR Summary

High Risk
Changes production CD (ECR :staging, GHCR :latest, ECS deploy ordering and concurrency); several paths are only fully exercised on first main push after merge.

Overview
Fixes Deploy so main runs are no longer dropped by a shared publish concurrency group (which could skip builds for commits that only that run would publish) and so mutable registry tags cannot move backward when CI finishes out of order.

Concurrency is now one group per run (github.run_id) for push/dispatch; builds run in parallel. Serialization moves to per-service jobs with queue: max: new stage for ECR :staging, manifest for GHCR :latest, and deploy for ECS ( deploy waits on stage so rollouts see the updated :staging).

docker-build-push splits validate / ECR tag push / GHCR push-by-digest, moves layer cache to a private GHCR registry ref (read on PRs; write only on trusted events; skipped for fork PRs), stamps org.opencontainers.image.revision, and returns ghcr-digest. deploy.yml stops tagging :staging and per-arch GHCR tags in build; it uploads digests and assembles :sha-<sha> from those digests instead of :latest-<arch> tags.

New tag-if-newer advances :staging / :latest only when the run’s commit is a strict git descendant of the revision on the current tag (from OCI labels), with retries on transient registry errors.

Reviewed by Cursor Bugbot for commit 4c62534. Bugbot is set up for automated code reviews on this repo. Configure here.

@augmentcode
Copy link
Copy Markdown

augmentcode Bot commented Jun 1, 2026

πŸ€– Augment PR Summary

Summary: This PR hardens the backend Deploy workflow against concurrency-related dropped builds and mutable-tag races.

Changes:

  • Reworks workflow concurrency: run-level uses unique github.run_id, while mutable-tag/rollout jobs serialize per service via queue: max.
  • Build job pushes immutable artifacts in parallel (ECR :sha/:run tags; GHCR pushed by digest only).
  • Adds a new tag-if-newer composite action to advance mutable tags (:staging, :latest) only when the candidate commit is a strict descendant of the currently-tagged commit.
  • Publishes multi-arch GHCR manifests from per-arch digests (avoids β€œFrankenstein” manifests from moving per-arch tags).
  • Moves build cache from GHA cache backend to a per-service/arch GHCR registry cache, written only on trusted events.
  • Adds retry wrappers around docker buildx imagetools create operations to mitigate transient registry failures.

Technical Notes: Images are labeled with org.opencontainers.image.revision to allow tag guards to resolve the commit behind existing tags.

πŸ€– Was this summary useful? React with πŸ‘ or πŸ‘Ž

Copy link
Copy Markdown

@augmentcode augmentcode Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review completed. 2 suggestions posted.

Fix All in Augment

Comment augment review to trigger a new review at any time.

Comment thread .github/actions/tag-if-newer/action.yml Outdated
Comment thread .github/workflows/deploy.yml Outdated
@codecov
Copy link
Copy Markdown

codecov Bot commented Jun 1, 2026

Codecov Report

βœ… All modified and coverable lines are covered by tests.
βœ… Project coverage is 59.07%. Comparing base (41f5267) to head (4c62534).
⚠️ Report is 7 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #8801   +/-   ##
=======================================
  Coverage   59.07%   59.07%           
=======================================
  Files        1344     1344           
  Lines      129750   129750           
  Branches     5868     5868           
=======================================
  Hits        76653    76653           
  Misses      52193    52193           
  Partials      904      904           
Flag Coverage Ξ”
apps.hash-ai-worker-ts 1.41% <ΓΈ> (ΓΈ)
apps.hash-api 0.00% <ΓΈ> (ΓΈ)
blockprotocol.type-system 40.84% <ΓΈ> (ΓΈ)
local.claude-hooks 0.00% <ΓΈ> (ΓΈ)
local.harpc-client 51.24% <ΓΈ> (ΓΈ)
local.hash-backend-utils 2.81% <ΓΈ> (ΓΈ)
local.hash-graph-sdk 9.63% <ΓΈ> (ΓΈ)
local.hash-isomorphic-utils 0.00% <ΓΈ> (ΓΈ)
rust.antsi 0.00% <ΓΈ> (ΓΈ)
rust.error-stack 90.87% <ΓΈ> (ΓΈ)
rust.harpc-codec 84.70% <ΓΈ> (ΓΈ)
rust.harpc-net 96.19% <ΓΈ> (ΓΈ)
rust.harpc-tower 67.03% <ΓΈ> (ΓΈ)
rust.harpc-types 0.00% <ΓΈ> (ΓΈ)
rust.harpc-wire-protocol 92.23% <ΓΈ> (ΓΈ)
rust.hash-codec 72.76% <ΓΈ> (ΓΈ)
rust.hash-graph-api 2.52% <ΓΈ> (ΓΈ)
rust.hash-graph-authorization 62.34% <ΓΈ> (ΓΈ)
rust.hash-graph-postgres-store 26.74% <ΓΈ> (ΓΈ)
rust.hash-graph-store 37.76% <ΓΈ> (ΓΈ)
rust.hash-graph-temporal-versioning 47.95% <ΓΈ> (ΓΈ)
rust.hash-graph-types 0.00% <ΓΈ> (ΓΈ)
rust.hash-graph-validation 83.45% <ΓΈ> (ΓΈ)
rust.hashql-ast 87.23% <ΓΈ> (ΓΈ)
rust.hashql-compiletest 28.26% <ΓΈ> (ΓΈ)
rust.hashql-core 79.28% <ΓΈ> (ΓΈ)
rust.hashql-diagnostics 72.53% <ΓΈ> (ΓΈ)
rust.hashql-eval 75.69% <ΓΈ> (ΓΈ)
rust.hashql-hir 89.06% <ΓΈ> (ΓΈ)
rust.hashql-mir 86.84% <ΓΈ> (ΓΈ)
rust.hashql-syntax-jexpr 94.06% <ΓΈ> (ΓΈ)

Flags with carried forward coverage won't be shown. Click here to find out more.

β˜” View full report in Codecov by Sentry.
πŸ“’ Have feedback on the report? Share it here.

πŸš€ New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • πŸ“¦ JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@codspeed-hq
Copy link
Copy Markdown

codspeed-hq Bot commented Jun 1, 2026

Merging this PR will not alter performance

βœ… 80 untouched benchmarks


Comparing t/sre-739-fix-backend-deploy-concurrency-dropped-builds-and-out-of (4c62534) with main (9eab148)

Open in CodSpeed

Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 16728da. Configure here.

Comment thread .github/actions/tag-if-newer/action.yml Outdated
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Reworks the backend Deploy GitHub Actions workflow to prevent dropped publishing runs and races around mutable Docker tags by switching to per-run concurrency at the workflow level and per-service serialization at the job level, assembling GHCR multi-arch manifests from pushed digests, and adding a β€œmove mutable tag only if newer” guard.

Changes:

  • Update deploy workflow concurrency and job structure (new stage job for ECR :staging, per-service serialization for mutable tags and ECS redeploys).
  • Push GHCR images by digest and assemble :sha-<sha> manifests from collected digests; advance :latest via a guard action.
  • Move build cache to GHCR registry cache refs and introduce a new tag-if-newer composite action.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
.github/workflows/deploy.yml Changes concurrency model; adds digest-based manifest assembly, a staging tag job, and guarded mutable-tag advancement.
.github/actions/tag-if-newer/action.yml New composite action to advance mutable tags only when the candidate commit is a strict descendant of the currently-tagged commit.
.github/actions/docker-build-push/action.yml Updates build/push behavior for GHCR digest pushes and GHCR registry-backed caching; adds digest output for downstream manifest assembly.

πŸ’‘ Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread .github/actions/docker-build-push/action.yml
Comment thread .github/workflows/deploy.yml
Comment thread .github/actions/docker-build-push/action.yml Outdated
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 1, 2026

Benchmark results

@rust/hash-graph-benches – Integrations

policy_resolution_large

Function Value Mean Flame graphs
resolve_policies_for_actor user: empty, selectivity: high, policies: 2002 $$25.6 \mathrm{ms} \pm 183 \mathrm{ΞΌs}\left({\color{gray}0.963 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: low, policies: 1 $$3.38 \mathrm{ms} \pm 14.0 \mathrm{ΞΌs}\left({\color{gray}-3.344 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: medium, policies: 1001 $$12.6 \mathrm{ms} \pm 91.1 \mathrm{ΞΌs}\left({\color{red}5.53 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: high, policies: 3314 $$43.7 \mathrm{ms} \pm 410 \mathrm{ΞΌs}\left({\color{gray}0.804 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: low, policies: 1 $$14.7 \mathrm{ms} \pm 173 \mathrm{ΞΌs}\left({\color{gray}-2.337 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: medium, policies: 1526 $$24.1 \mathrm{ms} \pm 138 \mathrm{ΞΌs}\left({\color{gray}2.53 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: high, policies: 2078 $$27.1 \mathrm{ms} \pm 175 \mathrm{ΞΌs}\left({\color{gray}3.17 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: low, policies: 1 $$3.76 \mathrm{ms} \pm 20.0 \mathrm{ΞΌs}\left({\color{gray}-0.558 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: medium, policies: 1033 $$13.6 \mathrm{ms} \pm 116 \mathrm{ΞΌs}\left({\color{red}5.00 \mathrm{\%}}\right) $$ Flame Graph

policy_resolution_medium

Function Value Mean Flame graphs
resolve_policies_for_actor user: empty, selectivity: high, policies: 102 $$3.72 \mathrm{ms} \pm 20.0 \mathrm{ΞΌs}\left({\color{gray}1.14 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: low, policies: 1 $$2.97 \mathrm{ms} \pm 14.3 \mathrm{ΞΌs}\left({\color{gray}-1.173 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: medium, policies: 51 $$3.33 \mathrm{ms} \pm 20.1 \mathrm{ΞΌs}\left({\color{gray}1.06 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: high, policies: 269 $$5.09 \mathrm{ms} \pm 36.0 \mathrm{ΞΌs}\left({\color{gray}1.22 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: low, policies: 1 $$3.55 \mathrm{ms} \pm 29.4 \mathrm{ΞΌs}\left({\color{gray}1.69 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: medium, policies: 107 $$4.08 \mathrm{ms} \pm 27.7 \mathrm{ΞΌs}\left({\color{gray}1.67 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: high, policies: 133 $$4.34 \mathrm{ms} \pm 31.7 \mathrm{ΞΌs}\left({\color{gray}-0.130 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: low, policies: 1 $$3.48 \mathrm{ms} \pm 17.6 \mathrm{ΞΌs}\left({\color{gray}1.16 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: medium, policies: 63 $$4.06 \mathrm{ms} \pm 30.1 \mathrm{ΞΌs}\left({\color{gray}1.68 \mathrm{\%}}\right) $$ Flame Graph

policy_resolution_none

Function Value Mean Flame graphs
resolve_policies_for_actor user: empty, selectivity: high, policies: 2 $$2.59 \mathrm{ms} \pm 15.7 \mathrm{ΞΌs}\left({\color{gray}-2.215 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: low, policies: 1 $$2.52 \mathrm{ms} \pm 14.6 \mathrm{ΞΌs}\left({\color{gray}-0.098 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: medium, policies: 1 $$2.57 \mathrm{ms} \pm 16.3 \mathrm{ΞΌs}\left({\color{gray}-2.592 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: high, policies: 8 $$2.85 \mathrm{ms} \pm 14.3 \mathrm{ΞΌs}\left({\color{gray}-3.680 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: low, policies: 1 $$2.67 \mathrm{ms} \pm 13.1 \mathrm{ΞΌs}\left({\color{gray}-3.018 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: medium, policies: 3 $$2.88 \mathrm{ms} \pm 20.1 \mathrm{ΞΌs}\left({\color{gray}-1.635 \mathrm{\%}}\right) $$ Flame Graph

policy_resolution_small

Function Value Mean Flame graphs
resolve_policies_for_actor user: empty, selectivity: high, policies: 52 $$3.02 \mathrm{ms} \pm 15.3 \mathrm{ΞΌs}\left({\color{gray}0.818 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: low, policies: 1 $$2.88 \mathrm{ms} \pm 43.9 \mathrm{ΞΌs}\left({\color{red}8.11 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: medium, policies: 25 $$2.98 \mathrm{ms} \pm 19.3 \mathrm{ΞΌs}\left({\color{gray}3.42 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: high, policies: 94 $$3.39 \mathrm{ms} \pm 18.6 \mathrm{ΞΌs}\left({\color{gray}1.82 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: low, policies: 1 $$2.99 \mathrm{ms} \pm 16.2 \mathrm{ΞΌs}\left({\color{gray}0.403 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: medium, policies: 26 $$3.32 \mathrm{ms} \pm 22.5 \mathrm{ΞΌs}\left({\color{gray}1.18 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: high, policies: 66 $$3.33 \mathrm{ms} \pm 18.0 \mathrm{ΞΌs}\left({\color{gray}1.83 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: low, policies: 1 $$2.96 \mathrm{ms} \pm 14.6 \mathrm{ΞΌs}\left({\color{gray}-0.092 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: medium, policies: 29 $$3.29 \mathrm{ms} \pm 17.6 \mathrm{ΞΌs}\left({\color{gray}0.061 \mathrm{\%}}\right) $$ Flame Graph

read_scaling_complete

Function Value Mean Flame graphs
entity_by_id;one_depth 1 entities $$54.8 \mathrm{ms} \pm 348 \mathrm{ΞΌs}\left({\color{gray}3.72 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;one_depth 10 entities $$45.6 \mathrm{ms} \pm 212 \mathrm{ΞΌs}\left({\color{gray}0.601 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;one_depth 25 entities $$50.1 \mathrm{ms} \pm 255 \mathrm{ΞΌs}\left({\color{gray}-0.862 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;one_depth 5 entities $$44.4 \mathrm{ms} \pm 234 \mathrm{ΞΌs}\left({\color{gray}0.636 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;one_depth 50 entities $$61.8 \mathrm{ms} \pm 403 \mathrm{ΞΌs}\left({\color{gray}0.736 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 1 entities $$62.6 \mathrm{ms} \pm 537 \mathrm{ΞΌs}\left({\color{gray}2.30 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 10 entities $$55.6 \mathrm{ms} \pm 264 \mathrm{ΞΌs}\left({\color{gray}1.13 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 25 entities $$103 \mathrm{ms} \pm 607 \mathrm{ΞΌs}\left({\color{gray}0.741 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 5 entities $$45.8 \mathrm{ms} \pm 258 \mathrm{ΞΌs}\left({\color{gray}0.103 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 50 entities $$289 \mathrm{ms} \pm 988 \mathrm{ΞΌs}\left({\color{gray}-0.346 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 1 entities $$18.9 \mathrm{ms} \pm 111 \mathrm{ΞΌs}\left({\color{gray}-2.341 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 10 entities $$19.9 \mathrm{ms} \pm 95.6 \mathrm{ΞΌs}\left({\color{gray}-2.205 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 25 entities $$20.3 \mathrm{ms} \pm 133 \mathrm{ΞΌs}\left({\color{gray}1.91 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 5 entities $$19.0 \mathrm{ms} \pm 118 \mathrm{ΞΌs}\left({\color{gray}-1.715 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 50 entities $$25.5 \mathrm{ms} \pm 147 \mathrm{ΞΌs}\left({\color{gray}2.58 \mathrm{\%}}\right) $$ Flame Graph

read_scaling_linkless

Function Value Mean Flame graphs
entity_by_id 1 entities $$19.2 \mathrm{ms} \pm 146 \mathrm{ΞΌs}\left({\color{gray}-0.168 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 10 entities $$19.4 \mathrm{ms} \pm 88.4 \mathrm{ΞΌs}\left({\color{gray}2.26 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 100 entities $$19.2 \mathrm{ms} \pm 108 \mathrm{ΞΌs}\left({\color{gray}-0.455 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 1000 entities $$20.1 \mathrm{ms} \pm 122 \mathrm{ΞΌs}\left({\color{gray}0.688 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 10000 entities $$26.7 \mathrm{ms} \pm 173 \mathrm{ΞΌs}\left({\color{gray}0.496 \mathrm{\%}}\right) $$ Flame Graph

representative_read_entity

Function Value Mean Flame graphs
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/block/v/1 $$33.8 \mathrm{ms} \pm 302 \mathrm{ΞΌs}\left({\color{gray}0.235 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/book/v/1 $$34.3 \mathrm{ms} \pm 274 \mathrm{ΞΌs}\left({\color{gray}2.20 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/building/v/1 $$34.3 \mathrm{ms} \pm 264 \mathrm{ΞΌs}\left({\color{gray}-1.582 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/organization/v/1 $$33.9 \mathrm{ms} \pm 329 \mathrm{ΞΌs}\left({\color{gray}0.162 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/page/v/2 $$34.1 \mathrm{ms} \pm 293 \mathrm{ΞΌs}\left({\color{gray}-0.168 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/person/v/1 $$34.2 \mathrm{ms} \pm 293 \mathrm{ΞΌs}\left({\color{gray}1.60 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/playlist/v/1 $$34.1 \mathrm{ms} \pm 268 \mathrm{ΞΌs}\left({\color{gray}2.41 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/song/v/1 $$35.2 \mathrm{ms} \pm 322 \mathrm{ΞΌs}\left({\color{gray}0.404 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/uk-address/v/1 $$34.6 \mathrm{ms} \pm 282 \mathrm{ΞΌs}\left({\color{gray}2.34 \mathrm{\%}}\right) $$ Flame Graph

representative_read_entity_type

Function Value Mean Flame graphs
get_entity_type_by_id Account ID: bf5a9ef5-dc3b-43cf-a291-6210c0321eba $$8.44 \mathrm{ms} \pm 35.3 \mathrm{ΞΌs}\left({\color{gray}-0.491 \mathrm{\%}}\right) $$ Flame Graph

representative_read_multiple_entities

Function Value Mean Flame graphs
entity_by_property traversal_paths=0 0 $$90.6 \mathrm{ms} \pm 423 \mathrm{ΞΌs}\left({\color{gray}0.020 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=255 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true $$144 \mathrm{ms} \pm 623 \mathrm{ΞΌs}\left({\color{gray}1.18 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false $$97.7 \mathrm{ms} \pm 556 \mathrm{ΞΌs}\left({\color{gray}-0.766 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true $$110 \mathrm{ms} \pm 601 \mathrm{ΞΌs}\left({\color{gray}1.88 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true $$116 \mathrm{ms} \pm 725 \mathrm{ΞΌs}\left({\color{gray}1.09 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true $$124 \mathrm{ms} \pm 564 \mathrm{ΞΌs}\left({\color{gray}1.86 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=0 0 $$105 \mathrm{ms} \pm 610 \mathrm{ΞΌs}\left({\color{gray}2.78 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=255 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true $$133 \mathrm{ms} \pm 525 \mathrm{ΞΌs}\left({\color{gray}2.26 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false $$111 \mathrm{ms} \pm 638 \mathrm{ΞΌs}\left({\color{gray}2.09 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true $$121 \mathrm{ms} \pm 673 \mathrm{ΞΌs}\left({\color{gray}2.69 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true $$124 \mathrm{ms} \pm 795 \mathrm{ΞΌs}\left({\color{gray}4.03 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true $$123 \mathrm{ms} \pm 493 \mathrm{ΞΌs}\left({\color{gray}1.78 \mathrm{\%}}\right) $$

scenarios

Function Value Mean Flame graphs
full_test query-limited $$191 \mathrm{ms} \pm 807 \mathrm{ΞΌs}\left({\color{gray}0.259 \mathrm{\%}}\right) $$ Flame Graph
full_test query-unlimited $$152 \mathrm{ms} \pm 828 \mathrm{ΞΌs}\left({\color{lightgreen}-18.217 \mathrm{\%}}\right) $$ Flame Graph
linked_queries query-limited $$41.3 \mathrm{ms} \pm 210 \mathrm{ΞΌs}\left({\color{gray}3.89 \mathrm{\%}}\right) $$ Flame Graph
linked_queries query-unlimited $$553 \mathrm{ms} \pm 1.09 \mathrm{ms}\left({\color{gray}-2.147 \mathrm{\%}}\right) $$ Flame Graph

@TimDiekmann TimDiekmann requested a review from a team June 2, 2026 09:00
Comment thread .github/workflows/deploy.yml
@TimDiekmann TimDiekmann added this pull request to the merge queue Jun 2, 2026
@github-merge-queue github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jun 2, 2026
@TimDiekmann TimDiekmann added this pull request to the merge queue Jun 2, 2026
@github-merge-queue github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jun 2, 2026
@TimDiekmann TimDiekmann added this pull request to the merge queue Jun 2, 2026
Merged via the queue into main with commit e660282 Jun 2, 2026
181 checks passed
@TimDiekmann TimDiekmann deleted the t/sre-739-fix-backend-deploy-concurrency-dropped-builds-and-out-of branch June 2, 2026 11:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/infra Relates to version control, CI, CD or IaC (area)

Development

Successfully merging this pull request may close these issues.

3 participants