Skip to content

SRE-728: Mirror images to GHCR alongside ECR#8781

Open
TimDiekmann wants to merge 17 commits into
t/sre-726-overhaul-external-services-dockerize-apps-for-self-hostingfrom
t/sre-728-publish-images-to-ghcr
Open

SRE-728: Mirror images to GHCR alongside ECR#8781
TimDiekmann wants to merge 17 commits into
t/sre-726-overhaul-external-services-dockerize-apps-for-self-hostingfrom
t/sre-728-publish-images-to-ghcr

Conversation

@TimDiekmann
Copy link
Copy Markdown
Member

@TimDiekmann TimDiekmann commented May 28, 2026

🌟 What is the purpose of this PR?

Publish HASH backend images to GHCR alongside the existing ECR push so self-hosters can docker pull directly. The build matrix is also restructured to build natively per-arch (arm64 + amd64), the backend CD jobs are merged into the existing deploy.yml workflow, and a handful of latent Docker-image fixes that surfaced once compose started actually pulling and running the published images are folded in.

🔗 Related links

  • SRE-728 (internal)
  • FE-752 — clean server/browser API-origin split (the hash-frontend GHCR image is not yet origin-portable; see Known issues)

🚫 Blocked by

🔍 What does this change?

deploy.yml workflow (merged from hash-backend-cd.yml)

  • Folded the backend CD jobs into the existing Deploy workflow — sourcemaps + backend image build/manifest/deploy now share one workflow, one setup job, one collector. hash-backend-cd.yml deleted.
  • Triggers extended with workflow_dispatch and merge_group.
  • Concurrency: PR / merge_group cancel-in-progress on a per-ref group; push / workflow_dispatch share a flat publish group so all publishing runs serialize regardless of ref. The previous cancel-in-progress: true on every event was incompatible with multi-arch GHCR pushes — concurrent main pushes would race on the moving :latest-<arch> tags.
  • Build matrix is service × arch (cartesian, native runners per arch — ubuntu-24.04-arm and ubuntu-24.04; no QEMU). Per-service config (paths, push targets, build args) lives in include: with push: [ecr, ghcr] declaring registries explicitly. A Compute targets step centralises the push policy and emits per-job should_build, push, ecr_tags, ghcr_tags outputs — eliminating the inline ${{ ... && ... || '' }} ternaries.
  • GHCR: each build pushes per-arch tags (:latest-<arch>, :sha-<sha>-<arch>); a follow-up manifest job joins them into multi-arch :latest / :sha-<sha> via docker buildx imagetools create. Image namespace: ghcr.io/hashintel/hash/<service> (canonical <owner>/<repo>/<image> pattern). Kratos and Hydra publish to ECR only.
  • ECR stays single-arch (Graviton-only staging) and main-only — the OIDC trust policy already rejects assumption from other refs; the workflow now matches and skips the auth step entirely on feature-branch dispatch.
  • Deploy matrix: three previous deploy jobs (graph / app / workers) collapsed into one matrix over all eight ECS services.
  • Deployments passed collector aggregates setup / sourcemaps / build / manifest / deploy. Stable name suitable as a Required branch-protection check.
  • Slack notifications split by failure type — standalone notify-slack-deploy fires on backend deploy failures on main; an in-passed step fires on merge-queue failures (audience: @devops vs @infra).
  • env-block trimmed: account-IDs and role-names stored separately, ARNs assembled in the steps. Drops the duplicated 469596578827 account ID.

docker-build-push composite action

  • Single-platform per call. Callers hand in fully-qualified ECR_TAGS / GHCR_TAGS (each may be empty) and a PLATFORM. Conditional ECR / GHCR login on tag-set non-emptiness.
  • BUILD_ARGS input restored for non-secret values (e.g. NEXT_PUBLIC_* URLs), with a description warning against using it for secrets.

Dockerfile fixes

Latent bugs uncovered once compose actually ran the published images:

  • hash-frontend: ENTRYPOINT used yarn-classic flags (--cache-folder / --global-folder) the repo's yarn berry (4.12.0) rejected — yarn start errored with Unknown Syntax Error: Command not found. Swap to the yarn-berry equivalents YARN_CACHE_FOLDER / YARN_GLOBAL_FOLDER as ENV. Plus chown -R .next because install -d was a no-op on the pre-existing directory (created by next build as root) and .next/trace was unwritable by the runtime user. Plus rm -rf .next/cache after turbo build (~2 GB of webpack cache that doesn't belong in the published image).
  • hash-ai-worker-ts: drop the ARG GOOGLE_CLOUD_WORKLOAD_IDENTITY_FEDERATION_CONFIG_JSON / ENV / if [ -n … ] block. The pattern would bake the JSON into image history and ENV the moment any caller passed it as a build-arg; the in-RUN export GOOGLE_APPLICATION_CREDENTIALS= was shell-scoped and never reached runtime anyway. GCP creds are now a runtime concern (mount file + set GOOGLE_APPLICATION_CREDENTIALS).
  • hash-integration-worker: drop stray ENV MISE_VERBOSE=1.

Cleanup

  • test.yml no longer needs its docker-build matrix — the merged deploy.yml validates the same images on every PR. Removes the build job, the dockers setup output, and the related affected-package query.
  • Delete the unused .github/actions/build-docker-images/ action — it was the structural counterpart to the ai-worker-ts secret footgun above.

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

  • does not modify any publishable blocks or libraries, or modifications do not need publishing

📜 Does this require a change to the docs?

  • are internal and do not require a docs change

🕸️ Does this require a change to the Turbo Graph?

  • do not affect the execution graph

⚠️ Known issues

  • The hash-frontend image still bakes NEXT_PUBLIC_* at build time (FE-752). Today the image hard-codes http://localhost:5001 / http://localhost:3000, which is fine for the staging/dev origin and the GHCR mirror's primary audience (self-hosters running it via compose). Proper origin portability is FE-752.

🐾 Next steps

  • SRE-733: Turbo-affected-packages filter for the backend build matrix (only build services whose source changed), mirroring the existing sourcemaps pattern in the same workflow. The current Compute targets step is structured so a future prep-job can feed it a pre-filtered matrix list.

🛡 What tests cover this?

No new tests — the workflow itself is the test surface. PR-mode runs all 12 (service, arch) build jobs without pushing; the Deployments passed collector aggregates the results.

❓ How to test this?

This PR (build-only validation):

  1. Wait for the workflow on this PR. All 12 build jobs + the Deployments passed collector should be green; manifest, deploy, and the standalone deploy-notify job skipped.

Ad-hoc validation from any branch (feature-branch dispatch):

gh workflow run deploy.yml --ref <branch>

ECR push is short-circuited on non-main (matches the OIDC trust policy). GHCR push and manifest creation run end-to-end. Verify:

docker pull ghcr.io/hashintel/hash/frontend:latest
docker manifest inspect ghcr.io/hashintel/hash/frontend:latest
# → arm64 + amd64 entries in the manifest list

Or via compose (after #8758 merges):

docker compose --profile hash up -d --pull always

After merge to main:

  • Workflow runs with full push intent. ECR and GHCR get fresh :staging / :latest / :sha-<sha> tags. ECS rolling-redeploys the eight backend services. Slack-notify fires on deploy failure.

@vercel
Copy link
Copy Markdown

vercel Bot commented May 28, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
hash Ready Ready Preview, Comment May 28, 2026 5:45pm
hashdotdesign-tokens Ready Ready Preview, Comment May 28, 2026 5:45pm
petrinaut Ready Ready Preview, Comment May 28, 2026 5:45pm

@github-actions github-actions Bot added area/deps Relates to third-party dependencies (area) area/apps > hash* Affects HASH (a `hash-*` app) area/infra Relates to version control, CI, CD or IaC (area) area/apps > hash-api Affects the HASH API (app) area/infra > docker type/eng > frontend Owned by the @frontend team type/eng > backend Owned by the @backend team area/tests New or updated tests area/apps area/apps > hash-graph type/legal Owned by the @legal team labels May 28, 2026
@TimDiekmann TimDiekmann changed the base branch from main to t/sre-726-overhaul-external-services-dockerize-apps-for-self-hosting May 28, 2026 11:13
@github-actions github-actions Bot removed area/deps Relates to third-party dependencies (area) area/apps > hash* Affects HASH (a `hash-*` app) area/apps > hash-api Affects the HASH API (app) area/infra > docker type/eng > frontend Owned by the @frontend team type/eng > backend Owned by the @backend team area/tests New or updated tests area/apps area/apps > hash-graph type/legal Owned by the @legal team labels May 28, 2026
@codspeed-hq
Copy link
Copy Markdown

codspeed-hq Bot commented May 28, 2026

Merging this PR will not alter performance

✅ 80 untouched benchmarks


Comparing t/sre-728-publish-images-to-ghcr (e55a185) with t/sre-726-overhaul-external-services-dockerize-apps-for-self-hosting (bf7508d)

Open in CodSpeed

Comment thread .github/actions/docker-build-push/action.yml Outdated
@codecov
Copy link
Copy Markdown

codecov Bot commented May 28, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (t/sre-726-overhaul-external-services-dockerize-apps-for-self-hosting@bf7508d). Learn more about missing BASE report.

Additional details and impacted files
@@                                           Coverage Diff                                           @@
##             t/sre-726-overhaul-external-services-dockerize-apps-for-self-hosting    #8781   +/-   ##
=======================================================================================================
  Coverage                                                                        ?   59.01%           
=======================================================================================================
  Files                                                                           ?     1342           
  Lines                                                                           ?   129455           
  Branches                                                                        ?     5849           
=======================================================================================================
  Hits                                                                            ?    76395           
  Misses                                                                          ?    52159           
  Partials                                                                        ?      901           
Flag Coverage Δ
apps.hash-ai-worker-ts 1.41% <ø> (?)
apps.hash-api 0.00% <ø> (?)
blockprotocol.type-system 40.84% <ø> (?)
local.claude-hooks 0.00% <ø> (?)
local.harpc-client 51.24% <ø> (?)
local.hash-backend-utils 2.81% <ø> (?)
local.hash-graph-sdk 9.63% <ø> (?)
local.hash-isomorphic-utils 0.00% <ø> (?)
rust.antsi 0.00% <ø> (?)
rust.error-stack 90.87% <ø> (?)
rust.harpc-codec 84.70% <ø> (?)
rust.harpc-net 96.18% <ø> (?)
rust.harpc-tower 67.03% <ø> (?)
rust.harpc-types 0.00% <ø> (?)
rust.harpc-wire-protocol 92.23% <ø> (?)
rust.hash-codec 72.76% <ø> (?)
rust.hash-graph-api 2.52% <ø> (?)
rust.hash-graph-authorization 62.34% <ø> (?)
rust.hash-graph-postgres-store 26.74% <ø> (?)
rust.hash-graph-store 37.76% <ø> (?)
rust.hash-graph-temporal-versioning 47.95% <ø> (?)
rust.hash-graph-types 0.00% <ø> (?)
rust.hash-graph-validation 83.45% <ø> (?)
rust.hashql-ast 87.23% <ø> (?)
rust.hashql-compiletest 28.26% <ø> (?)
rust.hashql-core 79.32% <ø> (?)
rust.hashql-diagnostics 72.53% <ø> (?)
rust.hashql-eval 75.69% <ø> (?)
rust.hashql-hir 89.06% <ø> (?)
rust.hashql-mir 86.74% <ø> (?)
rust.hashql-syntax-jexpr 94.06% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Two latent bugs in the runner stage that surfaced once compose started
pulling the published image (SRE-726):

- ENTRYPOINT invoked `yarn --cache-folder X --global-folder Y` — these
  are yarn-classic flags that yarn berry (the repo is on yarn@4.12.0)
  silently rejects, so `start` was parsed as a command name and yarn
  errored with "Unknown Syntax Error: Command not found". Skip yarn at
  runtime and invoke `next` directly from the hoisted workspace-root
  `node_modules`. WORKDIR still points at the app so next picks up its
  own next.config.js.

- `install -d` on `.next` was a no-op because the directory already
  exists from the builder stage's `next build`, leaving the
  root-owned contents (including `.next/trace`) unwritable by the
  runtime `frontend` user — next start aborted with EACCES on the
  trace file. Replace with an explicit `chown -R`.
@vercel vercel Bot temporarily deployed to Preview – petrinaut May 28, 2026 14:46 Inactive
@github-actions github-actions Bot added area/apps > hash* Affects HASH (a `hash-*` app) type/eng > frontend Owned by the @frontend team area/apps labels May 28, 2026
…tion

Image scan flagged a security footgun in `hash-ai-worker-ts`, a 2 GB
build-cache leak in `hash-frontend`, and a stray debug knob in
`hash-integration-worker`. None caused a live secret leak in the
currently published images, but the patterns were "wrong build args
away from leaking".

hash-ai-worker-ts:
  Drop the `ARG GOOGLE_CLOUD_WORKLOAD_IDENTITY_FEDERATION_CONFIG_JSON`
  / `ENV` pair from the runner stage along with the `if [ -n … ]`
  block that wrote the JSON to `/tmp/...` and `export`-ed
  `GOOGLE_APPLICATION_CREDENTIALS`. The `export` was shell-scoped
  (never reached the ENTRYPOINT) and the ARG/ENV combination would
  bake the JSON into both `docker history` and the final image as
  soon as any caller passed `--build-arg ...=$(cat creds.json)`.
  GCP credentials are runtime concerns — mount the file and set
  `GOOGLE_APPLICATION_CREDENTIALS` at runtime instead.

hash-frontend:
  `rm -rf /usr/local/src/apps/hash-frontend/.next/cache` after
  `turbo build`. The webpack/swc/fetch caches are runtime artefacts
  and shipped ~2 GB of dead bytes per architecture.

hash-integration-worker:
  Drop `ENV MISE_VERBOSE=1` (leftover from Dockerfile iteration).

Plus delete `.github/actions/build-docker-images/` — last caller (the
`test.yml` build matrix) was removed in the same change set, and the
action's `--build-arg GOOGLE_CLOUD_...` invocation was the structural
counterpart to the footgun above.
Reverts the direct-`next` invocation from 54dd725. Running next
straight from `node_modules/.bin/next` started the server but broke
SSR — request handling hit JSX-runtime / module-resolution mismatches
(`jsxDEV is not a function` from `@hashintel/design-system`, `EvalError`
in the edge runtime). `yarn start` keeps yarn's resolution pipeline,
which the build apparently relies on, and serves requests correctly.

The original `--cache-folder` / `--global-folder` classic-yarn flags
that caused the "Unknown Syntax Error: Command not found" on yarn
berry stay gone; they're replaced with the berry equivalents
`YARN_CACHE_FOLDER` and `YARN_GLOBAL_FOLDER` as ENV.

The `chown -R .next` from the previous commit stays — that was an
orthogonal permission fix needed regardless of which binary launches
next.
Fold all backend image build/manifest/deploy jobs into the existing
`Deploy` workflow alongside the sourcemaps flow, so one workflow owns
the post-PR deployment surface end-to-end.

- Add `workflow_dispatch:` trigger.
- Apply the backend-cd concurrency model to the whole workflow: PR /
  merge_group cancel-in-progress on the per-ref group, push and
  workflow_dispatch share a flat `publish` group. The previous
  unconditional `cancel-in-progress: true` is incompatible with
  publishing — concurrent main pushes would race on the moving GHCR
  `:latest-<arch>` tags.
- Pull in the `build` / `manifest` / `deploy` jobs from
  hash-backend-cd.yml verbatim (Compute-targets policy, `push: [ecr,
  ghcr]` per-service, native per-arch runners, manifest merge).
- Consolidate the collector: `Deployments passed` aggregates `setup` /
  `sourcemaps` / `build` / `manifest` / `deploy`. Stable name suitable
  as a `Required` branch-protection check.
- Slack notification split by failure type: the standalone
  `notify-slack-deploy` fires on backend deploy failures on main; the
  in-passed step fires on merge-queue failures. Different audiences,
  different conditions.

`hash-backend-cd.yml` removed — the merged workflow replaces it.

A turbo-driven affected-services matrix for the backend build (similar
to the existing sourcemaps pattern) is the natural follow-up; the
current static include lays the groundwork.
@cursor
Copy link
Copy Markdown

cursor Bot commented May 28, 2026

PR Summary

High Risk
Changes main-branch staging deploy paths, registry pushes, and ECS rollouts; misconfigured push policy or manifest jobs could break production images or deployments.

Overview
Consolidates backend image build, GHCR mirroring, multi-arch publishing, and ECS deploy into deploy.yml, replacing hash-backend-cd.yml and the PR docker build path in test.yml.

CI: Native service × arch builds (arm64/amd64) with a Compute targets step that decides ECR (main + arm64 only), GHCR per-arch tags, build-only on PR/merge_group, and skipped jobs when nothing would publish. docker-build-push now takes ECR_TAGS / GHCR_TAGS, PLATFORM, optional PUSH, and conditional ECR/GHCR login. A manifest job joins GHCR arch tags into :latest / :sha-*. deploy is one matrix over eight ECS services; concurrency serializes publish runs to avoid GHCR tag races. Deployments passed gates setup, sourcemaps, build, manifest, and deploy.

Dockerfiles: hash-frontend fixes Yarn Berry startup (env-based cache dirs), .next ownership, and strips .next/cache from the image. hash-ai-worker-ts drops GCP WIF JSON from build-args/image layers. hash-integration-worker drops MISE_VERBOSE.

Removed: build-docker-images composite action and test.yml docker build matrix.

Reviewed by Cursor Bugbot for commit e55a185. Bugbot is set up for automated code reviews on this repo. Configure here.

@augmentcode
Copy link
Copy Markdown

augmentcode Bot commented May 28, 2026

🤖 Augment PR Summary

Summary: This PR mirrors HASH backend Docker images to GHCR alongside the existing ECR publishing to improve self-hosting support.

Changes:

  • Merges the standalone backend CD workflow into .github/workflows/deploy.yml and expands triggers (adds workflow_dispatch, supports merge queue).
  • Restructures the image build pipeline into a service × arch matrix using native runners per arch (no QEMU).
  • Introduces GHCR publishing with per-arch tags and a follow-up manifest job that assembles multi-arch :latest and :sha-<sha> tags.
  • Refactors .github/actions/docker-build-push to accept explicit ECR/GHCR tag lists, platform selection, and optional push behavior.
  • Collapses ECS redeploys into a single deploy matrix and adds a stable “Deployments passed” collector suitable for branch protection.
  • Removes the old hash-backend-cd.yml workflow and eliminates redundant Docker-build validation from test.yml.
  • Applies Dockerfile fixes for the frontend runtime user permissions / Yarn Berry entrypoint, and removes a credential build-arg footgun from hash-ai-worker-ts.

Technical Notes: GHCR images are published under ghcr.io/<owner>/hash/<service>, with multi-arch manifests created via docker buildx imagetools create.

🤖 Was this summary useful? React with 👍 or 👎

Copy link
Copy Markdown

@augmentcode augmentcode Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review completed. 1 suggestion posted.

Fix All in Augment

Comment augment review to trigger a new review at any time.

@@ -93,20 +106,293 @@ jobs:
- name: Build sourcemaps
Copy link
Copy Markdown

@augmentcode augmentcode Bot May 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The sourcemaps job’s if: condition references github.event.pull_request..., which evaluates false on push / workflow_dispatch events; that means sourcemaps won’t run on main publishes (see .github/workflows/deploy.yml around the sourcemaps job if: at ~L68). Consider guarding the fork-only check so it still runs on non-PR events.

Severity: medium

Fix This in Augment

🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.

Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit ad008ba. Configure here.

Comment thread .github/workflows/deploy.yml
Two unrelated tweaks on the merged deploy workflow:

- Skip amd64 builds on `pull_request` events. arm64 + native runner
  catches almost all real bugs; arch-specific divergence in Rust/Node
  code is rare. merge_group still builds amd64 before any main push,
  so the safety gate stays intact. Halves PR runtime / runner cost.

- The `sourcemaps` job's `if:` referenced `github.event.pull_request.
  head.repo.full_name`, which is null on non-PR events — meaning the
  sourcemaps upload was silently skipped on main pushes (and dispatch
  / merge_group). Guard the fork-only check behind
  `github.event_name == 'pull_request'` so push/dispatch/merge_group
  always run; PRs still apply the fork filter.
@github-actions
Copy link
Copy Markdown
Contributor

Benchmark results

@rust/hash-graph-benches – Integrations

policy_resolution_large

Function Value Mean Flame graphs
resolve_policies_for_actor user: empty, selectivity: high, policies: 2002 $$28.7 \mathrm{ms} \pm 275 \mathrm{μs}\left({\color{gray}-1.017 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: low, policies: 1 $$3.52 \mathrm{ms} \pm 20.2 \mathrm{μs}\left({\color{gray}-0.337 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: medium, policies: 1001 $$13.5 \mathrm{ms} \pm 110 \mathrm{μs}\left({\color{gray}-3.282 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: high, policies: 3314 $$44.0 \mathrm{ms} \pm 403 \mathrm{μs}\left({\color{gray}-2.904 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: low, policies: 1 $$14.7 \mathrm{ms} \pm 128 \mathrm{μs}\left({\color{lightgreen}-6.232 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: medium, policies: 1526 $$25.6 \mathrm{ms} \pm 256 \mathrm{μs}\left({\color{gray}-0.729 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: high, policies: 2078 $$30.8 \mathrm{ms} \pm 262 \mathrm{μs}\left({\color{gray}2.30 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: low, policies: 1 $$3.91 \mathrm{ms} \pm 24.6 \mathrm{μs}\left({\color{gray}0.112 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: medium, policies: 1033 $$15.1 \mathrm{ms} \pm 102 \mathrm{μs}\left({\color{gray}-0.334 \mathrm{\%}}\right) $$ Flame Graph

policy_resolution_medium

Function Value Mean Flame graphs
resolve_policies_for_actor user: empty, selectivity: high, policies: 102 $$3.87 \mathrm{ms} \pm 24.4 \mathrm{μs}\left({\color{gray}-1.342 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: low, policies: 1 $$3.08 \mathrm{ms} \pm 19.2 \mathrm{μs}\left({\color{gray}-0.663 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: medium, policies: 51 $$3.44 \mathrm{ms} \pm 24.0 \mathrm{μs}\left({\color{gray}-1.350 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: high, policies: 269 $$5.38 \mathrm{ms} \pm 38.9 \mathrm{μs}\left({\color{gray}0.015 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: low, policies: 1 $$3.65 \mathrm{ms} \pm 20.5 \mathrm{μs}\left({\color{gray}-0.126 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: medium, policies: 107 $$4.30 \mathrm{ms} \pm 34.9 \mathrm{μs}\left({\color{gray}0.369 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: high, policies: 133 $$4.49 \mathrm{ms} \pm 27.3 \mathrm{μs}\left({\color{gray}-2.145 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: low, policies: 1 $$3.55 \mathrm{ms} \pm 29.4 \mathrm{μs}\left({\color{gray}-0.762 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: medium, policies: 63 $$4.19 \mathrm{ms} \pm 23.9 \mathrm{μs}\left({\color{gray}-1.552 \mathrm{\%}}\right) $$ Flame Graph

policy_resolution_none

Function Value Mean Flame graphs
resolve_policies_for_actor user: empty, selectivity: high, policies: 2 $$2.67 \mathrm{ms} \pm 15.3 \mathrm{μs}\left({\color{lightgreen}-5.315 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: low, policies: 1 $$2.59 \mathrm{ms} \pm 19.7 \mathrm{μs}\left({\color{gray}-1.149 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: medium, policies: 1 $$2.66 \mathrm{ms} \pm 15.6 \mathrm{μs}\left({\color{gray}-3.041 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: high, policies: 8 $$3.00 \mathrm{ms} \pm 25.4 \mathrm{μs}\left({\color{gray}-1.942 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: low, policies: 1 $$2.73 \mathrm{ms} \pm 16.3 \mathrm{μs}\left({\color{gray}-4.059 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: medium, policies: 3 $$3.04 \mathrm{ms} \pm 20.3 \mathrm{μs}\left({\color{gray}0.265 \mathrm{\%}}\right) $$ Flame Graph

policy_resolution_small

Function Value Mean Flame graphs
resolve_policies_for_actor user: empty, selectivity: high, policies: 52 $$3.03 \mathrm{ms} \pm 15.8 \mathrm{μs}\left({\color{gray}-1.715 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: low, policies: 1 $$2.83 \mathrm{ms} \pm 19.9 \mathrm{μs}\left({\color{gray}0.304 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: medium, policies: 25 $$3.03 \mathrm{ms} \pm 20.7 \mathrm{μs}\left({\color{gray}0.450 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: high, policies: 94 $$3.47 \mathrm{ms} \pm 18.2 \mathrm{μs}\left({\color{gray}-0.855 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: low, policies: 1 $$3.02 \mathrm{ms} \pm 20.1 \mathrm{μs}\left({\color{gray}0.392 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: medium, policies: 26 $$3.34 \mathrm{ms} \pm 20.0 \mathrm{μs}\left({\color{gray}-0.841 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: high, policies: 66 $$3.38 \mathrm{ms} \pm 18.6 \mathrm{μs}\left({\color{gray}-0.800 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: low, policies: 1 $$2.98 \mathrm{ms} \pm 19.7 \mathrm{μs}\left({\color{gray}-1.693 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: medium, policies: 29 $$3.40 \mathrm{ms} \pm 19.5 \mathrm{μs}\left({\color{gray}-0.558 \mathrm{\%}}\right) $$ Flame Graph

read_scaling_complete

Function Value Mean Flame graphs
entity_by_id;one_depth 1 entities $$57.2 \mathrm{ms} \pm 349 \mathrm{μs}\left({\color{gray}2.14 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;one_depth 10 entities $$48.2 \mathrm{ms} \pm 258 \mathrm{μs}\left({\color{gray}0.921 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;one_depth 25 entities $$53.6 \mathrm{ms} \pm 372 \mathrm{μs}\left({\color{gray}3.70 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;one_depth 5 entities $$46.4 \mathrm{ms} \pm 278 \mathrm{μs}\left({\color{gray}0.951 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;one_depth 50 entities $$66.6 \mathrm{ms} \pm 498 \mathrm{μs}\left({\color{gray}1.65 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 1 entities $$65.8 \mathrm{ms} \pm 429 \mathrm{μs}\left({\color{gray}1.96 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 10 entities $$60.4 \mathrm{ms} \pm 346 \mathrm{μs}\left({\color{gray}3.12 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 25 entities $$108 \mathrm{ms} \pm 569 \mathrm{μs}\left({\color{gray}0.715 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 5 entities $$49.7 \mathrm{ms} \pm 401 \mathrm{μs}\left({\color{gray}2.35 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 50 entities $$335 \mathrm{ms} \pm 1.24 \mathrm{ms}\left({\color{red}9.26 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 1 entities $$20.5 \mathrm{ms} \pm 119 \mathrm{μs}\left({\color{gray}0.044 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 10 entities $$21.0 \mathrm{ms} \pm 148 \mathrm{μs}\left({\color{gray}1.52 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 25 entities $$21.7 \mathrm{ms} \pm 132 \mathrm{μs}\left({\color{gray}-0.048 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 5 entities $$20.7 \mathrm{ms} \pm 124 \mathrm{μs}\left({\color{gray}2.31 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 50 entities $$26.2 \mathrm{ms} \pm 150 \mathrm{μs}\left({\color{gray}-1.019 \mathrm{\%}}\right) $$ Flame Graph

read_scaling_linkless

Function Value Mean Flame graphs
entity_by_id 1 entities $$20.4 \mathrm{ms} \pm 138 \mathrm{μs}\left({\color{gray}-0.889 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 10 entities $$20.4 \mathrm{ms} \pm 144 \mathrm{μs}\left({\color{gray}-1.651 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 100 entities $$20.3 \mathrm{ms} \pm 137 \mathrm{μs}\left({\color{gray}0.031 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 1000 entities $$21.4 \mathrm{ms} \pm 142 \mathrm{μs}\left({\color{gray}-3.020 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 10000 entities $$28.5 \mathrm{ms} \pm 232 \mathrm{μs}\left({\color{gray}1.44 \mathrm{\%}}\right) $$ Flame Graph

representative_read_entity

Function Value Mean Flame graphs
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/block/v/1 $$38.1 \mathrm{ms} \pm 341 \mathrm{μs}\left({\color{gray}0.155 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/book/v/1 $$36.9 \mathrm{ms} \pm 343 \mathrm{μs}\left({\color{gray}-1.103 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/building/v/1 $$37.4 \mathrm{ms} \pm 371 \mathrm{μs}\left({\color{gray}1.03 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/organization/v/1 $$36.1 \mathrm{ms} \pm 342 \mathrm{μs}\left({\color{lightgreen}-6.111 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/page/v/2 $$38.3 \mathrm{ms} \pm 356 \mathrm{μs}\left({\color{gray}-0.501 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/person/v/1 $$38.5 \mathrm{ms} \pm 394 \mathrm{μs}\left({\color{gray}2.26 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/playlist/v/1 $$36.4 \mathrm{ms} \pm 375 \mathrm{μs}\left({\color{gray}-3.673 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/song/v/1 $$34.9 \mathrm{ms} \pm 288 \mathrm{μs}\left({\color{gray}-4.679 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/uk-address/v/1 $$38.1 \mathrm{ms} \pm 372 \mathrm{μs}\left({\color{gray}-1.979 \mathrm{\%}}\right) $$ Flame Graph

representative_read_entity_type

Function Value Mean Flame graphs
get_entity_type_by_id Account ID: bf5a9ef5-dc3b-43cf-a291-6210c0321eba $$9.02 \mathrm{ms} \pm 66.2 \mathrm{μs}\left({\color{gray}-3.650 \mathrm{\%}}\right) $$ Flame Graph

representative_read_multiple_entities

Function Value Mean Flame graphs
entity_by_property traversal_paths=0 0 $$103 \mathrm{ms} \pm 745 \mathrm{μs}\left({\color{gray}-0.613 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=255 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true $$157 \mathrm{ms} \pm 770 \mathrm{μs}\left({\color{gray}-1.047 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false $$111 \mathrm{ms} \pm 625 \mathrm{μs}\left({\color{gray}0.733 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true $$121 \mathrm{ms} \pm 663 \mathrm{μs}\left({\color{gray}0.215 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true $$128 \mathrm{ms} \pm 657 \mathrm{μs}\left({\color{gray}-0.029 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true $$139 \mathrm{ms} \pm 707 \mathrm{μs}\left({\color{gray}1.31 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=0 0 $$109 \mathrm{ms} \pm 612 \mathrm{μs}\left({\color{gray}-1.844 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=255 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true $$139 \mathrm{ms} \pm 787 \mathrm{μs}\left({\color{gray}-1.825 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false $$116 \mathrm{ms} \pm 527 \mathrm{μs}\left({\color{gray}-2.115 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true $$127 \mathrm{ms} \pm 731 \mathrm{μs}\left({\color{gray}-1.261 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true $$128 \mathrm{ms} \pm 674 \mathrm{μs}\left({\color{gray}-1.483 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true $$128 \mathrm{ms} \pm 578 \mathrm{μs}\left({\color{gray}-0.956 \mathrm{\%}}\right) $$

scenarios

Function Value Mean Flame graphs
full_test query-limited $$186 \mathrm{ms} \pm 1.52 \mathrm{ms}\left({\color{red}12.9 \mathrm{\%}}\right) $$ Flame Graph
full_test query-unlimited $$146 \mathrm{ms} \pm 857 \mathrm{μs}\left({\color{lightgreen}-6.867 \mathrm{\%}}\right) $$ Flame Graph
linked_queries query-limited $$42.9 \mathrm{ms} \pm 193 \mathrm{μs}\left({\color{gray}-0.339 \mathrm{\%}}\right) $$ Flame Graph
linked_queries query-unlimited $$566 \mathrm{ms} \pm 1.20 \mathrm{ms}\left({\color{gray}-1.256 \mathrm{\%}}\right) $$ Flame Graph

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/apps > hash* Affects HASH (a `hash-*` app) area/apps area/infra Relates to version control, CI, CD or IaC (area) type/eng > frontend Owned by the @frontend team

Development

Successfully merging this pull request may close these issues.

2 participants