Added gift dimension to the web analytics data files#28854
Added gift dimension to the web analytics data files#28854jonatansberg wants to merge 5 commits into
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
Walkthrough
Suggested reviewers
🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
| Command | Status | Duration | Result |
|---|---|---|---|
nx run ghost:test:ci:integration |
✅ Succeeded | 2m 41s | View ↗ |
nx run ghost:test:integration |
✅ Succeeded | 2m 44s | View ↗ |
nx run ghost:test:legacy |
✅ Succeeded | 2m 51s | View ↗ |
nx run ghost:test:e2e |
✅ Succeeded | 2m 25s | View ↗ |
nx run-many --target=build --projects=tag:publi... |
✅ Succeeded | <1s | View ↗ |
nx run-many -t test:unit -p ghost |
✅ Succeeded | 30s | View ↗ |
nx run @tryghost/admin:build |
✅ Succeeded | 10s | View ↗ |
nx run-many -t lint -p ghost |
✅ Succeeded | 31s | View ↗ |
Additional runs (2) |
✅ Succeeded | ... | View ↗ |
💡 Verify your cache is correct by running tasks in a sandbox. Read docs ↗
☁️ Nx Cloud last updated this comment at 2026-06-30 23:50:50 UTC
jonatansberg
force-pushed
the
ber-3728-gift-links-analytics-datafiles
branch
from
b44b819 to
c890002
Compare
jonatansberg
changed the title
jonatansberg
force-pushed
the
ber-3728-gift-links-analytics-datafiles
branch
2 times, most recently
from
7a8290d to
84de5a6
Compare
jonatansberg
requested review from
9larsons,
EvanHahn,
cmraible and
troyciesco
as code owners
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 84de5a6d2c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| TOKEN "stats_page" READ | ||
| TOKEN "axis" READ | ||
|
|
||
| NODE gift_link_visits |
There was a problem hiding this comment.
This new endpoint is not included in TINYBIRD_PIPES in ghost/core/core/server/services/tinybird/tinybird-service.js (lines 47-72), and _generateToken only grants PIPES:READ scopes for that list. On sites using workspaceId/adminToken JWT auth, any Ghost server/UI call to api_gift_link_visits will be rejected even though the Tinybird datafile exists; add the pipe to the generated scope list before wiring consumers to it.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
This is valid and will need to be addressed at some point, or else Ghost Admin won't be able to access api_gift_link_visits.pipe. Doesn't need to be in this PR, but it will be required eventually.
| {% if defined(location) %} and location = {{ String(location, description="Location to filter on", required=False) }} {% end %} | ||
| {% if defined(pathname) %} and pathname = {{ String(pathname, description="Pathname to filter on", required=False) }} {% end %} | ||
| {% if defined(post_uuid) %} and post_uuid = {{String(post_uuid, description="Post UUID to filter on", required=False) }} {% end %} | ||
| {% if defined(gift) %} and gift != '' {% end %} |
There was a problem hiding this comment.
When api_kpis_v2 is called with only the new gift segment, this hit-level filter is calculated in pathname_pageviews, but finished_data only joins/uses that node when pathname or post_uuid is defined (lines 164 and 169). As a result, gift-only KPI responses fall back to b.pageviews from session-level data and count every pageview in sessions that had a gift hit, so a visitor who lands via a gift link and then browses other pages inflates gift pageviews; include defined(gift) in the condition that uses pathname_pageviews.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
I think this is valid — adding the endpoint tests should help confirm that.
Basically with this current implementation, this will return the total count of pageviews for any session that used a gift link, rather than the count of pageviews using gift links.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@ghost/core/core/server/data/tinybird/endpoints/api_gift_link_visits.pipe`:
- Around line 1-32: The new Tinybird pipe for gift link visits is missing from
the JWT scope allowlist, which will cause authenticated requests to fail. Update
the `TINYBIRD_PIPES` allowlist in `tinybird-service.js` to include
`api_gift_link_visits`, making sure the new `gift_link_visits` endpoint is
covered alongside the other pipe names used for JWT authorization.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: a5a82431-18c2-4aba-a797-86c7113df75d
📒 Files selected for processing (10)
ghost/core/core/server/data/tinybird/datasources/_mv_daily_pages.datasourceghost/core/core/server/data/tinybird/datasources/_mv_hits.datasourceghost/core/core/server/data/tinybird/endpoints/api_active_visitors_v2.pipeghost/core/core/server/data/tinybird/endpoints/api_gift_link_visits.pipeghost/core/core/server/data/tinybird/endpoints/api_kpis_v2.pipeghost/core/core/server/data/tinybird/endpoints/api_top_locations_v2.pipeghost/core/core/server/data/tinybird/endpoints/api_top_pages_v3.pipeghost/core/core/server/data/tinybird/pipes/filtered_sessions_v2.pipeghost/core/core/server/data/tinybird/pipes/mv_daily_pages.pipeghost/core/core/server/data/tinybird/pipes/mv_hits.pipe
jonatansberg
force-pushed
the
ber-3728-gift-links-analytics-datafiles
branch
from
84de5a6 to
79d0831
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 79d0831de3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| JSONExtractString(payload, 'member_status') as member_status, | ||
| JSONExtractString(payload, 'post_uuid') as post_uuid, | ||
| JSONExtractString(payload, 'post_type') as post_type, | ||
| JSONExtractString(payload, 'gift_link') as gift_link, |
There was a problem hiding this comment.
When these datafiles are deployed with the current Ghost tracker, this column is never populated: ghost_head only emits tb_site_uuid, tb_post_uuid, tb_post_type, tb_member_uuid, and tb_member_status, and the client only merges those tb_ attributes into page_hit payloads; I also found no other repo-wide writer for gift_link. As a result, every new gift_link != '' filter and api_gift_link_visits will return no rows for real gift-link traffic unless the gift-link token is added to the tracked payload before extracting it here.
Useful? React with 👍 / 👎.
cmraible
left a comment
cmraible
left a comment
There was a problem hiding this comment.
Can we add/update the Tinybird endpoint tests in ghost/core/core/server/data/tinybird/tests/*?
You'll need to update ghost/core/core/server/data/tinybird/fixtures/analytics_events.ndjson to add or update some rows to include a gift_link so there's some data to work with, then I'd like to see at least one test that includes the gift_link parameter as a filter for each modified endpoint + create host/core/core/server/data/tinybird/tests/api_gift_link_visits.yaml for the new endpoint.
The code looks okay to me, but it's really hard to review and be sure without the test coverage.
jonatansberg
force-pushed
the
ber-3728-gift-links-analytics-datafiles
branch
2 times, most recently
from
33611d2 to
5acdbc9
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5acdbc96b5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| {% if defined(location) %} and location = {{ String(location, description="Location to filter on", required=False) }} {% end %} | ||
| {% if defined(pathname) %} and pathname = {{ String(pathname, description="Pathname to filter on", required=False) }} {% end %} | ||
| {% if defined(post_uuid) %} and post_uuid = {{ String(post_uuid, description="Post UUID to filter on", required=False) }} {% end %} | ||
| {% if defined(gift_link) %} and gift_link != '' {% end %} |
There was a problem hiding this comment.
When this new gift_link predicate is inherited by api_top_pages_v2 in a tinybird:stats:version = v2 deployment, it only narrows filtered_sessions_v2 to sessions that had at least one gift hit; api_top_pages_v2.pipe then joins those sessions back to _mv_hits and groups all hits without an h.gift_link != '' predicate. A visitor who enters through a gift link and then opens another page will make that non-gift page appear in gift-filtered top content, so mirror the hit-level filter used in api_top_pages_v3/api_top_locations_v2 before exposing this segment.
Useful? React with 👍 / 👎.
jonatansberg
force-pushed
the
ber-3728-gift-links-analytics-datafiles
branch
from
5acdbc9 to
1fbc554
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1fbc554e71
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| 'api_top_devices_v2' | ||
| 'api_top_devices_v2', | ||
| // gift-link per-link usage | ||
| 'api_gift_link_visits' |
There was a problem hiding this comment.
When tinybird:stats.version is configured (for example v2), the normal Tinybird clients append that suffix to every requested endpoint (apps/admin-x-framework/src/utils/stats-config.ts:18 and ghost/core/core/server/services/stats/utils/tinybird.js:39-42), so a UI/server call for api_gift_link_visits becomes api_gift_link_visits_v2. This change only creates/scopes the unsuffixed pipe, so versioned stats deployments will hit a missing/unscoped endpoint unless the new pipe gets a versioned alias or callers explicitly bypass versioning.
Useful? React with 👍 / 👎.
ref https://linear.app/ghost/issue/BER-3746/integrate-gift-link-usage-tracking-with-analytics - gift-link usage is moving onto the existing analytics pipeline; this is the Tinybird side - _mv_hits gains a `gift_link` column (the gift-link token, extracted from the page_hit payload, kept in step with the materialised output ordering) - a `gift_link` segment is threaded through filtered_sessions_v2 and the v2 endpoints that filter _mv_hits directly, mirroring the existing post_uuid plumbing: omit it for all traffic, `gift_link=true` (or any truthy value) for gift-only, `gift_link=false`/`0` to exclude gift traffic - in api_kpis_v2, route gift-segmented queries through the hit-level pathname_pageviews node (defined(gift_link)) so gift pageviews count gift-link hits, not every pageview in a session that used a gift link - _mv_daily_pages (the daily rollup behind api_top_pages_v3) also gains `gift_link` as a dimension, so the segment works on the top-pages historical path instead of erroring on a missing column; non-gift queries merge across the gift_link split via uniqExactMerge and are unchanged - new api_gift_link_visits endpoint groups by gift_link for per-link visits / views / last_seen, registered in the tinybird-service JWT scope allowlist (TINYBIRD_PIPES) so authenticated Ghost calls to it are permitted - depends on the proxy passing gift_link through to Tinybird (separate PR); until then the column is empty and the segment matches nothing - endpoint tests: tag gift-link rows in fixtures/analytics_events.ndjson and add include (`gift_link=true`) and exclude (`gift_link=false`) cases to api_kpis_v2 / api_top_pages_v3 / api_top_locations_v2 / api_active_visitors_v2, plus a new tests/api_gift_link_visits.yaml. The kpis include case asserts gift pageviews count gift-link hits (2), not every pageview in a session that used a gift link (3)
jonatansberg
force-pushed
the
ber-3728-gift-links-analytics-datafiles
branch
from
1fbc554 to
188f09b
Compare
ref https://linear.app/ghost/issue/BER-3746/integrate-gift-link-usage-tracking-with-analytics - #28854 now resolves the gift_link param by value: 'false'/'0' selects traffic with no gift link ("not used"), any other value selects gift-link traffic ("used") - updates the filter option values from used/not_used to true/false so "not used" sends gift_link=false and returns the correct data instead of being treated as "used" - labels are unchanged ("used" / "not used"); only the underlying query value changed Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
ref https://linear.app/ghost/issue/BER-3746/integrate-gift-link-usage-tracking-with-analytics - web analytics always queries the version-suffixed endpoint (getStatEndpointUrl appends stats.version, and every analytics-enabled deployment runs v2), so the unversioned api_gift_link_visits pipe was never requested in any environment - renamed the pipe and its test to api_gift_link_visits_v2 and scoped only the v2 name, instead of shipping a base+v2 pair where the base is dead weight - the frontend hook still names api_gift_link_visits; the framework's version suffix resolves it to the v2 pipe
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3a280d251a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| // gift-link per-link usage (v2 only; web analytics always queries the | ||
| // version-suffixed endpoint, so the base name is never requested) | ||
| 'api_gift_link_visits_v2' |
There was a problem hiding this comment.
When tinybird:stats.version is unset, the browser helper leaves the endpoint name unsuffixed (apps/admin-x-framework/src/utils/stats-config.ts:17-18) and the server Tinybird client does the same (ghost/core/core/server/services/stats/utils/tinybird.js:39-42). This change only creates/scopes api_gift_link_visits_v2, so default/local stats configs without a version will request api_gift_link_visits.json and either 404 or be denied by the JWT scope once the per-link usage view is wired up.
Useful? React with 👍 / 👎.
ref https://linear.app/ghost/issue/BER-3746/integrate-gift-link-usage-tracking-with-analytics - api_top_pages / api_top_pages_v2 applied gift_link only via the filtered_sessions join, which selects whole sessions that had any gift hit and then counts all their pageviews — so the top content list was session-scoped and surfaced a gift session's non-gift pages (e.g. the homepage) under a gift-link filter - gift_link is a per-hit attribute like pathname, so apply it on the pipe's own hit scan (mirroring the pathname filter and api_top_pages_v3 / api_top_locations_v2), making views/visitors hit-scoped and consistent with the KPIs and every other gift-aware pipe - added gift_link include/exclude test cases proving a gift session's non-gift pageviews no longer appear; full datafiles suite passes 29/29
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: bd340367ec
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| 'api_top_devices_v2', | ||
| // gift-link per-link usage (v2 only; web analytics always queries the | ||
| // version-suffixed endpoint, so the base name is never requested) | ||
| 'api_gift_link_visits_v2' |
There was a problem hiding this comment.
When JWT Tinybird auth is enabled, adding this pipe to TINYBIRD_PIPES grants it to the same /tinybird/token response that only requires members.browse, while the normal gift-link API checks manage.gift_link(id) and edit.post(id) before exposing tokens. Because api_gift_link_visits_v2 returns raw gift_link values for the whole site with only the fixed site_uuid, any caller or integration that can obtain the stats token can enumerate gift-link tokens for posts it is not allowed to edit; keep this pipe out of the shared stats JWT or proxy it through a permission-checked endpoint.
Useful? React with 👍 / 👎.
ref https://linear.app/tryghost/issue/BER-3728/ Ghost uses the unsuffixed Tinybird stats endpoints by default, so gift-link analytics needs to be implemented on those live pipes as well as the experimental suffixed variants. The raw per-link usage pipe is intentionally left out of the shared stats JWT because it returns gift-link tokens and needs permission-checked access.
ref https://linear.app/tryghost/issue/BER-3728/ Gift-link analytics endpoints need JWT access in both unversioned and v2 Tinybird stats deployments.
2 participants
ref https://linear.app/ghost/issue/BER-3746/integrate-gift-link-usage-tracking-with-analytics
The Tinybird data-files slice of moving gift-link usage tracking onto the existing web-analytics pipeline. Independent of the producer/proxy PRs — can be reviewed and merged on its own (off
main).What this adds
1.
gift_linkcolumn on_mv_hits— extracted from thepage_hitpayload (JSONExtractString(payload, 'gift_link')), a plainStringlikepost_uuid. Added at the same position in bothmv_hits.pipe's materialised output and the_mv_hits.datasourceschema so the MV column mapping stays aligned. No existing consumer doesSELECT *from_mv_hits, so the new column is non-breaking.2. A
gift_linksegment (include / exclude) — a tri-state toggle mirroring the existingpost_uuidfilter plumbing: omitgift_linkfor all traffic, passgift_link=true(or any truthy value) for gift-only, orgift_link=false/0to exclude gift traffic. Renders as{% if defined(gift_link) %}{% if gift_link == 'false' or gift_link == '0' %} and gift_link = '' {% else %} and gift_link != '' {% end %}{% end %}(thepost_typepipes use the same param-value-conditional pattern). Threaded throughfiltered_sessions_v2(the shared chokepoint, so session-joined endpoints inherit it) plus the v2 endpoints that filter_mv_hitsdirectly:api_kpis_v2,api_top_pages_v3,api_top_locations_v2,api_active_visitors_v2. Lets operators segment "gift traffic" in the normal reports (Goal: segmentable). Inapi_kpis_v2, gift-only queries are routed through the hit-levelpathname_pageviewsnode so gift pageviews count gift-link hits, not every pageview in a session that merely used a gift link.3.
gift_linkon_mv_daily_pages—api_top_pages_v3is the one segmented endpoint that doesn't read_mv_hitsfor historical days; it reads the daily rollup_mv_daily_pages, which had nogift_linkcolumn, so the segment would have errored (UNKNOWN_IDENTIFIER) on any gift-filtered top-pages query.gift_linkis added as a dimension to the rollup (mv_daily_pages.pipeGROUP BY+ the datasource schema/sorting key). Non-gift queries are unchanged — they re-aggregate by(post_uuid, pathname)withuniqExactMerge, which collapses the gift split back to the same totals; gift queries selectgift_link != ''.4.
api_gift_link_visitsendpoint — groups bygift_link, scoped bysite_uuid(+ optionalpost_uuid/ date range), returningvisits = uniqExact(session_id),views = count(),last_seen = max(timestamp). Defaults to all-time, since a link's usage is its lifetime total. Feeds the per-link usage view (Goal: per-link tracking). Registered in theTINYBIRD_PIPESallowlist (tinybird-service.js) so JWT-authed Ghost calls to it aren't denied.Dependencies & sequencing
gift_link. Until the proxy lands,gift_linkis dropped before Tinybird, so the column is empty and the segment matches nothing — safe no-op._mv_hitsand_mv_daily_pagesto backfillgift_linkfor already-ingested rows is a follow-up populate (the rawpayloadretains it). Feature is unreleased, so no live audience.Not included
Tests
tb test run(the Tinybird endpoint tests). A fewfixtures/analytics_events.ndjsonrows are tagged with agift_linktoken (so unfiltered counts are unchanged), and each modified endpoint gets agift_link-filter case —api_kpis_v2,api_top_pages_v3,api_top_locations_v2,api_active_visitors_v2— plus a newtests/api_gift_link_visits.yaml. The kpis case pins the gift-only fix: gift pageviews count gift-link hits (2), not every pageview in a session that used a gift link (3). All 29 test files pass.v1 endpoints are left untouched (segment targets the v2 surface).