What the fleet costs and what it returns, in the terminal - #240
Conversation
`crawlproof dashboard` (also roi, tui) is a live hqtui dashboard over three feeds that have never been in the same place: the tracker for who arrived, the ad network for what was delivered, and CoinPay for what the bank actually did. Five screens — ROI, Traffic, Ads, Money, Spend — refreshing on a timer, plus --json for the same snapshot without a terminal. Two rules run through the arithmetic, because breaking either produces a flattering number that is false: Self-deal is not revenue. We advertise on our own slots, so ad spend and ad earnings are one dollar moving between two pockets. They are reported under Internal and counted as neither cost nor revenue. Personal money is not fleet cost. The bank feed carries groceries next to servers, so cost is the business scope only, joined from each ledger row's account to that account's effective_scope. Everything is normalised to a monthly rate and prorated onto the traffic window, because burn is a rate and the traffic side can be asked for an hour while the bank side only answers in weeks. Two token-authed reads to feed it: GET /api/ads/v1/earnings — the model /dashboard/ads/earnings already builds. Without it a client wanting fleet totals needs one request per campaign, and the account is past 170 of them. GET /api/tracker/v1/sites — /stats already names every project, but only inside the 400 it returns when the caller does not say which. A client should not have to parse an error message to find the fleet. loadEarnings now filters ad_campaigns, ad_slots, ad_ledger and ad_payouts by owner_id explicitly rather than leaning on RLS. For the dashboard that is a no-op narrowing of what RLS already allows; for the new route, which passes the service client, it is the security boundary. The dashboard reports what it cannot know as loudly as what it can: a site that did not answer is missing rather than zero, a vendor list built from one page of a longer ledger says so, and a fleet whose visits run 200x its pageviews — or whose average is really one busy site — says that too, next to the number. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HvWJ4336pxTFRdRbvsTQeD
ThreatCrush Security Scan39 finding(s) HIGH/CRITICAL: 2 | MEDIUM: 28 | LOW: 9
Snippets are redacted; ThreatCrush never prints matched credential material. |
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
|
Warning Review the following alerts detected in dependencies. According to your organization's Security Policy, it is recommended to resolve "Warn" alerts. Learn more about Socket for GitHub.
|
The revenue fix, the delivery fix and `crawlproof ad <url>` landed in #243, but the published version did not move with them: `crawlproof update` fetches @latest, so until this ships every box keeps reinstalling 0.1.0 and the dashboard keeps reporting a lifetime total as a monthly rate. This was pushed to #243's branch before the merge and the squash did not carry it, the same way #240 dropped bin/crawlproof.mjs. Landing it on its own. Claude-Session: https://claude.ai/code/session_01HvWJ4336pxTFRdRbvsTQeD Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 participant
crawlproof dashboard(aliasesroi,tui) — a live hqtui dashboard over the three feeds that have never been in the same place: the tracker for who arrived, the ad network for what was delivered, and CoinPay for what the bank actually did. Five screens (ROI, Traffic, Ads, Money, Spend), refreshing on a timer.--jsongives the same snapshot without a terminal.The two rules in the arithmetic
Both exist because breaking either produces a flattering number that is false.
Self-deal is not revenue. We advertise on our own slots, so ad spend and ad earnings are one dollar moving between two pockets. They are reported under Internal and counted as neither cost nor revenue.
Personal money is not fleet cost. The bank feed carries groceries next to servers. Cost is the business scope only, joined from each ledger row's
account_idto that account'seffective_scope— a ledger row names an account, not a scope, so both halves are required.Everything is normalised to a monthly rate and prorated onto the traffic window. Burn is a rate, and it is the only way the traffic side can be asked for an hour while the bank side only answers in weeks.
Two token-authed reads to feed it
GET /api/ads/v1/earnings?days=— the model/dashboard/ads/earningsalready builds. Without it, a client that wants fleet totals needs one request per campaign, and the account is past 170 of them.GET /api/tracker/v1/sites—/statsalready names every project, but only inside the 400 it returns when the caller does not say which. A client should not have to parse an error message to find the fleet.Fleet traffic is a client-side fan-out (one
/statscall per site, concurrency-capped at 8) rather than a server-side aggregate. Summing 52 projects inside one serverless request is how the tracker RPCs have timed out before; a slow client is a much better failure than a route that 504s for everyone. Measured at ~5s for 52 sites.Security
loadEarningsfiltered nothing by owner and relied on RLS. The new route passes the service client, which has no RLS, so it now filtersad_campaigns,ad_slots,ad_ledgerandad_payoutsbyowner_idexplicitly. For the existing dashboard that is a no-op narrowing of what RLS already allows; for the new route it is the security boundary.ad_ledgeris indexed on(owner_id, kind), so the filter is free.It says what it cannot know
A site that did not answer is missing, not zero. A vendor list built from one page of a longer ledger says so. A fleet whose visits run 200× its pageviews, or whose average is really one busy site, says that next to the number — a "visitor" is any non-crawler hit, which on a site with a machine-readable endpoint runs orders of magnitude above pages anyone read, so a per-pageview figure sits beside the per-visitor one.
Verified against live data
Ran end-to-end against a local server on the production environment, all 52 sites, all three feeds, zero source errors. All five screens captured through a pty harness. Real 1m answer: $4,448/mo business burn against $131/mo revenue, ROI −97%, 1.13 months of cover, $1.94 per pageview. Top vendors: OpenAI $720, DigitalOcean $603, Anthropic $337, Turso $308, Railway $279.
Full suite green: 2,129 tests (2,106 existing + 23 new),
tsc --noEmitclean.Notes
COINPAY_API_URLis the site origin in this repo but the CoinPay SDK's base must include/api. The CLI normalises either form — the failure otherwise is an HTML page parsed as JSON, which names neither cause.~/.coinpay.json, whichcoinpay auth loginwrites, orCOINPAY_SESSION_TOKEN). Without one the traffic and ads screens still work and the money panels say what is missing.npm run lintis broken on this branch and on master alike (next lintpasses "lint" as a directory). Not touched here.🤖 Generated with Claude Code
https://claude.ai/code/session_01HvWJ4336pxTFRdRbvsTQeD