feat(#520): opt-in live XMR/XTM price feed over Tor + fiat estimates on the earnings card - #646
Conversation
…on the earnings card The deferred auto half of #520 (PR #614 shipped only the static prices). dashboard.energy.price_feed (default false) fetches both spot prices from CoinGecko in the operator's currency — always over the bridge Tor SOCKS, 15-min throttle, fail-silent, last-good-prices kept, static config prices as the fallback. The earnings card states which price is in use (source + age), and the Monero / Tari / XvB tabs grow ≈-fiat rows once a price is known. New egress surfaces in the #170 posture + topology; pithead's closed-schema energy validation gains the boolean key. Hardening from security review: non-finite prices rejected (a hostile NaN would break /api/state serialization) and the currency label must be plainly alphabetic before it may leave the host as a query parameter. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…unused api_url knob Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… state.json Both PRs evolved _gen_state.py: #642 added the recorded raffle win, this branch enriched the worker power blocks. The rebased generator carries both and state.json is its real output again. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
a977c33 to
b2e9e7d
Compare
|
Review record (in-session, second pass on top of the authoring session's verifier/security/ponytail runs): read the egress core myself — parse_prices non-finite gate and the currency exfil-guard regex are real and correctly ordered before the URL; the price-feed egress row sits under the (host-networked) dashboard component and the topology carries it; pithead's closed-schema key list + boolean validation extend correctly, config.reference.json keeps the superset invariant. Rebased onto post-v1.9.0 develop: the fixture-generator conflict with #642 is resolved by merging BOTH changes (raffle win + enriched power blocks) and regenerating state.json from the generator — 208/208 frontend, full make test green, patch coverage 98%. Tracking issue #651 filed and linked (the deferred auto half of #520 had none). Known residual (response-size cap shared with the update-checker/XvB clients) stays a follow-up. Rides v1.9.1. |
… API fetches (#686) * security(#660): shared response-size cap for every external HTTP fetch All five external call sites — the GitHub release check (#224), the three XvB reads (stats, reward estimates, winners), and the CoinGecko price feed (#651) — read via a shared helper/http.py bounded_get that streams the body and cuts it at 1 MiB. Over-cap raises a RequestException subclass, so each client's existing fail-silent contract (None / keep last good) applies unchanged. Known residual from the #642/#646 security reviews; the local Telegram/healthchecks/monerod clients keep their own contracts and stay out of scope. Closes #660 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * security(#660): review fixes — scope the claim to the five capped clients, pin the at-cap boundary, widen the drift guard The changelog headline claimed "every external dashboard fetch" is size-capped, but the Tor heal probe, Telegram long-poll, and healthchecks ping keep their own contracts — the entry now names the actual scope: the update check, the price feed, and the XvB client's calls. A new tier-1 test pins the strictly-greater cap (a body of exactly max_bytes succeeds; a regression to >= fails it), and the wiring drift guard now rejects any direct requests.<verb>( in the client modules, not just requests.get(. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * style: ruff format on the at-cap boundary test Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* security(#660): shared response-size cap for every external HTTP fetch All five external call sites — the GitHub release check (#224), the three XvB reads (stats, reward estimates, winners), and the CoinGecko price feed (#651) — read via a shared helper/http.py bounded_get that streams the body and cuts it at 1 MiB. Over-cap raises a RequestException subclass, so each client's existing fail-silent contract (None / keep last good) applies unchanged. Known residual from the #642/#646 security reviews; the local Telegram/healthchecks/monerod clients keep their own contracts and stay out of scope. Closes #660 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(#596): per-worker RigForge new-version badge One fleet-wide, hourly, Tor-routed fetch of the latest RigForge release (UpdateChecker.latest_release_cached, gated on the same dashboard.check_for_updates flag) cached as raw {tag, url}; each rig's badge is derived at the render seam from its live reported version (rigforge_update_for), so derived state never outlives its inputs (#664) and a rig on X can never badge X available. compute_update normalizes the rig's bare 1.11.2 against the tag's v1.11.2. Badges render in the Workers Alive table and Worker Inspect; rigs reporting no version show no badge (unknown, not up-to-date). Closes #596 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(#596): review fixes — snapshot restore drops rigforge_release; docstring + catalog row - Snapshot restore popped `update` (#664) but not `rigforge_release`, so with check_for_updates flipped off a restored snapshot re-seeded stale per-worker badges until the first poll cycle. Pop it alongside `update`; the checker re-fetches on its own cadence. Extends the #664 restore test (fails without the pop). - update_checker docstring said the check is opt-in (default off) — the config default and docs/configuration.md say on by default. Docstring now matches. - testing-strategy scenario catalog (Section I) gains the #596 badge row: tier 1 covered today, live badge owed to the #597 gouda loaner session. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
…checks ping, Telegram getUpdates (#699) * security(#660): shared response-size cap for every external HTTP fetch All five external call sites — the GitHub release check (#224), the three XvB reads (stats, reward estimates, winners), and the CoinGecko price feed (#651) — read via a shared helper/http.py bounded_get that streams the body and cuts it at 1 MiB. Over-cap raises a RequestException subclass, so each client's existing fail-silent contract (None / keep last good) applies unchanged. Known residual from the #642/#646 security reviews; the local Telegram/healthchecks/monerod clients keep their own contracts and stay out of scope. Closes #660 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * security(#660): review fixes — scope the claim to the five capped clients, pin the at-cap boundary, widen the drift guard The changelog headline claimed "every external dashboard fetch" is size-capped, but the Tor heal probe, Telegram long-poll, and healthchecks ping keep their own contracts — the entry now names the actual scope: the update check, the price feed, and the XvB client's calls. A new tier-1 test pins the strictly-greater cap (a body of exactly max_bytes succeeds; a regression to >= fails it), and the wiring drift guard now rejects any direct requests.<verb>( in the client modules, not just requests.get(. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * security(#698): bound the remaining external GETs — Tor probe, Healthchecks ping, Telegram getUpdates The #660 cap now covers every external GET. The Tor egress probe and the Healthchecks ping are one-word conversions: both already treat any RequestException as failure, so an over-cap body follows the existing contract. The Telegram long-poll keeps its (connect, read) timeout tuple — bounded_get streams, but headers only arrive once the hold ends, so the read timeout still spans it — and gains a getUpdates limit of 10: the offset only advances after a batch is parsed, so an uncapped batch that tripped the size cap would be re-fetched forever and wedge the poll loop. Offset priming drains batch by batch to keep skipping the whole startup backlog. BoundedResponse grows the raise_for_status the long-poll already called, raising HTTPError to match requests' contract. Closes #698 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * style: ruff format on the at-cap boundary test Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(#698): pin the long-poll (connect, read) timeout tuple Issue #698's first hold-while-converting invariant — the read timeout must outlast Telegram's long-poll hold — had no test: dropping the tuple would abort every legitimate poll mid-hold and nothing would fail. Assert it behaviourally (read > requested hold) in the existing getUpdates wiring test instead of pinning the exact numbers. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* security(#660): shared response-size cap for every external HTTP fetch All five external call sites — the GitHub release check (#224), the three XvB reads (stats, reward estimates, winners), and the CoinGecko price feed (#651) — read via a shared helper/http.py bounded_get that streams the body and cuts it at 1 MiB. Over-cap raises a RequestException subclass, so each client's existing fail-silent contract (None / keep last good) applies unchanged. Known residual from the #642/#646 security reviews; the local Telegram/healthchecks/monerod clients keep their own contracts and stay out of scope. Closes #660 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(#596): per-worker RigForge new-version badge One fleet-wide, hourly, Tor-routed fetch of the latest RigForge release (UpdateChecker.latest_release_cached, gated on the same dashboard.check_for_updates flag) cached as raw {tag, url}; each rig's badge is derived at the render seam from its live reported version (rigforge_update_for), so derived state never outlives its inputs (#664) and a rig on X can never badge X available. compute_update normalizes the rig's bare 1.11.2 against the tag's v1.11.2. Badges render in the Workers Alive table and Worker Inspect; rigs reporting no version show no badge (unknown, not up-to-date). Closes #596 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(#597): one-click remote worker upgrade Fuses the two existing control templates: worker-apply's rig resolution (address + bearer from the HOST config, never the intent; charset guards; per-drain budget — one upgrade per drain) and the stack upgrade's throttled host-side target re-derivation (RigForge releases API over Tor, 10-min tag cache + pre-dial stamp so a flooding container costs at most one egress per window). POST :8082/upgrade with the host-derived tag; poll :8082/status matching change_id to applied/rolled_back/failed with a 12-min cap (builds ~10 min); rig-side throttle refusals map to a calm 'throttled' retry-later status. Dashboard: POST /api/control/worker-upgrade spools {worker, version} only and 202s immediately; a rig already reporting the target version no-ops without spooling (never burns the rig's 6h window). Inspect gains an arm→confirm Upgrade button where the #596 badge shows. Requires rig >= v1.11.2 (documented; earlier ancestry guard refuses fresh-clone upgrades). Per-rig only — no upgrade-all. Closes #597 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(#596): review fixes — snapshot restore drops rigforge_release; docstring + catalog row - Snapshot restore popped `update` (#664) but not `rigforge_release`, so with check_for_updates flipped off a restored snapshot re-seeded stale per-worker badges until the first poll cycle. Pop it alongside `update`; the checker re-fetches on its own cadence. Extends the #664 restore test (fails without the pop). - update_checker docstring said the check is opt-in (default off) — the config default and docs/configuration.md say on by default. Docstring now matches. - testing-strategy scenario catalog (Section I) gains the #596 badge row: tier 1 covered today, live badge owed to the #597 gouda loaner session. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(#597): pin the poll-cap timeout, GitHub tag derive, and rig-refusal paths Review fixes for #691. The sec-review's headline 90s poll cap gains a test seam (CONTROL_WU_POLL_CAP) and a stack case proving the timeout→accepted fallback; three new cases cover the GitHub tag derivation (fresh derive + cache write, unreachable-over-Tor refusal, no-usable-tag refusal) that every accept case had pre-cached away; one covers the rig non-202 refusal (where an old rig < v1.11.2 surfaces) incl. the 500-char cap on rig-supplied error text. testing-strategy Section I gains the #597 catalog row. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(#597): pin the non-JSON 400 and submit-failure 500 endpoint branches The develop merge narrowed the patch-coverage diff to this PR's own lines, surfacing the two uncovered error branches the review had flagged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Closes #651 (the tracking issue for the deferred auto half). Refs #520 (closed by #614 with the static half).
What
dashboard.energy.price_feed(default false): fetch the XMR + XTM spot prices live from CoinGecko in the operator'scurrency, replacing the staticxmr_price/tari_pricenumbers. Newservice/price_feed.pyfollows the Dashboard: "new release available" badge linking to the GitHub release (notify-only, no upgrade) #224 update-checker shape: 15-min self-throttle, fail-silent, keeps the last good prices on failure; the static config prices are the fallback until the first fetch lands. CoinGecko is the one keyless source quoting both coins in a single call (CoinDesk has no XTM; Yahoo has no stable free JSON API); Tari's CoinGecko id isminotari, verified live.Prices:footer on the earnings card —XMR USD 333.97 · XTM USD 0.000400 — live from CoinGecko over Tor, updated 12m ago/static, set in config.json/ feed-waiting.build_energypublishesprice_sourceprovenance.formatFiatPrice, no "USD 0.00").Privacy (#160/#270/#520 constraints)
socks5h, same route as the update check / XvB stats) — CoinGecko sees a Tor exit, never the host IP. No clearnet branch exists.Hardening (from the security-review pass)
parse_pricesrejects non-finite values:json.loadsacceptsNaN/Infinitytokens andNaN <= 0is False, so a hostile response could otherwise push invalid JSON into/api/stateand stall the whole dashboard.currencylabel (free-form printable ASCII, dashboard-committable via [Feature] Set electricity cost (dashboard.energy) from the dashboard, not just config.json #504) must be plainly alphabetic ([A-Za-z]{2,5}) before it may leave the host as a query parameter — a committed label can't become an exfil channel through the feed URL. Anything else: no request, static fallback.Tests
test_price_feed.py(parse incl. NaN/exfil guards, Tor proxying, throttle/fallback), config loader,build_energyprice-source matrix, egress posture,_sync_prices.coinFiat/formatFiatPrice/priceSourceLabelunits + EarningsCard render tests for the fiat rows and provenance line (203 pass).validate_energy_configgains price_feed cases (boolean-only rejection, closed-schema key list, valid block applies)._gen_state.pynow gives its workers real power data, so the frontend fixture regenerates honestly instead of the hand-edited energy block feat(#520): fold Tari revenue into the energy/profit net calculation #614 left behind.make lint/make test/make test-patch-coverage(90%) all green; adversarial verifier pass clean; ponytail review applied (dropped an unused constructor knob).Docs
configuration.md (new key + amended price rows), dashboard.md (price sources + fiat rows), privacy.md (egress table row), README feature bullet, and every stale "no price feed ships" claim removed (code docstrings, UI hint, pithead comment).
Follow-up candidates: response-size cap shared across the three external HTTP clients;
price_feededitability from the dashboard config editor (needs the #504 editable-keys treatment).🤖 Generated with Claude Code