chore(migrate): refresh pricing caches + weekly strict staleness workflow (stacked on #151) - #155
Merged
Conversation
added 2 commits
July 19, 2026 16:09
…t Two zero-dep TS tools (Node 24 type-stripping, same pattern as the existing validators), both wired into the lint aggregate: fixtures:check -- the replay fixture sets are the plugin's regression harness, and they rot in ways the build never saw: capture manifests referencing files that were never committed (a repo-root .gitignore build/ rule once silently swallowed three canned .next manifests), seed .phase-status.json files lagging behind the phases a skill declares, and asserters carrying syntax newer than the ambient python. Now CI checks: every fixture JSON parses, every asserter ast-parses (no bytecode litter), every manifest captures[]/api[] entry with status ok resolves to a file on disk (failed/skipped may be absent) and build.files exist, nothing under fixtures/ is gitignored, and every seed phase list exactly matches the owning skill's declared _phase set. Validated against all four in-flight fixture sets (heroku/gcp/vercel x2) -- and it immediately caught a real one: the vercel seed omitted the declared scaffold phase (fixed on that branch as d11db7b). pricing:staleness -- every pricing cache declares its own freshness contract (_meta.last_updated + staleness_days, or a Last updated line with a documented 30-day window) but nothing enforced it; caches quietly crossed their own thresholds and every estimate degraded to fallback accuracy. Warn-only in build (a stale cache must not fail unrelated PRs); --strict exits 1 for a scheduled freshness workflow. Currently reports 3/4 caches stale, which is the point.
…low All caches had crossed their own declared freshness windows (the new pricing:staleness tool reports 3/4 stale), so every estimate was degrading to cached_fallback accuracy. Refresh: spot-verified the highest-traffic rates against published pricing pages -- Fargate Linux/x86 us-east-1 per-second rates convert exactly to the cached $0.04048/vCPU-hr and $0.004445/GB-hr, and Heroku dyno flat rates (Eco $5 flat, Basic $7, Standard-1X $25, Standard-2X $50, Performance-M $250, Performance-L $500) match devcenter -- no rate changes needed, so this is a verified date bump with provenance recorded in _meta.last_verification / the cache header. Vendored copies synced. New .github/workflows/pricing-staleness.yml: weekly scheduled run of the zero-dep staleness tool with --strict (build keeps the warn-only mode so a stale cache never fails unrelated PRs; the schedule makes staleness a visible failure someone owns, prompting a awslabs#131-style data refresh).
This was referenced Jul 20, 2026
icarthick
added a commit
to herosjourney/startups
that referenced
this pull request
Jul 22, 2026
Clean merge (no conflicts) bringing the branch to current main. The two new zero-dep CI tools validated against main's current fixture sets: - fixtures:check OK (139 json, 9 asserters, 3 manifests / 51 refs, 10 seeds) - pricing:staleness reports 1/5 stale (heroku cache) — WARN-ONLY in build by design; awslabs#155 (stacked on this) does the actual refresh. Full mise build green.
Brings the branch to current main (after awslabs#151's staleness tool merged). awslabs#155's cache refresh now takes effect: pricing:staleness reports 5/5 fresh (was 1/5 stale), --strict exits 0, and the new weekly .github/workflows/pricing-staleness.yml is present. Also shared:sync'd vercel's vendored aws-infra-pricing.json to canonical (it lacked the new _meta.last_verification provenance field awslabs#155 adds). shared:check OK (4 trees), full build green.
icarthick
added a commit
to herosjourney/startups
that referenced
this pull request
Jul 22, 2026
Resolves conflicts after the workshop stack + awslabs#149/awslabs#151/awslabs#155/awslabs#162 landed: - heroku-to-aws SKILL.md + heroku-live-capture/README.md: took main (stale awslabs#148/awslabs#150 lineage; main has the merged workshop/what-if + Scenario C content). - gcp-live-capture/README.md: kept THIS branch's added 'Scenario C — derived GCP baseline' block (its genuine new content; main's side empty there). - migrate/README.md: took main (adds vercel + agent-advisor reqs; awslabs#149's gcloud GCP requirement lines already present above the conflict). New gcp-infra-pricing-cache.md auto-enrolls in pricing:staleness (6/6 fresh) and the seed-baseline fixtures pass fixtures:check (141 json / 10 asserters). Full build green.
icarthick
added a commit
to herosjourney/startups
that referenced
this pull request
Jul 22, 2026
Union-resolved mise.toml lint group: fixtures:assert (this PR) + fixtures:check + pricing:staleness (from awslabs#151/awslabs#155, now on main) all run in build. Registered the two gcp-live-capture asserters that landed via awslabs#149/awslabs#156 after this branch was cut (the PR's own registry-completeness guard correctly flagged them): - gcp-live-capture/check_expected_baseline.py (smoke — already exits 1 cleanly) - gcp-live-capture/check_expected_drift.py (smoke — added the same missing-artifact guard this PR applies to the other 4, replacing a FileNotFoundError traceback with a clean one-line FAIL + exit 1) fixtures:assert now 10/10 (3 golden, 7 smoke). Full build green.
5 tasks
icarthick
added a commit
that referenced
this pull request
Jul 23, 2026
Per product decision, remove the vercel-to-aws migration skill entirely. This is a forward-removal (not a git-revert): vercel landed via shared/cross-skill commits (#153/#157/#162/#155/#152/#176) that also carried heroku/gcp/sidebar work, so reverting those merge commits was not viable. Removed: - skills/vercel-to-aws/ (full skill, 57 files incl. its references/vendored/ copies) - fixtures/vercel-capture/ + fixtures/vercel-workshop/ (replay fixture sets) - Orphaned vercel report validator + its test + golden fixtures (lived at plugin level, not under the skill dir): scripts/validate-assessment-report.py, tests/test_validate_assessment_report.py, fixtures/assessment-report-{reference,stub}.html, fixtures/tier1-signals-reference.json (vercel prescan fixture, no consumers) - Deregistered from all 3 plugin manifests (.claude/.codex/.cursor-plugin): dropped vercel/next-js/nextjs/opennext keywords, the 'Vercel' mentions in descriptions, and the codex 'Assess my Vercel migration' example prompt - Stripped vercel from both READMEs (intro bullets, triggers, Vercel→AWS mapping table, trigger-table row, requirement bullet, DSL-architecture section: 'five skills'→'four', removed the resumability-ledger note) + fixtures/README registry line - Dropped vercel asserter entries from tools/run-asserters.py (10→6 asserters) - Stripped stale vercel cross-refs: heroku SKILL.md workshop-arch note, gcp schema-workshop-scenarios arch-defaults line, canonical workshop-invariants {SKILL_LABEL}/{INVENTORY} placeholders (shared:sync propagated to vendored copies) PRESERVED (heroku + gcp depend on these — #157's shared architecture): canonical skills/shared/workshop/workshop-invariants.md + skills/shared/estimate/pricing-mode.md and their heroku/gcp vendored copies. shared:check drops 4→3 trees cleanly (the sync tool auto-discovers consumers). Left legit non-skill refs: gcp's vercel.json detection signal, 'Vercel AI SDK' framework name, historical PR-context notes. Verified: shared:check OK (3 trees byte-identical), fixtures:assert 6/6, all validators green (heroku/gcp/agent-advisor), dprint + markdownlint clean, full mise run build green.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
--strict.Motivation: every cache had quietly crossed its own declared freshness window (#151's warn-only check reports 3 of 4 stale on main), so every estimate — including both fresh-agent estimate replays run this week — silently degraded to
cached_fallbackaccuracy. Nothing owned the refresh.Changes
Cache refresh (verified)
_meta.last_verificationand the markdown cache header. Vendored copies synced.pricing-cache.md(2026-07-17) was already inside its window; untouched.Weekly strict gate
.github/workflows/pricing-staleness.yml: scheduled Monday run of the zero-dep tool with--strict(checkout + node 24, no installs). Build keeps warn-only so a stale cache never fails unrelated PRs; the schedule makes staleness a visible failure someone owns, prompting the next fix: update Bedrock pricing cache and model lifecycle for July 2026 #131-style refresh.Relationship to other PRs
feat/fixtures-integrity-check) — the workflow runs thepricing-staleness.tstool that PR adds. Only the last commit (5d3140a) is new here.Test plan
mise run buildgreen on the branchpricing:stalenessreports 4/4 caches fresh after the refresh;--strictexits 0--strictexits 1 against main's stale state (the workflow would correctly fail today without this refresh)