-
Notifications
You must be signed in to change notification settings - Fork 0
roadmap and task breakdown
Thinking...
I'm mapping out a phased approach to enhance their Content-Finder tool, breaking it into discrete features that can be built incrementally with Claude Code while respecting the token constraints of a Claude Pro session.
For each stage, I'm aiming for self-contained, shippable units—roughly 200-600 lines of code changes across 1-3 files with accompanying tests—that can be completed in 1-2 focused coding sessions. The stages build on the existing Python CLI structure and ladder toward the broader vision of a learning-enabled briefing system with governance capabilities.
I'm organizing this into phases: first, foundation hardening to modularize and stabilize the core (refactoring the monolithic content_finder.py, adding config management, persistence, and structured logging), then moving into Phase 1 for feature enhancements like better source handling and scoring refinement.
Phase 2 focuses on synthesis quality—structured summaries, prompt versioning, and theme taxonomy—while Phase 3 tackles the newsletter consolidation pain point with source discovery and an unsubscribe dashboard. Phase 4 closes the loop with evaluation harnesses and feedback mechanisms to drive continuous improvement.
For Phase 5, I'm planning the output surfaces: an HTML renderer with collapsible themes, Markdown export for Obsidian integration, and static site publishing via GitHub Pages. I should also layer in token-budget guidance for each stage—roughly one Claude Code session per stage, keeping diffs under 500 lines of code plus tests, and using a cached project context file to avoid re-priming on each session. The key is scoping with planning mode first, then implementing incrementally.
This plan turns your current single-file content_finder.py into the broader briefing/learning system from the prior PRD, but broken into self-contained stages. Each stage is sized so it can be planned, implemented, tested, and merged inside roughly one Claude Code session on the Pro plan (i.e., one 5-hour usage window, often less).
I'll cover: working principles for staying within Pro limits, a CLAUDE.md you should land first, then the staged roadmap (Phase 0 → Phase 5) with deliverables, tests, and a token-budget hint per stage.
Pro's usage caps reset on a rolling ~5-hour window, so the goal is to make each stage fit in one window with headroom. A few habits that compound:
- Plan before coding, in plan mode. Use Claude Code's plan mode (or a manual "draft a plan first" turn) to produce a written task list before any file edits. This is cheap context-wise and prevents expensive rework. Reject any plan that touches more than ~3–4 files or exceeds ~500 lines of net change.
-
Land a
CLAUDE.mdearly. A short project guide drastically cuts the tokens Claude needs to re-orient each session. See the template below. - Keep stages PR-sized. One stage = one branch = one PR = one feature flag if needed. If a stage starts ballooning, split it. Smaller PRs are also easier to revert if Claude introduces a regression.
- Pre-write tests, then implement. Ask Claude to write the test cases first from the spec, get them red, then implement until green. This forces concreteness and reduces wandering.
- Cache the system prompt for synthesis calls. Once you start using prompt caching for the LLM-synthesis step (Phase 2), your runtime token cost drops too — separate concern from your coding-time token use, but both matter.
- Use the cheapest model that works for boilerplate. For mechanical refactors, scaffolding tests, regex work, and YAML edits, Claude Code's smaller model is fine. Save the larger model for design-heavy stages (synthesis prompts, taxonomy work, dedup logic).
-
Snapshot context with summaries. At the end of each session, ask Claude to write a 10-line "what changed and what's next" note into
docs/sessions/. The next session re-reads that instead of the whole repo.
Before Stage 0.1, spend ~15 minutes of one session creating CLAUDE.md at the repo root. This is the highest-ROI thing you can do for token efficiency. It should contain: project goal in two sentences, current architecture diagram in ASCII, the public CLI surface, the test command, the lint command, the file layout, and a "do not change without asking" list (e.g., the exact section names in the synthesized brief).
Token budget: trivial (<10% of a session).
The current script is fine for one user but every later feature gets cheaper if you split the script into modules and add persistence first. Resist adding features until Phase 0 is done.
What. Split the single file into a package: contentfinder/sources.py, scoring.py, dedup.py, synth.py, render.py, cli.py. Keep content_finder.py as a thin shim that imports from the package so existing cron jobs don't break.
Why first. Every later stage edits one of these concerns; keeping them isolated keeps Claude's context windows small (it only needs to read one module per stage).
Deliverables. New package; __init__.py re-exports; existing CLI behavior unchanged; updated README.md file map.
Tests. Snapshot test on the "plain ranked list" output for a fixture set of feeds — should be byte-identical before/after refactor (or diff-only on ordering).
Token budget: ~½ session if you let Claude do it module-by-module with one commit per module.
What. Move RSS_SOURCES, HN_QUERIES, KEYWORD_WEIGHTS, and the trusted-source weights into config/sources.yaml and config/scoring.yaml. Add a small loader with schema validation (pydantic or dataclasses + manual validation).
Why. Source/weight changes shouldn't require code edits. This also unlocks Phase 3, where the "user newsletter list" is just another YAML.
Deliverables. YAML files, loader, schema, migration of current values, doc snippet on adding a source.
Tests. Parse-and-validate test for each YAML; test that bad weights fail closed.
Token budget: ~⅓ session.
What. Introduce data/contentfinder.db with three tables to start: sources, items, runs. Each item gets a stable hash id, fetched-at, source id, raw fields, and a JSON column for extracted text and metadata. Each run records the timestamp, args, item ids included, and (later) prompt hash.
Why. Without persistence you can't do dedup across days, trend detection, credibility scoring, or evals. Everything from Phase 1 onward assumes this exists.
Deliverables. Schema migration (use plain sqlite3 + a tiny migration runner; don't add an ORM yet), repository functions, integration into the fetch path.
Tests. In-memory SQLite tests for insert/upsert/idempotency; test that re-running the same day doesn't duplicate rows.
Token budget: ~1 session. This is the largest Phase 0 stage; consider splitting schema and repo functions into two PRs if context gets tight.
What. Replace print debug with the logging module, JSON formatter, and write a per-run summary file under data/runs/<date>.json containing counts, timings, error list, and source health.
Why. You'll need this signal for source credibility (Phase 4) and for diagnosing dead feeds (Phase 1).
Deliverables. Logger config, run-summary writer, tiny runs CLI subcommand to print recent runs.
Tests. Verify run-summary file shape; verify error counters increment on simulated fetch failures.
Token budget: ~⅓ session.
The current scoring and dedup are good for ~10 sources but will get noisy as you add the user's newsletters. This phase makes the pipeline robust to dozens of heterogeneous sources.
What. Define a SourceAdapter protocol with fetch() -> Iterable[RawItem]. Implement adapters for: RSSAdapter, HNAdapter (existing logic moved), SubstackAdapter (a thin specialization of RSS that knows the /feed convention), and WebPageAdapter (uses trafilatura to extract from a homepage when no feed is declared).
Why. Phase 3's newsletter mapping will register sources of any of these types; the synth layer shouldn't care which.
Deliverables. Protocol, four adapters, registry that constructs adapters from sources.yaml.
Tests. Per-adapter unit tests with recorded HTTP fixtures (use responses or vcrpy). Crucially: a malformed-feed test that asserts the pipeline keeps going.
Token budget: ~1 session. Consider splitting if you add vcrpy cassettes (cassette generation can take a chunk of tokens).
What. Send If-None-Match / If-Modified-Since based on the last fetch; track per-source last_success_at, consecutive_failures, and surface a health flag (green/yellow/red) in the run summary.
Why. Required to scale to many feeds without hammering them; required for the unsubscribe dashboard.
Deliverables. Etag/Last-Modified columns on sources, conditional-fetch logic, health computation.
Tests. Simulate 304 response and assert no item ingestion; simulate three consecutive failures and assert source flips to red.
Token budget: ~½ session.
What. Add a canonical_url function (strip UTM, fragments, mobile prefixes, trailing slashes) and a SimHash over the extracted text. Replace existing dedup with a two-stage check: exact canonical URL match, then SimHash within a 72-hour window.
Why. The current ranked list often shows multiple takes on the same story; collapsing them is the single biggest perceived-quality win.
Deliverables. dedup.py with both stages, configurable Hamming-distance threshold, dedup metrics in the run summary.
Tests. Unit tests on the canonicalizer (table-driven); SimHash test with a hand-crafted near-dup pair.
Token budget: ~½ session.
What. Add an embedding column on items (use a small local model via sentence-transformers, or Voyage/OpenAI/Anthropic embeddings if you prefer hosted). After SimHash dedup, cluster remaining items in a 48-hour window using cosine similarity + a simple agglomerative pass; assign a cluster_id.
Why. Some near-duplicates are paraphrased (a Substack post and a Techmeme summary of it); only embeddings catch these. Clusters are also the unit the synthesis prompt should consume in Phase 2.
Deliverables. Embedding job (idempotent, only embeds new items), clustering pass, clusters table.
Tests. Synthetic cluster test with 5 known-similar items; threshold sweep documented in docs/clustering.md.
Token budget: ~1 session. Watch out for Claude wanting to add a vector DB — push back; pgvector/sqlite-vss can wait until you're past 100k items.
Now make the brief itself measurably better.
What. Replace the free-text per-item summarization with a JSON schema enforced via tool use: {tldr, what_changed, why_it_matters, claims[], code_or_api_changes[], numbers[], governance_signals[], open_questions[]}. Store the JSON in items.summary_json.
Why. Every later feature (drills, trend detection, credibility scoring) reads from this schema. Free text is a dead end.
Deliverables. Schema (Pydantic), tool definition, prompt, stored outputs, fallback if a field comes back empty.
Tests. Schema-conformance test on three real items; golden test that re-summarizing the same item produces the same tldr modulo minor variation (use a fuzz threshold, not equality).
Token budget: ~1 session. The prompt iteration takes the most tokens; do prompt drafts in a normal Claude chat first, then paste the final into Claude Code.
What. Store every prompt as a file under prompts/ with a version suffix. Each LLM call records prompt_id, model, temperature, input hash, output hash in the runs summary.
Why. Without this, you can't tell whether a brief got worse because of source drift, prompt drift, or model drift.
Deliverables. Prompt registry, call-logging wrapper around the Anthropic client.
Tests. Wrapper test that asserts log entries are emitted with all fields populated.
Token budget: ~⅓ session.
What. Add config/taxonomy.yaml with the initial themes from your README sections plus governance/agentic subnodes. After clustering, an LLM pass assigns each cluster to a theme (or proposes a new one with a flag for review).
Why. Replaces the current hard-coded section names with a maintainable structure; required for trend detection later.
Deliverables. Taxonomy YAML, assignment prompt, cluster_theme table, CLI command contentfinder taxonomy review to approve proposed new themes.
Tests. Assignment test on 10 fixed clusters with expected themes; assert "no new theme proposed" rate is reasonable.
Token budget: ~1 session.
What. Update the synthesis prompt to: (a) require inline citation tokens like [c12] mapping to cluster ids, (b) extract a "Contested" sub-section per theme when items disagree on numbers, dates, or claims. Render footnotes with cluster→source links.
Why. This is the single biggest trust feature. Once every claim is traceable, you'll feel comfortable unsubscribing in Phase 3.
Deliverables. Updated prompt, post-processing validator that fails the run if any citation token is unresolvable, footnote renderer.
Tests. Validator test: brief with bad citation should fail; brief with valid citations should render footnotes.
Token budget: ~½ session.
This is the phase that solves your stated pain point. Don't start it until Phase 2 is solid — you need synthesis quality high enough that you'd actually trust it as a replacement.
What. A CLI command contentfinder sources import <file> that takes a list of newsletter names or URLs (one per line) and creates pending entries in the sources table.
Why. Onboarding flow for the user's existing subscriptions.
Deliverables. CLI command, pending status, list/show/delete subcommands.
Tests. Idempotency on re-import; bad-line skipping with a warning.
Token budget: ~⅓ session.
What. For each pending source, attempt discovery: (1) try <name>.substack.com/feed, (2) fetch the homepage and look for <link rel="alternate" type="application/rss+xml">, (3) try common paths (/feed, /rss, /feed.xml, /atom.xml). Each candidate is fetched and validated (must parse, must contain ≥1 item from the last 60 days).
Why. Most of the user's newsletters are on Substack or have feeds; this maps them automatically.
Deliverables. discovery.py with the strategies, candidate scoring, discovery report per source.
Tests. Per-strategy unit tests with fixtures; integration test against three known sources from your current RSS_SOURCES.
Token budget: ~1 session.
What. Each discovered candidate gets a confidence score (0–1) based on: domain match with input, recency of items, title overlap with the user-provided name. CLI command contentfinder sources confirm walks the user through each pending source showing the top candidate and recent titles, asks [y/n/skip/manual]. Confirmed sources flip to active.
Why. Required guardrail — auto-discovery will be wrong sometimes and you must catch it before unsubscribing.
Deliverables. Confidence scorer, interactive CLI walker, active/needs_fallback states.
Tests. Scorer test on 5 hand-labeled cases.
Token budget: ~½ session.
What. For sources stuck in needs_fallback, document a setup that creates a forwarding address (Cloudflare Email Routing or SimpleLogin works), points it at a tiny inbound-email handler (start with a local IMAP poller against a dedicated mailbox — simpler than running an MX), parses incoming emails into items, and tags them as source-forwarded.
Why. Some newsletters genuinely don't have feeds; this is the only honest fallback.
Deliverables. imap_ingest.py, HTML cleanup using the same trafilatura pipeline, dedup against existing clusters.
Tests. Fixture-based test on 3 saved .eml files.
Token budget: ~1 session. Skip this stage if every newsletter you care about has a feed — discovered in 3.2.
What. A contentfinder unsubscribe command (or a tiny static HTML page) listing each source with: status pill, 14-day capture rate, items synthesized last 7 days, recommendation (safe_to_unsubscribe, keep_email, needs_fallback), and the unsubscribe URL if it can be found in feed metadata.
Why. Closes the loop. This is the surface that lets you act on the consolidation.
Deliverables. Recommendation logic (e.g., safe_to_unsubscribe requires 14 days green health and ≥3 items captured), CLI table renderer or static HTML.
Tests. Recommendation logic tests for each branch.
Token budget: ~½ session.
This is what turns the tool from a reader-replacement into something that improves your vibe-coding and frontier awareness.
What. Create evals/ with a frozen gold set of ~30 items with hand-written summaries, expected clusters, and expected themes. A contentfinder eval command runs the pipeline against the frozen inputs and reports deltas on summary fidelity (LLM-as-judge with rubric), dedup precision/recall, and theme accuracy.
Why. Without evals, every prompt change is a vibe call. Add this before any further prompt iteration.
Deliverables. Gold set (you write this; Claude assists), eval runner, baseline report, CI workflow that runs evals on PRs.
Tests. The eval harness is the tests for the synthesis stages from here on.
Token budget: ~1 session for the harness; gold-set authoring is mostly your time, not Claude's.
What. Each rendered brief includes a footer with 👍 / 👎 + free text per theme; the response is logged to data/feedback.db. A weekly contentfinder review command surfaces patterns.
Why. Feeds Phase 4.6 credibility scoring and prompt-iteration data.
Deliverables. Feedback ingestion endpoint or CLI prompt, storage, weekly digest command.
Tests. Round-trip test for feedback ingestion.
Token budget: ~½ session.
What. Append 3–5 concrete actions to each brief, generated from the day's structured summaries: a prompt to run, a repo to clone, a paper to read, an eval to write. Actions are tracked as todo items.
Why. Converts reading into doing — the actual point of the system.
Deliverables. Action-extraction prompt, todo store, contentfinder todo CLI.
Tests. Schema test on action shape; eval-backed test that actions cite a source cluster.
Token budget: ~½ session.
What. From high-signal items, generate optional 5-minute drills (reading comprehension, "predict the benchmark," tiny implementation challenges). Schedule via Leitner-style spaced repetition.
Why. Compounds learning over time.
Deliverables. Drill generator, SR scheduler, contentfinder drill CLI.
Tests. Scheduler unit tests for box transitions.
Token budget: ~1 session.
What. Rolling 7/30-day frequency per taxonomy node; z-score-based alerts when a theme spikes; "heating up / cooling down" section in the weekly digest.
Why. Surfaces the jagged frontier movement you can't see day to day.
Deliverables. Aggregation job, alert thresholds (configurable), digest section.
Tests. Synthetic time-series tests that hand-crafted spikes are detected.
Token budget: ~½ session.
What. Per source, compute timeliness (lead vs. lag on stories that later cluster), originality (broke vs. aggregated), signal density (claims per token in your structured summaries), and a feedback-weighted accuracy proxy. Use scores to weight items in synthesis.
Why. Stops loud-but-low-signal sources from dominating briefs.
Deliverables. Scoring job, score columns on sources, integration into synthesis ranking.
Tests. Scoring pipeline tests on a synthetic two-source scenario.
Token budget: ~1 session.
These can be interleaved into earlier phases when you want a morale boost.
A self-contained HTML template (Jinja2) producing a single-page brief with collapsible themes, footnote citations, and a sticky theme nav. ~½ session.
YAML frontmatter, wikilinks for entities (models, frameworks, regulations), date-stamped filename. ~⅓ session.
A workflow that publishes the latest brief as a static page at https://raidianblaster.github.io/Content-Finder/. Useful to read on phone. ~½ session.
If you do roughly one stage per day on average, the path from current state to a system that lets you actually unsubscribe is about 3–4 weeks of part-time evening work, with the unsubscribe milestone landing at Stage 3.5. The learning-loop phases (4.x) can spread over the following month.
A reasonable order:
- Pre-stage
CLAUDE.md(today). - Phase 0 in order — don't skip 0.3, you'll regret it.
- Stages 1.1 → 1.3, then 2.1, then 2.2, then 5.1 (you'll want a nicer brief by now).
- Stages 1.4 → 2.3 → 2.4.
- Phase 3 in full — this is the unsubscribe milestone.
- Stage 4.1 (evals) before any further prompt work.
- Remaining 4.x stages in feedback-driven order; let your weekly review tell you which to build next.
A few quick rules I'd hold to:
- One stage per usage window. If you're at 70% of the window and the stage isn't done, commit a WIP, write the session note, and stop. Resuming is cheaper than fighting a soft cap.
-
Keep
CLAUDE.md≤ 200 lines. Every session pays its cost. - Don't let Claude read the whole repo. Point it at the 2–3 files relevant to the stage.
- Batch trivial changes. Renames, format fixes, and dependency bumps can ride along inside another stage's PR rather than burning a session each.
- Use the smaller model for tests and YAML. Reserve the larger model for synthesis-prompt work, the dedup logic in 1.3/1.4, and the mapping confidence scorer in 3.3.
If a stage as written here looks too big once you're inside it (Stage 0.3 and Stage 1.4 are the most likely candidates), the right move is to split it on a natural seam — schema vs. integration, embedding vs. clustering — and ship the half you've got.