You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Migrate the Scrape pipeline step to a Pydantic AI agent as an SDK swap (single LLM call, no tools), establishing the agent shape so future PRs can grow it. Full rename Scrape → Fetch Details (breaking change) across status enum, file names, env vars, configure wizard, telemetry, docs, and diagrams.
Delivered as one PR on feature/fetch-details-agent with two commits:
Rename (status enum, files, env vars, configure wizard, recovery can_handle literal, docs, diagrams) + status data migration.
Agent migration (introduce episodes/agents/_model.py and episodes/agents/fetch_details.py, refactor fetch_details_step.py to call the agent, delete get_scraping_provider).
Acceptance criteria
episodes/agents/fetch_details.py exposes EpisodeDetails, get_agent(), run(html) and imports only Pydantic AI / Pydantic / stdlib / agents/_model.py (no Django, no DBOS, no episodes.models).
episodes/agents/_model.py provides a pure build_model(model_string, api_key) helper; recovery agent refactored to use it.
Status enum value scraping → fetching_details; data migration updates Episode.status and ProcessingStep.step_name rows.
Env vars: RAGTIME_SCRAPING_PROVIDER removed; RAGTIME_SCRAPING_MODEL → RAGTIME_FETCH_DETAILS_MODEL (Convention B, e.g. openai:gpt-4o-mini); RAGTIME_SCRAPING_API_KEY → RAGTIME_FETCH_DETAILS_API_KEY. .env.sample and core/management/commands/configure.py updated.
STEP_FUNCTIONS[Episode.Status.FETCHING_DETAILS] points at episodes.fetch_details_step.fetch_episode_details; PIPELINE_STEPS updated; get_scraping_provider deleted from episodes/providers/factory.py.
Fast-path skip and empty-field-only merge behavior preserved in the orchestrator.
What to build
Migrate the
Scrapepipeline step to a Pydantic AI agent as an SDK swap (single LLM call, no tools), establishing the agent shape so future PRs can grow it. Full renameScrape→Fetch Details(breaking change) across status enum, file names, env vars, configure wizard, telemetry, docs, and diagrams.Detailed plan:
doc/plans/2026-04-28-fetch-details-agent.md.Delivered as one PR on
feature/fetch-details-agentwith two commits:can_handleliteral, docs, diagrams) + status data migration.episodes/agents/_model.pyandepisodes/agents/fetch_details.py, refactorfetch_details_step.pyto call the agent, deleteget_scraping_provider).Acceptance criteria
episodes/agents/fetch_details.pyexposesEpisodeDetails,get_agent(),run(html)and imports only Pydantic AI / Pydantic / stdlib /agents/_model.py(no Django, no DBOS, noepisodes.models).episodes/agents/_model.pyprovides a purebuild_model(model_string, api_key)helper; recovery agent refactored to use it.scraper.py→fetch_details_step.py;agent.py/browser.py/deps.py/tools.py/resume.py→recovery_*.py.scraping→fetching_details; data migration updatesEpisode.statusandProcessingStep.step_namerows.RAGTIME_SCRAPING_PROVIDERremoved;RAGTIME_SCRAPING_MODEL→RAGTIME_FETCH_DETAILS_MODEL(Convention B, e.g.openai:gpt-4o-mini);RAGTIME_SCRAPING_API_KEY→RAGTIME_FETCH_DETAILS_API_KEY..env.sampleandcore/management/commands/configure.pyupdated.STEP_FUNCTIONS[Episode.Status.FETCHING_DETAILS]points atepisodes.fetch_details_step.fetch_episode_details;PIPELINE_STEPSupdated;get_scraping_providerdeleted fromepisodes/providers/factory.py.AgentStrategy.can_handle()literal flipped"scraping"→"fetching_details".ragtime-processing-pipeline,ragtime-processing-pipeline-with-recovery,ragtime-recovery(both.excalidrawand.svg).README.mdpipeline table,doc/README.mdstep descriptions,doc/features/2026-04-28-fetch-details-agent.md, planning + implementation session transcripts indoc/sessions/,CHANGELOG.md## 2026-04-28entry with**BREAKING**marker.Out of scope (future work)
Downloadstep to its own agentEpisode.scraped_htmlmodel fieldsummarize/extract/resolve/translate/embedstepsBlocked by
None - can start immediately.