Skip to content

v0.7.0 - Smart Routing v0, Extract Templates & Max Extract

Choose a tag to compare

@yogev-nimble yogev-nimble released this 26 Oct 06:19
· 152 commits to main since this release
10cddcd

Route a query to a method with a swappable decision model (experimental), run a vendor's saved site extractors, and race every extract backend for the fastest page.

🚀 What's New

Smart routing v0 (experimental): classify() and execute() (#20)

  • Opt-in with decider=, and the API may change. classify(query) picks the method (search, extract, research or browse) and returns a Route on that method's default auto chain. execute(route) runs it
  • Pick the model that decides: "keywords" (built-in keyword rules, no key, offline; it picks search or extract, and research or browse fall back to search), "jev" (TypeSafe's Jev), "llm" (any OpenAI-compatible chat model, needs the evals extra), or plug in your own with register_decider()
  • A failed or low-confidence decision falls back to search, with Route.fallback saying why
from web_router import Provider, WebRouter

router = WebRouter(decider="jev")  # TypeSafe's Jev, reads TYPESAFE_API_KEY
route = router.classify("Summarize https://example.com/pricing")
route.method  # e.g. "extract"
response = router.execute(route)

Extract Templates: run_template() (#28)

  • run_template(name, inputs, provider=...) runs a vendor's saved, site-specific extraction and returns the parsed record. templates() lists them and template() describes one, with the schemas of its inputs and output
  • provider is required: a template name belongs to one vendor, so there is no auto. Nimble is the first backend. The MCP server adds templates, template and run_template
  • An unknown name raises TemplateNotFoundError. A failed run or a 4xx/5xx page raises ProviderError, never an empty record
router = WebRouter()
template = router.template("walmart_pdp", provider="nimble")
template.input_schema  # what `inputs` takes
response = router.run_template("walmart_pdp", {"product_id": "123456789"}, provider="nimble")
response.data  # {'title': ..., 'price': ..., ...}

max: every extract backend at once (#29)

  • WebRouter(Provider.MAX).extract(url) sends the request to every backend auto would try and returns the first document back. response.provider names the one that answered
  • Every fetch is billed, so it costs up to one extract per configured backend per URL. It is never the default
  • provider_config={"max": {"hedge": 0.8}} runs the first backend alone for 0.8 s and starts the rest only if it hasn't answered
response = WebRouter(Provider.MAX).extract("https://example.com", format="markdown")
response.provider  # whichever backend answered first

Also new: benchmark_extract() compares extract backends on the same URLs, with a scan script in benchmarks/providers/extract.py (#7).

🐛 Fixes

  • Nimble enrichments (research with records) work as documented, and combinations Nimble refuses now raise before a run starts (#32)
  • Browser Use steps carry a real detail, read from the step's memory, and the action it took (#24)
  • benchmarks/providers/run.py saves rows as it goes, so a scan that dies keeps what it paid for (#7)

🔄 Changed

Browser Use now maps output_schema, max_steps and allowed_domains (#24). A browse call through auto that sets any of them was refused before; it now starts a Browser Use task.

Full Changelog: v0.6.0...v0.7.0