Skip to content

Storm 0.3.0

Latest

Choose a tag to compare

@trayanm trayanm released this 16 Nov 12:43
1ac5ca5

Storm 0.3.0 makes recipes, the way Storm reads a website's list without an LLM, something you can write, check and fix yourself, and lets them read more kinds of sites. Nothing leaves the system without Act, as before.

What's new

  • A guide to recipes: docs/recipes.md. When to use one, teaching and writing it, a walkthrough, every key, what to do when a site changes, and a troubleshooting table.
  • Edit a recipe on its page. Each version shows as YAML. Edit (or Edit from here on an older one) saves your change as the next version, checked like an import; Download saves a version as a file. Teach again starts from the recipe.
  • Try a recipe before a pipeline uses it. Try reads the list with the params you give and shows the items in a table, or what didn't match with a screenshot of the page. It changes nothing: the recipe isn't marked working or broken, and nobody is told. Dry runs of browser.extract now leave the recipe alone too.
  • "Load more" buttons and endless lists. Besides a next link and page numbers in the address, a recipe can press a "Load more" button (next: { kind: more, selector: … }) or scroll to the bottom (next: { kind: scroll }).
  • Detail pages. A recipe's detail section names fields to read on each item's own page (a job ad's description, a listing's floor). The new step browser.details opens them after data.dedupe, so only new items are opened, at the recipe's pace and on its site only.
  • Params explained where you use them. The Teach form asks for the Search results page and shows the address the recipe will keep as you type (https://www.bergfreunde.eu/s/<query>); Try shows the address it will open.
  • pnpm dev on Windows no longer fails with EPERM while the watchers start.

Upgrading from 0.2.0

git pull and ./docker/storm up. There are no new migrations or settings.

Requirements

Unchanged: Docker with Compose v2 and a Claude API key. Single user, AGPL-3.0.

Full list of changes: v0.2.0...v0.3.0