Taxonomy and Classification

Taxonomy and Classification

This page explains how Phase A decides where each bookmark belongs: where the taxonomy comes from, why it is frozen after the first run, how it is allowed to grow, how changes are detected without re-reading the whole library, and how the browser bar is protected from the library.

The README states the rules. This page is about the mechanism behind each rule and what happens when it is broken. For the files these rules read and write, see Ledger and File Formats; for what the appliers then do with the result, see Safety and Ownership Model.

Every collection, tag and bookmark named on this page is invented. Your install will contain none of them; see "Where the taxonomy comes from" for why.

Where the taxonomy comes from

The repository ships no taxonomy. There is no folder tree in the code, no tag list, no example categories that will turn up in your install. The first classification pass is the only source.

That pass reads the entire library and asks the model to derive two things that fit the saves it actually saw: a tree of collections and a vocabulary of tags. Both are written to ~/.raindrop-sync/taxonomy.json. The pass then creates each collection in Raindrop, records each taxonomy path against the collection id Raindrop returned in ~/.raindrop-sync/collection-map.json, classifies every item into exactly one path, writes the collections and tags back, and seeds the ledger in ~/.raindrop-sync/state.db with one row per bookmark. Collection ids are per-account integers generated by Raindrop; the map exists so that no later run ever has to look a collection up by name.

If your library is mostly Recipes, Woodworking, Research Papers and Travel, that is the tree you get. Somebody else's install, run on their library, produces a tree with nothing in common with yours. That is the point: a bundled taxonomy would be one person's interests imposed on another person's saves, which is the same failure as a catch-all bin, reached from the opposite direction.

The first pass is the only expensive one. It sends every title and URL in the library to the model. Every pass after it sends only the delta described below.

Why the taxonomy is pinned

After the first pass the tree is frozen. A routine run reads taxonomy.json, passes the tree into the prompt, and asks the model to place each new or changed item into it. A routine run never asks the model to design a tree.

The reason is not that re-derivation produces wrong answers. It is that it produces different answers. Ask a model to derive categories from the same few hundred titles on two different days and you get two trees that are almost, but not exactly, alike. Woodworking comes back as Woodworking and Furniture. Research Papers splits into Papers and Preprints. A subfolder moves up a level. Each output is individually defensible. Neither is wrong. But in this system a folder path is an identity, and every variation is a new identity:

• In Raindrop, a new path is a new collection. Items migrate between collections on every run, and the abandoned collections linger, because this tool does not delete.
• On the browser bar, both appliers resolve folder_path by walking folder names under the bar and creating whatever is missing. The file writer even derives a folder's GUID from its path. A renamed category is therefore not a rename at all. It is a second folder created beside the first, and since neither applier ever removes a node, the first folder stays. Re-derive nightly and the bar accumulates one near-duplicate of each category for every naming variation the model has ever produced.

That is what the README means by "unstable even when every individual write is correct". Every write did exactly what it was asked. The bar still churns, and the user's mental map of it is destroyed.

Pinning is a property of how the routine run is built: it supplies the tree and asks for placements only. Re-derivation still exists as a separate operation that you invoke on purpose; see "What a taxonomy_version bump means".

How the taxonomy is extended

Pinned does not mean closed. When a delta item fits nothing in the tree, the correct output is a new collection: created in Raindrop, added to collection-map.json, and named in the run summary so you see it happened. This is the response the design expects to something unfamiliar, not a fallback.

Suppose the tree holds Recipes, Woodworking, Research Papers and Travel, and a save arrives about keeping bees. Nothing fits. The run creates Beekeeping, files the item there, and says so. It does not file it under Recipes on the grounds that honey is food, and it does not create Other.

The rules around extension, and the reasoning:

• There is no bin. Noise, Junk, Misc, Other, Various and Uncategorised are forbidden collection names, and so is any catch-all wearing a friendlier one. A bookmark is a record of a deliberate act; a classifier that shrugs and bins what it does not recognise destroys the signal it exists to find. If the model genuinely cannot place something, that is information about a gap in the tree, and the gap is filled by adding a collection. CONTRIBUTING.md lists a catch-all category among the changes that will be declined.
• Placement is judged on the destination, never on the route. A utm_source parameter, an advertising click-id or a shortlink says how a bookmark arrived. It says nothing about whether it matters. Somebody who tapped an advertisement for a plumber and then saved the page was looking for a plumber, and that belongs in the library. Provenance is recorded as a tag (below) and must never drive placement.
• Dead links and slug titles are research problems. Follow the redirect, check the Wayback Machine, work out what the page was. Discarding is not an option because discarding is not available.
• Design for the next five hundred saves. A collection with two items today is fine if the interest is real. Real interests sprawl; a tree tuned to look tidy this week fights the library next year.

The line between extending and restructuring is sharp. Adding a leaf is extension and is allowed on any run. Renaming, merging, splitting or moving an existing collection is re-derivation, and needs a version bump.

The controlled tag vocabulary

Every classified item receives one to four tags drawn from the vocabulary in taxonomy.json, and nothing from outside it. Collections and tags answer different questions. A collection answers "what is this about", and there is exactly one answer per item. Tags answer "what kind of thing is it, and how did it arrive", and there can be several: recipe, tool-review, paper, itinerary, and the provenance tags paid-ad and short-link.

The vocabulary is controlled for the same reason the tree is pinned. Free-form tags drift exactly as a free-form tree does (recipes, recipe, Recipes), and the Raindrop API makes drift permanent: its tag write appends, it does not replace. A "correction" applied blindly leaves both the old tag and the new one on the item. The mechanics that follow from this, all in SKILL.md:

• Tag writes are read-modify-write. To replace, read the current tags, compute the target set, then write.
• Add and remove cannot be combined in one call; the API rejects it. They are split into two.
• Write-back is grouped by (collection, tagset), at most 150 bookmark ids per call.
• Provenance tags are facts about how a save arrived and are kept as such. They are never an input to placement.

The delta rule

A routine run classifies an item if, and only if, one of three things is true:

1. its id is absent from the ledger (NEW), or
2. its stored content_hash differs from the one just computed (EDITED), or
3. taxonomy_version has changed since the row was classified.

Everything else is skipped without a model call. On a steady-state day the delta is a handful of items.

The hash

content_hash = sha256(link | title | excerpt | note | type)

Only the first two slots carry data. The excerpt, note and type slots are present in the format and always empty, because the surface the classifier reads through does not return them. The format is stable even though three of its fields are unused, and that is deliberate: widening it is a breaking change (below).

Two things that look like reasonable inputs to this hash are not, and each was rejected for a specific reason.

Never a timestamp. The obvious incremental design is a watermark: remember lastUpdate from the last run and fetch everything newer. Raindrop bumps lastUpdate on every write to an item, including the write-back this job performs. So every item the run touched is "modified" as of that instant, the next run selects all of them, writes them again, and re-poisons the watermark. It never settles. There is no server-side rescue either: the documented search operators support < and > on created: and pointedly not on lastUpdate:, and date operators are day-granular, so no operator query could express an intra-day delta even if one existed. That path is deleted from the design rather than kept as a future optimisation.

Never curated values. The ledger stores two versions of a title and a link. The display columns hold the curated form: emoji stripped, over-long blurbs shortened, HTML entities decoded, tracking parameters removed. Those are what name the bookmark in the browser. The hash is computed from the raw values, verbatim from Raindrop. The first run of this project got that backwards, hashed the curated columns, and on the next run about 28% of the library re-flagged as EDITED, and would have done so on every run forever. The reason is simple once seen: a hash of an edited record cannot be reproduced from the source. The source still says Foo & Bar; the ledger says Foo & Bar; the digests will never agree. This is the same failure as the timestamp watermark, reached by a different road.

The invariant is two columns that intentionally disagree:

column	holds	used for
content_hash	sha256(raw_link|raw_title|||), verbatim source values	delta detection only
title, link	the curated display values	naming in the browser, desired.json, reading

A future maintainer who notices that a row's title does not match the string behind its hash will be tempted to "repair" the hash from the stored columns. That single act reintroduces the bug. The columns disagree by design.

Ordering

The hash is recorded as classified only after the item has been committed to desired.json. If the run crashes between classifying and committing, the item is simply classified again next time, at the cost of one redundant model call. Record the hash first and a crash produces a silent skip instead: the item is marked done, nothing ever flags it again, and there is no signal anywhere that it was missed. One redundant call is cheap; an unrecoverable skip is not.

What is and is not a hash input

Adding a real field to the hash, say a populated excerpt, invalidates every stored hash at once. The next run would see the entire library as EDITED and reclassify all of it. That is a taxonomy_version-class operation: explicit, user-invoked, never a silent upgrade shipped inside a routine run.

The mirror-image mistake also happened. Twice it was concluded that the hash could not be computed because a richer API surface was unavailable. Both times link and title were available the whole time; the empty slots are empty by design. A missing API token degrades full-library reconciliation, not hashing. Compute the hash from whatever authoritative source is already in hand. The error left a handful of rows with a null hash for two days, during which edit detection was blind for exactly those rows.

Reading the library without inventing a delete

The delta compares a fresh read against the ledger, so the read must be complete. RESEARCH.md finding 7 sets the rules: page ascending by creation date, because newest-first paging shifts the window when a bookmark is saved mid-run and an item silently falls between pages; or take the single export.csv render, which is atomic on the server side and immune to that race. A cheap "nothing changed" gate on changedBookmarksDate is in the design, but whether a tag-only edit advances that field is untested, so as built Phase A does a full read on every run. The design also calls for a forced full read after seven days regardless of any gate. See Research Status for the verification labels behind each of these.

The canary

Every run logs classified_count, the size of the delta it actually sent to the model. On a steady-state run that number is small. The skill's threshold is about fifty: above it, delta detection is broken, and the run is expected to say so loudly rather than reclassify the library.

That one number is the tell for every failure mode in this section. A poisoned watermark, a hash computed from curated values, a silently widened hash, a paging race that dropped items: each of them shows up as a delta that is dozens or hundreds on a library that barely changed. If you see it, stop. The cost is real model spend and, worse, a library that never settles. Troubleshooting covers how to find which of the four it is.

The promotion gate

Raindrop is a library. The bookmarks bar is a shortlist somebody built by hand. Those deserve different standards, and the gate between them is where the difference is enforced.

Phase A can stage a bookmark for the bar in two kinds of place:

• The managed mirror. A single top-level folder on the bar, with one child folder per top-level collection: Raindrop/Recipes, Raindrop/Woodworking, and so on. The skill keeps this flat, one level deep, because every extra level of nesting is one more hover menu between the reader and the bookmark.
• The user's own curated folders. These exist outside the managed folder, were built by hand, and track things like organisations and their primary sources.

Promotion into a curated folder is reject by default. What gets rejected, and why:

rejected	why
deep links that will rot: repository blob links, query-filtered views, anything whose URL encodes a moment	a curated folder is meant to be stable for years; a link that 404s in six months degrades it
commentary about a field rather than a primary source	the bar is where primary sources live; the commentary belongs in the library
one-off saves of early-stage products and vendor landing pages	a save is not an endorsement; a hand-built shortlist implies one
job postings and news articles	both are dated by construction
a proposed new curated folder holding fewer than three qualifying items	a folder with one item is a bookmark with extra clicks; three is the floor at which a folder earns its place

The skill also tells the classifier to ask before promoting if it hesitates. The reasoning is an asymmetry: a rejected item stays in Raindrop with its collection and tags and loses nothing, while a wrongly promoted one quietly degrades something a person spent years building. On the first real run eight promotions were proposed and seven were rejected. That ratio is the gate working, not failing.

Two consequences follow. First, desired.json is a far narrower set than "everything classified", and a run that classifies many items and promotes none is normal. Second, the gate is the only thing between the library and the bar, which is why it is adversarial rather than permissive. If the bar ever starts to look like the library, the gate has been loosened.

What a taxonomy_version bump means

taxonomy_version is an integer kept in the ledger's meta table. Bumping it is the explicit re-derivation operation, and it is the third condition of the delta rule: after a bump, every row in the ledger is due for classification.

What a bump commits you to:

• A full reclassification. Every item goes back to the model. That is the cost of a first pass, paid again, and it is why re-derivation is never scheduled.
• A full write-back to Raindrop. Collections are reassigned and tags recomputed for every item. Because the tag API appends, tags that no longer apply have to be removed in their own calls; a reclassification that only adds leaves the old vocabulary in place beside the new one.
• Old folders on the bar do not go away by themselves. Both appliers are additive. A category that was renamed or moved in the new tree gets a new folder; the folder built under the old name stays. The one mechanism for cleaning that up is ~/.raindrop-sync/reset_folders.json, which names a managed folder to tear down and rebuild. The native host passes those names to the extension, and the extension removes the named folder as a whole, including anything a person dropped into it by hand. It is only ever used for folders this tool owns end to end. That makes it the one removal path in the entire system, and it is covered on Safety and Ownership Model.

The same principle governs two neighbouring operations that are also version-class, even though they do not change the tree. Widening the content hash to a new field invalidates every stored hash, so it must be an explicit reclassification. And a data migration in the ledger, such as normalising the separator in a path-keyed column that different runs wrote as Parent / Child and Parent/Child, belongs to a deliberate, user-invoked pass with the rows re-validated afterwards. Neither is ever a silent fixup inside a routine sync. The rule underneath all three is the same one that pins the taxonomy: anything that touches every row at once is an event you start, not a side effect you discover.

A worked example

A library holding Recipes, Woodworking, Research Papers and Travel is classified on day one, producing taxonomy.json with those four collections, a few children under each, and a tag vocabulary. Over the next week:

• A new save about a hand plane arrives. It hashes to something the ledger has never seen (NEW), is classified into Woodworking / Hand Tools with tags tool-review, and is written back. The promotion gate looks at it, sees a product page, and rejects it for the bar. It stays in Raindrop, correctly filed.
• The user edits a recipe's title in Raindrop. Its raw title changes, so its hash changes (EDITED). It is reclassified, lands in the same collection, and is skipped by the gate again.
• A save about keeping bees arrives. Nothing fits. The run creates Beekeeping, files it there, and names the new collection in its summary.
• A save from an advertisement for a woodworking supplier arrives, with a click-id in the URL. It is filed under Woodworking / Suppliers, tagged paid-ad, and treated exactly like any other save from then on.
• Nothing else changed, so classified_count for the week's runs is 1, 1, 1 and 1. The canary is quiet.

At no point was the tree redesigned, and nothing on the bar moved.