v0.57.0 — smallweb stops being a catalogue
smallweb stops being a catalogue
/rss and /news find were carrying Kagi's Small Web as its catalogue: a 5MB
OPML of 33,000 personal blogs, plus a 1.1MB plain-text fork of the same list.
Both were marked search-only, which kept them out of a subscription but not out
of a search — and find loads every published list on every search, so each
search parsed 33,000 outlines. That is what took the reader down.
Kagi publishes those same blogs already merged. kagi.com/smallweb/feed is one
Atom document, newest first, several thousand recent posts with titles and
summaries, and that is what smallweb is now: one feed wearing a list's name.
/news add smallweb subscribes to it without fetching anything, /news rm smallweb takes it back out, and it reads on its own with
/news --feed kagi-small-web.
A catalogue can no longer be subscribed to by any route
Search-only guarded the list name, so add smallweb was refused while
add https://kagi.com/smallweb/opml quietly wrote 32,969 outlines into
news.opml — after which every /rss tried to fetch all of them. A URL is not
a smaller promise than a name, so the limit now sits on the number of feeds an
add subscribes to, whatever the source: named list, URL, or local file. It is
set well above every published list and well below a catalogue.
If you already have one in your subscriptions, ~/.moshcode/news.opml is the
file to look at — an import by URL lands untagged, so /news rm <list> cannot
find it.
Aggregated feeds no longer crowd out everyone else
Two guards that apply to every feed, not just this one. A feed read stops at 2MB
and parses what arrived, rather than failing outright at the document cap — a
10MB firehose used to be refused entirely when its newest few hundred entries
were exactly what was wanted. And no single feed may put more than 200 items
into a merged listing, because otherwise subscribing to an aggregator is the
same as unsubscribing from everything else: its same-day entries sort above
every other publisher's headlines.