Repository navigation
The first stable release. A final review ran every surface against the one production
deployment, a SvelteKit site on Vercel, and against its live index. It found that a
single prerendered redirect page had quietly switched off pruning for four weeks,
leaving over a thousand stale sections searchable. Most of this release closes that
class of silent failure: an indexing run that refuses to write now fails the build,
and a run that sees a page is excluded removes it whatever else went wrong. The rest
tightens the public boundary and settles the API that 1.0 commits to.
Upgrading from 0.11 needs action in three places. Config is now validated
strictly, so an unknown or misspelled key fails to load. URL patterns that the
matcher could never honour are rejected. A refused indexing run now fails the Vite
build. Each is marked Breaking: below.
Changed
- Breaking: config is validated strictly. An unknown key fails to load with
CONFIG_INVALID, naming the full path and suggesting the closest valid key
(ranking.minScoreRaito: unknown option. Did you mean ranking.minScoreRatio?).
Previously a typo was silently ignored and the setting it meant to change stayed
at its default. - Breaking:
transform.output,chunking.strategyandanalyticsare removed
keys. Each accepted one fixed value and was read by nothing. Delete them. - Breaking: unsupported URL patterns are rejected.
exclude,
source.build.excludeandranking.pageWeightssupport exact paths and a
trailing/*or/**. Anything else, such as/**/private/**,
/draft-*.htmlor a doubled slash like/docs//**, was compared as a literal and matched nothing. It now fails
config load. See "URL patterns" indocs/config.md. - Breaking: a refused indexing run fails. If
indexing.maxDeletionRatio
refuses a deletion plan, orrobots.txtcannot be read incrawlmode, the run
writes nothing, as before. It now also reportsstats.writesRefused
({ reason, detail }, otherwisenull).searchsocketVitePlugin()then fails
the build withINDEX_WRITES_REFUSEDinstead of logging "indexed" and deploying
over a frozen index, andsearchsocket indexexits4for a ratio refusal or
1for unreadable robots. New plugin options:failOnRefusedWrites(default
true) andacceptLargeDeletion(defaultfalse). - Breaking: CLI exit codes follow the documented table. Usage and config errors
exit2(were1), includingsearchsocket test --top-kabove 100 and a
searchsocket.config.tsthat fails to parse or throws while loading, which is now
CONFIG_INVALIDwith the underlying message. Failed
searchsocket testassertions exit3, but only when every query was retrieved:
a backend failure duringtestexits5rather than counting as failed
assertions. An unreachable, rate-limited or unconfigured backend exits5, which
now coverssearchandtestwithout credentials (previously1and3) and
statuswhen the health check fails (previously0). The full table is in the
README. - Breaking:
clean --remoteis plan-only until--apply. It previously
printed "dry-run only" after it had already deleted the local state directory. - Breaking: the public search endpoint refuses
debugandrankingOverrides
with 403, and validates numeric query parameters as whole values (topK=10abc
is a 400, not 10). A repeated singleton parameter is a 400.GET <api>/health
answers 503 when the backend is unhealthy. The browser client strips the two
refused fields, so type-correct calls keep working. - Breaking: MCP endpoints reject JSON-RPC batches with 400 /
-32600. One
anonymous batch could fan out into thousands of concurrent backend queries while
counting as a single request. The current MCP specification no longer has
batching. The standalonesearchsocket mcp --transport httpserver also refuses
a POST whose body Express's JSON parser did not accept (415), rather than letting
the MCP transport read the raw body itself: aContent-Typethe parser skipped
but the transport accepted, such asapplication/jsonfollowed by U+00A0,
reached the transport unchecked for batching and outside the body-size limit. - Breaking: queries are capped at 512 characters and page paths at 2,048
(MAX_QUERY_LENGTH,MAX_PATH_LENGTH). Longer input is a 400 over HTTP and a
validation error over MCP. Both measure the query after trimming, so the MCP
tools accept exactly the queries HTTP does. Query-aware excerpts consider at most 32 distinct
query terms. A 63 KB query previously cost around 12 seconds of CPU. - Pages a run sees as excluded are removed even when the run is not
authoritative. A page matched byexclude, blocked byrobots.txt, marked
noindexor weight 0, or served as a redirect stub is deleted from the index
even if an unrelated page's extraction failure blocks general pruning. Before,
one broken page kept every newly excluded page public. These deletions still
count towardsmaxDeletionRatio. An unreadablerobots.txtstill blocks
everything. This covers pages excluded bysource.build.exclude, and every
stored section of an excluded page, including older generations and pages
indexed before chunk manifests. A URL the same run also indexed is never
removed, and neither is one whose other source file failed to load or extract. - Prerendered redirect pages are skipped, not failed. A page whose original
HTML has no body text and a<meta http-equiv="refresh">is logged at info and
treated as excluded; SvelteKit prerenders exactly this for aredirect().
Previously each one was an extraction failure that disabled pruning for the whole
site. A page emptied bydropSelectorsis still an extraction failure. A stub
that redirects to its own URL, such asdocs.htmlpointing to/docs/, is
skipped without excluding/docs. --max-pagescounts URLs, not files. Source files that serve the same URL,
such asdocs.htmlanddocs/index.html, are loaded together, so a limited run
cannot load a redirect stub without the page it stands in for.- In manifest mode, a dynamic route that cannot expand blocks deletions. A
route with noparamValues, or values that do not fit, left an incomplete
inventory that could still authorise deleting live pages. The exception is a
route whose every URL is covered by the top-levelexcludeor
source.build.exclude(/blog/**covers/blog/[slug]). A route id is no longer
matched against patterns as if it were a URL, so/docs/*does not exclude
/docs/[...slug]. - Snippets no longer include the embedding prefix. Result snippets and
chunkTexthad started with theTitle — Sectiontext prepended for embedding,
and page-summary snippets repeated the title. Deep links built from those
snippets pointed at text that does not exist on the page. This is fixed at read
time, so existing indexes benefit without re-indexing. The prefix is removed only
whenchunking.prependTitleis on, and summary text only from actual
page-summary records, so a section that genuinely opens with the page title
keeps it. - Scroll-to-text lands on the section, not the table of contents. When a
heading on the page matches_ssk, the text target is searched from that
heading onwards; otherwise the first match outsidenav,asideand
role="navigation"wins. A documentation page repeats each heading in its "On
this page" list ahead of the content, so a result whose target was the heading
scrolled to the list. If_ssktcannot be found, the section heading is used.
Matching stops at the first eligible occurrence and looks up text nodes by binary
search, so a phrase repeated thousands of times on a long page no longer makes
the lookup quadratic. - Snippets are plain text. Markdown syntax was stripped by replacing every
#,>,*,_,|and-with a space, which left link syntax such as
[Settings](/docs/settings)in snippets and broke real words (right-click,
max_results). Only syntax is removed now. Code spans keep their contents
verbatim (__init__,a || b), table pipes are removed only on table rows, and
backslash escapes from HTML conversion (max\_results) are removed. The
conversion is linear, so pathological markdown can no longer make snippet
building quadratic. Stored snippets change, so the first run after upgrading
refreshes chunk metadata once, without re-embedding. - Browser clients drop malformed or unsafe result rows instead of failing the
whole response. An invalid envelope is still anInvalid search response. Result
URLs are checked: ajavascript:ordata:URL is dropped, and
buildResultUrlreturns""for one. llms.txtandllms-full.txtcarry a "Retrieval API" section naming the
search endpoint, and the MCP endpoint when it would answer. No key or variable
name is written.- Configured MCP transport, port and path now apply to the CLI.
searchsocket mcpanddev --mcppassed their own defaults, which always
overrodemcp.transport,mcp.http.portandmcp.http.path. searchsocket searchtakes the query positionally.--qstill works.- Breaking: the root export no longer exposes internals.
isServerless,
resolveScope,mergeConfigServerlessandrunMcpServerare removed; none was
documented.createUpstashStore()returns the newSearchStoreinterface rather
than the concrete Upstash class, so storage internals such as legacy-record
cleanup are no longer part of the public types.SearchEngineOptions.store
takes aSearchStoreReaderandIndexPipelineOptions.storea
SearchStoreWriter. - Breaking:
isSearchResponseis strict. It returnstrueonly when no row
would be dropped. Use the newparseSearchResponseto get the cleaned response
(ornullfor a broken envelope). Both are exported fromsearchsocket/client
andsearchsocket/svelte, alongsideisSafeResultUrl. - Public types are complete. Every type used in a public signature can now be
imported:SearchEngineOptions,IndexPipelineOptions,LoadConfigOptions,
ConfigOverrides,RelatedPage,RelationshipType,SiteStructureResult,
SiteTreeNode,ScoreBreakdown,RankingOverrides,OutgoingLink,
ScopeMode,SourceMode,PageHit,IndexWriteRefusal,
SearchSocketErrorOptions,SearchSocketHandleOptions,
SearchSocketAutoIndexOptions,SearchClient,SearchClientOptions,
LoggerOptions, and the
store typesSearchStore,SearchStoreReader,SearchStoreWriter,
StoreRecord,StoreMetadataUpdate,StoreQueryOptions,PageHashEntry,
ChunkHashEntry,ChunkInventoryEntryandRetrievalConstraints.searchsocket/clientexports a
namedSearchClientinterface. TheLoggerclass that
SearchEngineOptions.loggerandIndexPipelineOptions.loggeraccept is exported
fromsearchsocket, so those options can be used without an internal import. - Dependencies refreshed.
@modelcontextprotocol/sdknow requires^1.30.0,
and the lockfile picks up patched releases of the transitive runtime
dependencies.pnpm audit --prodreports no known vulnerabilities, down from 76. - Error codes:
CONFIG_INVALID(a config that loads but does not validate;
CONFIG_MISSINGnow means a file or required value is absent, such as the
variablescope.mode: "env"reads) and
INDEX_WRITES_REFUSEDare new. The unusedEMBEDDING_MODEL_MISMATCHand
EMBEDDING_FAILEDare removed.
Fixed
- Page titles could come from a heading that
dropSelectors,dropTagsor
data-search-ignoreremoved, because the title was derived before extraction
exclusions ran.data-search-ignoreon the content root itself was ignored. It
now excludes the page. - Index reads (page hashes, chunk scans, content hashes, page chunk fetches) retry
on 429, 5xx and non-JSON error bodies. The store talks to Upstash through its own
Requester(the SDK's documented transport hook), which keeps the HTTP status
the SDK's default client discards, so this works whatever the error body says,
for example a 503 answering{"error":"overloaded"}. A single transient Upstash
error used to fail the deploy. - Reading a page without a chunk manifest from a large scope scanned the whole
scope with no ceiling. Pages now also record how many sections they have. A page
too large to list its sections is read with page-filtered queries, and that
result is used only when the section count matches. Otherwise the read falls
back to a scan capped at 50 range requests, with an error that says whether
reindexing will help. The first run after upgrading refreshes page metadata once,
without re-embedding. instanceof SearchSocketErrorfailed across entry points. Each published entry
(searchsocket,searchsocket/sveltekit, the CLI, and the CommonJS and ESM
builds) bundles its own copy of the class, so an error thrown by one and checked
against the class imported from another was not recognised, and a config file
that threw a typed error was reported as an invalid config. The class now
recognises instances from every copy.deletePages()rethrew the raw SDK error rather than the classified storage
error.- With
serveMarkdownVariantson, a backend outage turned the app's own.md
routes into 500s. They now fall through to the app. searchsocket mcp --transport httpin public mode refused every request whose
Hostwas not localhost.searchsocket mcp --api-keycould not satisfy a config
withmcp.access: "public", because the key was applied after validation.searchsocket devcould silently miss every edit inside a directory created
after it started. With native file events, chokidar 5 on macOS intermittently
never subscribes to such a directory. Source trees (src/routes, content roots,
the config file) are now polled once a second. Build output stays on native
events.--no-pollrestores native events everywhere. "watcher ready" is printed
once, not once per missing root.- A non-finite score could reach a 200 response that the client then rejected;
such hits are dropped on the server. - CommonJS consumers received ESM type declarations. The exports map now points
requireat.d.cts. - Documentation: the MCP guide and the README no longer claim the route is on by
default or show a deployed connection without credentials, and the README's hook
examples import the full config rather than replacing it with a partial one; the
README and search UI guide describe row-level response validation instead of
whole-response rejection;indexing.maxDeletionRatioand the
URL pattern syntax are documented; CLI help names Upstash Vector;
SECURITY.mdlists 1.0.x.