Search Query Construction

Search query construction

How a search request becomes a Lucene query, an OData filter and a page of results, and the decisions src/controllers/search.js, src/search/ai-search.js and src/search/eagle-query.js point at rather than restate. Index schema: Search-Index-Reference. Evidence behind the retrieval choices: Search-Measurements.

The Lucene query

queryType: 'full', so + - * " ~ ( ) : are all Lucene operators.

Each term becomes (term OR term~1^0.5), the per-term groups are joined with AND, and a term* prefix is added on the last term only — the frontend searches on debounced keystrokes and results would otherwise thin out mid-typing.

The AND join is the whole query semantics: every term is mandatory, so a chunk is a candidate only if it contains all of them, and BM25 ranks only within what survives.

The OR inside each group is not redundant. A fuzzy term bypasses the query analyzer, so it matches only what is literally in the index; the plain term goes through en.microsoft and picks up stemming.

Three constants shape the result:

• buildQuery lowercases AND, OR and NOT. Lucene's operators are case-sensitive under queryType: 'full', and tokenize strips operator punctuation but cannot strip an operator word, so EAST TOBA AND MONIROSE HYDROELECTRIC PROJECT would build … AND AND AND … and 400 on a parse error. en.microsoft lowercases the index side, so a lowercased term still matches. Exposure is 2 of 72 labels across two sets, measured 2026-08-03.
• MIN_FUZZY_LENGTH is 4. Shorter terms get neither a fuzzy nor a prefix variant. Without the floor the query "the and of" returns a page of OCR debris, because the~1 matches a scanned fragment reading "th"; with fuzzy off it returns 0.
• MAX_TERMS is 16, and it protects recall rather than cost. Under a conjunction each extra term is a filter, not a ranking contribution, so terms past the cap can only reduce recall.

ANALYZER_STOPWORDS holds the 20 terms at or above MIN_FUZZY_LENGTH that en.microsoft removes at query time. They get neither variant: both ~1 and * bypass the query analyzer, so on a term the index does not hold they make the clause unsatisfiable, which is fatal under a conjunction. The plain term stays and analyzes away harmlessly. Regenerate the list with searchChunks({keywords: word, fuzzy: false}) — a count of 0 means the analyzer removed it; the Analyze API returns 403 for the app identity.

The ACL filter

Reads compose src/helpers/access-odata.js — filterFor(access, partitionField). Projects scope on id, everything else on projectId. OData has no false literal, so "matches nothing" is an empty flag and the caller issues no request at all. A null or empty filter is unrestricted.

Semantic reranking

Chunk results are reranked by Azure's semantic ranker on top of the Lucene query. How it is wired and how it degrades: Search-Measurements.

Highlighting

retrievable: false does not disable highlighting. @search.highlights returns marked fragments while the response carries no chunk text at all. This is why any probe that needs to read chunk text must read Cosmos, not the search service.

Project and document cards use the same mechanism rather than a browser-side regex, which marks words the index never hit and misses stemmed ones it did — en.microsoft matches flooding for flood, a regex marks neither. The client path is kept only for the Cosmos project-list read, which has no analyzer to ask, and for boundary-name lists that never touch the search API.

Field-level markup is dropped wherever the frontend substitutes text of its own — a rebuilt document title, a generated project description — because marking a phrase inside a sentence we wrote claims a match that did not happen. A fragment can arrive carrying a closing sentinel whose opener was trimmed off, rendering as a stray </mark> in an [innerHTML] binding, so fragments are balanced individually before they are joined.

Document search fans out through projects

A Cosmos document row does not carry projectName and an indexer reads one container, so the documents index cannot hold it. Dropping project-name matching would silently cost 77% of hits for "Ajax" and 66% for "pipeline" (measured 2026-08-04). searchDocuments recovers it with a second leg through the projects index, capped at MAX_PROJECT_FANOUT (25) ids and run under the caller's own document ACL.

The two legs must be disjoint, which is what makes stateless paging possible: leg two carries not search.ismatch(<leg one's own query>, …), so the union is leg one followed by leg two and skip - direct.count is exact. The excluded query has to be leg one's byte for byte — same terms, same fuzzy arm, same trailing *, same fields, queryType: 'full' — or the legs stop being complements and pages overlap and drop rows. Overlapping legs leave 53 of 250 matching documents unreachable from any page for keywords=pattullo at pageSize=10 over pages 0–25.

Leg two's documents matched on their project's name, not their own metadata, so they carry no @search.highlights and their cards render unmarked.

Deletes

The _ts high-water mark cannot see deletes and no data source declares a deletion detection policy, so a hard-deleted row stays searchable until it is deleted from the index explicitly. That is application work — aiSearch.deleteFromIndex in deleteProject and deleteDocument, and deleteChunksForDocument for chunk text. Deleting from the index is a write, so the app's identity needs Search Index Data Contributor, not Reader.

deleteChunksForDocument pages, because a search returns at most 1000 keys and a document can hold more chunks. It loops until the reported count is exhausted, capped at 25 rounds. Deletes here are not read-your-write, so re-seeing keys is expected; a total that does not fall between rounds is the stop condition, not a retry.

Search controller decisions

Project id spaces

Three id spaces meet on a search row and none is derived from another: _id is the Eagle ObjectId (eagle-public routes and eagle-api's /api/project/{id} accept only that), id/projectId is the DEMI project id and Cosmos partition key, and trackProjectId is the Track master id — null, never the DEMI id, for an Eagle-only project, which the merge writes under an eagle-<ObjectId> key. Deriving one from another silently matches nothing rather than erroring.

Unresolved project ids pass through as literals

&project= carries an Eagle ObjectId and the indexes hold DEMI ids, so the controller resolves it before any filter is built. An ObjectId with no project row passes through unchanged rather than being refused: the seed admits documents parented by a ProjectNotification, whose ObjectId-shaped _id is itself the partition key. A literal that matches nothing is safe; dropping the key would widen to the whole corpus.

Why the fuzzy parameter is ignored

fuzzy is forced on. eagle-public hard-codes fuzzy=false on every search and eagle-search fuzzes anyway, so honouring the wire value would make DEMI answer a strictly smaller set than the service it replaces. The fuzzy arm cannot outrank the exact one, and the parameter stays accepted so saved URLs do not 400.

Page size clamping

pageSize is clamped with parsedPageSize >= 1 ? parsedPageSize : 10 rather than Math.max(1, ... || 10), so abc (NaN), 0 and -1 all land on the documented default of 10; the Math.max form would turn -1 into a one-row page, inventing a fourth behaviour. Unclamped, pageSize=abc returns the entire visible corpus and pageSize=-5&pageNum=3 reaches the service as {top: -5, skip: -15}. A second Math.min(…, 5000) caps the effective page above the clamp, so an unindexed Cosmos read cannot be asked for an unbounded page either; indexed reads hit the lower MAX_PAGE_ROWS guard first.

Why large pages are refused, not truncated

runSearch sends $skip, clamped at MAX_SKIP (100,000), and fills a page from consecutive SERVICE_MAX_TOP (250) requests up to MAX_PAGE_ROWS (500); the controller computes skip = pageNum * pageSize in the caller's own row unit. Above MAX_PAGE_ROWS the only alternatives are a 400 or twenty round trips per keystroke against a 1-SU service. The guard fires on keyword, criteria, or dataset=Document searches. The keyword test is load-bearing: storage.service.ts and projects.component.ts both call getAllFull(1, 1000000) for the projects map with no keywords and no criteria, so they take the Cosmos list path and never reach the guard.

Totals are measured, never the page length

res.json is wrapped once, so the eagle envelope and the analytics event are emitted from one place. searchResultsTotal is emitted only where a total was actually measured — absent means "not measured", while 0 is a claim about the index. Falling back to the page length makes pageSize=500 with no pageNum report 500 for a registry of any size, collapsing eagle-public's pager to one page.

The dropped-keys report

meta.dropped is omitted when nothing was dropped and otherwise always carries both .filter and .sort. A dropped filter widened the result set; a dropped sort left the caller reading an arbitrary order believing it is the one they asked for — flattening the two would say neither. Without the report, a discarded filter answers 200 with a plausible full-corpus page, signalled nowhere but the log.

Project reads split between Cosmos and the index

A bare project list is a read, not a search, so it stays on Cosmos in the repository's own order; keywords or criteria go to AI Search. Both branches serve the same corpus: every row the caller's read[] ACL admits, whatever system the row came from.

Native document lists choose a store

GET /documents serves one project's plain list and an ids= read from Cosmos. That means a project or ids parameter with no q, no sort, no displayNameContains, and no isFeatured, legislation or date filter. Every other list, cross-project lists without q included, pages in the index. The Cosmos list reads by overfetch-and-slice against a repository that clamps maxItemCount to 1000, and the container indexes neither datePosted nor displayName, so it cannot page or order a large project. The trade for the index path: list metadata is stale by up to the indexer's PT5M pass. Visibility is not, because setDocumentPublished and the project cascade write ACLs straight into the index. Update-form images are left out on both paths.

Native GET /documents

Paging: page (from 1), pageSize, count=true. The index returns the page and @odata.count in one request, and x-total-count is that count. On the index path a pageSize above 500, a filter with no index column, a contains filter with no index column, or any flag parameter (see "Native list filters") is a 400. The re-read does not apply CONTAINS, so a text filter the index cannot match would otherwise vanish; it is refused.

The index returns only ids. The rows are re-read from Cosmos by id, which applies each record's own field dials, and put back in index order. A stale index makes a page short; it does not leak.

Order, by orderby, always ending in id so pages do not overlap:

Request	Order
sort named	that key's index column (table below), then id
q, no sort	relevance (search.score() desc), then id
no q, no sort	datePosted desc, then id

Sort keys, each ascending or descending (- prefix): datePosted, displayName (column displayNameSort), sortOrder, type, milestone, projectPhase, documentAuthorType, isPublished, legislation, isFeatured, documentAuthor (column documentAuthorSort). Most rows hold sortOrder 0, so a sortOrder sort breaks ties by datePosted desc, then displayNameSort asc, then id. Any other key is a 400.

Filters: project (alias of projectId), typeId, milestoneId, projectPhaseId and documentAuthorTypeId take several values. legislation takes several values, each matching ^\d{1,4}$. isFeatured takes one value, true or false. datePostedStart and datePostedEnd take one YYYY-MM-DD day each; the end day is included. displayNameContains is a name match, described under "Contains filters". These values reach the index as unquoted literals, so each is checked before any OData is built, and a bad value is a 400. Unchecked, legislation=1) or (true would close the ACL clause and expose @odata.count.

The searchVis clause is sent only when q is present. Its definition: Search-Index-Reference "searchVis on documents".

Native list filters, shared by every kind

Every native list (/projects, /documents, /updates, /notifications) parses its query with parseNativeQuery (src/helpers/native-query.js). A parameter the kind does not declare, or a control parameter (page, pageSize, count, sort, q, ids) or single-value filter given twice, is a 400. A sort key outside the kind's list is a 400, cannot sort on: <key>.

A sort or filter on a field the caller cannot see is a 400 before any read. A staff-only field answers a caller below staff with [<entity>] cannot filter on a field this caller cannot see: <field>. Every sort key has an included path in the container's index policy (azure/modules/cosmos-nosql.bicep); Cosmos cannot serve an ORDER BY on an unindexed path.

Five filter shapes:

Shape	Wire	Rule
exact	field=value	A repeatable filter takes several values, at most 100, matched as in. Others take one.
range	fieldStart, fieldEnd	One YYYY-MM-DD calendar day each, UTC. Start is from 00:00:00.000, end to 23:59:59.999, so the end day is included. A bad day is a 400.
contains	fieldContains	Case-insensitive substring. Trimmed; an empty value is ignored; over 200 characters is a 400.
presence	hasField	true or false only, else 400. true means the field holds a value other than null or ''; false is the exact complement.
flag	field	true or false only, else 400 <field> takes only true or false. false also matches a row where the field is missing.

Every clause also honours the record's own dial on the field: a record that hides a field below the caller's level never matches a filter on it, so count=true cannot reveal that value.

Contains filters

On a Cosmos read the filter is CONTAINS(c.<field>, text, true), a substring match. On an index path the match is the word match described in "and[nameContains] matches a fragment of the name". Two filters use it: documents displayNameContains, over the columns displayName, documentFileName and fileNameTokens, and projects nameContains with q, over name and nameTokens. A filter the documents index has no column for is a 400, cannot match text in <field> here.

Lists with q=

On /projects, /updates and /notifications a list with q reads at most 250 ranked ids from the index and pages them in memory, so its total is capped at 250. The index returns ranked ids only. Filters with an index column also narrow the index request; every filter, range, contains and presence clause is then applied again in the Cosmos re-read of those ids, with the dials. A list without q is read from Cosmos alone. /documents is described above.

Native GET /projects

Default order name asc. Sort keys: name, dateUpdated, proponentName, projectType, sector, region, currentPhaseName.name, eacDecisionSort. Phase sorts on the phase's name, not its id. eacDecisionSort holds eacDecision.name; it is a separate field so that a project with no decision does not gain an eacDecision object that readers show as a decision. src/scripts/backfill-project-sort-fields.js fills it on rows written earlier.

Repeatable filters: projectType, region, projectState, proponentName, sector, eacDecision, currentPhaseName, CEAAInvolvement. The last three name the List ref and match its _id. sector is an exact match. projectType matches both stored spellings (see Search-Index-Reference on PROJECT_TYPE_SPELLINGS). Single-value filters: regionalDistrict, municipality, electoralDistrict.

pcp is repeatable and filters on the project's comment periods: open, upcoming, closed or pending. pending is a period that starts within the next 7 days. The result holds projects with a period in any named status that the caller may read. Another value is a 400, pcp takes open, upcoming, closed, pending. GET /search/counts does not accept pcp.

Ranges: decisionDateStart, decisionDateEnd, dateUpdatedStart, dateUpdatedEnd. Contains: nameContains on name. With q the index applies the word match (see "Contains filters") and the Cosmos re-read of the ranked ids still applies the substring check; without q it is a substring match on Cosmos.

Cosmos drops a row that lacks its ORDER BY field, so upsert writes every sort field it can fill (sortFieldFills, src/repositories/projects.js): proponentName, projectType, sector and region as null when absent; dateUpdated from dateAdded as an ISO time, or null when dateAdded is absent or does not parse; currentPhaseName as {_id: null, name: null}. A field the row already holds, null included, is left alone.

Native GET /updates

Default order dateAdded desc. Sort keys: dateAdded, dateUpdated, publishDate, headline, type, pinned, status. The mirror stores pinned and complianceAndEnforcement as booleans (false when Eagle sets none) and status as null when there is none. src/scripts/backfill-update-sort-fields.js fills rows written earlier.

Filters: project (alias of projectId, the Eagle project id as stored; repeatable, at most 100 ids), type (repeatable), dateAddedStart, dateAddedEnd (a YYYY-MM-DD day each, the end day included), complianceAndEnforcement=true|false, and hasDocumentUrl=true|false. A null documentUrl and an empty string both count as no document. Callers below staff see published rows only.

Native GET /notifications

Default order notificationReceivedDate desc. Sort keys: notificationReceivedDate, name, type, region, decision, proponent, subType.

Filters: type, region and decision are repeatable; pcp takes one value, and none is a real value. A notification has no project axis, so role ACL alone decides which rows a caller sees.

Native GET /documents/passages

One row per chunk hit, for keyword search inside documents. q is required. pageSize is at most 100, count=true and page work as above, ids= and sort are not accepted. Filters: project, documentId and the four id filters above.

The chunks index answers the query (searchChunks). The parent gate then drops chunks whose document the caller cannot read, and the count is net of what it dropped (src/search/chunk-query.js, see "The parent document is the chunk gate"). A parent-field filter on a scope the backfill has not fully stamped answers 503 with the reason, and logs a warning: a 200 with zero rows would read as "no such documents".

Chunk filters answer on the chunk's own copy

and[type], and[milestone], and[projectPhase] and and[documentAuthorType] filter dataset=DocumentChunk directly: buildFilter maps each wire key onto a field the chunks index already carries (typeId, milestoneId, projectPhaseId, documentAuthorTypeId — DOCUMENT_FACETS in src/search/eagle-query.js), and emits one clause, same as any other chunk filter. No document-index lookup runs first. Field reference and how the copy stays current: Search-Index-Reference "Chunk filters carry a copy of their parent document's fields".

Three things are not answered on the chunk row, and all three take the same path out.

The first is a facet the live index turns out not to carry. Expressibility is read from the service, not from the chunks.json in the deployed package — nothing on the deploy path applies that file, so between an app release and the index PUT the two disagree, and emitting typeId eq … against an index without the field is a 400 the route answers as a 502 over the whole tab. The answer is cached for ten minutes, so widening the index starts working without a restart; an answer of "the probe could not run" is held for 30 seconds instead, because that is a hiccup and not a fact about the index.

Only a field the probe vouched for is sent. A field it could not reach is treated as inexpressible and takes the same path out as one the index does not carry, because the two possible mistakes do not cost the same: guessing that the field is there is a 400 that answers the whole tab as 502, while dropping it costs one filter that the documents index usually answers anyway. The probe asks one question per field and asks them all at once, so a field whose question failed is the only one left undecided — one field's hiccup drops one filter, not all four.

The second is a column the live index carries but this request's own rows do not all hold a value for yet. The schema probe only answers "does the column exist" — it says nothing about whether the rows behind this scope have been written. A chunk backfill-chunk-parent-fields.js has not reached holds null in all four parent columns, so typeId eq '<id>' over it is not a narrower answer, it is a non-match, and a 200 built that way reads as "this project has no documents of that type" rather than as a backfill still running. liveChunkFacets in src/search/chunk-query.js measures this directly, with a stale-chunk count taken over the caller's own ACL and project scope, and withholds all four facets together the moment that count comes back nonzero: one facet answering zero poisons the whole conjunction, so applying the carried columns and dropping the rest would be the same wrong answer at a smaller blast radius. This is the common case during a backfill run, not the exception the schema miss above is — most facets withheld while a backfill is behind take this path.

The third is a document-only key: datePostedStart, datePostedEnd, isFeatured, legislation, documentSource. A chunk carries no copy of those.

Update-form images are filtered out by default

An image uploaded through an Update's gallery or featured-image field is a Document with documentSource: 'UPDATE'. It shows on its Update only, so buildFilter adds documentSource ne 'UPDATE' to every dataset=Document query, unless the caller's own filter already names documentSource — a caller who asks for a source has said which ones they want, and the default steps aside. Legacy rows carry '' or no documentSource at all; the clause is a ne, not eq 'PROJECT', because null ne 'UPDATE' is true in Azure AI Search, so those rows are unaffected.

All three are resolved through the documents index — ask which documents match, scope the chunk query to their ids — under that index's own ACL, not the chunk query's. A document-scoped credential compares documentId on chunks and id on documents, so reusing the chunk clause here is a 400 naming a field the documents index does not carry, which the route answers as a 502. The list of ids is capped at 249, one service request's worth. Over the cap the key stays in dropped rather than being applied to an arbitrary prefix. Project-scoped, which is how eagle-public sends these, the set is a handful of documents and the filter applies; corpus-wide, a date range does not. So during a deploy or a backfill window a facet degrades to the behaviour it had before the chunk copy existed — capped, but applied — rather than to no filter at all. A key neither index can express, documentAuthor for one, is dropped and named in meta.dropped.filter.

Deep Search can be degraded three ways

meta[0].degraded on dataset=DocumentChunk carries any of these, and more than one at a time:

• missing: [field] — the live index could not answer a column the app selected, so the search layer dropped it and retried once. The page is narrower than asked for.

• reasons: ['chunk-parent-fields-unstamped'] — the facet filter was WITHHELD, not applied: this request's own scope is mid-backfill, and the documents-index recovery that should have answered it in the chunk filter's place was over the 249-document scope cap, so there is nowhere left to go and the page really is short. This is rarer than a plain schema miss: when the recovery lands under the cap, the filter is answered in full through the documents index and nothing is marked at all — most requests, since eagle-public sends these project-scoped.

• reasons: ['chunk-parent-fields-unknown'] — the same withheld filter, for a question nobody could answer rather than for an answer. A live-index probe that could not run, or a stale-chunk count the service could not give, withholds the facet exactly as a positive count does, and when the documents-index recovery is over the cap as well the page is just as short. It is reported separately because it clears differently: the unstamped mark clears when the backfill finishes, this one clears as soon as the service answers again. A column the probe reported absent is not one of these — that is a settled fact about the index, it clears when someone applies the index definition. Any two of these reasons can appear together — a probe that died while the count that did answer came back positive, for one.

• reasons: ['chunk-parent-fields-missing'] — the same withheld filter, because the live chunks index does not carry the column at all. That is what a deploy looks like before someone applies azure/search/indexes/chunks.json: the app knows the facet, the service does not. Like the other two it is only raised when the documents-index recovery was over the cap as well, since that is the only case where the page is actually short. It is named separately because it clears differently again — nobody waits it out, an operator applies the index definition — and because without it the wire shape of this case is a key in dropped.filter with no reason beside it, which is exactly what an ordinary over-cap drop looks like.

The mark is raised over the rows the caller asked for, not over the corpus. The count behind it is taken under the same filterFor clause the page was selected with, plus the request's project scope when it carries one, so a backfill running in one project does not banner every other project's page for its duration. The index-wide number is kept only as the free short-circuit it can answer: it is a superset of any scope, so zero there is zero everywhere, and null there still means "cannot say" and marks nothing. Counts are cached per access-and-project scope for the same ten minutes the schema answer is, in a map bounded at 32 scopes. A count of "cannot say" — the service failed, or the index carries no stamp column — is held for 30 seconds instead, the same window an unanswered schema probe gets, so an unanswerable scope is not pinned for ten minutes.

Unknown is not zero in either direction. A per-scope count of null withholds the facet exactly as a positive count does — the facet is dropped and routed to the documents-index recovery either way — but only a measured backlog raises chunk-parent-fields-unstamped: it is true only when the count came back positive, never when it came back null. An unanswered count is not evidence the backfill is behind, so it is reported as chunk-parent-fields-unknown instead, which is a claim about the question rather than about the corpus.

The reason carries no count. This route answers anonymous callers and a backfill statistic is not something a caller can act on; the named reason is the whole of it.

The state is read off the stamp — parentFieldsVersion below CHUNK_PARENT_FIELDS_VERSION, or absent — never off the four values. A document with no List refs at all legitimately stamps four nulls, and prod has many, so "all four are null" is permanently true of correctly stamped chunks: a probe counting those never reaches zero and the mark it gates never clears. A live index that carries the four fields but not the stamp column answers "unknown", which leaves the page unmarked rather than claiming the backfill is done.

Chunk paging is a window

Chunk rows are grouped by parent document, so a page of documents costs a window of chunks, and the window is both the fetch and the paging unit. skip and top must share that unit or consecutive pages leave gaps. The window is capped at SERVICE_MAX_TOP, not MAX_PAGE_ROWS, because this query runs on every debounced keystroke and anything larger costs a second request. pageSize is a fetch knob for this dataset, not a row count.

The parent document is the chunk gate

A chunk is a fragment of its document, so a caller who cannot read the document cannot read its text. The chunk's own read[] is an ingest-time snapshot and DEMI cascades a document's ACL onto chunks only on a publish transition, so a stale chunk ACL cannot leak text on its own. count is reported net of what the page withheld and floored at the surviving chunk count, never at the row count — one row can carry a dozen matches.

A failed search is never an empty one

Every branch answers 502 on failure rather than 200 with []: a 200 asserts "0 results" as a fact about the index, and the analytics table records it as a zero-result search. No branch falls back to another source — falling through to a keywordless Cosmos read would let one and[centroid]=x from any anonymous caller turn a keyword search into an arbitrary page of the corpus.

The summary endpoint's gates

GET /api/search/summary is on authMiddleware, not passiveAuthMiddleware: a <mark> fragment discloses a phrase, a synthesised cross-chunk answer discloses substance. Three gates are load-bearing — the AI Search ACL filter, chunksRepo.getById re-applying the ACL at the database, and the parent-document check the chunk search path also applies. The model is never called when retrieval returns nothing; that is the grounding guarantee.

and[nameContains] matches a fragment of the name

and[nameContains] renders as search.ismatch(..., 'full', 'any') against name on dataset=Project or displayName on dataset=Document. It is a substring filter, not the field eq value exact match and[name] and and[displayName] already give. Sending it on any other dataset, or with a value over 200 characters, is a 400.

The value is split into words on whitespace and on punctuation — commas, brackets, hyphens and the rest — so Kitimat LNG, Phase 2 splits into four words, not one string with a comma in it; commas are not a list separator here. The apostrophe is the one character that stays inside a word, so dam's does not split. Each word of two or more characters is matched as a prefix, so Kitimat LNG, Phase 2 asks the index for Kitimat*, LNG*, Phase* and 2 in the name. A word the query analyzer drops as a stopword (of, and, the, and the rest of the en.microsoft list) matches whole instead of as a prefix, because a starred stopword demands a literal the index does not hold and would zero the whole match under the AND join. Every word has to match. A value that is empty, or holds no letters or digits at all, adds no filter — the same answer an empty cell gets.

GET /search/counts refuses and[nameContains] the same as every other /search filter — see "GET /search/counts answers one number per tab" below.

GET /search/counts answers one number per tab

GET /search/counts?keywords=&prefix=&datasets=Project,Document,RecentActivity,ProjectNotification,CommentPeriod gives eagle-public's unified search page a match count per record-type tab in one request instead of one full result page per tab. Same guards as /search (passiveAuthMiddleware then credentialsMiddleware): the counts are ACL-scoped, so credentials load before any filter is built.

The response is a one-element array:

[{
  "counts": { "Project": 4, "Document": 206, "RecentActivity": 13, "ProjectNotification": null, "CommentPeriod": 9 },
  "meta": [{ "unavailable": ["ProjectNotification"], "degraded": [], "cached": false }]
}]

A count of null means unknown, never zero. A leg that fails answers null and names itself in meta[0].unavailable; the request still answers 200, because one dead tab must not take the other four with it. meta[0].degraded names a leg answered from the Cosmos container count instead of the index — RecentActivity falls back this way when its index is unset or missing; a fallback answer is a different question (CONTAINS on the container, not BM25 on the index) so the caller is told. ProjectNotification has no fallback: unreachable there means null, not a container count. CommentPeriod is Cosmos-only and has no index leg at all: its count is commentPeriodsRepo.countAll under the caller's access, keywords and no status filter — the same predicate the keywordless, status-less list read counts against — and never null, since there is no index call to fail.

keywords (or q) and prefix take the same defaults as /search — prefix=false turns off prefix matching, on by default. datasets narrows which counts the response carries; all five are always computed, because the page asks for all five on every keystroke pause and a narrower request would otherwise split one cache entry into several that each pay full price.

This endpoint takes no filters. Every /search filter parameter — and[...], dataset, pageSize, sortBy, pageNum, fuzzy, project, top, docIds, period — is refused with a 400: a badge is one number for all four record types at once, so there is no filter it could apply, and accepting one that is silently ignored would answer a wrong count with nothing to say so. The 400 covers /search parameters too, not just unknown keys — keywords, q, prefix and datasets are the whole allow-list.

Counts are cached 45 seconds, keyed on keywords, prefix and an access fingerprint (tier, level, roles, teams, project scope, compartment, credentials), with the promise cached rather than the value so concurrent identical requests share one fan-out instead of one each. The document count reuses searchDocuments's own two-leg total (see "Document search fans out through projects" above), so it equals searchResultsTotal for the same query on /search.

Unsupported parameters 400, inexpressible filter keys drop

A parameter this endpoint does not read is refused. page=2 for pageNum=1, or an unparsed filter key, would otherwise answer 200 with page one or the whole corpus — a wrong answer that looks fine. A filter key the index cannot express is genuinely inexpressible, so it is dropped, logged, and named in meta.dropped.

Query parser shapes

andParams reads both wire shapes: querystring.parse, the dispatcher's query parser, gives the literal key and[type], while a caller that sends a nested req.query.and = {type: 'x'} shape also works. Handling one shape silently drops every filter and returns the whole visible corpus where a filtered handful was expected.

Default order and relevance

search: '*' has no stable order, so a keywordless page gets DEFAULT_ORDER plus an id tiebreak; without it page 2 repeats and omits rows from page 1, which reads as data loss. A keyword search gets search.score() desc, id asc instead — relevance made explicit so it can be paged, since an alphabetical default would sort the ranking away. chunks gets no $orderby at all: every field in it is sortable: false, so deep chunk pages are not stable, and $orderby would override semantic reranking on the one index that has it.

Eagle mirror datasets, _schemaName and x-total-count

dataset also accepts User, Group, Inspection, InspectionElement and InspectionItem. These are Cosmos list reads of the Eagle mirrors under the caller's access, not index queries. They take no keywords: q answers 400. User has no project axis, so a project filter answers no rows. and[inspection] filters InspectionElement and InspectionItem by the parent inspection's Eagle id, and and[element] filters InspectionItem by its element.

dataset=Item fetches one record by Eagle id. It requires _id and _schemaName (one of the model names above, plus Comment, CommentPeriod, ProjectNotification and RecentActivity) and takes nothing else. It answers a bare array of zero or one record, not the search envelope. _schemaName on any other dataset answers 400.

and[eaoStatus] applies to dataset=Comment and takes Pending, Published, Rejected or Deferred, comma-separated for several. Any other value answers 400.

Every search response carries an x-total-count header with the measured total, the same number as meta[0].searchResultsTotal. On dataset=Item it is the length of the array. The header is exposed to CORS. HEAD /search returns the headers with no body.

Reads of the user, group and inspection mirrors

These routes take passiveAuthMiddleware.strict and credentialsMiddleware, the same as GET /documents. strict answers a rejected Bearer token with its 401 instead of serving the anonymous view; GET /search takes the lenient passiveAuthMiddleware. Each list route has a by-id route at /:id.

Route	Filter
GET /users	none
GET /groups	project
GET /inspections	project
GET /inspection-elements	inspection
GET /inspection-items	inspection, element

Sort keys of the mirror lists. Each key is in the container's index policy.

Route	Sort keys
GET /comment-periods	dateStarted, dateCompleted, isPublished
GET /organizations	name, companyType, companyLegal
GET /users	displayName, orgName, email, phoneNumber
GET /groups	name

companyLegal on organizations and email and phoneNumber on users are staff-only. A caller below staff who sorts on one gets a 400, for example [users] cannot filter on a field this caller cannot see: email.

By-id reads of groups take ?project=. The other by-id reads of inspection kinds take ?inspection=, the partition key.