Search

Search

Lexicon search has four user-facing modes. Treat them as lenses over the same search entries rather than separate indexes.

Modes

Mode	Use it for
hybrid	Default search. Combines path/name token matches, phrase matches, metadata, references, and semantic scores when embeddings are available.
semantic	Natural-language meaning search. Best with --embedding-provider mlx and a cached embedding index.
token	Deterministic path, name, type, default, and reference lookup. Useful for editor tooling and exact-ish IDs.
lexical	Phrase and fuzzy text search over IDs, notes, comments, defaults, and metadata without synonym rules.

Scopes

Scope	Search space
own	The declared document graph only. Fastest and most predictable.
live	Search own first, then expand likely hits through resolved lemma context. Good for interactive search over inherited context.
full	Materialize the resolved lemma search space, then search it. Traversal counts depth and budget at each child edge and stops repeated inherited structure so recursive types do not expand forever.

--depth, --candidates, and --budget bound live and full traversal.

Deterministic Search

swift run lexicon search Examples/search-demo.lexicon submit order \
	--mode hybrid \
	--limit 5 \
	--embedding-provider none

Useful deterministic examples:

Query	Expected kind of match
submit order	API operation, such as demo.api.order.submit.
payment decline	API or UI error concepts around declined payments.
free shipping	Campaign or feature terms.
checkout button	UI button terms.
purchase completed event	Analytics event terms.

Semantic Search

On Apple platforms, the base build can use NaturalLanguage sentence embeddings. For MLX-backed semantic search, build the CLI with the MLXSearch package trait:

swift run --traits MLXSearch lexicon search Examples/search-demo.lexicon \
	"customer asks for money back after purchase" \
	--mode semantic \
	--embedding-provider mlx \
	--embedding-model TaylorAI/bge-micro-v2

The first MLX semantic search builds a document embedding cache and logs indexing progress to stderr. Pass --embedding-cache to choose a cache path, or --rebuild-embeddings to force regeneration.

Natural-language queries are useful when the user's phrase does not share tokens with the vocabulary:

Query	Likely match
customer asks for money back after purchase	refund request terms
late delivery after carrier delay	delivery delay terms
card issuer rejected transaction	payment decline terms
basket threshold delivery cost	free shipping campaign terms
stock availability nearly sold out	inventory stock terms

Example Result Snapshots

The examples below use Examples/search-demo.lexicon. They are useful for sanity-checking ranking changes, but semantic results can vary when the embedding model changes. The semantic examples were generated with TaylorAI/bge-micro-v2.

Hybrid Examples

Query	Top result
submit order	demo.api.order.submit
payment decline	demo.ui.checkout.error.payment_declined
free shipping	demo.feature.campaign.free_shipping
refund status	demo.api.refund.status
customer loyalty	demo.api.customer.loyalty
delivery delay	demo.analytics.event.delivery_delayed
checkout button	demo.ui.checkout.button
low stock badge	demo.ui.product.card.badge.low_stock
escalate urgent payment	demo.support.ticket.escalate
purchase completed event	demo.analytics.event.purchase_completed
guest checkout	demo.feature.checkout.guest
seasonal recommendation	demo.feature.recommendation.seasonal

Semantic Examples

Query	Top result
customer asks for money back after purchase	demo.api.refund.request
late delivery after carrier delay	demo.fulfillment.delivery.delay
card issuer rejected transaction	demo.api.payment.decline
popular products recommendation	demo.feature.recommendation.trending
holiday weather products	demo.feature.recommendation.seasonal
lapsed customers returning offer	demo.feature.campaign.winback
basket threshold delivery cost	demo.feature.campaign.free_shipping
reserve funds with card issuer	demo.api.payment.authorize
stock availability nearly sold out	demo.api.inventory.stock
inventory reservation during checkout	demo.api.inventory.reservation
help article checkout delivery questions	demo.support.knowledge_base.article
product description benefits fit materials	demo.content.product.description

Token Examples

Query	Top result
demo api order submit	demo.api.order.submit
demo api payment decline	demo.api.payment.decline
demo ui checkout button primary	demo.ui.checkout.button.primary
demo ui product card badge low stock	demo.ui.product.card.badge.low_stock
demo analytics event checkout started	demo.analytics.event.checkout_started
demo support ticket status	demo.support.ticket.status
demo fulfillment warehouse ship	demo.fulfillment.warehouse.ship
demo feature checkout save card	demo.feature.checkout.save_card
demo content campaign email	demo.content.campaign.email
demo type component banner	demo.type.component.banner
demo type channel push	demo.type.channel.push
demo api inventory reservation	demo.api.inventory.reservation

Lexical Examples

Query	Top result
card issuer rejected the transaction	demo.api.payment.decline
urgent billing or payment issues	demo.support.ticket.escalate
late delivery notification	demo.fulfillment.delivery.delay
support agents	demo.api.order.status
customer submits an order	demo.api.order.submit
carrier pickup to delivery	demo.fulfillment.delivery.tracking
quantity is limited	demo.ui.product.card.badge.low_stock
save payment method for a future checkout	demo.feature.checkout.save_card
shopper search query	demo.analytics.event.search_performed
returning shopper offer	demo.feature.campaign.winback
email and push messages	demo.ui.account.settings.notifications
basket threshold delivery cost	demo.feature.campaign.free_shipping

Names-Only Search

swift run lexicon search commerce.lexicon checkout button \
	--mode token \
	--names-only

Use --names-only for editor completion-style workflows where notes, comments, defaults, and references should not influence ranking.

Source-Only Search

swift run lexicon search commerce.lexicon submit \
	--source-only

By default, search composes imports. Use --source-only when you need to inspect only the file being edited.

Practical Search Workflow

1. Start with --mode hybrid --scope own.
2. If a concept is inherited through types, retry with --scope live.
3. If a query is natural-language and token-poor, use --mode semantic.
4. If ranking seems noisy, use --names-only or switch to token.
5. If results are unexpectedly missing, run lexicon validate and check composition diagnostics first.