refactor(cli): shared pluggable extractor

[akijakya]

Summary

Introduces a shared, pluggable OASF extractor in the client module and moves the natural-language search decomposer onto it, so both dirctl and (in a later step) the server gateway share one extractor abstraction over either an in-process local backend or a remote gRPC OASF-SDK server.

This is the foundation slice of the AI Catalog natural-language search epic (#1903): it establishes the abstraction and refactors dirctl onto it, with no behavior change to dirctl search on the local-assets path.

Why

Today the OASF extractor is a CLI-only concern: it lives in cli/internal/extractor as an in-process library over locally-provisioned assets (~89 MB), and the NL decomposition lives in cli/internal/nlsearch. The server gateway (a different module) can't reach any of it, and there's no way to reach the extractor over the network. To bring NL search to the AI Catalog UI, the extraction has to run on a backend that owns the extractor — and a browser can't host the model — so the extractor needs to be:

• shared across cli and server (lives in client, which both already import), and
• pluggable between a local in-process backend and a remote gRPC OASF-SDK server, chosen at runtime.

What changed

New client/extractor package

• Extractor interface + ExtractOptions, returning the SDK's extractor.Result so downstream consumers are backend-agnostic.
• localExtractor — wraps the in-process oasf-sdk/pkg/extractor over provisioned assets. Load, LoadConfigured, Provision, SmokeCheck, and the asset/config helpers move here from cli/internal/extractor.
• remoteExtractor — a gRPC ExtractorService client that maps ExtractResponse back into the SDK Result shape.
• ResolveExtractor / ResolveConfigured — pick the backend: remote-first when RemoteAddr is set, else provisioned local assets, else one actionable error naming both fixes.

New client/nlsearch package

• Decompose / DecomposeWithMinScore now take the Extractor interface + ExtractOptions instead of the concrete *sdk.Extractor and variadic sdk.QueryOption.

Config

• client/config.Extractor gains a persisted RemoteAddr field (empty ⇒ local assets).

CLI rewire (init, import, search, routing)

• All four callers now use client/extractor + client/nlsearch; cli/internal/extractor and cli/internal/nlsearch are deleted.
• search/routing resolve via ResolveConfigured; routing's semantic-only embedding weights pass through as local-backend tuning.

Dependencies

• Adds buf.build/gen/go/agntcy/oasf-sdk/grpc/go (pinned to the same buf commit as the already-required protocolbuffers/go). go mod tidy for client, cli, and tests.

Design notes

• The core SearchService stays NL-agnostic. The planner lives in clients (today dirctl; the gateway later), not the server.
• Remote backend can't set per-call embedding weights. The routing path's semantic-only weights are a construction-time, local-only knob; when a caller resolves to a remote server it uses the server's default weighting. Documented, and it has no effect on this PR (local path unchanged).
• DRY: one decomposer and one extractor abstraction, shared via the client module — no new module.

Testing

• New unit tests: backend resolution (chooseBackend / ResolveExtractor), remoteExtractor response→Result mapping (fake gRPC client), and Decompose tier/threshold filtering + keyword wrapping + option forwarding (the decomposer had zero tests before). Existing extractor tests were moved so coverage is preserved.
• task lint:go — 0 issues across all 7 modules.
• client and cli module test suites — green.
• Note: server/store/oci → TestIntegrationOCIStoreWorkflow fails on task test:unit, but it is pre-existing and unrelated — that module is byte-identical to the base commit, and the test requires a live OCI registry at localhost:5555 (normally provided by the e2e testenv), so it fails on any plain unit-test run.

Scope / follow-ups

• dirctl init does not yet expose a flag to set RemoteAddr; wiring the remote extractor into dirctl and the daemon/gateway is [Feature]: Extractor config (RemoteAddr) + resolver wiring for dirctl and gateway #1906.
• The backend POST /v1/search endpoint ([Feature]: Backend POST /v1/search endpoint (proto + gateway handler, extract-then-filter) #1905), Helm deployment of the OASF-SDK server ([Feature]: Helm: optional OASF-SDK extractor server deployment + config #1907), UI wiring ([Feature]: AI Catalog UI: wire the search bar to /v1/search #1908), and e2e ([Test]: e2e for AI Catalog /v1/search — local (assets) + remote (OASF-SDK server) #1909) build on this.

Closes #1904

[Copilot]

Pull request overview

This PR lays the groundwork for sharing natural-language search extraction across dirctl and (later) the server gateway by introducing a shared, pluggable extractor abstraction in the client module and refactoring CLI NL decomposition to depend on that abstraction.

Changes:

• Added client/extractor with a backend-agnostic Extractor interface plus local (assets) and remote (gRPC) implementations, and resolution helpers (ResolveExtractor / ResolveConfigured).
• Moved NL decomposition into client/nlsearch, updating dirctl search/routing callers to use the shared interface and options.
• Updated shared config to persist an optional RemoteAddr, and adjusted module dependencies/tests accordingly.

Reviewed changes

Copilot reviewed 23 out of 30 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
tests/go.mod	Adds indirect dependency for generated OASF-SDK gRPC bindings used by new remote extractor tests.
tests/go.sum	Records checksums for the new generated gRPC bindings dependency.
client/go.mod	Adds direct deps needed by shared extractor (gRPC bindings, zerolog) and updates indirect deps via tidy.
client/go.sum	Records checksums for newly pulled transitive dependencies.
client/config/config.go	Persists extractor.remote_addr in the shared config schema.
client/config/config_test.go	Adds coverage ensuring RemoteAddr survives save/load round trips.
client/nlsearch/decompose.go	Refactors decomposition to consume client/extractor.Extractor + ExtractOptions.
client/nlsearch/decompose_test.go	Adds unit tests for filtering and option forwarding without requiring a real model.
client/extractor/extractor.go	Introduces the backend-agnostic Extractor interface and shared option/result types.
client/extractor/config.go	Extends extractor config with RemoteAddr and updates package-level docs.
client/extractor/config_test.go	Adds tests for default resolution and local config validation.
client/extractor/assets.go	Minor comment tweak in teardown guard path.
client/extractor/assets_test.go	Adds tests for provision detection and teardown safety guards.
client/extractor/local.go	Implements local backend wrapper and moves provisioning/load/smokecheck into client.
client/extractor/load_test.go	Adds tests ensuring load paths error clearly when unconfigured/unprovisioned.
client/extractor/resolve.go	Adds backend selection + configured resolver, including remote-first selection.
client/extractor/resolve_test.go	Adds unit tests for backend selection and resolution behavior.
client/extractor/remote.go	Implements remote backend and maps gRPC responses into SDK result shapes.
client/extractor/remote_test.go	Adds mapping tests using a fake gRPC client.
client/extractor/provision_integration_test.go	Adds an opt-in integration test for provisioning + smoke check.
cli/cmd/search/nlsearch.go	Rewires CLI NL search to resolve the shared extractor and call shared decomposition.
cli/cmd/routing/nlsearch.go	Rewires routing NL search to shared extractor/decomposer, keeping local-only tuning knobs.
cli/cmd/init/run.go	Switches init to use the shared extractor package.
cli/cmd/init/options.go	Switches init options wiring to shared extractor package.
cli/cmd/init/run_test.go	Updates init tests to reference shared extractor package.
cli/cmd/import/config.go	Switches import enricher to shared extractor package for local load/provisioning.
cli/go.mod	Adds indirect dependency for generated OASF-SDK gRPC bindings.
cli/go.sum	Records checksums for new gRPC bindings dependency.
cli/internal/extractor/provision.go	Deleted; functionality moved to client/extractor.
cli/internal/extractor/load.go	Deleted; functionality moved to client/extractor.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[codecov]

Codecov Report

❌ Patch coverage is 61.49068% with 62 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
client/extractor/local.go	57.4%	16 Missing and 7 partials ⚠️
client/extractor/resolve.go	48.6%	18 Missing and 1 partial ⚠️
client/extractor/remote.go	77.6%	9 Missing and 4 partials ⚠️
cli/cmd/routing/nlsearch.go	0.0%	3 Missing ⚠️
cli/cmd/search/nlsearch.go	0.0%	3 Missing ⚠️
cli/cmd/import/config.go	50.0%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[paralta]

LGTM 👍