Skip to content

feat: Pre-computed feature vectors for online serving#6463

Merged
ntkathole merged 1 commit into
feast-dev:masterfrom
ntkathole:precompute_feature_service
Jun 2, 2026
Merged

feat: Pre-computed feature vectors for online serving#6463
ntkathole merged 1 commit into
feast-dev:masterfrom
ntkathole:precompute_feature_service

Conversation

@ntkathole

@ntkathole ntkathole commented Jun 2, 2026

Copy link
Copy Markdown
Member

What this PR does / why we need it:

Problem

When a get_online_features() request spans multiple feature views, the online server issues a separate store read per feature view. For latency-sensitive use cases, we need improvement that cannot be achieved by optimizing the existing read path alone.

Fixes #6185

Solution

Introduce pre-computed feature vectors - an opt-in feature (precompute_online=True on FeatureService) that stores all of a service's features for each entity as a single serialized protobuf blob. At read time, the server fetches one blob per entity instead of N reads per feature view, reducing the operation to O(1).

Key design decisions:

  • Store-agnostic: Core fast-path logic (read/write_precomputed_vector) lives in the base OnlineStore class, working with all backends (Redis, DynamoDB, PostgreSQL, etc.)
  • Automatic refresh: Pre-computed vectors are rebuilt automatically after materialize, materialize-incremental, and push -- no separate manual step required
  • Strict error handling: When precompute_online=True, there is no silent fallback to per-FV reads; missing or stale vectors raise a RuntimeError, making problems visible immediately
  • Schema-aware: A fingerprint of feature names detects schema changes and rejects stale vectors with column-order-independent comparison
  • Per-FV TTL enforcement: Individual feature view TTLs are checked within the pre-computed blob

Impact

Here are the real numbers from the tuned server (4 CPU, 9 workers), same pod, same run, same service (benchmark_customer_service, 6 FVs, 29 features):

Batch Size (rows per request) p99 Regular (per-FV) p99 Precomputed Speedup
1 10.74ms 1.70ms 6.3x
5 44.91ms 3.00ms 15.0x
10 60.37ms 2.07ms 29.2x
50 120.12ms 7.49ms 16.0x
Batch Size p50 Regular p50 Precomputed
1 5.95ms 0.98ms
5 9.66ms 1.37ms
10 16.60ms 1.81ms
50 60.07ms 5.27ms
100 85.58ms 9.48ms
500 218.79ms 40.25ms
  • 5-15x latency reduction: In-pod benchmarks show p50=0.66ms, p99=0.94ms for single-entity reads (down from ~5-10ms with per-FV reads across 6 feature views)
  • Zero changes to read API: Existing get_online_features() calls work unchanged; the fast path activates transparently when precompute_online=True
  • No regression on regular path: The non-precomputed per-feature-view read path is completely untouched
  • Opt-in only: Defaults to False; no impact on existing deployments

@ntkathole ntkathole self-assigned this Jun 2, 2026
@ntkathole
ntkathole requested review from a team as code owners June 2, 2026 07:31
@ntkathole
ntkathole requested review from HaoXuAI, franciscojavierarceo and shuchu and removed request for a team June 2, 2026 07:31
@ntkathole
ntkathole force-pushed the precompute_feature_service branch from 6246f28 to b967bc2 Compare June 2, 2026 07:39
Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
@ntkathole
ntkathole force-pushed the precompute_feature_service branch from b967bc2 to d5ed6ca Compare June 2, 2026 13:46
@ntkathole
ntkathole merged commit 8011550 into feast-dev:master Jun 2, 2026
24 of 29 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Pre-computed feature vectors for online serving

2 participants