This roadmap details the planned features and priorities for Spice.ai Open Source, aligning with the mission to empower developers to build decision-making, data-driven AI applications. It is regularly refined based on community feedback, customer needs, and strategic goals.
To propose features or report issues, please file an issue or connect with us on Slack. Your input drives our direction.
- Track active bugs on GitHub.
- Report new bugs via this issue template and share details on Slack for faster resolution.
Focus: Production-Grade HA.
- Spice Cayenne (GA): Production-ready columnar storage format for accelerated datasets.
- Active-Active HA (GA): Production-ready multi-node deployment with zero-downtime failover.
- Distributed Query (GA): Stable multi-node query execution for large-scale workloads.
- Backup & Point-in-Time Recovery: Snapshot restore for accelerated datasets.
- Accelerated Dataset Distribution: Replicate accelerated datasets across executor nodes.
Focus: Schema Management & Search.
- Distributed Search (Alpha): Federated vector and full-text search across multiple nodes.
- Schema Registry (Initial): Versioning and backward compatibility checks.
- Hybrid Search Ranking: Configurable fusion strategies (Reciprocal Rank Fusion (RRF) weights, learned ranking).
Focus: Reactive Actions & Event Processing.
- Actions (Drasi-based): Reactive event-driven actions triggered by data changes.
- Webhooks & Event Notifications: Push-based data change alerts for downstream consumers.
Focus: Enterprise Security & Compliance.
- Data Masking & Anonymization: Dynamic PII redaction at query time.
- OIDC Token Verification: Validate identity tokens from enterprise providers (Okta, Azure AD, etc.).
Focus: Governance.
- Policy Engine (Cedar-based):
- Role-based security for fine-grained access control.
- Data access policies to enforce compliance and governance.
- Audit Logging: Persistent, immutable query and access logs for compliance.
- Resource Quotas: Per-user/tenant query limits and throttling.
Focus: Extensibility & Plugin Architecture.
- Extensible Middleware: Pluggable extensions for dynamic customization.
- Custom Tool Providers: User-defined MCP tool implementations.
Focus: Encryption.
- Customer-Managed Keys (BYOK): Encryption key management for sensitive workloads.
- Data-at-Rest Encryption: Encrypted storage for accelerated datasets.
These are prioritized based on community demand and strategic alignment. Share your feedback to influence their inclusion.
- Delta Lake: Write support for transactional data lakes.
- Elasticsearch: Integration for search and analytics workloads.
- MotherDuck: Hybrid cloud DuckDB connector.
- BigQuery: Support for Google Cloud's data warehouse.
- Google Docs: Experimental connector for collaborative data sources.
- Key/Value API:
- SlateDB data accelerator for low-latency storage.
- PostgreSQL API: Native compatibility with PostgreSQL clients.
- Vision Processing: Support for image and video.
- Custom Model Integration: Framework for user-defined ML models.
- Model Versioning & A/B Testing: Canary deployments and version management for models.
- Hallucination Detection: Fact-checking LLM responses against source data.
- Faceted Search: Aggregations, filters, and counts for enterprise search UX.
- Data Lineage: Track data provenance and transformations across the pipeline.
- Schema Registry & Evolution: Versioning and backward compatibility checks.
- Observability: Comprehensive metrics, logging, and tracing.
- CLI & SDKs: Streamlined developer experience with improved tooling.
- Data Accelerators: Additional engines for specialized workloads.
This roadmap is dynamic and evolves with community input and market needs. Thank you for contributing to the Spice.ai ecosystem!