Lightweight and Fast Feature Store Powered by Go
Quickstart · Architecture · Benchmark
oomstore allows you to:
- Define features with YAML.
- Store features in databases of choice.
- Retrieve features for both online serving and offline training, fast.
Please see our docs for more details.
- 🍼 Simple. Being serverless and CLI-friendly, users can be productive in hours, not months.
- 🔌 Composable. We support your preferred databases of choice.
- ⚡ Fast. Benchmark shows oomstore performs QPS > 50k and latency < 0.3 ms with Redis.
- 🌊 Streaming. We support streaming features to ensure your predictions are up-to-date.
See Architecture for more details.
-
Install
oomclifollowing the guide. -
oomcli initto initialize oomstore. Make sure there is a~/.config/oomstore/config.yamlas below.
online-store:
sqlite:
db-file: /tmp/oomstore.db
offline-store:
sqlite:
db-file: /tmp/oomstore.db
metadata-store:
sqlite:
db-file: /tmp/oomstore.dboomcli apply -f metadata.yamlto register metadata. See metadata.yaml below.
kind: Entity
name: user
description: 'user ID'
groups:
- name: account
category: batch
description: 'user account'
features:
- name: state
value-type: string
- name: credit_score
value-type: int64
- name: account_age_days
value-type: int64
- name: 2fa_installed
value-type: bool
- name: txn_stats
category: batch
description: 'user txn stats'
features:
- name: count_7d
value-type: int64
- name: count_30d
value-type: int64
- name: recent_txn_stats
category: stream
snapshot-interval: 24h
description: 'user recent txn stats'
features:
- name: count_10min
value-type: int64- Import CSV data to Offline Store, then sync from Offline to Online Store
oomcli import \
--group account \
--input-file account.csv \
--description 'sample account data'
oomcli import \
--group txn_stats \
--input-file txn_stats.csv \
--description 'sample txn stats data'
oomcli sync --group-name account --revision-id 2
oomcli sync --group-name txn_stats --revision-id 4- Push stream data to both Online and Offline Store.
oomcli push --group recent_txn_stats --entity-key 1006 --feature count_10min=1- Fetch features by key.
oomcli get online \
--entity-key 1006 \
--feature account.state,account.credit_score,account.account_age_days,account.2fa_installed,txn_stats.count_7d,txn_stats.count_30d,recent_txn_stats.count_10min+------+---------------+----------------------+--------------------------+-----------------------+--------------------+---------------------+------------------------------+
| user | account.state | account.credit_score | account.account_age_days | account.2fa_installed | txn_stats.count_7d | txn_stats.count_30d | recent_txn_stats.count_10min |
+------+---------------+----------------------+--------------------------+-----------------------+--------------------+---------------------+------------------------------+
| 1006 | Louisiana | 710 | 32 | false | 8 | 22 | 1 |
+------+---------------+----------------------+--------------------------+-----------------------+--------------------+---------------------+------------------------------+
- Generate training datasets via Point-in-Time Join.
oomcli join \
--feature account.state,account.credit_score,account.account_age_days,account.2fa_installed,txn_stats.count_7d,txn_stats.count_30d,recent_txn_stats.count_10min \
--input-file label.csv+------------+---------------+---------------+----------------------+--------------------------+-----------------------+--------------------+---------------------+------------------------------+
| entity_key | unix_milli | account.state | account.credit_score | account.account_age_days | account.2fa_installed | txn_stats.count_7d | txn_stats.count_30d | recent_txn_stats.count_10min |
+------------+---------------+---------------+----------------------+--------------------------+-----------------------+--------------------+---------------------+------------------------------+
| 1002 | 1950236233000 | Hawaii | 625 | 861 | true | 11 | 36 | |
| 1003 | 1950411318000 | Arkansas | 730 | 958 | false | 0 | 16 | |
| 1004 | 1950653614000 | Louisiana | 610 | 1570 | false | 12 | 26 | |
| 1005 | 1950166137000 | South Dakota | 635 | 1953 | false | 7 | 30 | |
| 1006 | 1950403162000 | Louisiana | 710 | 32 | false | 8 | 22 | 1 |
| 1007 | 1950160030000 | New Mexico | 645 | 37 | true | 5 | 40 | |
| 1008 | 1950274859000 | Nevada | 735 | 1627 | false | 12 | 51 | |
| 1009 | 1949958846000 | Kentucky | 650 | 88 | true | 11 | 23 | |
| 1010 | 1949920686000 | Delaware | 680 | 1687 | false | 2 | 39 | |
+------------+---------------+---------------+----------------------+--------------------------+-----------------------+--------------------+---------------------+------------------------------+
See Quickstart for more complete details.
- Amazon DynamoDB
- Redis
- TiKV
- Cassandra
- PostgreSQL
- MySQL
- TiDB
- SQLite
- Snowflake
- Amazon Redshift
- Google BigQuery
- PostgreSQL
- MySQL
- TiDB
- SQLite
- PostgreSQL
- MySQL
- TiDB
- SQLite
Feel free to join the community for questions and feature requests!
oomstore is highly inspired by Feast.