feature-store

A multi-language library that allows you to persist features wherever you want and to deserialize them to whatever data input format you need.

feature-store is the fastest way to a Feature Store. It makes no assumptions on your current infrastructure, tooling, and workflows. There is nothing to host, maintain, or monitor. Just install the library and turn any storage into a Feature Store.

Examples

feature-store is implemented in many languages. The following examples are for TypeScript.

Persisting Feature Values

import { AWS } from 'aws-sdk'
import { FeatureStore, FeatureValue } from '@feats/feature-store'

const redshift = new AWS.Redshift({ ...yourConfig })
const dynamo = new AWS.DynamoDB({ ...yourConfig })

// create your feature store(s) - as few or many as you want
const offlineStore = new FeatureStore({ driver: redshift })
const onlineStore = new FeatureStore({ driver: dynamo })

// create your feature value(s)
const featureValue = FeatureValue.from({
  entity: {
    name: 'customer',
    value: '123'
  },
  feature: {
    name: 'basket_value_usd_cents',
    value: 2395
  },
  // optional
  eventTime: new Date(2020, 08, 23, 14, 05)
})

// persist your feature value(s)
await Promise.all([
  offlineStore.persist(featureValue),
  onlineStore.persist(featureValue)
])

Learn more e.g. register features, version features, and manage your features

Loading Feature Values

import * as tf from '@tensorflow/tfjs'
import { AWS } from 'aws-sdk'
import { FeatureStore, FeatureValue } from '@feats/feature-store'

const loadFeatures = async (featureStore, type) => {
  return {
    input: await featureStore.load(['customer/basket_value_usd_cents'], type)
    label: await featureStore.load(['customer/age_in_years'], type)
   }
}

// for training
const features = await loadFeatures(offlineStore, tf.Tensor)
await model.fit(features.input, features.label)

// in production
const features = await loadFeatures(onlineStore, tf.Tensor)
await model.predict(features.input, features.label)

Learn more e.g. Point-in-Time correctness

Why feature-store?

• ✅ Zero Infrastructure Dependencies: Use with any storage, database, message broker you want
• ✅ Zero Tooling Dependencies: Use with any ML framework you want
• ✅ Zero Runtime Dependencies: Use locally, in notebooks, in the cloud, or whereever you want
• ✅ Zero Org Dependencies: Use with whatever ML/Data Science workflows and teams you want

This means for you:

• ✅ No setup, maintainance, monitoring, or downtimes
• ✅ No new platform or infrastructure to learn
• ✅ No platform, vendor, tooling lock-in
• ✅ No unsupported workflows; extend, integrate, and make it fit your needs

Usage

Pick your feature-store implementation. See available languages below.

You can use different languages for persisting and deserializing, e.g. you can persist via feature-store-ts and deserialize via feature-store-py.

• Python: feature-store-py
• Node/JS/TS: feature-store-ts

Supported Persistence Systems

• AWS
  • DynamoDB (available in: Python, Node.js)
  • Redshift (available in: Python, Node.js)
  • S3 (available in: Python, Node.js)
• GCP
  • BigTable (available in: Python, Node.js)
  • Memorystore (available in: Python, Node.js)
  • BigQuery (available in: Python, Node.js)
  • CloudStorage (available in: Python, Node.js)
• Kafka (available in: Python, Node.js)
• Redis (available in: Python, Node.js)
• Cassandra (available in: Python, Node.js)
• FileSystem (available in: Python, Node.js)
• SQLite (available in: Python, Node.js)
• indexedDB (available in: Node.js)

Supported Deserialization Formats

• Python: Numpy, Pandas, Tensorflow, PyTorch, and many more ...
• Node.js: Tensorflow-js

Extend

Are you missing support for a persistence system or deserialization format?

See this this guide and learn how to easily implement it yourself.