Feast 0.10 on onpremise k8s environment

[Atamos]

Hi,
I have some doubt regarding a production ready feast deploy.
I would like deploy feast 0.10 not 0.9 on on-premise k8s installation.
But , i really miss the point.

The doc is not very clear about it. There is a message on website that you can deploy on k8s only the 0.9 but there are some workarounds ? Obviously without big query and so on ?

I can't really imagine what steps are involved to deploy feast 0.10 in production environment.

Anyway , i very like what are you doing with 0.10 version. It's an amazing improvment.

[woop]

Thank you @Atamos for the kind words!

I need to get a better idea of the reason you'd like to deploy Feast to K8s in the first place. Do you want to have a single feature serving API for all consumers? The current Feast 0.10 design means you don't have to deploy anything to your Kubernetes cluster at all. You can run Feast primarily as an SDK/CLI, and ensure that your online store stays up to date.

Also, contributors like @qooba are working on adding Redis support here as well (#547), if that is important to you.

[Atamos]

To be clear, all this is refered at feast 0.10 version.
Ok, So maybe i'm in wrong direction. I'm try to implement a ml level 2 pipeline and I would like to do with feast as layer between our data lake and the beginning of ml pipeline.

What that i can't understand is how to put feast in the middle.
In the case of ingestion:
In my immagination, there are some jobs previuosly developed by some data engenieer that runs as batch from airflow for example. These jobs extract some data from a single data , produce some features and ingest in feature store.

At the other side , there are some ci pipeline triggered by github actions for example for models training that read those features from feast and use historical data for repetable training job for example.

This suppose that feast shoul be reacheable from the 2 points as ingress and otuput for a features.

Before moving to realtime ingestion or inferences , i would like understand well how feast can fit in those ipotetical architecture.

What i miss is how use feast for this result.. maybe i'm completly wrong interpretated the flow , and feast should be deployed into feature script ? but at that point how i can reuse those features for other projects and so on.

If you have time to explain your vision about that i will be very greaceful

I know slack it can be a more appropiate place for this discussion but maybe it can be useful for someone else.

[qooba]

Hi @Atamos,

I also need to use Feast with on-premise k8s and definitely recommend to use it with Kubeflow platform.
I have prepared short video that shows the Feast v. 0.10 usage example with k8s:
https://www.youtube.com/watch?v=CeYR0INmPhs
Here you have article:
https://qooba.net/2021/05/22/feast-with-ai-feed-your-mlflow-models-with-feature-store/
and the code is on my github:
https://github.com/qooba/mlflow-feast
Where you have k8s deployments (this is minimal dev deployment :) ).
In your case the model training and materialization should be done in kubeflow-pipelines (or airflow etc.). In the example for simplicity I have used Jupyter.

I hope this will help :)

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.