feat: Feast extractor

[szczeles]

Summary of Changes

This PR provider Extractor for Feast feature store, announced in amundsen-io/amundsen#815. Apart from FeatureTables definitions, the extractor also pushes the metadata collected by Feast as programmatic descriptions, so they look like this on the Frontend:

The new "extra" dependency is added: feast (python sdk for Feast), it is distributed on ASF licence.

Fixes amundsen-io/amundsen#815

Tests

Unit tests for the extractor class.

Documentation

A sample job with loader definition, doc strings for FeastExtractor class and extract method

CheckList

Make sure you have checked all steps below to ensure a timely review.

• PR title addresses the issue accurately and concisely. Example: "Updates the version of Flask to v1.0.2"
  • In case you are adding a dependency, check if the license complies with the ASF 3rd Party License Policy.
• PR includes a summary of changes.
• PR adds unit tests, updates existing unit tests, OR documents why no test additions or modifications are needed.
• In case of new functionality, my PR adds documentation that describes how to use it.
  • All the public functions and the classes in the PR contain docstrings that explain what it does
• PR passes make test

[feng-tao]

thanks, I will take a look.

[feng-tao]

@szczeles great contribution, I wonder how easy to setup databuilder using kubeflow job? Given most people used Airflow for orchestration daily, I am interested in knowing and trying with kubeflow as well.

[feng-tao]

lgtm

[feng-tao]

interesting to know you yield the prog description in the same extractor.

[szczeles]

Actually it was quite easy, since both tables and prog descriptions use same class. I was thinking once about creating multiple extractors - one for table and second for prog descirptions, but decided to not overcomplicate.

[feng-tao]

cc @dikshathakur3119 for FYI as this will be easier for Lyft internal programmatic description use case.

[dikshathakur3119]

it is super useful

[feng-tao]

is there any order in feast for entity vs feature? or you just put the entity column first before feature column?

[szczeles]

Feast doesn't define ordering, these are just different properties of the feature table. I put entities first, because they are like "primary keys" of the table.

[feng-tao]

It would be good to evangelize in the feast community as well given both projects are in LFAI incubation!

[szczeles]

@szczeles great contribution, I wonder how easy to setup databuilder using kubeflow job? Given most people used Airflow for orchestration daily, I am interested in knowing and trying with kubeflow as well.

Thanks! It's super easy to setup in Kubeflow. Kubeflow Pipelines is a system to orchestrate and schedule run of different containers, so I have built one Docker image with databuilder, added the scripts inside and I'm calling them one by one in the pipeline like this:

It would be good to evangelize in the feast community as well given both projects are in LFAI incubation!

Absolutely! As soon as the feature is merged I'm going to evangelize there :-) The lack of user interface for features exploration was noticed by Feast devs as well, see: https://docs.feast.dev/#problems-feast-does-not-yet-solve, so it can be a valuable add-on.

[feng-tao]

thanks @szczeles for the info!