Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move source types into their own package and discover them using java.util.ServiceLoader #61

Closed
tims opened this issue Jan 11, 2019 · 1 comment
Assignees
Labels
kind/feature New feature or request

Comments

@tims
Copy link
Contributor

tims commented Jan 11, 2019

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
The ingestion sources are currently mingled in with all the other ingestion code in the transforms package. This makes it hard to distinguish what all the sources are and makes it trickier for people to see how they might extend feast to support new sources.

Describe the solution you'd like
I'd like to create a source package in the ingestion module, and create common factory type class that can create FeatureIO.Read instances and have a function for returning their type string. New source implementations need only implement this interface and use an @AutoService annotation to make it available.

The import job can then load all available source factories from the class path and find the one with the matching type string against the import spec.

This follows the same pattern we use for storage types and allows us a path for including new source types as external dependencies, or moving existing ones into their own module. And then making them available simply by adding their jars on the class path.

@woop
Copy link
Member

woop commented Sep 1, 2019

This issue isn't relevant any more due to changes in Feast 0.3.0.

@woop woop closed this as completed Sep 1, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/feature New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants