-
Notifications
You must be signed in to change notification settings - Fork 8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
generate_embeddings management command #128
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The approach here feels like it may cause some problems down the road.
Instead of writing the fixture data to an intermediary fixture, then loading that fixture - what about writing the data directly to the database, then using the dumpdata command to export the fixtures?
Notably - the logic where data_to_write
is built, is very subject to breakage if the schema changes.
thinking out loud
It's a shame we can't use the existing embedding provider classes to perform the embedding (as opposed to importing the libraries here and doing them). It's not really doable because the providers need a user object...
03af06c
to
c22f71f
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
FANTASTIC! Really great stuff. Excellent use of the prompts to employ an existing user.
The purpose of this PR is to add a management command called
generate_embeddings
. It allows the user to easily selectthe embedding model service, and based on their selection the user is presented with the available embedding models. Once the user provides their username and password, the API key provided for the selected service is used to generating the embeddings.