Image suggestions with LLM - #6190
Conversation
ef761cd to
4141a21
Compare
taitus
left a comment
There was a problem hiding this comment.
Hi @stirkac
Thanks for the PR 👏 . I've left a few comments on the code. They're mostly minor suggestions and questions.
Feel free to ask if anything needs clarification.
I still need to go through the tests and documentation, but I’ll do that shortly.
67cb30c to
4268c56
Compare
|
I believe I addressed the raised concerns, along with UX improvements to the feature. Turned out to be a bit more complex that initially expected due to some edge cases. |
taitus
left a comment
There was a problem hiding this comment.
Hi @stirkac 👋
Thanks again for the PR! I've left a few comments on the code (mostly small suggestions, questions, and potential simplifications).
I've also attached a small patch in one of the comments with a possible alternative approach, just as a proposal to make the discussion easier.
Let me know what you think, and happy to iterate or clarify anything 👍
| def create | ||
| @resource_type = params[:resource_type] | ||
| @resource_id = params[:resource_id] | ||
| @resource_attributes = params.require(@resource_type.parameterize.underscore).permit!.except(:subtitle) |
e8cd6fa to
e3626cd
Compare
There was a problem hiding this comment.
Hi, @stirkac 👋
Thanks a lot for the PR and for all the iterations so far 🙏
Most of my remaining comments are small documentation suggestions, plus a few minor adjustments to the specs.
For the feedback on the tests, I've shared a .patch in this comment that applies all the discussed changes to the specs in one go:
image-suggestions-tests-and-docs.patch
While reviewing the test coverage, I noticed that although all the components are being tested, we were missing a system spec. I've added one under spec/system/ in the patch mainly as an example (it may need further tweaks)
The patch also updates an existing system spec (system/admin/settings_spec.rb) to include the AI Image Suggestions setting.
Let me know what you think, and thanks again!
9aec0ed to
dfea25b
Compare
dfea25b to
aa5d64b
Compare
aa5d64b to
5f064e3
Compare
Introduce the Pexels client dependency used to search and download stock images. Also bump ruby_llm to 1.9.x to support the LLM-based query generation used for image suggestions.
Introduce the `llm.use_ai_image_suggestions` feature flag and expose it in the LLM settings tab in the admin interface. Also document the required `pexels_access_key` in the secrets example configuration.
Introduce ImageSuggestions::Pexels as a small wrapper around the Pexels API to search and download stock images. This client is used by the image suggestions flow to retrieve images that can later be attached as if they were uploaded by the user.
Introduce ImageSuggestions::Llm::Client to generate Pexels search queries from the title and description fields of a resource. The prompt definition is stored in `llm_prompts.yml`.
Introduce image suggestions next to the upload image button using stock images from the Pexels service. The feature uses an LLM to extract context from the title and description fields of the resource and generate a search query for the Pexels API. Selected images are downloaded through the server and attached as if they were uploaded by the user. In imageable.js we remove the "_method" parameter when serializing the form data because the request reuses the edit form. If left in place Rails would interpret the request as PATCH/PUT instead of creating a new image suggestion.
Document the image suggestions feature, including usage instructions, screenshots and links from the project SUMMARY.
5f064e3 to
15022ca
Compare
Objectives
Since users might not have their own image to include on their purpose, we introduce AI-powered image suggestions next to the upload image button, to allow stock images from Pexels API to be added to the resource.
We reuse as much as possible of the existing infrastructure:
Images::NestedComponentto render the suggestions buttonVisual Changes
Notes
Read through docs/en/features/image_suggestions.md for a deeper explanation.