feat: add AI-based asset description generation endpoint

[go-wyvern]

No description provided.

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a new API endpoint that leverages artificial intelligence to generate comprehensive descriptions for assets. The endpoint is capable of processing multimodal input, combining textual context with image data to produce detailed descriptions. These AI-generated descriptions are specifically optimized for use in embedding and search functionalities, aiming to improve asset discoverability and management.

Highlights

• New AI-based Endpoint: Added a new API endpoint, /asset/description, dedicated to generating asset descriptions using artificial intelligence.
• Multimodal Input Support: Implemented support for multimodal input, allowing the AI to process both text and image data to create contextual descriptions.
• Search and Embedding Optimization: Designed the generated asset descriptions to be suitable for embedding and search operations, enhancing discoverability.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

• docs/openapi.yaml
  • Added a new POST endpoint /asset/description to the OpenAPI specification.
  • Defined the request body schema to support multimodal input, including input_text and input_image types.
  • Included example requests for generating descriptions.
  • Specified the response schema for the AI-generated asset description.

Activity

• No human activity has been recorded on this pull request yet.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces a new endpoint /asset/description for generating asset descriptions using AI. The OpenAPI specification for this endpoint is well-structured. I have two suggestions for improvement:

1. Add 403 and 429 error responses to handle quota and rate limiting, which is consistent with other AI endpoints in the API.
2. Refactor the request body schema to use a discriminator for the oneOf keyword, which will make the API more robust and easier for client code generators to work with.

[Copilot]

Pull request overview

Adds an OpenAPI specification entry for a new endpoint that generates an AI-based description for an asset from text + image input, intended for embedding/search workflows.

Changes:

• Document POST /asset/description under Assets to generate an AI description from multimodal input.
• Define request/response schemas and include an example payload/response.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[fennoai]

Good start on the AI description endpoint, but several issues need addressing before merge: the endpoint should live under /aigc/ (consistent with all other AI endpoints), input fields lack size constraints (security/cost risk), format: uri is incorrect for data: URIs, 429/403 responses are missing (inconsistent with the rest of the AIGC spec), and the example text should be in English.