AI Assistant Chat : Attach images to discussions

[n2ojim]

Scope & Context

We want to enhance the AI assistant's capabilities by allowing users to copy and paste multiple images directly into the chat. This feature aims to enable the AI to perform visual data extraction to automatically populate and update record fields based on the provided images (e.g., property details, invoices, business cards, or floor plans). It could also provide more context in any conversation.

Current behavior

Currently, the AI assistant chat interface primarily accepts text input. Users cannot copy and paste multiple images directly into the chat to provide visual context for data entry. If a user has an image containing data they want to encode into a record, they must manually read the image and type the information out themselves.

Expected behavior

Users should be able to paste (Ctrl+V or Cmd+V) one or multiple images from clipboard or files directly into the AI assistant chat input box.
The pasted images should appear as small thumbnails above or within the input field before sending. Once sent, the AI should be able to analyze the images (using vision models), extract the relevant data, and seamlessly encode or update the information into the currently viewed record (like a Property record) or create new records.

Technical inputs

• Modify the AI chat input component to listen for paste events and intercept e.clipboardData containing image files (image/*).
• Implement an image upload mechanism to store these images (either temporarily or as standard attachments linked to the message thread).
• Update the UI to display thumbnail previews of the pasted images with a "remove/delete" button before submission.
• Update the AI integration backend to support multimodal inputs (e.g., passing image URLs or base64-encoded images to the LLM vision API).
• Ensure the AI system prompt and tool definitions are equipped to handle visual data extraction and map it correctly to the workspace's metadata to perform update or create mutations.

[BOHEUS]

@n2ojim you can add images as attachments, regarding the LLM vision API, that's entirely on provider's side

[n2ojim]

I am using Gemini. With files I can upload an image, but not with pasing from clipboard.

[Amresh-01]

can i work on this issue.