Image caption generation fails

[rafaldabrowskipagepro]

Describe the bug

I've recently installed the AI assist package in order to be able to auto-generate the alt text on images. It worked once, but the next day when I tried to generate the alt text on another image I got two different errors

This is the error I'm getting on draft documents (doesn't matter if there's a published version or not):

This is the one I'm getting on published documents:

The assist package has been added to the plugins list with no additional configuration - assist()

Here's the image schema

export default defineType({
  name: "customImage",
  type: "image",
  fields: [
    defineField({
      title: "Caption",
      name: "caption",
      type: "string"
    }),
    defineField({
      title: "Alternative text",
      name: "alt",
      type: "string",
    })
  ],
  options: {
    metadata: ["lqip"],
    aiAssist: {
      imageDescriptionField: "alt"
    }
  },
  preview: {
    select: {
      subtitle: "alt",
      media: "asset"
    },
    prepare: ({ subtitle = "", media }) => ({
      title: "Image",
      subtitle,
      media
    })
  }
})

To Reproduce

Steps to reproduce the behavior:

1. Go to an image in a studio
2. Click on 'Generate image description'
3. See error

Expected behavior

The image caption is generated without errors

Which versions of Sanity are you using?

@sanity/cli (global) 4.3.0 (latest: 4.10.2)
@sanity/assist 5.0.0 (up to date)
@sanity/block-tools 3.70.0 (up to date)
@sanity/dashboard 4.1.4 (latest: 5.0.0)
@sanity/document-internationalization 1.1.2 (latest: 4.0.0)
@sanity/eslint-config-studio 5.0.2 (up to date)
@sanity/icons 3.7.4 (up to date)
@sanity/image-url 1.2.0 (up to date)
@sanity/types 4.3.0 (latest: 4.10.2)
@sanity/ui 3.1.0 (latest: 3.1.8)
@sanity/vision 4.8.1 (latest: 4.10.2)
sanity 4.3.0 (latest: 4.10.2)

What operating system are you using?

Mac OS 15.5 (24F74)

Which versions of Node.js / npm are you running?

Node v22.19.0

• Can I access something that's not mine, or something I shouldn't have access to?
• Can I disable something for other people? If the answer to either of those two questions are "yes", then you're probably dealing with a security issue. Note that even if you answer "no" to both questions, you may still be dealing with a security issue, so if you're unsure, just email us at [security@sanity.io](mailto:security@sanity.io.

[jackgregory]

I'm getting the exact same issue on v5.0.2.

[jackgregory]

After further testing and use it seems this issue occurs on images which have been uploaded prior to AI assist being activated for the project and then using the Generate image description action. If I replace this image with another it works and it works on newly uploaded or selected images. But doesnt work if I use the action again after changing or uploading a new image.

Edit: Seems to not work on published documents, if I make an edit to a document and try the action again it works.

[andrewnick]

We have this issue as well on
"sanity": "4.15.0",
"@sanity/assist": "^5.0.2"

It looks like the call to generate the caption assist/tasks/generate-caption/staging?projectId=xxxxxx&tag=sanity.studio
is trying to create this with the draft id when the draft doesn't exist yet.

[valik351]

I guess the simplest fix would be to create a draft if there is no draft, before running generateCaptionsActions

[pedrobonamin]

Thank you for reporting, this seems to be the same as #660

Creating a draft before trying to generate the caption should fix it, however, this should be handled by the plugin. We will be working on a fix for it.

[theocerutti]

hey any update on this ? thanks!