[google-drive] Require local PNG inspection for URL thumbnails

[florence-oai]

Within codex, I saw that the model wasn't doing any visual iteration loop.

This is the thumbnail response. It only has the content URL and no image asset. So I prompted the model to curl the png url to do visual inspection.

{
  "slideObjectId": "slide01",
  "contentUrl": "https://lh7-us.googleusercontent.com/docsdf/AFQj2d42XRZv39WYewOcp5OGKexAPgQJtgsVW3kS5QO8gpgxYPaiDJjCrtndMIF_0MKocU08grf8JYQIUeKpHZwYcWO66Du_jTbyx4PT81r_lZb5CIDaS3SFqR4T56tjK-HfAitrUS0fVv0yyP47AXeZ5wQiWKpwQ4C4LkeCGgofv7OemCo4=s1600",
  "width": 1600,
  "height": 900,
  "mimeType": "image/png",
  "thumbnailSize": "LARGE",
  "structuredContent": {
    "slideObjectId": "slide01",
    "contentUrl": "https://lh7-us.googleusercontent.com/docsdf/AFQj2d42XRZv39WYewOcp5OGKexAPgQJtgsVW3kS5QO8gpgxYPaiDJjCrtndMIF_0MKocU08grf8JYQIUeKpHZwYcWO66Du_jTbyx4PT81r_lZb5CIDaS3SFqR4T56tjK-HfAitrUS0fVv0yyP47AXeZ5wQiWKpwQ4C4LkeCGgofv7OemCo4=s1600",
    "width": 1600,
    "height": 900,
    "mimeType": "image/png",
    "thumbnailSize": "LARGE"
  },
  "content": [
    {
      "type": "text",
      "text": "Slide thumbnail for slide slide01. Size: 1600x900. Use the accompanying internal image asset pointer as the primary image reference for visual inspection when it is present. Use contentUrl as a direct-fetch fallback when needed. Treat this thumbnail as a point-in-time snapshot of the slide, and fetch a fresh thumbnail after any slide write before relying on it for verification. Use the accompanying MIME type when decoding or rendering the image instead of assuming it is always PNG, and inspect the image itself instead of relying only on dimensions or metadata."
    },
    {
      "type": "image_asset_pointer",
      "asset_pointer": "sediment://file_00000000e93871fbb3db29325729e87f"
    }
  ]
}

model finally does visual iteration loop now

Screen.Recording.2026-04-14.at.11.39.40.AM.mov

[florence-oai]

@Aismit I notice this instruction prompts the model to inspect the returned image directly, but then I notice there's no image base 64 returned in the response. I'd love some review on this new instruction.

[Aismit]

Yeah this may be stale but we are using connector file reference object now

[florence-oai]

Thank you!

[Aismit]

lgtm