Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,9 @@ Use this guidance when the user asks to refresh, update, replace, or repair a ch

- A chart refresh or replacement is only complete when the chart slide was re-read after the write and the visual result was checked with a thumbnail when the image itself matters.
- Do not claim success just because the write call succeeded or because adjacent chart labels changed.
- When thumbnail responses include an `image_asset_pointer`, image content part, or other rendered image artifact, inspect that returned image directly instead of downloading the `contentUrl` just to look at the same slide.
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Aismit I notice this instruction prompts the model to inspect the returned image directly, but then I notice there's no image base 64 returned in the response. I'd love some review on this new instruction.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah this may be stale but we are using connector file reference object now

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

- When thumbnail responses include an `image_asset_pointer`, image content part, or other rendered image artifact, inspect that returned image directly.
- If the thumbnail response only exposes a thumbnail URL or `contentUrl`, curl the rendered image to a local PNG, for example `curl -L "$contentUrl" -o /tmp/slides-thumb-<slide-id>.png`, and inspect that local image before claiming the chart visual was verified.
- After chart or layout edits, fetch a fresh thumbnail and re-curl any URL-backed thumbnail to a new local PNG before final success.
- When the refresh or replacement changed visible layout, placeholder cleanup, or nearby styling, follow [visual-change-loop](./visual-change-loop.md) and keep the review loop going through the third fresh visual pass.
- Verify both of these things before declaring success:
- the chart graphic itself now matches the intended source chart
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@ Use this recipe whenever a Slides write can change anything the user will see, e
2. Start with a thumbnail.
- Fetch a `LARGE` thumbnail when spacing, clipping, or shape alignment matters.
- Inspect the image returned by `get_slide_thumbnail` directly. That may appear as an `image_asset_pointer`, an image content part, or another rendered image artifact in the tool response; do not require base64 bytes before starting visual review.
- If the thumbnail response only provides a thumbnail URL or `contentUrl`, save the rendered image locally with `curl -L "$contentUrl" -o /tmp/slides-thumb-<slide-id>.png` before visual review.
- Use the returned image artifact or the saved local PNG as the visual source of truth for overlap, clipping, padding, footer collisions, text rendering, and shape alignment.
- Use that thumbnail as the primary visual signal, but not as the only signal for overflow or collision checks.
- Write down the 2-4 concrete visible issues you are fixing in the next pass.

Expand All @@ -40,6 +42,7 @@ Use this recipe whenever a Slides write can change anything the user will see, e

5. Verify immediately.
- Fetch another thumbnail right after the write.
- If the fresh thumbnail is URL-backed, curl it to a new local PNG and inspect that new file before declaring the pass verified.
- Confirm both text and non-text visual targets actually changed.
- Re-read the live slide structure after the write when text boxes, wrapping, or neighboring layout relationships were in scope.
- If the write fixed the main text but left stale bars, arrows, borders, wrapping, or collisions, the slide is not done.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,8 @@ If a dedicated visual-iteration tool exists in the runtime, use it. Otherwise, e
- When the tool returns an `image_asset_pointer`, image content part, thumbnail artifact, or other rendered image in `content`, treat that as analyzable visual input for this workflow and inspect it directly.
- Do not wait for image bytes, a data URL, or base64 image content before reviewing the slide. A returned image artifact from `get_slide_thumbnail` is the slide screenshot for this loop.
- If `get_slide_thumbnail` succeeds, treat that as the visual verification path for this workflow even if the transcript view looks metadata-shaped. Do not abandon the thumbnail loop just because the runtime shows a thumbnail artifact, asset pointer, URL, or metadata wrapper instead of inline pixels in the message body.
- The response may also include `contentUrl` metadata, but prefer the returned image content instead of downloading the URL or switching to another image-analysis path.
- If the thumbnail response only exposes a thumbnail URL or `contentUrl`, curl the rendered image to a local PNG before visual review, for example `curl -L "$contentUrl" -o /tmp/slides-thumb-<slide-id>.png`.
- Inspect the saved local PNG with the available image-viewing path before diagnosing overlap, clipping, padding, footer collisions, text rendering, or before claiming visual verification.
- Do not switch to deck export, PDF rendering, or other fallback rendering paths when the thumbnail tool already succeeded. Only use a fallback path if the thumbnail action itself failed or is unavailable.

4. Diagnose concrete visual problems.
Expand Down Expand Up @@ -100,6 +101,7 @@ If a dedicated visual-iteration tool exists in the runtime, use it. Otherwise, e

6. Verify immediately.
- Call `get_slide_thumbnail` again after every batch update.
- If the fresh thumbnail is URL-backed, curl the new URL to a local PNG and inspect that new file. Do not reuse a PNG from before the write.
- State which issues are now fixed, which issues remain, and whether the pass introduced any new regressions.
- Confirm the targeted issue cluster is actually fixed before moving on.
- Verify that small target or benchmark text changed along with the main headline value when both were in scope.
Expand Down
4 changes: 3 additions & 1 deletion plugins/google-drive/skills/google-slides/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,9 @@ Confirm the runtime exposes the relevant Google Slides actions before editing:
- For slide summaries or inspection, do not rely on text extraction alone when a slide contains charts, graphs, screenshots, diagrams, or image-heavy content.
- Use `get_slide_thumbnail` alongside text/structure reads when visual evidence matters so the summary reflects both what the slide says and what the slide shows.
- If a candidate slide has little useful extracted text but may still contain relevant evidence in a chart, screenshot, or diagram, inspect its thumbnail before ruling it out.
- If the thumbnail response includes an `image_asset_pointer`, image content part, or other rendered image artifact, inspect that returned image directly as the slide image input. The response may also include `contentUrl` metadata; prefer the returned image content for visual understanding instead of downloading that URL or relying only on metadata.
- If the thumbnail response includes an `image_asset_pointer`, image content part, or other rendered image artifact, inspect that returned image directly as the slide image input.
- If the thumbnail response only exposes a thumbnail URL or `contentUrl`, curl the rendered image to a local PNG, for example `curl -L "$contentUrl" -o /tmp/slides-thumb-<slide-id>.png`, then inspect that local image before making or reporting visual judgments.
- Do not claim visual inspection from slide JSON, text extraction, geometry, or thumbnail metadata alone.
- Treat the slide page size as a hard boundary for every shape, text box, image, and color band you create.
- If the write can change anything visible on the slide, such as text wrapping, shape styling, arrow direction, accent bars, chart placement, or connector styling, follow [visual-change-loop](./visual-change-loop.md) before the first `batch_update`.

Expand Down