-o image_detail high option

[simonw]

Refs:

• Initial plugin design #1 (comment)
• https://platform.openai.com/docs/guides/images?api-mode=responses&format=url#specify-image-input-detail-level

[simonw]

Can be low, high, or auto. OpenAI default is auto but in my experiment this can result in surprising high cost operations, so I'm going to default to low instead.

[simonw]

Without -o image_detail high gpt-4o used 323 input tokens, with that option it used 1343 input tokens:

[
  {
    "id": "01jpx31b3m580xbb2c4h7ve18e",
    "model": "openai/gpt-4o",
    "prompt": "describe",
    "system": null,
    "prompt_json": null,
    "options_json": {},
    "response": "The image shows two pelicans in flight against a clear blue sky. Their wings are outstretched, and they are gliding gracefully through the air. The pelicans have long beaks and large wings, with their bodies and feathers visible in detail. The lighting captures their natural coloring and features.",
    "response_json": {
      "id": "resp_67ddbea034888191bb4e4b067fbbd8100c48efbc78d30253",
      "created_at": 1742585504.0,
      "error": null,
      "incomplete_details": null,
      "instructions": null,
      "metadata": {},
      "model": "gpt-4o-2024-08-06",
      "object": "response",
      "output": [
        {
          "id": "msg_67ddbea13680819181ddfb52dc74c2aa0c48efbc78d30253",
          "content": [
            {
              "annotations": [],
              "text": "The image shows two pelicans in flight against a clear blue sky. Their wings are outstretched, and they are gliding gracefully through the air. The pelicans have long beaks and large wings, with their bodies and feathers visible in detail. The lighting captures their natural coloring and features.",
              "type": "output_text"
            }
          ],
          "role": "assistant",
          "status": "completed",
          "type": "message"
        }
      ],
      "parallel_tool_calls": true,
      "temperature": 1.0,
      "tool_choice": "auto",
      "tools": [],
      "top_p": 1.0,
      "max_output_tokens": null,
      "previous_response_id": null,
      "reasoning": {
        "effort": null,
        "generate_summary": null
      },
      "status": "completed",
      "text": {
        "format": {
          "type": "text"
        }
      },
      "truncation": "disabled",
      "usage": {
        "input_tokens": 323,
        "input_tokens_details": {
          "cached_tokens": 0
        },
        "output_tokens": 61,
        "output_tokens_details": {
          "reasoning_tokens": 0
        },
        "total_tokens": 384
      },
      "user": null,
      "store": true
    },
    "conversation_id": "01jpx31b3h24rcdrpaa8esca59",
    "duration_ms": 2950,
    "datetime_utc": "2025-03-21T19:31:43.979337+00:00",
    "input_tokens": null,
    "output_tokens": null,
    "token_details": null,
    "conversation_name": "describe",
    "conversation_model": "openai/gpt-4o",
    "schema_json": null,
    "attachments": [
      {
        "id": "69caf7de384d865612b63cec6663a3cfff56a39197e68da6ffb105d78dce7f6f",
        "type": "image/jpeg",
        "path": null,
        "url": "https://static.simonwillison.net/static/2025/two-pelicans.jpg",
        "content_length": null
      }
    ]
  },
  {
    "id": "01jpx320db7q21rwtr8jzeq65x",
    "model": "openai/gpt-4o",
    "prompt": "describe",
    "system": null,
    "prompt_json": null,
    "options_json": {
      "image_detail": "high"
    },
    "response": "The image depicts two pelicans flying against a clear blue sky. Their wings are spread wide, showcasing their long wingspans. The pelicans have large, distinctive bills and predominantly brown and white plumage. They are gracefully gliding, creating a sense of motion and freedom.",
    "response_json": {
      "id": "resp_67ddbeb650a481919ec93f05a4f64bbf0c826dee442eab74",
      "created_at": 1742585526.0,
      "error": null,
      "incomplete_details": null,
      "instructions": null,
      "metadata": {},
      "model": "gpt-4o-2024-08-06",
      "object": "response",
      "output": [
        {
          "id": "msg_67ddbeb7cea88191ba733d9519601b040c826dee442eab74",
          "content": [
            {
              "annotations": [],
              "text": "The image depicts two pelicans flying against a clear blue sky. Their wings are spread wide, showcasing their long wingspans. The pelicans have large, distinctive bills and predominantly brown and white plumage. They are gracefully gliding, creating a sense of motion and freedom.",
              "type": "output_text"
            }
          ],
          "role": "assistant",
          "status": "completed",
          "type": "message"
        }
      ],
      "parallel_tool_calls": true,
      "temperature": 1.0,
      "tool_choice": "auto",
      "tools": [],
      "top_p": 1.0,
      "max_output_tokens": null,
      "previous_response_id": null,
      "reasoning": {
        "effort": null,
        "generate_summary": null
      },
      "status": "completed",
      "text": {
        "format": {
          "type": "text"
        }
      },
      "truncation": "disabled",
      "usage": {
        "input_tokens": 1343,
        "input_tokens_details": {
          "cached_tokens": 0
        },
        "output_tokens": 57,
        "output_tokens_details": {
          "reasoning_tokens": 0
        },
        "total_tokens": 1400
      },
      "user": null,
      "store": true
    },
    "conversation_id": "01jpx320dam9cd2wqffvf5nbb0",
    "duration_ms": 2639,
    "datetime_utc": "2025-03-21T19:32:06.106828+00:00",
    "input_tokens": null,
    "output_tokens": null,
    "token_details": null,
    "conversation_name": "describe",
    "conversation_model": "openai/gpt-4o",
    "schema_json": null,
    "attachments": [
      {
        "id": "69caf7de384d865612b63cec6663a3cfff56a39197e68da6ffb105d78dce7f6f",
        "type": "image/jpeg",
        "path": null,
        "url": "https://static.simonwillison.net/static/2025/two-pelicans.jpg",
        "content_length": null
      }
    ]
  }
]