Prompt caching with Batch API ?

[clonecorp]

Hey there,

I see I get a system.0.cache_control: Extra inputs are not permitted or messages.0.cache_control: Extra inputs are not permitted when trying to use the cache_control with the Batch API. Is this by design? Looking at the docs I should be able to: https://docs.anthropic.com/en/docs/build-with-claude/message-batches#can-i-use-the-message-batches-api-in-conjunction-with-other-betas

My code:

await anthropic.beta.messages.batches.create({
  requests: [
    {
      custom_id: some_id,
      params: {
        betas: ['prompt-caching-2024-07-31', 'message-batches-2024-09-24'],
        model: 'claude-3-5-sonnet-20240620',
        max_tokens: 100,
        system: [
          {
            text: very_long_text,
            type: 'text',
            cache_control: {
              type: 'ephemeral'
            }
          }
        ],
        messages: [
          {
            role: 'user',
            content: 'What is this very long text?',
          }
        ]
      }
    },
  ]
})

[berezovyy]

@clonecorp you should move the "betas" field out of the request:

await anthropic.beta.messages.batches.create({
  betas: ['prompt-caching-2024-07-31', 'message-batches-2024-09-24'],
  requests: [
    {
      custom_id: some_id,
      params: {
        model: 'claude-3-5-sonnet-20240620',
        max_tokens: 100,
        system: [
          {
            text: very_long_text,
            type: 'text',
            cache_control: {
              type: 'ephemeral'
            }
          }
        ],
        messages: [
          {
            role: 'user',
            content: 'What is this very long text?',
          }
        ]
      }
    },
  ]
})

P.S. Keep in mind, that there is min tokens requirement for the prompt to be cached

[clonecorp]

@clonecorp you should move the "betas" field out of the request

Ahhh, I see. It works now. Thanks!

[yolpsoftware]

Getting the same error, but I have no "betas" enabled.