Skip to content

Conversation

@esnible
Copy link
Member

@esnible esnible commented Feb 12, 2025

This fixes problems with Replicate returning Error during 'replicate/ibm-granite/granite-3.0-8b-instruct' model call: litellm.UnprocessableEntityError: ReplicateException - {"title":"Version disabled","detail":"This deployment is currently disabled:\n- This deployment has been disabled in its settings.","status":422}

This fixes Granite 3.1 so that it generates prompts as expected.

This PR introduces some new problems that will need to be fixed here or in a follow-up.

  • With this fix applied, the 9-react.pdl example gets the correct answer, although it fails with a new error after that.
  • With this fix applied, examples/weather/weather.pdl fails with a JSON decoding error.

I started investigating the new errors. Probably they reflect places the example prompts don't produce the expected output. Unfortunately, while testing, I started getting 402 Payment Required from Replicate even though I had an API key.

Signed-off-by: Ed Snible <snible@us.ibm.com>
Signed-off-by: Ed Snible <snible@us.ibm.com>
@esnible esnible requested review from mandel and vazirim February 12, 2025 01:41
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please remove the word replicate from line 697? Thanks!

Signed-off-by: Ed Snible <snible@us.ibm.com>
@esnible esnible merged commit c419a02 into IBM:main Feb 12, 2025
6 checks passed
@esnible esnible deleted the no-more-granite30-instruct branch February 12, 2025 14:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants