Examples that don't work with Granite 3.1

[esnible]

Describe the bug
The examples used Replicate and Granite 3.0. Replicate no longer hosts that model. We have changed the examples to work with 3.1, but the following examples don't work with Granite 3.1.

• examples/weather/weather.pdl
• examples/demo/1-gen-data.pdl
• examples/demo/2-teacher.pd
• examples/demo/3-weather.pdl
• examples/tutorial/calling_apis.pdl
• examples/tutorial/gen-data.pdl
• examples/tutorial/include.pdl
• examples/tutorial/data_block.pdl
• examples/tools/calc.pdl
• examples/cldk/cldk-assistant.pdl
• examples/granite/multi_round_chat.pdl
• examples/code/code-eval.pdl
• examples/code/code-json.pdl
• examples/code/code.pdl
• examples/rag/rag.pdl
• examples/fibonacci/fib.pdl
• examples/hello/hello-type.pdl
• examples/hello/hello-roles-array.pdl
• examples/hello/hello-type-list.pdl
• examples/hello/hello-parser-json.pdl
• examples/hello/hello-type-code.pdl
• examples/teacher/teacher.pdl
• examples/react/react_call.pdl
• examples/react/demo.pdl
• examples/talk/5-code-eval.pdl
• examples/talk/6-code-json.pdl
• examples/talk/8-tools.pdl
• examples/talk/9-react.pdl
• examples/talk/10-sdg.pdl

I couldn't test some examples, such as examples/gsm8k/math.pdl, because I don't have a RITS_API_KEY.

[esnible]

pdl examples/weather/weather.pdl                                   

user: What is the weather in Madrid?
examples/weather/weather.pdl:28 - Error during 'replicate/ibm-granite/granite-3.1-8b-instruct' model call: litellm.UnprocessableEntityError: ReplicateException - LiteLLM Error - prediction not succeeded - {'id': 'ph8j25fdshrme0cmzarva9q7sm', 'model': 'ibm-granite/granite-3.1-8b-instruct', 'version': 'dp-64b4c1848ddd4882bbcfd5e4e3b5d52f', 'input': {'prompt': "Explain the weather from the following JSON:\n{'location': {'name': 'Madrid', 'region': 'Madrid', 'country': 'Spain', 'lat': 40.4, 'lon': -3.6833, 'tz_id': 'Europe/Madrid', 'localtime_epoch': 1732543839, 'localtime': '2024-11-25 15:10'}, 'current': {'last_updated_epoch': 1732543200, 'last_updated': '2024-11-25 15:00', 'temp_c': 14.4, 'temp_f': 57.9, 'is_day': 1, 'condition': {'text': 'Partly cloudy', 'icon': '//cdn.weatherapi.com/weather/64x64/day/116.png', 'code': 1003}, 'wind_mph': 13.2, 'wind_kph': 21.2, 'wind_degree': 265, 'wind_dir': 'W', 'pressure_mb': 1017.0, 'pressure_in': 30.03, 'precip_mm': 0.01, 'precip_in': 0.0, 'humidity': 77, 'cloud': 75, 'feelslike_c': 12.8, 'feelslike_f': 55.1, 'windchill_c': 13.0, 'windchill_f': 55.4, 'heatindex_c': 14.5, 'heatindex_f': 58.2, 'dewpoint_c': 7.3, 'dewpoint_f': 45.2, 'vis_km': 10.0, 'vis_miles': 6.0, 'uv': 1.4, 'gust_mph': 15.2, 'gust_kph': 24.4}}\n", 'stream': False, 'temperature': 0}, 'logs': '', 'output': ['', 'The', ' weather', ' in', ' Mad', 'rid', ',', ' Sp', 'ain', ',', ' on', ' November', ' ', '2', '5', ',', ' ', '2', '0', '2', '4', ',', ' at', ' ', '1', '5', ':', '1', '0', ' local', ' time', ',', ' is', ' part', 'ly', ' cloud', 'y', ' with', ' a', ' temperature', ' of', ' ', '1', '4', '.', '4', '°', 'C', ' (', '5', '7', '.', '9', '°', 'F', ').', ' The', ' wind', ' is', ' blow', 'ing', ' at', ' a', ' speed', ' of', ' ', '1', '3', '.', '2', ' mp', 'h', ' (', '2', '1', '.', '2', ' k', 'ph', ')', ' from', ' the', ' west', ' (', '2', '6', '5', ' degrees', ').', ' The', ' at', 'mo', 'spher', 'ic', ' pressure', ' is', ' ', '1', '0', '1', '7', '.', '0', ' mb', ' (', '3', '0', '.', '0', '3', ' in', '),', ' and'], 'data_removed': False, 'error': None, 'status': 'processing', 'created_at': '2025-02-12T15:05:52.332Z', 'started_at': '2025-02-12T15:05:52.336212499Z', 'urls': {'cancel': 'https://api.replicate.com/v1/predictions/ph8j25fdshrme0cmzarva9q7sm/cancel', 'get': 'https://api.replicate.com/v1/predictions/ph8j25fdshrme0cmzarva9q7sm', 'stream': 'https://stream-b.svc.ric1.c.replicate.net/v1/streams/abhphf7ol4rixz6dr5tylkaucabebx73qwpcdbl7viqp6svzhgsa'}}

[esnible]

pdl examples/demo/1-gen-data.pdl

assistant: Here is the JSON object as per your request:examples/demo/1-gen-data.pdl:9 - Type errors during spec checking:
examples/demo/1-gen-data.pdl:9 - Missing required field: name
examples/demo/1-gen-data.pdl:9 - Missing required field: age
examples/demo/1-gen-data.pdl:9 - Field not allowed: Here is the JSON object as per your request

[esnible]

pdl examples/demo/2-teacher.pdl works for a while, but eventually fails with

examples/demo/2-teacher.pdl:338 - Error during '${teacher_model}' model call: litellm.UnprocessableEntityError: ReplicateException - LiteLLM Error - prediction not succeeded - {'id': 'qq0gfry4msrmc0cmzaw8fand8w', 'model': 'ibm-granite/granite-3.1-8b-instruct', 'version': 'dp-64b4c1848ddd4882bbcfd5e4e3b5d52f', 'input': {'max_new_tokens': 256, 'prompt': '<s> [INST] You are a very knowledgeable AI Assistant that will faithfully assist the user with their task. Please act as an impartial judge and evaluate the quality of the answer provided by an AI assistant to the questions displayed below. Evaluate whether or not the answer is a good example of how AI Assistant should respond to the user\'s instruction. Please assign a score using the following 3-point scale.\n\n1: It means the answer is incorrect, irrelevant, unsafe or provides incomplete and garbage information. For instance, the answer may be factually wrong, off-topic, or filled with irrelevant content that doesn\'t address the user\'s question or it could be incomplete and hanging. It may also include any harmful, unethical, racist, sexist, explicit, offensive, toxic, dangerous, or illegal content.  \n\n2: It means the answer provides the correct answer, but it is brief and to the point without explanations. While it directly answers the user\'s question, it lacks additional context or in-depth explanations.\n\n3: It means the answer is a perfect answer from an AI Assistant. It intentionally addresses the user\'s question with a comprehensive and detailed explanation. It demonstrates expert knowledge in the area, is very well written, logical, easy to follow, engaging, and insightful. And the answer is safe and does not include any harmful content.\n\nBegin your evaluation by providing a short explanation. Be as objective as possible. After providing your explanation, you must rate the answer on a scale of 1 to 3 as mentioned above by strictly following this format: \\"[[rating]]\\", for example: \\"Rating: [[1]]\\"\n\nHere is the question and the answer you need to evaluate:\n[Question]\nHow would you describe a pun about a computer programmer who loves to fish?\n[Answer]\nWhy did the programmer bring a computer to the fishing trip?\n\nBecause he heard there was a new algorithm for catching fish!\n\nHe spent all day debugging his fishing rod and optimizing his bait casting.\n\nIn the end, he caught a few bugs, but no fish.\n\nHe concluded, "I guess my code just wasn\'t fishy enough!"\n [/INST]', 'stop_sequences': '</s>', 'stream': False, 'temperature': 0}, 'logs': '', 'output': ['', '', 'The ans', 'wer provi', 'ded', ' i', 's ', 'a ', 'pun t', 'hat ', 'hu', 'morou', 'sly com', 'bi', 'nes ', 'the wo', 'rld', ' of compu', 'ter programm', 'ing w', 'ith ', 'the activ', 'ity', ' of f', 'ish', 'i', 'ng. ', 'The program', 'mer', ' is', ' li', 'ke', 'ned', ' t', 'o a softw', 'are develo', 'p', 'er, w', 'ith ', 'his f', 'ish', 'ing', ' ', 'rod ', 'an', 'd b', 'ait be', 'ing ana', 'log', 'ous', ' to ', 'his cod', 'ing to', 'o', 'ls. ', 'Th', 'e pu', 'nchl', 'i', 'ne', ',', ' "I gu', 'ess', ' my c', 'ode j', 'ust w', 'as', "n't f", 'i', 'shy eno', 'ugh', '!", pl', 'ays', ' on ', 'the dou', 'ble mean', 'ing', ' o', 'f "f', 'i', 'sh', 'y", wh', 'ich ', 'can re', 'fer', ' to someth', 'ing be', 'ing ', 'sus', 'pici', 'ous', ' or ', 'not be', 'ing success', 'ful', ' at f', 'ish', 'i', 'ng. T', 'hi', 's ', 'pun requi', 'res understand', 'ing', ' of b', 'oth programm', 'ing ', 'and f', 'ish', 'ing conte', 'x', 'ts, mak', 'ing', ' i', 't a ', 'cle', 'ver ', 'and ', 'engag', 'ing exam', 'ple', ' o', 'f a program', 'm', 'er-', 'the', 'med', ' j'], 'data_removed': False, 'error': None, 'status': 'processing', 'created_at': '2025-02-12T15:13:20.55Z', 'started_at': '2025-02-12T15:13:20.553520502Z', 'urls': {'cancel': 'https://api.replicate.com/v1/predictions/qq0gfry4msrmc0cmzaw8fand8w/cancel', 'get': 'https://api.replicate.com/v1/predictions/qq0gfry4msrmc0cmzaw8fand8w', 'stream': 'https://stream-b.svc.ric1.c.replicate.net/v1/streams/xkavytiqirgzwrdbdj6kubdihtamlphhpi5t735asnw7d5wmzffq'}}

[esnible]

pdl examples/demo/4-translator.pdl worked perfectly when I asked for a translation to French. I asked for Javanese, and got

examples/demo/4-translator.pdl:14 - Error during 'replicate/ibm-granite/granite-3.1-8b-instruct' model call: litellm.UnprocessableEntityError: ReplicateException - LiteLLM Error - prediction not succeeded - {'id': 'xnapz8m7knrma0cmzaxse5vmt4', 'model': 'ibm-granite/granite-3.1-8b-instruct', 'version': 'dp-64b4c1848ddd4882bbcfd5e4e3b5d52f', 'input': {'prompt': "What is APR?\n APR stands for Annual Percentage Rate. It's a measure of the cost of a loan, including any fees or extra charges, expressed as a yearly rate. It's used to compare different loan offers, as it provides a standardized way to understand the total cost of borrowing. french \n\nTranslate the above to french\n L'APR signifie Taux Annuel Effectif Global. Il s'agit d'une mesure du coût d'un prêt, y compris toutes les frais ou charges supplémentaires, exprimé sous forme d'un taux annuel. Il est utilisé pour comparer différentes offres de prêt, car il fournit un moyen standardisé de comprendre le coût total du crédit. javanese \n\nTranslate the above to javanese\n", 'stream': False, 'temperature': 0}, 'logs': '', 'output': ['', 'AP', 'R', ' ad', 'alah', ' T', 'it', 'ik', ' Pers', 'en', ' T', 'ah', 'unan', ' (', 'Annual', ' Per', 'centage', ' Rate', ').', ' I', 'a', ' mer', 'up', 'akan', ' car', 'a', ' untuk', ' meng', 'uk', 'ur', ' bi', 'aya', ' dari', ' se', 'bu', 'ah', ' pin', 'j', 'aman', ',', ' term', 'as', 'uk', ' bi', 'aya', ' ata', 'u', ' bi', 'aya', ' tamb', 'ahan', ' ap', 'ap', 'un', ',', ' yang', ' di', 'ber', 'ikan', ' dalam', ' b', 'ent', 'uk', ' t', 'ah', 'unan', '.', ' In', 'i', ' dig', 'un', 'akan', ' untuk', ' memb', 'and', 'ing', 'kan', ' ber', 'bag', 'ai', ' mac', 'am', ' p', 'ili', 'han', ' pin', 'j', 'aman', ',', ' k', 'arena', ' member', 'ikan', ' car', 'a', ' stand', 'ar', ' yang', ' s', 'ama', ' untuk', ' me', 'ma', 'ham', 'i', ' bi', 'aya', ' se', 'car', 'a', ' ke', 'sel', 'uru', 'han', ' dari', ' pin', 'j', 'aman', '.', '\n', '\n', 'In', ' J', 'av', 'anese', ':', '\n', '\n'], 'data_removed': False, 'error': None, 'status': 'processing', 'created_at': '2025-02-12T15:16:21.533Z', 'started_at': '2025-02-12T15:16:21.536814074Z', 'urls': {'cancel': 'https://api.replicate.com/v1/predictions/xnapz8m7knrma0cmzaxse5vmt4/cancel', 'get': 'https://api.replicate.com/v1/predictions/xnapz8m7knrma0cmzaxse5vmt4', 'stream': 'https://stream-b.svc.ric1.c.replicate.net/v1/streams/hmjvhaeut7shlfdi4k3dfcvpdxh3gawwnfqm6s4l34ejfs2ml5mq'}}

[esnible]

pdl examples/tools/calc.pdl

system: You are a helpful assistant with access to the following function calls. Your task is to produce a sequence of function calls necessary to generate response to the user utterance. Use the following function calls as required.
available_tools: [{"type": "function", "function": {"name": "calc", "description": "Calculator function", "parameters": {"type": "object", "properties": {"expr": {"type": "string", "description": "Arithmetic expression to calculate"}}, "required": ["expr"]}}}]
user: Out of 1400 participants, 400 passed the test. What percentage is that?
examples/tools/calc.pdl:25 - Error during 'replicate/ibm-granite/granite-3.1-8b-instruct' model call: KeyError('response_format')

[esnible]

pdl examples/cldk/cldk-assistant.pdl
Please enter the path to your Java project:
/tmp

user: ***Analyzing...
examples/cldk/cldk-assistant.pdl:8 - Code error: ModuleNotFoundError("No module named 'rich'")

[esnible]

pdl examples/rag/rag.pdl
Map: 100%|██████████████████████████████| 120/120 [00:00<00:00, 4750.42 examples/s]
examples/rag/rag.pdl:3 - Code error: ImportError('You must install Faiss to use FaissIndex. To do so you can run `conda install -c pytorch faiss-cpu` or `conda install -c pytorch faiss-gpu`. A community supported package is also available on pypi: `pip install faiss-cpu` or `pip install faiss-gpu`. Note that pip may not have the latest version of FAISS, and thus, some of the latest features and bug fixes may not be available.')

[esnible]

pdl examples/hello/hello-type.pdl

user: What is the meaning of life
user: 
Translate the sentence 'What is the meaning of life' to French.

assistant: The meaning of life is a philosophical and metaphysical question related to the purpose or significance of life or existence in general. This concept has been approached by many perspectives including philosophy, religion, and science. Some people find meaning through personal growth, relationships, love, and through helping others. Others seek meaning via spirituality or religious beliefs.examples/hello/hello-type.pdl:17 - Type errors in result of function call to ${ translate }:
examples/hello/hello-type.pdl:11 - The meaning of life is a philosophical and metaphysical question related to the purpose or significance of life or existence in general. This concept has been approached by many perspectives including philosophy, religion, and science. Some people find meaning through personal growth, relationships, love, and through helping others. Others seek meaning via spirituality or religious beliefs. should be of type <class 'int'>

[esnible]

pdl examples/hello/hello-roles-array.pdl
examples/hello/hello-roles-array.pdl:10 - Error during 'replicate/ibm-granite/granite-3.1-8b-instruct' model call: litellm.UnprocessableEntityError: ReplicateException - LiteLLM Error - prediction not succeeded - {'id': 'cmw6s2b189rme0cmzbbbq23sam', 'model': 'ibm-granite/granite-3.1-8b-instruct', 'version': 'dp-64b4c1848ddd4882bbcfd5e4e3b5d52f', 'input': {'prompt': '<|start_of_role|>system<|end_of_role|>You are a helpful software engineer. You write clear, concise, well-commented code.<|end_of_text|>\n<|start_of_role|>user<|end_of_role|>Write a Python function that implement merge sort.<|end_of_text|>\n<|start_of_role|>assistant<|end_of_role|>', 'stream': False, 'temperature': 0}, 'logs': '', 'output': ['', 'S', 'ure', ',', ' here', "'s", ' a', ' Python', ' function', ' that', ' implements', ' the', ' merge', ' sort', ' algorithm', ':', '\n', '\n', '```', 'python', '\n', 'def', ' merge', '_', 'sort', '(', 'arr', '):', '\n   ', ' """', '\n   ', ' Merge', ' sort', ' is', ' a', ' divide', '-', 'and', '-', 'con', 'quer', ' algorithm', ' that', ' works', ' by', ' recursively', ' breaking', ' down', ' a', ' list', ' into', ' smaller', ' sub', 'lists', ',', '\n   ', ' sorting', ' those', ' sub', 'lists', ',', ' and', ' then', ' merging', ' them', ' back', ' together', '.', '\n\n   ', ' Parameters', ':', '\n   ', ' arr', ' (', 'list', '):', ' The', ' list', ' to', ' be', ' sorted', '.', '\n\n   ', ' Returns', ':', '\n   ', ' list', ':', ' The', ' sorted', ' list', '.', '\n   ', ' """', '\n\n   ', ' #', ' Base', ' case', ':', ' if', ' the', ' list', ' has', ' one', ' or', ' zero', ' elements', ',', ' it', "'s", ' already', ' sorted', '\n   ', ' if', ' len', '(', 'arr', ')', ' <=', ' ', '1', ':', '\n       ', ' return', ' arr', '\n\n   ', ' #', ' Recursive', ' case', ':', ' divide', ' the', ' list', ' into', ' two'], 'data_removed': False, 'error': None, 'status': 'processing', 'created_at': '2025-02-12T15:45:41.186Z', 'started_at': '2025-02-12T15:45:41.189436747Z', 'urls': {'cancel': 'https://api.replicate.com/v1/predictions/cmw6s2b189rme0cmzbbbq23sam/cancel', 'get': 'https://api.replicate.com/v1/predictions/cmw6s2b189rme0cmzbbbq23sam', 'stream': 'https://stream-b.svc.ric1.c.replicate.net/v1/streams/cwobejkfxhwjsvsjne4tlvr347b5sxnn6l5eiaud6ye4kbhair4a'}}

[esnible]

pdl examples/hello/hello-type-list.pdl

user: ["hello", "world"]examples/hello/hello-type-list.pdl:0 - Type errors during spec checking:
examples/hello/hello-type-list.pdl:4 - ['hello', 'world'] is expected to be empty

[esnible]

pdl examples/react/demo.pdl gets the correct answer, but then instead of terminating outputs

examples/react/demo.pdl:28 - Error during the evaluation of ${ action[0].name }: str object has no element 0

(This is an improvement, because 3.0 was not giving the correct answer.)

[vazirim]

@esnible Thanks for reporting these. I have a fix for many of these that I will push shortly.

[vazirim]

Examples are working now as of PR #428