[5/N][RAG Demo fixes] more prompt and input name indexing fixes

cookbooks/RAG-with-Model-Graded-Eval/rag.aiconfig.yaml

@@ -3,7 +3,7 @@ description: ''
 metadata:
   models: {}
   parameters:
-    context: " New England open to neutrals.\nFrom that time nothing entered or left\
+    context0: " New England open to neutrals.\nFrom that time nothing entered or left\
       \ the blockaded coast except swift\nprivateers, or occasional fast-sailing vessels\
       \ which risked capture\nin the attempt. Toward the close of the year Admiral\
       \ Warren extended\nhis blockade eastward. Notice of the extension was given\
@@ -71,146 +71,215 @@ metadata:
       the protests of the Federalists, manufactures had been forced upon\nthem by\
       \ national legislation until New Engl\n17, 1813; Annals, 1813\u20131814, p.\
       \ 2781.\n"
-    query: in July, flour sold at Boston for _?
+    context1: " antic during the\nwinter months.\n\nWith it went the tale of Napoleon\u2019\
+      s immense disaster. October 23 he\nbegan his retreat; November 23 he succeeded\
+      \ in crossing the Beresina\nand escaping capture; December 5 he abandoned what\
+      \ was still left of\nhis army; and December 19, after travelling secretly and\
+      \ without rest\nacross Europe, he appeared suddenly in Paris, still powerful,\
+      \ but in\ndanger. Nothing could be better calculated to support the Russian\n\
+      mediation in the President\u2019s mind. The possibility of remaining without\n\
+      a friend in the world while carrying on a war without hope of success,\ngave\
+      \ to the Czar\u2019s friendship a value altogether new.\n\nOther news crossed\
+      \ the ocean at the same time, but encouraged no hope\nthat England would give\
+      \ way. First in importance, and not to be trifled\nwith, was the British official\
+      \ announcement, dated December 26, 1812,\nof the blockade of the Chesapeake\
+      \ and Delaware. Americans held that\nthis blockade was illegal,[18]--a blockade\
+      \ of a coast, not of a port; a\npaper-\n the Indian signers to declare war on\
+      \ the British.\nNeither Harrison\u2019s nor Jackson\u2019s treaty embraced the\
+      \ chief body of\nhostile Indians; but Harrison\u2019s treaty served another\
+      \ purpose of no\nsmall value in appearing to remove an obstacle to negotiation\
+      \ with\nEngland.\n\n\n\n\nCHAPTER XI.\n\n\nBADLY as the United States fared\
+      \ in the campaign of 1813, their\nsituation would have been easy had they not\
+      \ suffered under the\nannoyances of a blockade continually becoming more stringent.\
+      \ The\ndoctrine that coasts could be blockaded was enforced against America\n\
+      with an energy that fell little short of demonstration. The summer was\nwell\
+      \ advanced before the whole naval force to be used for the purpose\ncould be\
+      \ posted at the proper stations. Not until May 26 did Admiral\nWarren issue\
+      \ at Bermuda his proclamation of \u201Ca strict and rigorous\nblockade of the\
+      \ ports and harbors of New York, Charleston, Port Royal,\nSavannah, and of the\
+      \ river Mississippi,\u201D which completed the blockade\nof the coast, leaving\
+      \ only the ports of \nblockade, one of the grievances against which the war\
+      \ was waged;\nbut whatever they might choose to call it, they could not successfully\n\
+      disprove its efficiency, or deny that it made Chesapeake Bay, Delaware\nRiver,\
+      \ and the Vineyard Sound little better than British waters. Export\nof American\
+      \ produce from the Chesapeake and Delaware ceased.\n\nThe blockade, though serious\
+      \ beyond all other military measures,\nroused less attention and less protest\
+      \ than another measure of the\nBritish government which had the character of\
+      \ a profitable insult. A\ncircular dated November 9, addressed to the governors\
+      \ of West Indian\ncolonies by the British government, authorized them to issue\
+      \ licenses\nfor importation of necessary supplies during the war,--a precaution\n\
+      commonly taken to meet the risk of famine in those regions. The\nGovernor of\
+      \ the Bermudas, in issuing a proclamation January 14, 1813,\npublished the circular,\
+      \ which contained one unusual provision:[19]--\n\n    \u201CWhatever importations\
+      \ are proposed to be m\n nor any\nmeasures that could be taken by sea or land,\
+      \ prevented a constant\ntraffic between Halifax and the New England ports not\
+      \ blockaded. The\nUnited States government seemed afraid to interfere with it.\
+      \ The\nnewspapers asserted that hundreds of Americans were actually in Halifax\n\
+      carrying on a direct trade, and that thousands of barrels of flour were\nconstantly\
+      \ arriving there from the United States in vessels carrying\nthe Swedish or\
+      \ other neutral flag. In truth the government could do\nlittle to enforce its\
+      \ non-intercourse, and even that little might prove\nmischievous. Nothing could\
+      \ be worse than the spirit of the people on\nthe frontier. Engaged in a profitable\
+      \ illicit commerce, they could only\nbe controlled by force, and any force not\
+      \ overwhelming merely provoked\nviolence or treason. The Navy Department had\
+      \ no vessels to send there,\nand could not have prevented their capture if vessels\
+      \ in any number had\nbeen sent. The Secretary of War had abandoned to the State\
+      \ governments\nthe defence \n ruining the peasantry. We drive all their cattle,\
+      \ and of\ncourse ruin them. _My_ hands are clean; but it is hateful to see the\n\
+      poor Yankees robbed, and to be the robber.\u201D\n\nCompared with the widespread\
+      \ destruction which war brought on these\nregions half a century afterward,\
+      \ the injury inflicted by the British\nnavy in 1813 was trifling, but it served\
+      \ to annoy the Southern\npeople, who could offer no resistance, and were harassed\
+      \ by incessant\nmilitia-calls. To some extent the same system of vexation was\
+      \ pursued\non the Northern coast. The Delaware River was blockaded and its\n\
+      shores much annoyed. New York was also blockaded, and Nantucket with\nthe adjacent\
+      \ Sounds became a British naval station. There Sir Thomas\nHardy, Nelson\u2019\
+      s favorite, officer, commanded, in his flag-ship the\n\u201CRamillies.\u201D\
+      \ Hardy did not encourage marauding such as Cockburn\npractised, but his blockade\
+      \ was still stringent, and its efficiency was\nproved by the failure of Decatur\u2019\
+      s efforts to evade it.\n\nDecatur commanded a squad\ned States outside of New\
+      \ England did\nnot exceed $150,000 a month, or at the rate of $1,800,000 a year.[384]\n\
+      \nNo ordinary operations of war could affect the United States so\nseverely\
+      \ as this inexorable blockade. Every citizen felt it in every\naction of his\
+      \ life. The farmer grew crops which he could not sell,\nwhile he paid tenfold\
+      \ prices for every necessity. While the country\nwas bursting with wealth, it\
+      \ was ruined. The blockade was but a part\nof the evil. The whole coast was\
+      \ systematically swept of the means of\nindustry. Especially the Virginians\
+      \ and Marylanders felt the heavy hand\nof England as it was felt nowhere else\
+      \ except on the Niagara River. A\nlarge British squadron occupied Chesapeake\
+      \ Bay, and converted it into\na British naval station. After the month of February,\
+      \ 1813, the coasts\nof Virginia and Maryland enjoyed not a moment\u2019s repose.\
+      \ Considering the\nimmense naval power wielded by England, the Americans were\
+      \ fortunate\nthat their chief losses were confined to the farm-yards and \n\
+      n be given to it.\u201D\n\nAlthough Madison pointed to the notorious supply\
+      \ of food for the\nBritish forces in Canada as one of the motives for imposing\
+      \ an embargo,\nno one supposed that motive to be decisive. Other laws already\
+      \ forbade\nand punished such communication with the enemy; and experience proved\n\
+      that a general embargo would be no more effective than any special\nprohibition.\
+      \ The idea that England could be distressed by an embargo\nseemed still less\
+      \ likely to influence Government. Congress knew that\nRussia, Prussia, Denmark,\
+      \ Sweden and Norway, Spain, and South America\nwere already open to English\
+      \ commerce, and that a few days must decide\nwhether Napoleon could much longer\
+      \ prevent Great Britain from trading\nwith France. The possibility of distressing\
+      \ England by closing Boston\nand Salem, New Bedford and Newport to neutral ships\
+      \ was not to be\nseriously treated.\n\nWhatever was the true motive of the President\u2019\
+      s recommendation,\nCongress instantly approved it. The next day, December 10,\
+      \ the Hou\nculty in obtaining it; but he would hardly have arranged a system\n\
+      of signals as visible to Decatur as to himself. Even had he done so, he\nmight\
+      \ have employed men in his own service as well as Americans for the\npurpose.\
+      \ Decatur\u2019s letter admitted that he had made great exertions to\ndetect\
+      \ the culprits, but without success.\n\nThe rigor of the British blockade extended\
+      \ no farther north than\nthe Vineyard and Nantucket. Captain Broke in the \u201C\
+      Shannon,\u201D with a\ncompanion frigate, cruised off Boston harbor rather to\
+      \ watch for\nships-of-war than to interfere with neutral commerce. Along the\
+      \ coast\nof Maine an illicit trade with the British provinces was so actively\n\
+      pursued that one of the few American sloops-of-war, the \u201CEnterprise,\u201D\
+      \ncruised there, holding smugglers, privateers, and petty marauders in\ncheck.\
+      \ On no other portion of the coast would an armed national vessel\nhave been\
+      \ allowed to show itself, but the \u201CEnterprise,\u201D protected\nby the\
+      \ bays and inlets of Maine, and favored by the absence of a\nst India colonies\
+      \ were kept in constant discomfort and starvation\nby swarms of semi-piratical\
+      \ craft darting in and out of every channel\namong their islands; but the people\
+      \ of England could have borne with\npatience the punishment of the West Indies\
+      \ had not the American\ncruisers inflicted equally severe retribution nearer\
+      \ home.\n\nGreat Britain was blockaded. No one could deny that manifest danger\n\
+      existed to any merchant-vessel that entered or left British waters.\nDuring\
+      \ the summer the blockade was continuous. Toward the close of\n1812 an American\
+      \ named Preble, living in Paris, bought a small vessel,\nsaid to have belonged\
+      \ in turn to the British and French navy, which\nhe fitted as a privateer-brig,\
+      \ carrying sixteen guns and one hundred\nand sixty men. The \u201CTrue-Blooded\
+      \ Yankee,\u201D commanded by Captain Hailey,\nsailed from Brest March 1, 1813,\
+      \ and cruised thirty-seven days on\nthe coasts of Ireland and Scotland, capturing\
+      \ twenty-seven valuable\nvessels; sinking coasters in the very bay of Dublin;\
+      \ la\ntion that the secretary\nintended to throw upon them the whole responsibility\
+      \ for the failure of\nan expedition which he as well as they knew to be hopeless.\n\
+      \nDoubtless a vigorous general might still have found means if not to\ntake\
+      \ Montreal, at least to compel the British to evacuate Upper Canada;\nbut Wilkinson\
+      \ was naturally a weak man, and during the descent of the\nriver he was excessively\
+      \ ill, never able to make a great exertion.\nEvery day his difficulties increased.\
+      \ Hardly had his flotilla begun\nits descent, when a number of British gunboats\
+      \ commanded by Captain\nMulcaster, the most energetic officer in the British\
+      \ naval service\non the Lake, slipping through Chauncey\u2019s blockade, appeared\
+      \ in\nWilkinson\u2019s rear, and caused him much annoyance. Eight hundred British\n\
+      rank-and-file from Kingston and Prescott were with Mulcaster, and at\nevery\
+      \ narrow pass of the river, musketry and artillery began to open\non Wilkinson\
+      \ from the British bank. Progress became slow. November 7,\nMacomb was landed\
+      \ on "
+    query0: in July, flour sold at Boston for _?
+    query1: When and why did the british Blockade happen?
     remember_chat_context: false
 name: Rag Demo With Model-graded Eval
 prompts:
-- input: 'Answer the following question using the provided context. Question: {{query}}
-    Context: {{context}}'
+- input: 'Answer the following question using the provided context. Question: {{query1}}
+    Context: {{context1}}'
   metadata:
     model:
       name: gpt-3.5-turbo
       settings: {}
     parameters: {}
-  name: generate_baseline
-  outputs:
-  - data: In July, flour sold at Boston for $11.87 a barrel.
-    execution_count: 0
-    metadata:
-      created: 1707342850
-      id: chatcmpl-8pk9qicvDEenHkHyOJZPUivwEYbwm
-      model: gpt-3.5-turbo-0613
-      object: chat.completion.chunk
-      raw_response:
-        content: In July, flour sold at Boston for $11.87 a barrel.
-        role: assistant
-      role: assistant
-    output_type: execute_result
+  name: generate_1
+  outputs: []
 - input: "Given the following question, and answer, does the answer satisfactorily\
-    \ answer the question? \n\nQuestion: {{query}}\nAnswer: {{generate_baseline.output}}"
+    \ answer the question? \n\nQuestion: {{query1}}\nAnswer: {{generate_1.output}}"
   metadata:
     model: gpt-4
     parameters: {}
     remember_chat_context: false
-  name: evaluate_relevance
-  outputs:
-  - data: Yes, the answer satisfactorily answers the question.
-    execution_count: 0
-    metadata:
-      created: 1707342854
-      id: chatcmpl-8pk9uOnWR9uM4MFKXgR7xUsvCdtfv
-      model: gpt-4-0613
-      object: chat.completion.chunk
-      raw_response:
-        content: Yes, the answer satisfactorily answers the question.
-        role: assistant
-      role: assistant
-    output_type: execute_result
+  name: evaluate_relevance_1
+  outputs: []
 - input: "Given the following question, context, and answer, does the answer adhere\
     \ strictly to the context? \n\nGive a faithfulness verdict (YES or NO) with an\
-    \ explanation.\n\nQuestion: {{query}}\nAnswer: {{generate_baseline.output}}\n\n\
-    Verdict:\nExplanation:"
+    \ explanation.\n\nQuestion: {{query1}}\nContext: {{context1}}\nAnswer: {{generate_baseline.output}}\n\
+    \nVerdict:\nExplanation:"
   metadata:
     model: gpt-4
     parameters: {}
     remember_chat_context: false
-  name: evaluate_faithfulness_baseline
-  outputs:
-  - data: 'NO Explanation: The context does not provide any information on the price
-      at which flour was sold in Boston in July.'
-    execution_count: 0
-    metadata:
-      created: 1707342862
-      id: chatcmpl-8pkA2opsT7yYITM9ZeNEb6Ptmpxni
-      model: gpt-4-0613
-      object: chat.completion.chunk
-      raw_response:
-        content: 'NO Explanation: The context does not provide any information on
-          the price at which flour was sold in Boston in July.'
-        role: assistant
-      role: assistant
-    output_type: execute_result
+  name: evaluate_faithfulness_1
+  outputs: []
 - input: 'Given the following answer, is the answer self-consistent and easy to understand?
-    Answer: {{generate_baseline.output}}'
+    Answer: {{generate_1.output}}'
   metadata:
     model: gpt-4
     parameters: {}
     remember_chat_context: false
-  name: evaluate_coherence
-  outputs:
-  - data: Yes, the answer is self-consistent and easy to understand. It clearly states
-      that in the specified month and location, flour was sold for a specific price.
-    execution_count: 0
-    metadata:
-      created: 1707342885
-      id: chatcmpl-8pkAPsRKRDAkMWTblxgNHkAUvyA7Y
-      model: gpt-4-0613
-      object: chat.completion.chunk
-      raw_response:
-        content: Yes, the answer is self-consistent and easy to understand. It clearly
-          states that in the specified month and location, flour was sold for a specific
-          price.
-        role: assistant
-      role: assistant
-    output_type: execute_result
-- input: 'Answer the following question using the provided context. Review your answer.
-    Consider all the claims in your answer. Is each one justified by the context.
-    Revise your answer to accord ONLY with the context, paying careful attention to
-    details. Review your answer again. Does it answer the question? Exclude details
-    that are irrelevant to the question. If you cannot answer the question, simply
-    state that and nothing else. Question: {{query}} Context: {{context}} ANSWER:
-    REVISED ANSWER:'
+  name: evaluate_coherence_1
+  outputs: []
+- input: "Answer the following question using the provided context. Review your answer.\
+    \ Consider all the claims in your answer. Is each one justified by the context.\
+    \ Revise your answer to accord ONLY with the context, paying careful attention\
+    \ to details. Review your answer again. Does it answer the question? Exclude details\
+    \ that are irrelevant to the question. If you cannot answer the question, simply\
+    \ state that and nothing else. Question: {{query1}}\nContext: {{context1}} \n\
+    ANSWER: \nREVISED ANSWER:"
   metadata:
     model: gpt-3.5-turbo
     parameters: {}
-  name: generate_improved
-  outputs:
-  - data: Based on the context, there is no specific information provided about the
-      price of flour in July.
-    execution_count: 0
-    metadata:
-      created: 1707342892
-      id: chatcmpl-8pkAWtSxabIaQgTVHTc7o4XrybUFj
-      model: gpt-3.5-turbo-0613
-      object: chat.completion.chunk
-      raw_response:
-        content: Based on the context, there is no specific information provided about
-          the price of flour in July.
-        role: assistant
-      role: assistant
-    output_type: execute_result
+  name: generate_improved_1
+  outputs: []
 - input: "Given the following question, context, and answer, does the answer adhere\
     \ strictly to the context? \n\nGive a faithfulness verdict (YES or NO) with an\
-    \ explanation.\n\nQuestion: {{query}}\nAnswer: {{generate_improved.output}}\n\n\
-    Verdict:\nExplanation:"
+    \ explanation.\n\nQuestion: {{query1}}\nContext: {{context1}}\nAnswer: {{generate_1.output}}\n\
+    \nVerdict:\nExplanation:"
+  metadata:
+    model: gpt-4
+    parameters: {}
+  name: evaluate_faithfulness_improved_1
+  outputs: []
+- input: 'Is this text succinct? Succinct means it does not contain extraneous information.
+
+    Give a verdict (YES or NO) with an explanation.
+
+
+    Text: {{generate_1.output}}
+
+
+    Verdict:
+
+    Explanation:'
   metadata:
     model: gpt-4
     parameters: {}
-  name: evaluate_faithfulness_improved
-  outputs:
-  - data: 'YES Explanation: The answer accurately lines up with the context where
-      there is no specific information provided about the price of flour in July.'
-    execution_count: 0
-    metadata:
-      created: 1707342924
-      id: chatcmpl-8pkB2pDap3Bld3Lc1aEDheOHJs6vP
-      model: gpt-4-0613
-      object: chat.completion.chunk
-      raw_response:
-        content: 'YES Explanation: The answer accurately lines up with the context
-          where there is no specific information provided about the price of flour
-          in July.'
-        role: assistant
-      role: assistant
-    output_type: execute_result
+  name: evaluate_succinctness_1
+  outputs: []
 schema_version: latest