[15/N][RAG Demo fixes] [dev] benchmarking {gpt3.5,gpt4} X {generate_V1, generate_V2}

[jonathanlastmileai]

[15/N][RAG Demo fixes] [dev] benchmarking {gpt3.5,gpt4} X {generate_V1, generate_V2}

5 trials each. With GPT4, generate prompt v2 does slightly better on the benchmark.

GPT3.5, V1

GPT3.5, V2

GPT4, V1

GPT4, V2

---

Stack created with Sapling. Best reviewed with ReviewStack.

• -> [15/N][RAG Demo fixes] [dev] benchmarking {gpt3.5,gpt4} X {generate_V1, generate_V2} #1206
• [14/N][RAG Demo fixes] [dev] run 5 trials over 4 queries #1204
• [13/N][RAG Demo fixes] [dev] aggregate metrics over all queries to review which dont work well #1203
• [12/N][RAG Demo fixes] query_index no longer needed #1202
• [11/N][RAG Demo fixes] add N-trial evals #1200