POC: Performance Improvements

[AkhileshNegi]

Is your feature request related to a problem? Please describe.
We recently migrated Glific flow to use AI Platform endpoint so we can run benchmarks, and overall didn't see much improvements

setup the sheet & flow correctly to capture the time and after 2400+ Q&A pair with AI platform end point vs 3000+ Q&A pair with Glific filesearch here are results:

Using AI platfrom webhook
https://docs.google.com/spreadsheets/d/1u0mtdt1nMLxE_NIR9BKD-pYhsXs_YstrFgF9AAnz6TA/edit?gid=41605576#gid=41605576

Using Filesearch within Glific
https://docs.google.com/spreadsheets/d/1u0mtdt1nMLxE_NIR9BKD-pYhsXs_YstrFgF9AAnz6TA/edit?gid=1678521389#gid=1678521389

Left: Base.png: results from Filesearch within Glific
Right: platform.png : results from Using AI platfrom webhook

If anything the performance is degraded by few seconds,

Describe the solution you'd like
Find way to improve latency and validate the approach by running benchmarks against the base data here

[jerome-white]

See Discord for existing discussions. Gist:

• Cache OpenAI client instances
• Remove tests for assistant and thread validity

[jerome-white]

Also mentioned in PR comment:

          Consider avoiding this test in favor of `client.beta.threads.runs.create_and_poll` reporting the error

Originally posted by @jerome-white in #40 (comment)

[AkhileshNegi]

We talked it over internally—there’s not much we can do right now. We’ll be adding Langfuse soon, which should give us better visibility into latency. Once that’s in place, we’ll revisit this.