faster chatting (WIP)

[ddorian]

Description: Making chatting faster by removing small and big things.

Checklist:

• My PR is ready for code review
• I have added some tests, if applicable, and run the whole test suite, including linting tests
• I have updated the documentation, if applicable

Some benchmarks on my workload: (100 runs, but some are "hi" and some are "find intent/slots".
No change::

Removing deepcopy:

As we can see, there are still things that take time just like my last pull request #648 like initial_string_from_tokens(), tokens = [c["token"] for c in cache], having the hot-path execute last in the functions, using some mutable data structures when they're never modified (and recreating them). This was using a small model, I'll check with big model to see if they take more % time.

What do you think ?

[adrienball]

Hi @ddorian ,
Thanks for this PR.
We initially used these deepcopys in order to ensure the integrity of the cache and protect it from being mutated in the client code. That is indeed expensive.
Can you fix the small linting issue (unused deepcopy import)? Then I will merge this.
Cheers.

[adrienball]

🎉

[ddorian]

Thank you.
What about my other suggestions ? initial_string_from_tokens and small stuff that add up ? Should I create a pull request for them ?