Adjust top_p & temp depending on type of query

[vGebs]

Adjusting the temperature and top_p parameters can change the output of the model in various ways. Here are some scenarios where you might want to alter them:

Temperature:
Creative Writing: If you're using the model for creative writing or brainstorming and you want diverse, unexpected responses, a higher temperature like 0.8 might be appropriate. This introduces more randomness in the outputs.
Technical Writing: If you're generating technical or professional content where consistency and focus are crucial, a lower temperature like 0.2 might be better. This makes the output more deterministic and focused.

Top_p (Nucleus Sampling):
High-Stakes Decision Making: If you're using the model to help with high-stakes decisions where accuracy is critical, a lower top_p value like 0.1 could be useful. This restricts the model's responses to the most probable tokens.
Exploratory Conversations: If you're having an exploratory conversation and want a good balance between diversity and relevance, a higher top_p value like 0.9 could be suitable. This allows the model to consider a broader range of token possibilities.

Remember that both parameters control the trade-off between diversity and determinism in the model's responses. They should be adjusted according to the specific requirements of your task. As the documentation suggests, it's generally recommended to adjust either temperature or top_p, not both, to avoid confusion and unexpected behavior.