Lowering temperature to reduce AI hallucinations

[MalikMAlna]

With a current default temperature of 0.7 the AI is much more likely to hallucinate while completing tasks. Lowering the temperature to 0.0 should reduce the likelihood of hallucination substantially.

[MalikMAlna]

Also changed the n parameter in the context_agent function to top_results_num to be more readable. Updated the docstring for it too.

[Podidiving]

I think 0.0 is a bit extreme. Are you sure it works better? Maybe the best way is to let the user decide which value suits better by introducing additional command-line argument?

+ I'm not quite sure of the intuition for fixing different constants (0.5 and 0.7). Seems to me like some kind of magic constants

[MalikMAlna]

I mean, I'm not against only setting it to something that's less than 0.7, but the temperature is essentially the variability of the text response. So for creative tasks it's better if its higher, but worse if the task is objective, fact-based. OpenAI outlines this distinction quite clearly in the docs.

If you want more accurate and deterministic results, you lower that to 0.

So @Podidiving, I don't consider that extreme at all if one wishes to avoid hallucinations occurring because the bot decided to get less consistent with its task responses.

[Podidiving]

Yeah, I totally agree on the fact that if you want your model to be more deterministic, you set temperature low. But in some cases you may want to get more creative response. That's why I suggested to introduce temperature as a parameter, which user can adjust

[MalikMAlna]

@Podidiving I'm not sure I understand you. It is already a parameter with a default value that I changed to 0.0 instead of having it be 0.7 by default.

[Podidiving]

All I'm saying is that maybe it's better to make this parameter customisable.

E.g. in .env file:

TEMPERATURE=0.0

and in babyagi.py: temperature = float(os.getenv("TEMPERATURE, 0.0))

Something like that

P.s. default parameter is 0.5 and not 0.7 actually. And I don't really get it why here it is changed

[MalikMAlna]

Alright, there you go, @Podidiving .

[MalikMAlna]

Also, it was getting called as 0.7 as the temperature. So it was defaulting to 0.7, which is waaaay too high for most task agent use cases.