Default temperature

[woctezuma]

Is there a reason why the default temperature is 0.7?

I have not checked thoroughly yet, but this page shows different temperatures for "detectability baselines" (shown below). It seems that the temperature is chosen closer to:

• 0.9 for the 117M and 345M models,
• 0.75 for the 762M and 1542M models.

Model	Temperature 1	Top-K 40
117M	88.29%	96.79%
345M	88.94%	95.22%
762M	77.16%	94.43%
1542M	74.31%	92.69%

[minimaxir]

No technical reason; in my testing 0.7 was the safest for a fine-tuned model.

Keep in mind that finetuned GPT-2 is different from the raw GPT-2.

[woctezuma]

For info, I read the table wrong. :'(

those numbers are accuracies, not temperatures