Beam search numbers and Temperature??

[ZDDWLIG]

What are these two parameters for：“Beam search numbers” and “Temperature”

[schxar]

temperature is always about next token possibility chance，higher means more random，lower means always the same ，i don't know beam search numbers either。

[Korner83]

A beam search is most often used to maintain tractability in large systems with insufficient amount of memory to store the entire search tree. For example, it has been used in many machine translation systems. (The state of the art now primarily uses neural machine translation based methods.)

I have also asked Mini-Gpt about it :) here is what it says:

Beam search is a technique used in large language models to generate text. In beam search, the model generates text by considering a set of possible words or tokens at a time, rather than all possible words. This is done by “beaming” a set of words together, creating a beam of possible words. The beam is then gradually narrowed down by considering only the best words, based on some predefined scoring function.

The size of the beam affects the quality of the generated text. A larger beam size results in a wider range of possible words, but may lead to lower quality text. A smaller beam size results in a narrower range of possible words, but may lead to better quality text. The beam size is a hyperparameter that can be adjusted in the model.

In practice, beam search numbers are used to control the beam size in the model. A beam size of 1 means that the model generates text by considering only one word at a time. A beam size of 2 means that the model generates text by considering two words at a time. The larger the beam size, the more words will be considered at a time, and the more diverse the generated text will be.

You can check it out more in depth:
https://www.width.ai/post/what-is-beam-search

[ZDDWLIG]

Thanks, now I get it