Why does "validation bpb" happen at step 0?

[vinjn]

[svlandeg]

Hi @vinjn! I'll transfer this to the Discussions, as we'd like to keep Issues focused on bug reports and features/TODOs.

[karpathy]

why wouldn't it?

[jwnsu]

It serves a good purpose to get a sense what initial bgb is (i.e. random initialization). base_train is a long running process consuming huge resources, regular readings of various metrics are essential to gauge whether the training is improving results. For example, if 2nd reading of bgb is not improving from initial bgb, it indicates there are likely some coding/process/approach issues that need to be fixed and the costly training can be stopped right away.