update evolution

[jpata]

• allow to interleave attention layers
• make the fitness rely more on loss and less on match_frac, iqr and runtime

TODO:

• run for 24h on A100 to check the latest fitness results and compare to standalone hackable model+training, architecture tuning #460

Top 10 models by fitness:
 1. Gen  3 - Fitness: 0.4112 - Val Loss: 0.9669 - Match Frac: 0.6891 - VRAM: 3905.0 MB - Key: i(55,128,256,default,dropout=0.0)|{pid:f(8,128,512,pos=T,dropout=0.2)*2,reg:g(8,128,512,pos=T,dropout=0.0)*3+g(8,128,512)+f(8,128,512,dropout=0.2)*4}|o(8,256,default)
 2. Gen  4 - Fitness: 0.4085 - Val Loss: 0.9633 - Match Frac: 0.6722 - VRAM: 1597.9 MB - Key: i(55,128,256,default,dropout=0.0)|f(16,128,512)*2|o(8,256,default)
 3. Gen  6 - Fitness: 0.4082 - Val Loss: 0.9701 - Match Frac: 0.6706 - VRAM: 1552.3 MB - Key: i(55,128,256,default,dropout=0.0)|f(16,128,256)+f(16,128,512)|o(8,256,default)
 4. Gen  6 - Fitness: 0.4073 - Val Loss: 0.9815 - Match Frac: 0.6972 - VRAM: 1295.9 MB - Key: i(55,128,256,default,dropout=0.0)|f(16,128,512)|o(8,256,default)
 5. Gen  5 - Fitness: 0.4065 - Val Loss: 0.9681 - Match Frac: 0.6726 - VRAM: 1597.9 MB - Key: i(55,128,256,default,dropout=0.0)|f(16,128,512)*2|o(8,256,default)
 6. Gen  6 - Fitness: 0.4061 - Val Loss: 0.9760 - Match Frac: 0.6766 - VRAM: 1598.5 MB - Key: i(55,128,256,default,dropout=0.0)|f(16,128,512)*2|o(8,256,default,rg={pt:direct,eta:direct,sin_phi:direct,cos_phi:direct,energy:direct},dropout=0.15,embedding_dim=128)
 7. Gen  1 - Fitness: 0.4051 - Val Loss: 1.0430 - Match Frac: 0.7294 - VRAM: 3954.0 MB - Key: i(55,128,256,default,dropout=0.2)|h(16,128,512,pos=T)|o(8,256,default,dropout=0.2)
 8. Gen  4 - Fitness: 0.4034 - Val Loss: 1.0366 - Match Frac: 0.7045 - VRAM: 1609.2 MB - Key: i(55,128,256,default,dropout=0.2)|f(16,128,512)*2|o(8,256,default,rg={pt:direct,eta:direct,sin_phi:direct,cos_phi:direct,energy:direct},dropout=0.15,embedding_dim=128)
 9. Gen  5 - Fitness: 0.4029 - Val Loss: 1.0030 - Match Frac: 0.7487 - VRAM: 1285.4 MB - Key: i(55,128,256,default)|g(32,128,512,pos=T,dropout=0.2)|o(8,256,default)
10. Gen  2 - Fitness: 0.4013 - Val Loss: 1.0439 - Match Frac: 0.6599 - VRAM: 1492.3 MB - Key: i(55,128,256,default,dropout=0.0)|s(8,128,512,pos=T)*2|o(8,256,default)