Apfel++ benchmarks #181

giacomomagni · 2023-03-16T10:44:09Z

This PR is to introduce APFEL++ in yadmark.
This will be needed to benchmark polarized SF.

giacomomagni · 2023-03-16T10:47:49Z

@Radonirinaunimi Yadmark is running,
but numbers are obviously crazy, so there is still some work to to in apfelpy_utils, it would be great if you can take a look as well (I will not commit anything else today or tomorrow so feel free to push).

Note that while adding this I noticed that we still have to fix all the couplings for polarised SF in case of CC and NC
adjusting the parity violating terms.

src/yadmark/benchmark/external/apfelpy_utils.py

Radonirinaunimi · 2023-03-16T12:54:47Z

@Radonirinaunimi Yadmark is running, but numbers are obviously crazy, so there is still some work to to in apfelpy_utils, it would be great if you can take a look as well (I will not commit anything else today or tomorrow so feel free to push).

Note that while adding this I noticed that we still have to fix all the couplings for polarised SF in case of CC and NC adjusting the parity violating terms.

Thanks a lot @giacomomagni! I can indeed take it from here.

benchmarks/runners/apfelpy_.py

src/yadmark/benchmark/external/apfelpy_utils.py

Co-authored-by: Giacomo Magni <39065935+giacomomagni@users.noreply.github.com>

src/yadmark/benchmark/external/apfelpy_utils.py

src/yadism/coefficient_functions/fonll/kernels.py

src/yadmark/benchmark/external/apfelpy_utils.py

Radonirinaunimi · 2023-04-27T08:07:01Z

benchmarks/runners/apfelpy_.py

+        self.run(
+            self.theory_updates_zm(pto),
+            self.obs_updates_pol(),
+            ["ToyLH_polarized"],


Using ToyLH_polarized significantly worsen the agreement; at LO the difference can be up to 12%. This also happens when using that set to benchmark against Apfel.

it seems that using ToyLH_polarized enehanced the difference in g1 at NNLO, which was barely visible before.

can you break the difference down by channel? (using genpdf) how about using the good old $x(1-x)$?

we can try, the difference should originate from the Non Singlet coefficient.

here you have. At NNLO using -p toy_pol and using a finer Q grid (nQ=1000) for Apfel++:

gonly_pol:

────────── g1_total ────────── x Q2 yadism yadism_error APFEL++ percent_error 0 0.000012 10.000000 1.567862e-04 7.992130e-12 1.567876e-04 -0.000894 1 0.000049 10.000000 4.635223e-04 6.373798e-12 4.635245e-04 -0.000470 2 0.000204 10.000000 1.072596e-03 1.879000e-11 1.072594e-03 0.000148 3 0.000853 10.000000 1.955818e-03 3.772849e-11 1.955810e-03 0.000395 4 0.003562 10.000000 2.601327e-03 7.953587e-11 2.601333e-03 -0.000248 5 0.014874 10.000000 1.670687e-03 1.505371e-10 1.670663e-03 0.001465 6 0.062102 10.000000 -3.127464e-03 2.028302e-10 -3.127201e-03 0.008424 7 0.194737 10.000000 -6.261686e-03 1.752376e-10 -6.261719e-03 -0.000526 8 0.336842 10.000000 -3.545204e-03 4.788348e-11 -3.545221e-03 -0.000486 9 0.478947 10.000000 -1.226541e-03 2.452783e-11 -1.226448e-03 0.007615 10 0.621053 10.000000 -2.511471e-04 4.300323e-12 -2.512205e-04 -0.029240 11 0.763158 10.000000 -2.113743e-05 3.250476e-13 -2.090429e-05 1.115271 12 0.905263 10.000000 -1.089370e-07 6.238962e-15 -8.873470e-08 22.767103 13 0.001000 3.000000 1.905295e-03 4.628793e-11 1.905296e-03 -0.000049 14 0.001000 7.388431 2.258469e-03 5.386868e-11 2.258471e-03 -0.000051 15 0.001000 18.196305 1.738741e-03 4.083879e-11 1.738742e-03 -0.000071 16 0.001000 44.814047 1.572657e-03 3.648971e-11 1.572659e-03 -0.000134 17 0.001000 110.368500 1.329227e-03 3.050560e-11 1.329231e-03 -0.000236 18 0.001000 271.816686 1.147919e-03 2.608454e-11 1.147925e-03 -0.000490 19 0.001000 669.432950 1.011382e-03 2.277513e-11 1.011394e-03 -0.001116 20 0.001000 1648.686404 9.156301e-04 2.044882e-11 9.156547e-04 -0.002681 21 0.001000 4060.401956 8.726769e-04 1.934155e-11 8.727295e-04 -0.006031 22 0.001000 10000.000000 8.932759e-04 1.965916e-11 8.933740e-04 -0.010984

lonly_pol ie light quarks only:

────────── g1_total ────────── x Q2 yadism yadism_error APFEL++ percent_error 0 0.000012 10.000000 -0.000291 -2.251389e-11 -0.000287 1.305583 1 0.000049 10.000000 -0.000344 -4.627033e-11 -0.000340 1.095596 2 0.000204 10.000000 -0.000373 -1.629789e-11 -0.000373 0.057539 3 0.000853 10.000000 -0.000240 4.838951e-12 -0.000254 -5.772452 4 0.003562 10.000000 0.001481 5.130205e-11 0.001418 4.454185 5 0.014874 10.000000 0.012064 1.868573e-10 0.011870 1.632602 6 0.062102 10.000000 0.050398 7.749967e-10 0.049997 0.800746 7 0.194737 10.000000 0.109347 2.073501e-09 0.108995 0.322261 8 0.336842 10.000000 0.126687 5.465003e-09 0.126531 0.123587 9 0.478947 10.000000 0.110273 4.325416e-09 0.110178 0.086218 10 0.621053 10.000000 0.072825 2.807101e-09 0.072871 -0.063313 11 0.763158 10.000000 0.031264 8.198026e-10 0.031163 0.323404 12 0.905263 10.000000 0.004238 9.329867e-11 0.004395 -3.578833 13 0.001000 3.000000 0.000064 4.590512e-12 0.000041 56.256089 14 0.001000 7.388431 -0.000302 9.068681e-12 -0.000322 -6.256600 15 0.001000 18.196305 -0.000009 6.725702e-12 -0.000024 -61.740713 16 0.001000 44.814047 0.000060 4.396884e-12 0.000046 30.844171 17 0.001000 110.368500 0.000175 3.627017e-12 0.000164 6.923428 18 0.001000 271.816686 0.000250 3.047234e-12 0.000241 3.877718 19 0.001000 669.432950 0.000290 2.564404e-12 0.000283 2.767274 20 0.001000 1648.686404 0.000275 2.045430e-12 0.000269 2.471179 21 0.001000 4060.401956 0.000128 1.240913e-12 0.000122 4.640097 22 0.001000 10000.000000 -0.000245 -3.842776e-14 -0.000250 -1.908960

so was the problem nQ?

so was the problem nQ?

Yes! I am just now investigating if nQ=1000 is too much or it is enough to use nQ=200.

so was the problem nQ?

But only partly, just to be clear the problem is still there but there is not much we can do.
We are implementing the analytical relation for NS g1 = F3 while Aplel++ is not. In the bench you see the difference, which is small but not fully invisible.

But this is an analytical relation and I am not sure we'll ever do anything about this.

Also note that now with nQ=1000, the agreement for the full NNLO is now ~per-mille level (except for when the absolute results are ~zero).

felixhekhorn

please fix pre-commit as much as you can

src/yadmark/benchmark/external/apfelpy_utils.py

src/yadmark/benchmark/runner.py

Radonirinaunimi · 2023-04-28T12:52:27Z

As for the pre-commit, most of the pydocstyle fails are not related to this PR and would better be addressed in a different PR.

As for black & isort, they modify all the pylint labels as follows:

           from .external import (  # pylint:disable=import-error,import-outside-toplevel
                apfel_utils,
            )

Do we want to introduce these?

felixhekhorn · 2023-04-28T13:35:20Z

As for the pre-commit, most of the pydocstyle fails are not related to this PR and would better be addressed in a different PR.

agreed

As for black & isort, they modify all the pylint labels as follows:
           from .external import (  # pylint:disable=import-error,import-outside-toplevel
                apfel_utils,
            )
Do we want to introduce these?

if they want so, yes

init apfelxx bench

48a8720

giacomomagni requested review from felixhekhorn, RoyStegeman and Radonirinaunimi March 16, 2023 10:44

giacomomagni added benchmarks Benchmark (or infrastructure) related enhancement New feature or request labels Mar 16, 2023

giacomomagni marked this pull request as draft March 16, 2023 10:48

giacomomagni commented Mar 16, 2023

View reviewed changes

src/yadmark/benchmark/external/apfelpy_utils.py Outdated Show resolved Hide resolved

alecandido reviewed Mar 16, 2023

View reviewed changes

benchmarks/runners/apfelpy_.py Show resolved Hide resolved

giacomomagni commented Mar 16, 2023

View reviewed changes

src/yadmark/benchmark/external/apfelpy_utils.py Outdated Show resolved Hide resolved

felixhekhorn and others added 8 commits March 22, 2023 14:53

Update src/yadmark/benchmark/external/apfelpy_utils.py

c7e114f

Co-authored-by: Giacomo Magni <39065935+giacomomagni@users.noreply.github.com>

Merge branch 'polarized-coeffs' into apfelxx_bench

216bf90

Make benchmark working

ecb5b3e

fix thr and em charge

880ec63

some cleaning

9472f92

make it working agin

38b78e3

fix F3 EM to be 0

cb812c6

fix g4 and gL couplings

0badd0b

giacomomagni commented Apr 19, 2023

View reviewed changes

src/yadmark/benchmark/external/apfelpy_utils.py Outdated Show resolved Hide resolved

Radonirinaunimi added 4 commits April 19, 2023 21:46

Make some cleaning

a56cfa7

Make CC work for unpolarised

43b3169

Reduce Cyclomatic complexity

e3755c3

Fix unit tests

c05a374

felixhekhorn reviewed Apr 20, 2023

View reviewed changes

src/yadism/coefficient_functions/fonll/kernels.py Outdated Show resolved Hide resolved

Start propagate is_pv in some files

35a1a1f

Radonirinaunimi reviewed Apr 20, 2023

View reviewed changes

src/yadmark/benchmark/external/apfelpy_utils.py Outdated Show resolved Hide resolved

Radonirinaunimi mentioned this pull request Apr 21, 2023

Implement the light part of the polarized coefficient functions #173

Merged

4 tasks

Propagate everywhere

7ec9138

Fix CC light/charm/bottom

f334a60

giacomomagni commented Apr 24, 2023

View reviewed changes

src/yadmark/benchmark/external/apfelpy_utils.py Outdated Show resolved Hide resolved

giacomomagni commented Apr 24, 2023

View reviewed changes

src/yadmark/benchmark/external/apfelpy_utils.py Outdated Show resolved Hide resolved

giacomomagni commented Apr 24, 2023

View reviewed changes

src/yadmark/benchmark/external/apfelpy_utils.py Outdated Show resolved Hide resolved

Radonirinaunimi and others added 5 commits April 24, 2023 11:19

Apply some of the comments from CR

3319565

Rename tabulating of NC & EM

f644231

allow for generic PDF in apfelpy utils

4aa0a36

make CC working

537c2c4

fix pdf name in apfelpy utils

d9ec896

Radonirinaunimi reviewed Apr 27, 2023

View reviewed changes

giacomomagni marked this pull request as ready for review April 28, 2023 09:54

giacomomagni added 2 commits April 28, 2023 12:06

Merge branch 'polarized-coeffs' into apfelxx_bench

de68752

black and isort

4df0334

giacomomagni changed the base branch from polarized-coeffs to master April 28, 2023 10:11

giacomomagni changed the base branch from master to polarized-coeffs April 28, 2023 10:11

fix name shortcut

64efdfc

giacomomagni requested review from Radonirinaunimi and felixhekhorn April 28, 2023 10:28

Set nQ to a more reasonable value

4d15c43

Radonirinaunimi approved these changes Apr 28, 2023

View reviewed changes

felixhekhorn requested changes Apr 28, 2023

View reviewed changes

src/yadmark/benchmark/external/apfelpy_utils.py Show resolved Hide resolved

src/yadmark/benchmark/external/apfelpy_utils.py Outdated Show resolved Hide resolved

src/yadmark/benchmark/runner.py Show resolved Hide resolved

Radonirinaunimi force-pushed the apfelxx_bench branch from 116ada7 to 4d15c43 Compare April 28, 2023 12:31

Remove non-applicable To do

3bf2a64

Fix imports where pylint labels exist

a5d9004

felixhekhorn approved these changes Apr 28, 2023

View reviewed changes

giacomomagni merged commit c9b6515 into polarized-coeffs Apr 28, 2023
3 of 4 checks passed

giacomomagni deleted the apfelxx_bench branch April 28, 2023 15:49

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Apfel++ benchmarks #181

Apfel++ benchmarks #181

giacomomagni commented Mar 16, 2023

giacomomagni commented Mar 16, 2023 •

edited

Radonirinaunimi commented Mar 16, 2023

Radonirinaunimi Apr 27, 2023

giacomomagni Apr 27, 2023

felixhekhorn Apr 27, 2023

giacomomagni Apr 28, 2023

giacomomagni Apr 28, 2023 •

edited

felixhekhorn Apr 28, 2023

Radonirinaunimi Apr 28, 2023

giacomomagni Apr 28, 2023

Radonirinaunimi Apr 28, 2023

Radonirinaunimi Apr 28, 2023

felixhekhorn left a comment

Radonirinaunimi commented Apr 28, 2023

felixhekhorn commented Apr 28, 2023

Apfel++ benchmarks #181

Apfel++ benchmarks #181

Conversation

giacomomagni commented Mar 16, 2023

giacomomagni commented Mar 16, 2023 • edited

Radonirinaunimi commented Mar 16, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

giacomomagni Apr 28, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

felixhekhorn left a comment

Choose a reason for hiding this comment

Radonirinaunimi commented Apr 28, 2023

felixhekhorn commented Apr 28, 2023

giacomomagni commented Mar 16, 2023 •

edited

giacomomagni Apr 28, 2023 •

edited