URGENT - master_june24 does not use gpucpp_june24 (and upgrading fails codegen)

[valassi]

Another issue introduced in #830 and being reviewed in #882.

Hi @oliviermattelaer this is a question related to the WIP #882 merge of master into master_june24

I have noticed that master_june24 currently uses this commit

git log --oneline HEAD
eef200f94 (HEAD) Merge remote-tracking branch 'upstream/gpucpp_826' into gpucpp_wrap
57e617cf3 Merge branch 'gpucpp' into gpucpp_wrap
10378b3c0 allow for second exporter to have access to all variable used in the fortran exporter
f9f957918 Fix validity time check for UFO pickle (#97)
8a8622b26 update py8 test

Conversely gpucpp_june24 is more recent (which as you see is also more recent than gpucpp_wrap, which is more recent than eef200f94)

git log --oneline origin/gpucpp_june24
def7fa691 (origin/gpucpp_june24) sync gpuccp_june24 with gpucpp
1e2aa4bc3 remove FIXME associated to goodjet and add appropriate code for proper debugging
09c96dd17 allow for second exporter to have access to all variable used in the fortran exporter
9abf6a3ad Merge pull request #113 from valassi/valassi_volatile
f274cab55 Workaround for SIGFPE crashes in function rotxxx (madgraph5/madgraph4gpu#855): add 'volatile' to prevent optimizations
0b8678984 Merge pull request #112 from valassi/valassi_uninitialised111
18696c1cf Merge pull request #110 from valassi/valassi_leak109
4f8fbb7f3 Workaround for issue #111 reported by valgrind (initialise goodjet array in function setclscales in reweight.f)
f6d90fa58 Fix memory leak #109 in madevent_driver.f (close file dname.mg)
37d18713d fix change in cpp template within the test (spaces)
0e5e483d4 fixing pythia8 iotest
849194f00 fix some tests (template changed)
1a9e6dd4f fixing an issue in the fortran side if no vector size
e2e790051 add a possibility to have multiple type of factorization (no actual code change at this stage)
942dab7c1 secure the code
45f8410e7 (origin/gpucpp_fix_scan_vector) Merge branch 'gpucpp_june24' into gpucpp_fix_scan_vector
0959edf96 Merge branch 'gpucpp_wrap' into gpucpp_june24
63d2d396c (origin/gpucpp_wrap) remove useless file + debug statement
ecb0a03fb add gpu in the autocompletion for group_subproceses
82f078bdc change makefile to recompile code as needed if vector size is changed
10378b3c0 allow for second exporter to have access to all variable used in the fortran exporter
f9f957918 Fix validity time check for UFO pickle (#97)
8a8622b26 update py8 test

That is to say
gpucpp_june24 > gpucpp_wrap > eef200f94

I have tried to upgrade master_june24 to use gpucpp_june24 (in eba31fd in #882), but this fails codegen

Command "import /data/avalassi/GPU2023/madgraph4gpuX/MG5aMC/TMPOUT/CODEGEN_mad_ee_mumu.mg" interrupted in sub-command:
"output madevent_simd ../TMPOUT/CODEGEN_mad_ee_mumu --hel_recycling=False --vector_size=32" with error:
AttributeError : 'SIMD_ProcessExporter' object has no attribute 'write_source_makefile'
Please report this bug on https://bugs.launchpad.net/mg5amcnlo
More information is found in 'MG5_debug'.
Please attach this file to your report.

Can you please clarify? Can we start getting rid of some of these branches, to simplify the situation? (eg gpucpp_wrap can be removed if it is already superseded by gpucpp_june24?)

Note: for the moment for merging master_june24 and master I will work on merging gpucpp into eef200f94 but I will not touch gpucpp_june24. Let me know how that sounds...

Thanks
Andrea

[oliviermattelaer]

@valassi Is this still relevant?

[valassi]

@valassi Is this still relevant?

Hi @oliviermattelaer yes unfortunately this is still relevant.

And I would say that it is urgent, because the situation with all these branches is very messy and diverging even more. I repeat, the problem with branches was there before I started my work: there is a master_june24 but this was not using gpuccp_june24. And trying to use gpucpp_june24 against master_june24 as-is produced the codegen errors above. (This still happens if I merge gpucpp_june24 into my own branch).

For development in #882, I am using https://github.com/valassi/mg5amcnlo/commits/valassi_gpucpp_june24/

I have just produced two PRs in mg5amcnlo for info

• (WIP: NEEDED?) Merge remnants of gpucpp_june24 and gpucpp_wrap into gpucpp mg5amcnlo/mg5amcnlo#120 against gpucpp_june24: here maybe you can understand better my changes, but the result is UNUSABLE for me at the moment
• master_june24 completion: nb_warp_used (and a SUBset of gpucpp_june24) into gpucpp mg5amcnlo/mg5amcnlo#121 against gpucpp: this is USABLE for me against my 882... and indeed it is a possibility to consider, but I would merge that ONLY when we also agree to merge 882 into master (otherwise again we get master not in sync with gpucpp)

Can you try to have a look at this issue please?

Command "import /data/avalassi/GPU2023/madgraph4gpuX/MG5aMC/TMPOUT/CODEGEN_mad_gg_tt.mg" interrupted in sub-command:
"output madevent_simd ../TMPOUT/CODEGEN_mad_gg_tt --hel_recycling=False --vector_size=32" with error:
AttributeError : 'SIMD_ProcessExporter' object has no attribute 'write_source_makefile'
Please report this bug on https://bugs.launchpad.net/mg5amcnlo
More information is found in 'MG5_debug'.
Please attach this file to your report.

Thanks
Andrea

[oliviermattelaer]

Ok I can take a look at the status of the gpucpp_june24 branch
and make sure that it is working with master_june24

[valassi]

Hi @oliviermattelaer I had a look at this #886 again.

The status is that gpucpp_june24 still does not work for me with master_june24.

I also had a look at the version which you use in the CI pR #865 ie mg5amcnlo/mg5amcnlo@942dab7 The status is that these changes are all in gpucpp_june24 IIUC, so they do not add anything an ddo not solve the problem.

So my question: can I just ignore gpucpp_june24? Rephrase: the work I did on master_june24 works and is self consistent with the gpucpp* modified version I have, I do not need anything from gpucpp_june24. If there is anything you need from gpucpp_june24, I suggest that you look at that AFTER we merge #882. Does this makes sense?

Thanks
Andrea

[valassi]

PS Note, effectively gpucpp_june24 is replaced by my valassi_gpucpp_june24. This should be approved here mg5amcnlo/mg5amcnlo#121

[valassi]

This can be closed (and unpinned).

I have now merged into gpucpp what I needed for june24/channelid.
This is mg5amcnlo/mg5amcnlo#121 (branch valassi_gpucpp_june24 merged into gpucpp).

Note: there are still some commits which were maybe used by #830 and which are now in mg5amcnlo/mg5amcnlo#120 (this remains open). But functionally june24 is now in master and all looks good.

CLosing and unpnning