Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sci-physics/pythia: add 8.3.09, 9999 #32056

Closed
wants to merge 2 commits into from
Closed

Conversation

APN-Pucky
Copy link
Contributor

Closes: https://bugs.gentoo.org/862103
Signed-off-by: Alexander Puck Neuwirth alexander@neuwirth-informatik.de

In the last half year I accumulated some changes compared to the most recent ebuild 8.3.07-r1.

These changes are:

  1. hepmc2 and hepmc3 use flags to switch between versions, however I would not mind too much droping hepmc2 support, but I think it is a nice to have.
  2. PYTHIADIR is /usr/share/Pythia8 instead of /usr/share/pythia8. First letter in caps is also the default by pythia. So typical tools using pythia expect it with a capital first letter.
  3. Added share/Pythia8/pdfdata installation
  4. Fix a bug by using EPREFIX as absolute instead of relative path for --prefix-lib

@gentoo-bot
Copy link

Pull Request assignment

Submitter: @APN-Pucky
Areas affected: ebuilds
Packages affected: sci-physics/pythia

sci-physics/pythia: @gentoo/sci-physics

Linked bugs

Bugs linked: 862103


In order to force reassignment and/or bug reference scan, please append [please reassign] to the pull request title.

Docs: Code of ConductCopyright policy (expl.) ● DevmanualGitHub PRsProxy-maint guide

@gentoo-bot gentoo-bot added assigned PR successfully assigned to the package maintainer(s). bug linked Bug/Closes found in footer, and cross-linked with the PR. labels Jul 26, 2023
@gentoo-repo-qa-bot
Copy link
Collaborator

Pull request CI report

Report generated at: 2023-07-26 19:10 UTC
Newest commit scanned: a6e5188
Status: ✅ good

There are existing issues already. Please look into the report to make sure none of them affect the packages in question:
https://qa-reports.gentoo.org/output/gentoo-ci/a4973bc13a/output.html

Copy link
Member

@AndrewAmmerlaan AndrewAmmerlaan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, some minor comments.

There is no need to drop the x86 keyword for these changes.

IUSE="doc examples fastjet +hepmc3 hepmc2 lhapdf root test zlib"
RESTRICT="!test? ( test )"
REQUIRED_USE="
^^ ( hepmc3 hepmc2 )
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this be ?? instead of ^^? I.e. 'zero or one' instead of 'exactly one'.

Comment on lines 16 to 22
https://www.hepforge.org/archive/lhapdf/pdfsets/v6.backup/${LHA_VER}/CT10.tar.gz
https://www.hepforge.org/archive/lhapdf/pdfsets/v6.backup/${LHA_VER}/MRST2007lomod.tar.gz
https://www.hepforge.org/archive/lhapdf/pdfsets/v6.backup/${LHA_VER}/NNPDF23_nlo_as_0119_qed_mc.tar.gz
https://www.hepforge.org/archive/lhapdf/pdfsets/v6.backup/${LHA_VER}/NNPDF23_nnlo_as_0119_qed_mc.tar.gz
https://www.hepforge.org/archive/lhapdf/pdfsets/v6.backup/${LHA_VER}/cteq66.tar.gz
https://www.hepforge.org/archive/lhapdf/pdfsets/v6.backup/${LHA_VER}/cteq6l1.tar.gz
https://www.hepforge.org/archive/lhapdf/pdfsets/v6.backup/${LHA_VER}/unvalidated/MRST2004qed.tar.gz
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are redirected to ../downloads/.. instead of ../archive/..:

RedirectedUrl: version 8.3.09: SRC_URI: permanently redirected: https://www.hepforge.org/archive/lhapdf/pdfsets/v6.backup/6.2.1/CT10.tar.gz -> https://www.hepforge.org/downloads/lhapdf/pdfsets/v6.backup/6.2.1/CT10.tar.gz
RedirectedUrl: version 8.3.09: SRC_URI: permanently redirected: https://www.hepforge.org/archive/lhapdf/pdfsets/v6.backup/6.2.1/MRST2007lomod.tar.gz -> https://www.hepforge.org/downloads/lhapdf/pdfsets/v6.backup/6.2.1/MRST2007lomod.tar.gz
RedirectedUrl: version 8.3.09: SRC_URI: permanently redirected: https://www.hepforge.org/archive/lhapdf/pdfsets/v6.backup/6.2.1/NNPDF23_nlo_as_0119_qed_mc.tar.gz-> https://www.hepforge.org/downloads/lhapdf/pdfsets/v6.backup/6.2.1/NNPDF23_nlo_as_0119_qed_mc.tar.gz
RedirectedUrl: version 8.3.09: SRC_URI: permanently redirected: https://www.hepforge.org/archive/lhapdf/pdfsets/v6.backup/6.2.1/NNPDF23_nnlo_as_0119_qed_mc.tar.gz -> https://www.hepforge.org/downloads/lhapdf/pdfsets/v6.backup/6.2.1/NNPDF23_nnlo_as_0119_qed_mc.tar.gz
RedirectedUrl: version 8.3.09: SRC_URI: permanently redirected: https://www.hepforge.org/archive/lhapdf/pdfsets/v6.backup/6.2.1/cteq66.tar.gz -> https://www.hepforge.org/downloads/lhapdf/pdfsets/v6.backup/6.2.1/cteq66.tar.gz
RedirectedUrl: version 8.3.09: SRC_URI: permanently redirected: https://www.hepforge.org/archive/lhapdf/pdfsets/v6.backup/6.2.1/cteq6l1.tar.gz -> https://www.hepforge.org/downloads/lhapdf/pdfsets/v6.backup/6.2.1/cteq6l1.tar.gz
RedirectedUrl: version 8.3.09: SRC_URI: permanently redirected: https://www.hepforge.org/archive/lhapdf/pdfsets/v6.backup/6.2.1/unvalidated/MRST2004qed.tar.gz ->https://www.hepforge.org/downloads/lhapdf/pdfsets/v6.backup/6.2.1/unvalidated/MRST2004qed.tar.gz
RedirectedUrl: version 9999: SRC_URI: permanently redirected: https://www.hepforge.org/archive/lhapdf/pdfsets/v6.backup/6.2.1/CT10.tar.gz -> https://www.hepforge.org/downloads/lhapdf/pdfsets/v6.backup/6.2.1/CT10.tar.gz
RedirectedUrl: version 9999: SRC_URI: permanently redirected: https://www.hepforge.org/archive/lhapdf/pdfsets/v6.backup/6.2.1/MRST2007lomod.tar.gz -> https://www.hepforge.org/downloads/lhapdf/pdfsets/v6.backup/6.2.1/MRST2007lomod.tar.gz
RedirectedUrl: version 9999: SRC_URI: permanently redirected: https://www.hepforge.org/archive/lhapdf/pdfsets/v6.backup/6.2.1/NNPDF23_nlo_as_0119_qed_mc.tar.gz -> https://www.hepforge.org/downloads/lhapdf/pdfsets/v6.backup/6.2.1/NNPDF23_nlo_as_0119_qed_mc.tar.gz
RedirectedUrl: version 9999: SRC_URI: permanently redirected: https://www.hepforge.org/archive/lhapdf/pdfsets/v6.backup/6.2.1/NNPDF23_nnlo_as_0119_qed_mc.tar.gz -> https://www.hepforge.org/downloads/lhapdf/pdfsets/v6.backup/6.2.1/NNPDF23_nnlo_as_0119_qed_mc.tar.gz
RedirectedUrl: version 9999: SRC_URI: permanently redirected: https://www.hepforge.org/archive/lhapdf/pdfsets/v6.backup/6.2.1/cteq66.tar.gz -> https://www.hepforge.org/downloads/lhapdf/pdfsets/v6.backup/6.2.1/cteq66.tar.gz
RedirectedUrl: version 9999: SRC_URI: permanently redirected: https://www.hepforge.org/archive/lhapdf/pdfsets/v6.backup/6.2.1/cteq6l1.tar.gz -> https://www.hepforge.org/downloads/lhapdf/pdfsets/v6.backup/6.2.1/cteq6l1.tar.gz
RedirectedUrl: version 9999: SRC_URI: permanently redirected: https://www.hepforge.org/archive/lhapdf/pdfsets/v6.backup/6.2.1/unvalidated/MRST2004qed.tar.gz -> https://www.hepforge.org/downloads/lhapdf/pdfsets/v6.backup/6.2.1/unvalidated/MRST2004qed.tar.gz

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They are all over the place. i.e.

https://www.hepforge.org/archive/lhapdf/pdfsets/v6.backup/6.2.1/CT10.tar.gz -> https://lhapdf.hepforge.org/downloads?f=pdfsets/v6.backup/6.2.1/CT10.tar.gz
https://www.hepforge.org/downloads/lhapdf/pdfsets/v6.backup/6.2.1/CT10.tar.gz -> https://lhapdf.hepforge.org/downloads?f=pdfsets/v6.backup/6.2.1/CT10.tar.gz
lhapdfsets.web.cern.ch/lhapdfsets/current/CT10.tar.gz -> https://lhapdfsets.web.cern.ch/current/CT10.tar.gz

all give the same checksum in the end. Not sure which to pick but since LHAPDF uses http://lhapdfsets.web.cern.ch/lhapdfsets/current/ this would probably be the most reliable?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

current sounds like it is a moving target, so this is not an option as it would break the SRC_URI every time a new version is released. We need something that includes some version number and ideally doesn't cause a RedirectedUrl warning, but this might not be possible.

Copy link
Contributor Author

@APN-Pucky APN-Pucky Jul 29, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, I'll switch to downloads then, since the unvalidated path also does not exist at the cern url.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll update these while I am at it:

 * failed fetching file: pythia-6.4.28.tar.xz, uri: https://dev.gentoo.org/~bicatali/distfiles/pythia-6.4.28.tar.xz
 * failed fetching file: lutp0613man2.pdf, uri: http://home.thep.lu.se/~torbjorn/pythia/lutp0613man2.pdf
 * failed fetching file: pythia8245.tgz, uri: http://home.thep.lu.se/~torbjorn/pythia8/pythia8245.tgz

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See direct links in https://pythia.org/releases/, for lhapdfsets, I think using http://lhapdfsets.web.cern.ch/lhapdfsets/current/ should be stable (see https://lhapdf.hepforge.org/#sets).

@gentoo-repo-qa-bot
Copy link
Collaborator

Pull request CI report

Report generated at: 2023-07-29 17:25 UTC
Newest commit scanned: 5e3b06a
Status: ✅ good

There are existing issues already. Please look into the report to make sure none of them affect the packages in question:
https://qa-reports.gentoo.org/output/gentoo-ci/f5b7bc0bc2/output.html

Closes: https://bugs.gentoo.org/862103
Signed-off-by: Alexander Puck Neuwirth <alexander@neuwirth-informatik.de>
@APN-Pucky APN-Pucky force-pushed the pythia branch 2 times, most recently from 309c0bc to af0290a Compare July 29, 2023 19:11
@gentoo-repo-qa-bot
Copy link
Collaborator

Pull request CI report

Report generated at: 2023-07-29 19:25 UTC
Newest commit scanned: af0290a
Status: ✅ good

There are existing issues already. Please look into the report to make sure none of them affect the packages in question:
https://qa-reports.gentoo.org/output/gentoo-ci/8f0b19ed53/output.html

Copy link
Member

@AndrewAmmerlaan AndrewAmmerlaan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Conceptually LGTM, haven't tested though since I'm travelling at the moment.

# workaround to official pythia-split not having a pythia subdir
src_unpack() {
mkdir -p "${S}"
cd "${S}"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

|| die

@amadio
Copy link
Member

amadio commented Aug 4, 2023

I'm getting non-empty git diff after applying this patch and running rm Manifest && pkgdev manifest for files pythia8245.tgz and MRST2004qed.tar.gz.

@APN-Pucky
Copy link
Contributor Author

APN-Pucky commented Aug 4, 2023

I'm getting non-empty git diff after applying this patch and running rm Manifest && pkgdev manifest for files pythia8245.tgz and MRST2004qed.tar.gz.

MRST2004qed.tar.gz links seem to be dead (I still had it cached) and is no longer in /unvalidated/ (changed that now). pythia8245.tgz changed by switching to pythia.org.

@gentoo-repo-qa-bot
Copy link
Collaborator

Pull request CI report

Report generated at: 2023-08-04 15:01 UTC
Newest commit scanned: 44982cf
Status: ✅ good

There are existing issues already. Please look into the report to make sure none of them affect the packages in question:
https://qa-reports.gentoo.org/output/gentoo-ci/cba40d95a8/output.html

Signed-off-by: Alexander Puck Neuwirth <alexander@neuwirth-informatik.de>
https://lhapdfsets.web.cern.ch/lhapdfsets/current/NNPDF23_nnlo_as_0119_qed_mc.tar.gz
https://lhapdfsets.web.cern.ch/lhapdfsets/current/cteq66.tar.gz
https://lhapdfsets.web.cern.ch/lhapdfsets/current/cteq6l1.tar.gz
https://www.hepforge.org/downloads/lhapdf/pdfsets/v6.backup/${LHA_VER}/MRST2004qed.tar.gz
Copy link
Member

@amadio amadio Aug 4, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can the tests maybe run with the split MRST2004qed_proton MRST2004qed_neutron files? The split files are available in https://lhapdfsets.web.cern.ch/lhapdfsets/current/. In any case, shouldn't these all be installed by lhapdf itself?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I'll check where it is needed and if it works with the other ones.

@gentoo-repo-qa-bot
Copy link
Collaborator

Pull request CI report

Report generated at: 2023-08-04 15:46 UTC
Newest commit scanned: c093b6a
Status: ✅ good

There are existing issues already. Please look into the report to make sure none of them affect the packages in question:
https://qa-reports.gentoo.org/output/gentoo-ci/3e5d2cbd9f/output.html

@amadio
Copy link
Member

amadio commented Aug 4, 2023

Now it looks fine, I will run with tests too then I can merge this. Thanks for the work!

@APN-Pucky
Copy link
Contributor Author

APN-Pucky commented Aug 4, 2023

I changed some of the PDFs, but it seems to me like the tests (mainXX.cc) are just compiled but not run in src_test, so the pdfs don't matter in the end. I used

        https://lhapdfsets.web.cern.ch/lhapdfsets/current/NNPDF31_nnlo_as_0118_luxqed.tar.gz
        https://lhapdfsets.web.cern.ch/lhapdfsets/current/PDF4LHC15_nlo_asvar.tar.gz
        https://lhapdfsets.web.cern.ch/lhapdfsets/current/CT14qed_proton.tar.gz
        https://lhapdfsets.web.cern.ch/lhapdfsets/current/NNPDF23_nlo_as_0119_qed.tar.gz
        https://lhapdfsets.web.cern.ch/lhapdfsets/current/NNPDF23_nnlo_as_0119_qed.tar.gz

and it succeeded, but the tests also succeeded with previous different pdfs.

@AndrewAmmerlaan
Copy link
Member

Great work, Thanks 👍

@APN-Pucky
Copy link
Contributor Author

APN-Pucky commented Sep 20, 2023

I have another package (herwig:7) that needs/check for some pdfs during installation and running. What would you think about grouping those direct downloaded lhapdf sets in a package lhapdf-sets?

Here is what my first attempt would look like.
https://gitlab.com/APN-Pucky/gentoo-hep-forge/-/blob/master/sci-physics/lhapdf-sets/lhapdf-sets-0.ebuild?ref_type=heads

Give a user has already installed one of those packages manually (eg. # lhapdf install cteq66), portage complains about Detected file collisions(s), but still installs them without problems.

@AndrewAmmerlaan
Copy link
Member

I have another package (herwig:7) that needs/check for some pdfs during installation and running. What would you think about grouping those direct downloaded lhapdf sets in a package lhapdf-sets?

It's a bit weird to check for a pdf file in the configure phase, is it really required?
My experience with these packages is limited to occasionally merging a PR, I am not familiar with what it does or how it does it. Therefore I defer the decision to you (or maybe someone else in the team has an opinion on this @gentoo/sci )

Here is what my first attempt would look like. https://gitlab.com/APN-Pucky/gentoo-hep-forge/-/blob/master/sci-physics/lhapdf-sets/lhapdf-sets-0.ebuild?ref_type=heads

Are these pdf files unusually big? We usually don't introduce USE flags for toggling the installation of small (documentation) files.
If this makes sense we can separate the pdf files into a separate package, but I don't think we need all those USE flags.

@APN-Pucky
Copy link
Contributor Author

APN-Pucky commented Sep 20, 2023

Are these pdf files unusually big?

I'd say yes extracted:

/usr/share/LHAPDF $ du -sh *
32M	CT10
27M	cteq66
976K	cteq6l1
4.0K	lhapdf.conf
60M	MSHT20lo_as130
64M	MSHT20nlo_as118
21M	MSTW2008nlo68cl
12M	nCTEQ15HQ_208_82
102M	NNPDF23_lo_as_0130_qed
159M	NNPDF40_nlo_as_01180
44K	pdfsets.index

The non-extracted tar's are of about 30M (just to be sure, they are not documentation, but grid data, Parton Distribution Function).

Herwig:7 won't work without CT14nlo and CT14lo.

For reference Herwig is a bit special and has its own Programming via ThePEG's .in files. Could maybe be patched though:

Making install in Merging
 /bin/mkdir -p '/var/tmp/portage/sci-physics/herwig-7.2.3-r3/image/usr/share/Herwig/Merging'
 /usr/lib/portage/python3.11/ebuild-helpers/xattr/install -c -m 644 LEP91-Analysis.in LHC7-Z-Analysis.in LHC7-W-Analysis.in LHC7-J-Analysis.in LHC7-T-Analysis.in LHC8-H-Analysis.in FactorCMWScheme.in LinearCMWScheme.in LEP-Merging.in LHC-W-Merging.in LHC-Z-Merging.in LHC-H-Merging.in LHC-T-Merging.in LHC-J-Merging.in '/var/tmp/portage/sci-physics/herwig-7.2.3-r3/image/usr/share/Herwig/Merging'
 /bin/mkdir -p '/var/tmp/portage/sci-physics/herwig-7.2.3-r3/image/usr/bin'
 /bin/mkdir -p '/var/tmp/portage/sci-physics/herwig-7.2.3-r3/image/usr/bin'
 /bin/mkdir -p '/var/tmp/portage/sci-physics/herwig-7.2.3-r3/image/usr/share/Herwig'
 /bin/mkdir -p '/var/tmp/portage/sci-physics/herwig-7.2.3-r3/image/usr/share/Herwig'
  /bin/sh ../libtool   --mode=install /usr/lib/portage/python3.11/ebuild-helpers/xattr/install -c Herwig '/var/tmp/portage/sci-physics/herwig-7.2.3-r3/image/usr/bin'
 /usr/lib/portage/python3.11/ebuild-helpers/xattr/install -c herwig-config '/var/tmp/portage/sci-physics/herwig-7.2.3-r3/image/usr/bin'
 /usr/lib/portage/python3.11/ebuild-helpers/xattr/install -c -m 644 Makefile-UserModules '/var/tmp/portage/sci-physics/herwig-7.2.3-r3/image/usr/share/Herwig'
 /usr/lib/portage/python3.11/ebuild-helpers/xattr/install -c -m 644 DIS.in DIS-Matchbox.in GammaGamma.in ILC.in ILC-MSSM.in ILC-MUED.in ILC-RS.in LEP.in LEP-Matchbox.in LHC-ADD.in LHC-CEX.in LHC-GammaGamma.in LHC-ResolvedGammaGamma.in LHC.in LHC-Matchbox.in LHC-LQ.in LHC-MSSM.in LHC-MUED.in LHC-NMSSM.in LHC-Powheg.in LHC-RPV.in LHC-RS.in LHC-Sextet.in LHC-TRP.in LHC-TTBA.in LHC-MB.in LHC-ZP.in TVT.in TVT-Powheg.in TVT-TTBA.in LHC-LH.in LHC-LHTP.in LHE.in LHE-POWHEG.in LHE-MCatNLO.in LHE-FxFx.in LHE-MGMerging.in CMSSM40.1.1.slha NMSSM.spc ADD.model '/var/tmp/portage/sci-physics/herwig-7.2.3-r3/image/usr/share/Herwig'
 /usr/lib/portage/python3.11/ebuild-helpers/xattr/install -c -m 644 Leptoquark.model LH.model LHTP.model MSSM.model MUED.model NMSSM.model RPV-Bi.model RPV-Tri.model RS.model Sextet.model TTBA.model Zprime.model RPV-BI.slha RPV-TRI.slha RPV-UDD.slha '/var/tmp/portage/sci-physics/herwig-7.2.3-r3/image/usr/share/Herwig'
libtool: install: /usr/lib/portage/python3.11/ebuild-helpers/xattr/install -c .libs/Herwig /var/tmp/portage/sci-physics/herwig-7.2.3-r3/image/usr/bin/Herwig
Creating repository
Error: 'set /Herwig/Partons/HardLOPDF:PDFName CT14lo': PDF not installed. Try 'lhapdf install'.
make[5]: *** [Makefile:1211: install-data-hook] Error 1
make[4]: *** [Makefile:1077: install-data-am] Error 2
make[3]: *** [Makefile:1024: install-am] Error 2
make[2]: *** [Makefile:857: install-recursive] Error 1
make[1]: *** [Makefile:1017: install] Error 2
make: *** [Makefile:521: install-recursive] Error 1
 * ERROR: sci-physics/herwig-7.2.3-r3::hep-forge failed (install phase):
 *   emake failed
 * 
 * If you need support, post the output of `emerge --info '=sci-physics/herwig-7.2.3-r3::hep-forge'`,
 * the complete build log and the output of `emerge -pqv '=sci-physics/herwig-7.2.3-r3::hep-forge'`.
 * The complete build log is located at '/var/tmp/portage/sci-physics/herwig-7.2.3-r3/temp/build.log'.
 * The ebuild environment file is located at '/var/tmp/portage/sci-physics/herwig-7.2.3-r3/temp/environment'.
 * Working directory: '/var/tmp/portage/sci-physics/herwig-7.2.3-r3/work/Herwig-7.2.3'
 * S: '/var/tmp/portage/sci-physics/herwig-7.2.3-r3/work/Herwig-7.2.3'

@APN-Pucky
Copy link
Contributor Author

APN-Pucky commented Sep 20, 2023

We usually don't introduce USE flags for toggling the installation of small (documentation) files.

My idea behind the use flags was to make the dependency explicit. Apart from taking up some space installing more than the minimal collection of PDFs is no problem, which is quite a draw back for containerizing it, IMO.

What do you think about grouping the PDFs use-flag, i.e. atm. USE="pythia herwig" for their required PDF collections (though the logic is in the technically wrong package then)?

Or one could just call lhapdf install ... in the ebuild, but that requires care with installing in the right prefix.

@AndrewAmmerlaan
Copy link
Member

The non-extracted tar's are of about 30M (just to be sure, they are not documentation, but grid data, Parton Distribution Function).

aaaah right, this makes a lot more sense now, I though we were talking about plain .pdf files.

What do you think about grouping the PDFs use-flag, i.e. atm. USE="pythia herwig" for their required PDF collections (though the logic is in the technically wrong package then)?

I don't know, this might be confusing for users.

Or one could just call lhapdf install ... in the ebuild, but that requires care with installing in the right prefix.

This might be the best solution, it seems the easiest and therefore least likely to break in the future. With sed it is usually easy to make install commands respect some prefix/destdir if they don't already do so.

@amadio
Copy link
Member

amadio commented Sep 20, 2023

sci-physics/geant uses a separate package for data, I think it's fine to split the datasets into a separate package, especially since they are used not only by lhapdf, but also other generators.

@APN-Pucky
Copy link
Contributor Author

APN-Pucky commented Sep 20, 2023

This might be the best solution

Turns out this does not work since it would require network access to fetch the data via the web. (I can trick it to think of gentoo's $DISTDIR as of a CVMFS and fetch it from there, but then lhapdf install is only a complicated mv ... && tar xfz ...)

I think it's fine to split the datasets into a separate package

The problem is more if we want a use flag for each of the pdf-set or just install all in one. I don't think installing all makes sense, since pythias tests would already be ~100M. (here a permanent link https://gitlab.com/APN-Pucky/gentoo-hep-forge/-/blob/f96ce260c3762f835b4b044b094e509a58abae48/sci-physics/lhapdf-sets/lhapdf-sets-0.ebuild)

@AndrewAmmerlaan
Copy link
Member

Maybe we can do something like sci-geosciences/GeographicLib-data::guru, i.e. use USE_EXPAND. If I remember correctly there was a similar discussion about the GeographicLib(-data) long long ago, where we reached some conclusion in the form of a separate package with some USE_EXPAND based sorting of the individual data sets. I don't remember why this package never made it into ::gentoo though, there probably was some reason to move GeographicLib but not GeographicLib-data.

What does the upstream lhapdf install do? Does it install all the files or just some subset?
When in doubt I usually check what upstream and other distro's are doing.

@APN-Pucky
Copy link
Contributor Author

APN-Pucky commented Sep 20, 2023

Maybe we can do something like sci-geosciences/GeographicLib-data::guru, i.e. use USE_EXPAND

Looks like atm sci-geosciences/GeographicLib-data::guru just has a bunch of USEs (grouped by IUSE) https://github.com/gentoo/guru/blob/master/sci-geosciences/GeographicLib-data/GeographicLib-data-0-r1.ebuild (sort of like my lhapdf-sets). I could also group them like IUSE_LHAPDF_SETS.

What does the upstream lhapdf install do?

lhapdf install cteq66 only pulls the cteq66.tar.gz from the https and unpacks them to LHAPDF_DATA_PATH, i.e. /usr/share/LHAPDF. lhapdf install will (always) install only one PDF. It also checks that it exists in /usr/share/LHAPDF/pdfsets.index, but a recent version of that is shipped by the install tarball too. When the python part of lhapdf install was broken, wget https://.../cteq66.tar.gz && tar xfz *.gz in /usr/share/LHAPDF did the same job. Only for very recent PDFs one needs to update with lhapdf update first to get a newer index.

@AndrewAmmerlaan
Copy link
Member

Looks like atm sci-geosciences/GeographicLib-data::guru just has a bunch of USEs (grouped by IUSE)

Yes but they are grouped in USE_EXPANDS: https://github.com/gentoo/guru/tree/master/profiles/desc

andrew@andrew-gentoo-pc ~ % eix geographiclib
* sci-geosciences/GeographicLib
Available versions:  (~)1.52-r2(0/19)^t {doc examples python test PYTHON_TARGETS="python3_10 python3_11"}
Homepage:            https://sourceforge.net/projects/geographiclib/
Description:         C++ library for converting geographic coordinate systems

* sci-geosciences/GeographicLib-data [1]
Available versions:  (~)0-r1 {GEOIDS_DATASETS="egm84-15 egm84-30 egm96-5 egm96-15 +egm2008-1 egm2008-5 egm2008-2-5" GRAVITY_MODELS="egm84 egm96 +egm2008 wgs84" MAGNETIC_MODELS="emm2010 emm2015 emm2017 igrf11 igrf12 wmm2010 +wmm2020 wmm2015v2"}
Homepage:            https://sourceforge.net/projects/geographiclib/
Description:         Datasets for GeographicLib

[1] "guru" /var/db/repos/guru

Found 2 matches

It might make sense to do something similar here so users can set LHPADF_DATASETS="data sets I want" instead of a whole bunch of USE flags in package.use/.

Then again the approach in sci-physics/geant-data is just to install everything, we might want to keep a consistent approach with these data packages.

@APN-Pucky
Copy link
Contributor Author

I see they are defined in profiles/make.defaults.

I think since we don't want to ever provide all of them, just letting the user/dependencies choose packages is best.

@APN-Pucky
Copy link
Contributor Author

APN-Pucky commented Sep 21, 2023

I don't remember why this package never made it into ::gentoo though

Modifying USE_EXPAND needs to be discussed on the gentoo-dev mail list and it looks like it is primarily used for features and never data:
https://github.com/gentoo/gentoo/blob/97ce24fb60631e53601559d7e9acfdced79bf730/profiles/base/make.defaults#L13-L14C45

I have it working here https://gitlab.com/APN-Pucky/gentoo-hep-forge/-/blob/master/sci-physics/lhapdf-sets/lhapdf-sets-0.ebuild?ref_type=heads. Should I PR it to ::science, given that ::gentoo tries to keep the number of packages manageable? Also packages that would require it, I'd rather PR to ::science than to pollute ::gentoo.

@AndrewAmmerlaan
Copy link
Member

I have it working here https://gitlab.com/APN-Pucky/gentoo-hep-forge/-/blob/master/sci-physics/lhapdf-sets/lhapdf-sets-0.ebuild?ref_type=heads. Should I PR it to ::science, given that ::gentoo tries to keep the number of packages manageable? Also packages that would require it, I'd rather PR to ::science than to pollute ::gentoo.

Sure, lets add it to ::sci first. This makes it easier to change later if we change our mind. And if you want, in ::sci you can add the USE_EXPAND without having to go through the ML first.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
assigned PR successfully assigned to the package maintainer(s). bug linked Bug/Closes found in footer, and cross-linked with the PR.
Projects
None yet
5 participants