Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

gfal2: init at 2.22.2; gfal2-python: init at 1.12.2; gfal2-util: init at 1.8.1 #200100

Merged
merged 3 commits into from
Apr 5, 2024
Merged

gfal2: init at 2.22.2; gfal2-python: init at 1.12.2; gfal2-util: init at 1.8.1 #200100

merged 3 commits into from
Apr 5, 2024

Conversation

ShamrockLee
Copy link
Contributor

@ShamrockLee ShamrockLee commented Nov 7, 2022 •
edited

Description of changes

Gfal2, Grid File Access Library v2, provides a layer of abstraction and unified (POSIX-like) way to access different kinds of GRID storage systems.

https://dmc-docs.web.cern.ch/dmc-docs/gfal2/gfal2.html

This PR includes the C library (gfal2), its Python binding (gfal2-python) and command-line utilities (gfal2-util). Now we have gfal-ls and gfal-copy in Nixpkgs!

This PR also mark xrootd as broken. Due to #169677, the package is not usable.

Cc: @veprbl

Things done
  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 22.11 Release Notes (or backporting 22.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
    • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
  • Fits CONTRIBUTING.md.

@ShamrockLee ShamrockLee changed the title gfal2: init at 2.21.1; gfal2-python: init at 1.12.0; gfal2-util: init at 1.8.0 gfal2: init at 2.21.1; gfal2-python: init at 1.12.0; gfal2-util: init at 1.8.0; xrootd: meta.broken = true Nov 8, 2022
@ShamrockLee
Copy link
Contributor Author

Edited the manually-generated patch file and zeroize the timestamp to reduce PII leakage.

@ShamrockLee ShamrockLee changed the title gfal2: init at 2.21.1; gfal2-python: init at 1.12.0; gfal2-util: init at 1.8.0; xrootd: meta.broken = true gfal2: init at 2.21.1; gfal2-python: init at 1.12.0; gfal2-util: init at 1.8.0 Nov 16, 2022
@ShamrockLee ShamrockLee marked this pull request as draft November 16, 2022 23:26
@ShamrockLee
Copy link
Contributor Author

@ofborg build gfal2.passthru.tests.gfal2Minimal

@ShamrockLee
Copy link
Contributor Author

ShamrockLee commented Nov 17, 2022 •
edited

@ofborg build gfal2.passthru.tests.gfal2Http

@ShamrockLee
Copy link
Contributor Author

@ofborg build gfal2.passthru.tests.gfal2Sftp

@ShamrockLee
Copy link
Contributor Author

@ofborg build gfal2.passthru.tests.gfal2Xrootd
@ofborg build gfal2.passthru.tests.gfal2Mock
@ofborg build gfal2.passthru.tests.gfal2Dcap
@ofborg build gfal2.passthru.tests.gfal2File

@ShamrockLee
Copy link
Contributor Author

@ofborg build dcap

@ShamrockLee ShamrockLee marked this pull request as ready for review November 17, 2022 12:03
@ShamrockLee
Copy link
Contributor Author

Fixed the Darwin build.

@ShamrockLee ShamrockLee mentioned this pull request Nov 17, 2022
Open
@stale stale bot added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label May 21, 2023
@stale stale bot removed the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Nov 7, 2023
@ShamrockLee ShamrockLee marked this pull request as ready for review November 13, 2023 20:45
@ShamrockLee ShamrockLee changed the title gfal2: init at 2.21.1; gfal2-python: init at 1.12.0; gfal2-util: init at 1.8.0 gfal2: init at 2.22.2; gfal2-python: init at 1.12.2; gfal2-util: init at 1.8.1 Mar 27, 2024
@ShamrockLee
Copy link
Contributor Author

ShamrockLee commented Mar 27, 2024 •
edited

Update to the newest tagged versions and add more tests.
Mark broken on fail-to-build Python versions of gfal2-python. gfal2-python 1.12.2 builds on Python <= 3.11, which is better than before.

@ShamrockLee
Copy link
Contributor Author

Let's see if we could make it work on Darwin.

@ofborg build gfal2.tests.gfal2-minimal

@ShamrockLee
Copy link
Contributor Author

Let's see if we could make it work on Darwin.

@ofborg build gfal2.tests.gfal2-minimal

A lot of compilation error is generated even without any gfal2 plug-ins. Set meta.platform to linux as I don't have a Mac device to debug it.

@ShamrockLee
Copy link
Contributor Author

According to the gfal2 people, the errors encountered by OfBorg, is originated from the GTest library.

Let's add doCheck = stdenv.hostPlatform.isLinux; to gfal2 and see if we could at least get gfal-ls and gfal-copy work on Darwin.

@ofborg build gfal2.tests.gfal2-minimal

@ShamrockLee ShamrockLee marked this pull request as draft April 2, 2024 12:07
@ShamrockLee
Copy link
Contributor Author

ShamrockLee commented Apr 2, 2024 •
edited

Oops! I forgot to git add the changes. (Photo credit: Rohit Sam (@urslimshady) / Twitter)

@ofborg build gfal2.tests.gfal2-minimal

Meme `git commit` and `git push` without `git add`

@ShamrockLee
Copy link
Contributor Author

The aarch64-darwin build jobs have been queued for about a day. x86_64-darwin passing the functionality test (gfal2-util.tests, targeting gal2-copy and gfal2-ls through XRootD-based source) should be a strong indication that gfal2-util builds and runs on Darwin.

Mark the PR ready for review.

@ShamrockLee ShamrockLee marked this pull request as ready for review April 3, 2024 15:12
veprbl
veprbl approved these changes Apr 3, 2024
View reviewed changes
Copy link
Member

@veprbl veprbl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ShamrockLee
gfal2-util, python3Packages.gfal2-util: init at 1.8.1
fc61cff 
Add also passthru.fetchGfal2. This fetcher is currently used only by
passthru.tsets, but capable to be used as a domain-specific
real-world fetcher.
@ShamrockLee
Copy link
Contributor Author

Fixup:

Fix unintended shadowing of previousAttrs.passthtru in the overrideAttrs part of python3Packages.gfal2-util.

Fix error: gfal2-util should use buildPythonPackage or toPythonModule if it is to be part of the Python packages set.

Sorry for the loose ends.

@veprbl veprbl merged commit a849117 into NixOS:master Apr 5, 2024
24 checks passed
@ShamrockLee ShamrockLee mentioned this pull request Apr 8, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@veprbl veprbl veprbl approved these changes

@FRidh FRidh Awaiting requested review from FRidh

@jonringer jonringer Awaiting requested review from jonringer

Assignees
No one assigned
Labels
6.topic: python 8.has: package (new) 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 11.by: package-maintainer
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@ShamrockLee @veprbl