3.11.0.29

• Memory checks for different resource types
  • Discussed in JIRA ticket ATLASPANDA-1051 (“Pilot RSS kill threshold should depend on subresource, not just PQ.maxrss”)
• Same memory limits are used for cgroups as for different resource types
  • All subprocesses - including payload - are in their own cgroup (“subprocesses”)
  • Tested in production at MWT2
  • Discussed in JIRA ticket ATLASPANDA-1251 (“Implement memory restrictions on just the pilot payload using cgroup v2”)
• Resolved JIRA ticket ATLASPANDA-1483 (“Traceback in pilot for remoteio timeout”)
  • An exception was caught in the setup command which could indicate a worker node issue / severe apptainer issue, but for remote file open that is not the case and should not result in a non-zero pilot exit code.
  • Example job: 6794333602
• JIRA ticket ATLASPANDA-1496 (“Add retries for pilot stage-out”)
  • Added --stageout-attempts pilot argument
  • Likely needs some further development
• Debugging issue with OIDC token seen with long running job at MWT2
  • Now decoding token after refresh to print out issue and expiry time
• Worker node map
  • Pilot now adds /usr/sbin to PATH if missing (needed to find lspci command at BNL_GPU)
  • Discussed in JIRA ticket ATLASPANDA-1368
• Internal improvements
  • Refactoring of main pilot module
  • Pylint updates - Current average score across all 215 pilot modules: 9.77/10
  • Added GitHub Action workflow using new tool VeriCode, capable of running different linters and other tools
    • Currently, it verifies that no individual pilot module has a pylint score of less than 8 (it would fail the PR)
    • More info at https://pypi.org/project/vericode/