Skip to content

Comments

Installation update; expand torch and cuda versions#806

Merged
RasmusOrsoe merged 99 commits intographnet-team:mainfrom
RasmusOrsoe:torch_update
Jun 23, 2025
Merged

Installation update; expand torch and cuda versions#806
RasmusOrsoe merged 99 commits intographnet-team:mainfrom
RasmusOrsoe:torch_update

Conversation

@RasmusOrsoe
Copy link
Collaborator

@RasmusOrsoe RasmusOrsoe commented Jun 2, 2025

This PR updates GraphNeT to be compatible with latest PyTorch and CUDA versions.

Instead of a single pytorch version, we now offer three (2.7, 2.6, 2.5) and cuda versions 11.8 and 12.{1,4,6,8} (see installation matrix below). These versions are collectively offered in the python range of 3.9 to 3.11. Once #808 is merged, we can offer the latest python versions too. This PR closes #799 and closes #798.

image

The most relevant changes are highlighted below.

No requirement files!

Up until now, we've relied on requirement files in the installation procedure, e.g. pip install -r requirements/torch_cu121.txt -e .[torch,develop]. The purpose of these files was primarily to point at the correct Python wheels. This scheme does not scale well to ranges of pytorch and cuda versions. Instead, the installation procedure has been altered to use the installation matrix to inject the correct wheel links in conjunction with specific entries in setup.py that each represent a particular choice in versions. The setup.py entries has the form torch-26-cu124 for pytorch 2.6.0 with cuda 12.4. (See installation matrix output above).

Changes to GitHub Actions

  • Installation action: The action now accepts pytorch versions as argument. A boolean is introduced which allows graphnet to be installed inside a virtual environment.
  • IceTray runner: Now runs the latest icetray docker image (1.13 + python 3.10). The action has been modified to install graphnet via a virtual environment, which is more aligned with the proposed installation procedure for a graphnet + icetray installation. Comments has been added to highlight why setting environment variables is needed for a successful import of icecube within the github action environment. The methodology of setting the environment variables is adjusted to be (hopefully) compatible with future icetray docker images.

Changes to Testing

The testing suite on pull requests has been subject to change. Here, the aim has been to balance between testing a wider range of pytorch versions vs. our limited testing abilities on the free tier of github actions. The new testing scheme is shown below

  • Ubuntu 22.4 Runner: (Newest compatible python, $torch_version, cpu) - example scripts only
  • macOS 15 Runner: (Newest compatible python, newest torch version, cpu) - example scripts only
  • IceTray Runner: (latest docker image, newest pytorch version, cpu) - full suite of tests + coverage report
    Note that the macOS15 runner tests just a single PyTorch version; this is because macOS runners have a minute multiplier of 10 (!) see here.

Minor Changes

  • docformatter has been updated from 1.7.5 to 1.7.7 for pre-commit>=4.0 compatibility see here. The new version required updates of docstrings in several modules of the repository, which accounts for the majority of changes in this PR.
  • The verbose argument in torch.optim.lr_scheduler.LRScheduler was deprecated in torch==2.7.0 and has been removed from graphnet.training.callbacks.PiecewiseLinearLR
  • The default value for num-workers in training examples has changed to 2 to improve unit tests, as GitHub runners are limited to 2 cores.

Copy link
Collaborator

@Aske-Rosted Aske-Rosted left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice contribution! Some very minor doc-string things (which weren't caused by this PR I realise...), and one question about the default behavior of github action.

name: Unit tests - macOS
runs-on: macos-13
build-macos-examples:
# Every macOS test 'costs' 10 minutes of runner time. Minimize!
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is multiplied by a factor 10, this makes it sound like a flat cost.

use_vm:
description: "Should the installation happen in a virtual environment named venv?"
required: true
default: false
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the reason why the default is not true if this is the recommended action?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It costs upwards of 30 seconds of runner time, so limiting the usage is economical for us. Using it on the icetray runner makes the most sense as the pathing there is a little different.

The new i3 files will appear as copies of the original i3 files but with
reconstructions added. Original i3 files are left untouched.
The new i3 files will appear as copies of the original i3 files but
with reconstructions added. Original i3 files are left untouched.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe also remove the "but"

import and download pre-converteddatasets for training of deep learning based
methods in GraphNeT.
Inheriting subclasses are data-specific implementations that allow the
user to import and download pre-converteddatasets for training of deep
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing space.

@RasmusOrsoe RasmusOrsoe merged commit b126d65 into graphnet-team:main Jun 23, 2025
8 checks passed
@RasmusOrsoe RasmusOrsoe mentioned this pull request Jul 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants