Example: PyTorch Lightning integration #17

LeoRoccoBreedt · 2025-05-28T14:45:11Z

Description

Include a summary of the changes and the related issue.

Related to: <ClickUp/JIRA task name>

Any expected test failures?
Python 3.13 not supported for PyTorch (yet).

Add a [X] to relevant checklist items

❔ This change

adds a new feature
fixes breaking code
is cosmetic (refactoring/reformatting)

✔️ Pre-merge checklist

Refactored code (sourcery)
Tested code locally
Precommit installed and run before pushing changes
Added code to GitHub tests (notebooks, scripts)
Updated GitHub README
Updated the projects overview page on Notion

🧪 Test Configuration

OS: Windows
Python version: 3.12
Neptune version: 0.13.0
Affected libraries with version: torch, lightning

Summary by Sourcery

Add a PyTorch Lightning integration example notebook for Neptune Scale and update the CI workflow to run tests on it.

New Features:

Add Neptune + PyTorch Lightning example notebook demonstrating integration usage

CI:

Include the new PyTorch Lightning notebook in the test-notebooks CI workflow

Summary by Sourcery

Provide a Neptune Scale integration example for PyTorch Lightning by adding a Jupyter notebook, a Python script with supporting requirements and run script, and update CI workflows to test the new integration.

New Features:

Add Neptune + PyTorch Lightning example notebook, script, requirements, and run_examples.sh for demonstration

CI:

Include the PyTorch Lightning example notebook in the test-notebooks workflow
Include the PyTorch Lightning scripts directory in the test-scripts workflow

sourcery-ai · 2025-05-28T14:45:24Z

Reviewer's Guide

This PR integrates a new PyTorch Lightning example into the repository by adding a detailed example notebook and script (with supporting requirements and run script) that demonstrate NeptuneScaleLogger usage, and updates the CI workflows to include these assets in automated tests.

Class Diagram for LitModel (PyTorch Lightning Module)

classDiagram
    class LightningModule {
        <<PyTorch Lightning>>
        +log()
        +log_dict()
    }
    class LitModel {
        +List training_step_outputs
        +List validation_step_outputs
        +List test_step_outputs
        +int linear
        +float learning_rate
        +float decay_factor
        +torch.nn.Linear layer_1
        +torch.nn.Linear layer_2
        +torch.nn.Linear layer_3
        +NeptuneScaleLogger logger
        +__init__(linear, learning_rate, decay_factor)
        +forward(x) Tensor
        +configure_optimizers() List
        +training_step(batch) Dict
        +on_train_epoch_end()
        +validation_step(batch, batch_idx) Dict
        +on_validation_epoch_end()
        +test_step(batch) Dict
        +on_test_epoch_end()
    }
    LightningModule <|-- LitModel

Class Diagram for MNISTDataModule (PyTorch Lightning DataModule)

classDiagram
    class LightningDataModule {
        <<PyTorch Lightning>>
    }
    class MNISTDataModule {
        +int batch_size
        +tuple normalization_vector
        +Dataset mnist_train
        +Dataset mnist_val
        +Dataset mnist_test
        +__init__(batch_size, normalization_vector)
        +prepare_data()
        +setup(stage)
        +train_dataloader() DataLoader
        +val_dataloader() DataLoader
        +test_dataloader() DataLoader
    }
    LightningDataModule <|-- MNISTDataModule

File-Level Changes

Change	Details	Files
Add PyTorch Lightning integration example notebook demonstrating NeptuneScaleLogger usage	Created a notebook with setup instructions, model and DataModule definitions, and logging steps Included sections for hyperparameter logging, model summary, and run closure Documented installation options and live metadata exploration	`integrations-and-supported-tools/pytorch-lightning/notebooks/Neptune_PyTorch_Lightning.ipynb`
Add PyTorch Lightning integration script for automated example execution	Implemented a script defining LitModel and MNISTDataModule with NeptuneScaleLogger Configured the trainer, logging calls, and a main() entry point Structured training and testing flow with live Neptune logging	`integrations-and-supported-tools/pytorch-lightning/scripts/Neptune_Pytorch_Lightning.py`
Add support files for example script execution	Created requirements.txt listing dependencies (Lightning, Neptune, torch, etc.) Added run_examples.sh to install requirements and execute the example script	`integrations-and-supported-tools/pytorch-lightning/scripts/requirements.txt` `integrations-and-supported-tools/pytorch-lightning/scripts/run_examples.sh`
Include new PyTorch Lightning example in CI testing workflows	Appended the new notebook path to the test-notebooks CI workflow Appended the new scripts directory to the test-scripts CI workflow	`.github/workflows/test-notebooks.yml` `.github/workflows/test-scripts.yml`

Tips and commands

Interacting with Sourcery

Trigger a new review: Comment @sourcery-ai review on the pull request.
Continue discussions: Reply directly to Sourcery's review comments.
Generate a GitHub issue from a review comment: Ask Sourcery to create an
issue from a review comment by replying to it. You can also reply to a
review comment with @sourcery-ai issue to create an issue from it.
Generate a pull request title: Write @sourcery-ai anywhere in the pull
request title to generate a title at any time. You can also comment
@sourcery-ai title on the pull request to (re-)generate the title at any time.
Generate a pull request summary: Write @sourcery-ai summary anywhere in
the pull request body to generate a PR summary at any time exactly where you
want it. You can also comment @sourcery-ai summary on the pull request to
(re-)generate the summary at any time.
Generate reviewer's guide: Comment @sourcery-ai guide on the pull
request to (re-)generate the reviewer's guide at any time.
Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
pull request to resolve all Sourcery comments. Useful if you've already
addressed all the comments and don't want to see them anymore.
Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
request to dismiss all existing Sourcery reviews. Especially useful if you
want to start fresh with a new review - don't forget to comment
@sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

Enable or disable review features such as the Sourcery-generated pull request
summary, the reviewer's guide, and others.
Change the review language.
Add, remove or edit custom review instructions.
Adjust other review settings.

Getting Help

Contact our support team for questions or feedback.
Visit our documentation for detailed guides and information.
Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

sourcery-ai

Hey @LeoRoccoBreedt - I've reviewed your changes - here's some feedback:

Consider refactoring the Lightning model and DataModule definitions into a shared utility module to avoid duplicating the same code in both the notebook and script.
The installation instructions rely on a custom PyTorch Lightning fork; please clarify version compatibility or plan to switch to the official release once supported to reduce long-term maintenance overhead.

Here's what I looked at during the review

🟡 General issues: 5 issues found
🟢 Security: all looks good
🟢 Testing: all looks good
🟢 Complexity: all looks good
🟢 Documentation: all looks good

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

integrations-and-supported-tools/pytorch-lightning/scripts/Neptune_Pytorch_Lightning.py

…ch-lightning