Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

upload checkpoint files to neptune from stream #17430

Merged
merged 12 commits into from
Apr 27, 2023

Conversation

AleksanderWWW
Copy link
Contributor

@AleksanderWWW AleksanderWWW commented Apr 21, 2023

What does this PR do?

Fixes (issue on neptune-client repo):
neptune-ai/neptune-client#884

Before submitting
  • Was this discussed/agreed via a GitHub issue? (not for typos and docs)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure your PR does only one thing, instead of bundling different changes together?
  • Did you make sure to update the documentation with your changes? (if necessary)
  • Did you write any new necessary tests? (not for typos and docs)
  • Did you verify new and existing tests pass locally with your changes?
  • Did you list all the breaking changes introduced by this pull request?
  • Did you update the CHANGELOG? (not for typos, docs, test updates, or minor internal changes/refactors)

PR review

Anyone in the community is welcome to review the PR.
Before you start reviewing, make sure you have read the review guidelines. In short, see the following bullet-list:

Reviewer checklist
  • Is this pull request ready for review? (if not, please submit in draft mode)
  • Check that all items from Before submitting are resolved
  • Make sure the title is self-explanatory and the description concisely explains the PR
  • Add labels and milestones (and optionally projects) to the PR so it can be classified

@github-actions github-actions bot added the pl Generic label for PyTorch Lightning package label Apr 21, 2023
@AleksanderWWW AleksanderWWW changed the title upload checkpoint files from stream upload checkpoint files to neptune from stream Apr 21, 2023
Copy link
Contributor

@Blaizzy Blaizzy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@AleksanderWWW AleksanderWWW marked this pull request as ready for review April 21, 2023 17:23
Copy link
Member

@awaelchli awaelchli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this @AleksanderWWW
I left a comment for fixing the tests, let me know if you need help with it.
We could also add an entry in the changelog in src/lightning/pytorch/CHANGELOG.md in the "#Fixed" section. Thanks!

src/lightning/pytorch/loggers/neptune.py Show resolved Hide resolved
@awaelchli awaelchli added bug Something isn't working logger: neptune community This PR is from the community labels Apr 22, 2023
@awaelchli awaelchli added this to the 2.0.x milestone Apr 22, 2023
@AleksanderWWW
Copy link
Contributor Author

@awaelchli Thank you for the feedback! Let me know if the relevant commit is enough to move forward, or if there's anything else I should do :)

@carmocca
Copy link
Contributor

Tests are failing. They need to be fixed

FAILED loggers/test_all.py::test_loggers_fit_test_all[NeptuneLogger] - AttributeError: 'NoneType' object has no attribute 'from_stream'
FAILED loggers/test_neptune.py::TestNeptuneLogger::test_after_save_checkpoint - FileNotFoundError: [Errno 2] No such file or directory: 'path/to/models/last'
FAILED loggers/test_neptune.py::TestNeptuneLogger::test_neptune_leave_open_experiment_after_fit - AttributeError: 'NoneType' object has no attribute 'from_stream'
FAILED loggers/test_neptune.py::TestNeptuneLogger::test_neptune_log_metrics_on_trained_model - AttributeError: 'NoneType' object has no attribute 'from_stream'

@awaelchli awaelchli self-assigned this Apr 24, 2023
@mergify mergify bot removed the has conflicts label Apr 24, 2023
@AleksanderWWW
Copy link
Contributor Author

@carmocca I think I fixed the tests (they pass locally at least), but the pre-commit failure I see now doesn't seem to have anything to do with my changes and they only appeared after I merged the master branch.

Copy link
Contributor

@carmocca carmocca left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

cc @Blaizzy for a final stamp from Neptune

@mergify mergify bot added the ready PRs ready to be merged label Apr 27, 2023
@Borda Borda enabled auto-merge (squash) April 27, 2023 16:21
@Borda Borda requested a review from awaelchli April 27, 2023 16:21
@Borda Borda merged commit ef9df05 into Lightning-AI:master Apr 27, 2023
75 checks passed
@AleksanderWWW AleksanderWWW deleted the neptune/improve-checkpointing branch April 28, 2023 08:31
Borda pushed a commit that referenced this pull request May 9, 2023
Borda pushed a commit that referenced this pull request Jun 1, 2023
lantiga pushed a commit that referenced this pull request Jun 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working community This PR is from the community logger: neptune pl Generic label for PyTorch Lightning package ready PRs ready to be merged
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants