-
Notifications
You must be signed in to change notification settings - Fork 6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allow for the creation of unique subfolders in the remote_workdir
#57
Allow for the creation of unique subfolders in the remote_workdir
#57
Conversation
@wjcunningham7, @santoshkumarradha, @cjao: This should be good to go I think! I just tested it out, and it worked as expected. Since we will want to mimic this approach in other executors, please make sure the naming and such are to your liking. When it's merged, can we also get a version update? Thanks! |
Codecov ReportPatch coverage:
Additional details and impacted files@@ Coverage Diff @@
## develop #57 +/- ##
===========================================
+ Coverage 84.06% 84.43% +0.36%
===========================================
Files 2 2
Lines 295 302 +7
===========================================
+ Hits 248 255 +7
Misses 47 47
☔ View full report in Codecov by Sentry. |
remote_workdir
remote_workdir
Wow this was super fast and perfect ! 🙏 CC : @wjcunningham7 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Last comment is that the unique directory should be conveyed to Slurm using the --chdir
option.
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
EDIT: Fixed and ready to go!! I confirmed that everything works correctly in a "real" scenario on Perlmutter. Alright, I'm a bit stumped. Entirely unrelated to this PR, I'm having that issue where I cannot unpickle things on the remote machine (Perlmutter). I tried nuking my entire conda env again on both local and remote but with no luck. That's making it a bit hard to test this and give me confidence that everything is working perfectly. I also think I must have introduced an error somewhere when switching from Going to have to tackle this tomorrow because I have no brain cells left right now... |
for more information, see https://pre-commit.ci update test black fix test Move cache_dir kwarg
remote_workdir
remote_workdir
remote_workdir
remote_workdir
Closes #56 and combines a separate bugfix I was going to introduce in #58.