-
Notifications
You must be signed in to change notification settings - Fork 43
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Refactor and streamline copy_files
handling
#1759
Conversation
This commit fixes the style issues introduced in e425213 according to the output from Black, isort and Prettier. Details: #1759
copy_files
handlingcopy_files
handling: Part 1/3
Thanks, @Nekkrad! Much appreciated. I'll likely go ahead and merge this in the evening here so that it can unblock a few other PRs (including one of yours). I'll make any changes to your PR to ensure everything passes after I do --- no need for you to do anything there. In any case, I wanted to communicate this minor but notable change in terms of how files are handled. |
Haven't had time to look at this in details (I am off this week). I think your last sentence summarise very well my first thoughts about this:
Which is very positive |
@tomdemeyere No worries. Try not to open GitHub too much on your time off. 😉 Much of this refactoring was due to nice feature additions you made, so thank you! I will go ahead and merge this shortly, but when you return, don't hesitate to reach out if there are any issues/concerns. |
copy_files
handlingcopy_files
handling
Alright, this sucked up half a week of my life but in the end, there should be no major breaking changes (while also resolving #1742). There are some gnarly |
Closes #1742.
Note:
AppFuture
s do not get implicitly resolved Parsl/parsl#3108Motivation
In short, the
copy_files
handling was in desperate need of a behind-the-scenes overhaul. There are several reasons for this.Path()
on aFuture
, which means any recipes doingPath()
(or string concatenation of paths) in aFlow
are going to crash when dispatched over a workflow engine.copy_decompress_files
,copy_decompress_tree
,copy_decompress_files_from_dir
.prev_dir: str | Path
positional argument is often gettingdict
entries thrown into, as a result ofpw_copy_files
and such (see the test suite).Because of this mess, things were in desperate need of a revamp, which this PR addresses.
User-Facing Changes
In most cases, there are no breaking changes to users and is mostly an internal refactor. Some positional arguments of
prev_dir
have been renamedcopy_files
for consistency with the type hint, although no user changes are expected if used as positional. Some internal functions have slightly modified behavior, such ascopy_decompress_files
. Refer to the function signature for more.