-
Notifications
You must be signed in to change notification settings - Fork 111
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ENH+BF: get_parent_paths - make / into sep option and consistently use "/" as path separator #6963
Conversation
…e / as path separator Code review of this function came up while reviewing the datalad#6942 and where get_parent_paths, which was pretty much "avoided" while resolving infinite recursion of `get_submodules_ -> get_content_info -> get_submodules_`, was re-introduced specifically within get_submodules_ . BF: use consistently "sep" instead of / for check within the loop, and then os.sep to check if points to up/down folders. ENH: - sep became an option so we could use it also with os.sep if so desired on paths which are not POSIXed. By default it would remain "POSIX" (although checks were not checking for the POSIX endings before). - The test became parametric to test for various scenarios of using \ vs / vs "the default" (/ - POSIX).
… get_parent_paths get_submodules_ returns native paths within "path" which makes sense. But then we take back relative path and not convert back to POSIX. Now we would!
hm, this
is odd since |
Apparently (now to me) pathlib.*PosixPath() does not do "conversion" from native form to POSIX - it would happily preserve \ on Windows within POSIX filename. As a result we would still be passing through Windows paths down. PurePath.as_posix() is the one we need to do such conversion, so use it for paths and submodules paths (come out native from get_submodules_()). Also do not overload paths and then create some path_strs -- place into a more clearly named posix_paths variable.
Codecov Report
@@ Coverage Diff @@
## maint #6963 +/- ##
==========================================
+ Coverage 89.97% 90.93% +0.95%
==========================================
Files 354 354
Lines 46261 46517 +256
==========================================
+ Hits 41624 42299 +675
+ Misses 4637 4218 -419
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. |
Woohoo, it is ready to please windows ppl |
One thing upfront: This change should make the code no worse than it was before. It should be fine to merge if that is the goal. In general I very much dislike the idea of fixing up any path processing helper in datalad that needs to take We have been hit by countless "there is more to path-processing-than-replacing-slash-with-backslash" issues over the years. Also this code is unlikely to properly handle encoding and other handling of unsupported chars etc. |
I hear you @mih and would welcome RF/BF which would introduce proper use/manipulation of pathlib objects instead of path strings. It was not the goal of this PR though which was initially to fix inconsistency thus making it work properly on Windows, then due to added extra checks detected additional issues with taken approach and those were fixed. It indeed should not make code worse from what I see, and it should make DataLad perform better on Windows. Overall I will take your comment as blessing then and will proceed with the merge. |
Code review of this function came up while reviewing the
#6942 and where get_parent_paths,
which was pretty much "avoided" while resolving infinite recursion of
get_submodules_ -> get_content_info -> get_submodules_
, was re-introducedspecifically within get_submodules_ .
BF: use consistently "sep" instead of / for check within the loop, and then
os.sep to check if points to up/down folders.
ENH:
sep became an option so we could use it also with os.sep if so desired
on paths which are not POSIXed. By default it would remain "POSIX"
(although checks were not checking for the POSIX endings before).
The test became parametric to test for various scenarios of using \ vs /
vs "the default" (/ - POSIX).