Join GitHub today
GitHub is home to over 36 million developers working together to host and review code, manage projects, and build software together.Sign up
Filter redesign (forward slashes on Windows) #257
As discussed in #191, this PR redesigns the filter system.
TL;DR; This is a breaking change that simplifies filters on all platforms. Filters will always use forward slashes, even on Windows. Please test and provide feedback.
How filters currently work
How this PR implements filtering
def build_filter(regex): if os.path.isabs(regex): return AbsoluteFilter(regex) else: return RelativeFilter(os.getcwd(), regex) class Filter(object): def __init__(self, pattern): self.pattern = re.compile(pattern) def match(self, path): os_independent_path = path.replace(os.path.sep, '/') return self.pattern.match(os_independent_path) class AbsoluteFilter(Filter): def match(self, path): abspath = os.path.realpath(path) return super().match(abspath) class RelativeFilter(Filter): def __init__(self, root, pattern): super().__init__(pattern) self.root = root def match(self, path): abspath = os.path.realpath(path) relpath = os.path.relpath(abspath, self.root) return super().match(relpath) class AlwaysMatchFilter(Filter): def __init__(self): super().__init__("") def match(self, path): return True class DirectoryPrefixFilter(Filter): def __init__(self, directory): os_independent_dir = directory.replace(os.path.sep, '/') pattern = re.escape(os_independent_dir + '/') super().__init__(pattern)
Importantly, the filter pattern is no longer transformed as a path before being compiled into a regex! This means we can now run the same filtering code on Windows and Unix. To use filters that match relative paths, we turn that path into a relative path.
One remaining difference is that Windows doesn't seem to support Unix-like symlinks. The relevant test cases still fail, because
This is a breaking change on all systems. Because the filter regexes are no longer treated as paths, there are some subtle changes on Unix (e.g., this filter will no longer work:
These are not:
Please test this PR on your real projects. I am determined to push this through because it fixes many problems, but I'm hesitant to merge before it's been properly validated. And if you have any suggestions regarding the behaviour of these filters, now is the time to share them.
@@ Coverage Diff @@ ## master #257 +/- ## ========================================= + Coverage 88.76% 89.5% +0.73% ========================================= Files 13 13 Lines 1478 1515 +37 Branches 267 269 +2 ========================================= + Hits 1312 1356 +44 + Misses 108 105 -3 + Partials 58 54 -4
referenced this pull request
May 22, 2018
Test on mingw64 with mingw64-x86_64-python3
test relative path with regex : pass
test abs path with unix syntax : pass
test abs path with windows syntax: needed a check here
Thank you @lisongmin for your tests.
The “bad escape” error from the regex module doesn't look very user friendly, and only occurs on recent Python 3 versions. I'll try to implement a warning when a user writes a filter that cannot be a regex for a path, e.g. if it contains an escaped backslash
The Absolute and Relative filters describe what kind of path they are trying to match. So we do not transform the filter, but transform the path into the format that the filter is expecting.