Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make GlobTokenizer resilient against invalid path characters #1354

Open
stffabi opened this issue Nov 11, 2016 · 1 comment
Open

Make GlobTokenizer resilient against invalid path characters #1354

stffabi opened this issue Nov 11, 2016 · 1 comment

Comments

@stffabi
Copy link
Contributor

stffabi commented Nov 11, 2016

During the last couple of weeks several issues (#1316, #1342, #1323, #1245) were created because the GlobTokenizer couldn't work with identifiers that contained some special characters, like exclamation, at... and so one.
The issues have been fixed on a case by case basis and the characters have been added to the identifier RegEx, which is used in the GlobTokenizer.

The GlobTokenizer should be more resilient against such characters. A proposal has been mentioned in #1316.

Quote: #1316 (comment)

The best would be to make the GlobTokenizer resilient against invalid file/path characters. The tokenizer shouldn't be responsible for any file/path character checking, that's the job of the filesystem. It should be independent of the filesystem and not have any filesystem dependent code in it. E.g. currently there's a WindowsRoot token, in order to support this, the tokenizer mainly prevents the use of : in a path on an unix system. The Globber definition has some special characters, *, ?, /, \. Apart from these the tokenizer should allow all characters to be used in an identifier and later on the specific filesystem should decide if it's valid or not.

I'm going to provide a PR with an implementation, so we could further discuss this.

@stffabi
Copy link
Contributor Author

stffabi commented Nov 11, 2016

I've started with the implementation, but won't be able to create the PR before around Tuesday.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant