Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fs: utf8 fast paths don't accept all valid utf8 values #49888

Open
CanadaHonk opened this issue Sep 26, 2023 · 5 comments
Open

fs: utf8 fast paths don't accept all valid utf8 values #49888

CanadaHonk opened this issue Sep 26, 2023 · 5 comments
Labels
fs Issues and PRs related to the fs subsystem / file system.

Comments

@CanadaHonk
Copy link
Member

UTF8 and UTF-8 are also valid but not accepted by any existing utf8 fast paths (#48658, #49884)

@Uzlopak
Copy link
Contributor

Uzlopak commented Sep 27, 2023

Seems there needs to be a call to normalizeEncoding

@CanadaHonk
Copy link
Member Author

CanadaHonk commented Sep 27, 2023

I don't think that would be needed, there's only 4 cases where that returns utf8 and it's with the values: utf8, utf-8, UTF8, UTF-8. It might be worth making a small util to check for those (not sure if the function call would add overhead or not).

@Uzlopak
Copy link
Contributor

Uzlopak commented Sep 27, 2023

yeah, but you could also have mixed cases, like 'Utf-8' right?

@CanadaHonk
Copy link
Member Author

Oh right as a final check. I don't think fast paths should have to worry about that for now at least as I doubt it's common and would add measurable overhead?

@Uzlopak
Copy link
Contributor

Uzlopak commented Sep 27, 2023

I once wrote for testing all permutations of uppercase/lowercase into a Map and it improved the performance of normalizeEncoding. This sounds like a use case for normalizeEncoding.

@mertcanaltin mertcanaltin added the fs Issues and PRs related to the fs subsystem / file system. label Sep 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
fs Issues and PRs related to the fs subsystem / file system.
Projects
None yet
Development

No branches or pull requests

3 participants