Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better support for / separators on Windows, improve wchar filename coverage #1787

Merged
merged 9 commits into from
Jan 11, 2021
Merged

Better support for / separators on Windows, improve wchar filename coverage #1787

merged 9 commits into from
Jan 11, 2021

Conversation

sylveon
Copy link
Contributor

@sylveon sylveon commented Jan 6, 2021

The first part of this PR fixes a bug with the default formatting. When not using /FC the __FILE__ macro can take strange values such as C:\Projects\TranslucentTB\TranslucentTB\tray\../dynamicloader.hpp. Under the current code, spdlog would print it in the log as ../dynamicloader.hpp. With this PR, it now correctly prints dynamicloader.hpp.

As a bonus, I added an extension point for users: they can change what spdlog considers to be a folder separator, if for example they want to revert to the old behavior.

I've also added an AppVeyor worker which tries building spdlog with wide filenames enabled, and adjusted the unit tests accordingly. If I didn't forget anything, I believe it should pass.

@sylveon
Copy link
Contributor Author

sylveon commented Jan 6, 2021

Crud a test broke - it's currently 4:30am here so I will look at it tomorrow.

@sylveon
Copy link
Contributor Author

sylveon commented Jan 7, 2021

All green now! I also fixed an issue where win_eventlog_sink would silently discard errors from ReportEvent or MultiByteToWideChar if WCHAR to UTF-8 was enabled. Now they are correctly handled.

Also fixed a security bug in utf8_to_wstrbuf that could result in a potential buffer overflow/stack overrun. The buffer was a narrow char buffer but the size was passed as if it was a wide char buffer, making it seem twice as big than it really is. I fixed the issue by using a proper wide character buffer.

Copy link
Owner

@gabime gabime left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this pr. Not sure I want to allow folder separator under windows be '\' and '/'. (e.g. what would happen if a folder path constains the '/' char as part of the path?)

Please see my comments

example/example.cpp Outdated Show resolved Hide resolved
example/example.cpp Outdated Show resolved Hide resolved
example/example.cpp Outdated Show resolved Hide resolved
example/example.cpp Outdated Show resolved Hide resolved
example/example.cpp Outdated Show resolved Hide resolved
@@ -118,7 +118,7 @@ SPDLOG_INLINE std::tuple<filename_t, filename_t> file_helper::split_by_extension
}

// treat cases like "/etc/rc.d/somelogfile or "/abc/.hiddenfile"
auto folder_index = fname.rfind(details::os::folder_sep);
auto folder_index = fname.find_last_of(details::os::folder_seps_filename);
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will break if folder_seps_filename is more than single char. Let's use the original rfind()

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's the intention: each character in folder_seps_filename is considered as a possible folder separator. find_last_of searches for any character in folder_seps_filename instead of searching for the entirety of the string.

include/spdlog/details/os-inl.h Show resolved Hide resolved
{
result_size = ::MultiByteToWideChar(CP_UTF8, MB_ERR_INVALID_CHARS, str.data(), str_size, NULL, 0);
}

if (result_size > 0)
{
target.resize(result_size);
result_size = ::MultiByteToWideChar(CP_UTF8, MB_ERR_INVALID_CHARS, str.data(), str_size, (LPWSTR)target.data(), result_size);
result_size = ::MultiByteToWideChar(CP_UTF8, MB_ERR_INVALID_CHARS, str.data(), str_size, target.data(), result_size);

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why remove the (LPWSTR) cast ? Does target.data() guaranteed to be (LPWSTR) ?

Copy link
Contributor Author

@sylveon sylveon Jan 9, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, because now it uses a wmemory_buffer, which as the w prefix says, is always wide.

The previous cast was questionable (casting from non-wide to wide is a frequent source of bug)

size_t search_offset = 0;
do
{
auto token_pos = path.find(folder_sep, search_offset);
auto token_pos = path.find_first_of(folder_seps_filename, search_offset);
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will break if folder_seps_filename is more than single char. Let's use the original find()

#else
SPDLOG_CONSTEXPR static const char folder_sep = '/';
#define SPDLOG_FOLDER_SEPS "/"
Copy link
Owner

@gabime gabime Jan 9, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What would happen if a folder name in windows contains the "\" character ? Let's keep the orig code

Copy link
Contributor Author

@sylveon sylveon Jan 9, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Folder names can't contain this character.

return rv != nullptr ? rv + 1 : filename;
// if the size is 2 (1 character + null terminator) we can use the more efficient strrchr
// the branch will be elided by optimizations
if (sizeof(os::folder_seps) == 2)
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please post a dedicated pr for this optimization.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change doesn't make sense without the rest of the folder separator changes

@sylveon
Copy link
Contributor Author

sylveon commented Jan 9, 2021

(e.g. what would happen if a folder path constains the '/' char as part of the path?)

Windows already disallows this and CreateFile (the Win32 API that is ultimately called) interprets / as a valid folder separator. This is easily confirmed by seeing that the tests pass correctly: some tests use / in file paths, and they do not fail
image

@gabime gabime merged commit 46d4181 into gabime:v1.x Jan 11, 2021
@gabime
Copy link
Owner

gabime commented Jan 11, 2021

Thanks @sylveon . Merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants