Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Redirect using C output instead of C++ output #8391

Merged
merged 2 commits into from
Feb 17, 2024

Conversation

Pentarctagon
Copy link
Member

Apparently redirecting stdout/stderr also results in std::cout/std::cerr being redirected, but not the reverse. This is not compatible with using boost's tee.

Fixes #8108
Fixes #8255


The uncommented code in log.cpp results in the following being logged to file:

0
1
2
3
4
wesnoth: src/wesnoth.cpp:1131: int main(int, char**): Assertion `false && "5"' failed.

The commented code in log.cpp results in the following being logged to file (stdout/std::cout is missing):

0
1
3
wesnoth: src/wesnoth.cpp:1131: int main(int, char**): Assertion `false && "5"' failed.

The commented code is based on the 1.16 code from log_windows, so I'm not sure why stdout doesn't get redirected correctly (it's neither in the log file nor present on the terminal - I've no idea where it's getting sent to):

void log_file_manager::do_redirect_single_stream(const std::string& file_path,


@Wedge009 would you be able to test if this works the same on Windows 10 1903 or later where the path to the log file contains non-ascii characters?

src/log.cpp Outdated
// redirect to a file
// IMPORTANT: apparently redirecting stderr/stdout will also redirect std::cerr/std::cout, but the reverse is not true
// redirecting std::cerr/std::cout will *not* redirect stderr/stdout
if(setvbuf(stdout, NULL, _IONBF, 2) == -1) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you want to set stdout to be unbuffered?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Otherwise the stdout/std::cout output doesn't make it to the log file, at least for the example in wesnoth.cpp where it almost immediately terminates with an assert. stderr/std::cerr is unbuffered by default.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm… std::cout and/or the C FILE API might have a software buffer on top of that, so maybe it's fine…

src/log.cpp Outdated Show resolved Hide resolved
@Pentarctagon Pentarctagon marked this pull request as draft February 12, 2024 18:37
Comment on lines +195 to +245
// IMPORTANT: apparently redirecting stderr/stdout will also redirect std::cerr/std::cout, but the reverse is not true
// redirecting std::cerr/std::cout will *not* redirect stderr/stdout
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this is technically true, but… the way it's stated is maybe a tad misleading.

Redirecting std::cerr/std::cout is in the iostream, replacing their stream buffer with something else. Redirecting stderr/stdout is done at a lower level, essentially making both streams be the same stream.

I'm not sure what's a better way to put this…

@Wedge009
Copy link
Member

would you be able to test if this works the same on Windows 10 1903 or later where the path to the log file contains non-ascii characters?

Looks like this is still being worked on?

@Pentarctagon
Copy link
Member Author

It should at least build now. The unit tests won't run since the assert is there in wesnoth.cpp, but that's fine for now.

@Wedge009 Wedge009 added the Engine General game engine issues that do not fit in any other category. label Feb 13, 2024
@Wedge009
Copy link
Member

I'm not sure what I'm supposed to test. I haven't tried non-ASCII path because I get the assertion failure in Win10 with VS 2022. On Win7 with VS 2019 I don't even get that far - nothing opens, nothing displays.

@gfgtdf
Copy link
Contributor

gfgtdf commented Feb 13, 2024

The test is about whether the logfile is createsd and contains all messages incluusing the assertion error message.

@Pentarctagon
Copy link
Member Author

@Wedge009 the assertion failure is intended - part of the test is whether the text of the assertion is correctly added to the log file.

@Wedge009
Copy link
Member

Sorry I missed that detail. Without even considering the complication of non-ASCII log paths, I don't seem to get anything recorded in the log file. Just an empty log file is created and that's all. Your code is tested as-is in Win10 with VS2022 compilation, I haven't made any modifications.

@Pentarctagon
Copy link
Member Author

Weird. How are you running it?

@Wedge009
Copy link
Member

Same as usual, from the VS IDE. Same result when running from the command-line. The only argument used is specifying the data-dir.

@Pentarctagon
Copy link
Member Author

Is there anything printed on the command line?

@Wedge009
Copy link
Member

Wedge009 commented Feb 14, 2024

No, because everything should be redirected to the log by default. If I set --no-log-to-file then I see the numbers 0-4 inclusive printed, but no assertion message (it shows as an assertion failed dialogue instead - same as for normal log-to-file mode - and yes, it's release-mode compilation).

@Pentarctagon
Copy link
Member Author

If you comment out lines 198-228, and then uncomment lines 230-240, does it work?

@Wedge009
Copy link
Member

0, 1, and 3 are printed in the log file, followed by the normal log. Assertion is not printed - again, it's shown as a dialogue.

I didn't try moving my user directory to a path with non-ASCII characters. Is there an easy way to tell Wesnoth to use a specific directory for the log file?

@CelticMinstrel
Copy link
Member

It sounds to me that the assertion not being printed is intentional – Windows is trying to be smart and show it to the user in a more direct way.

@Wedge009
Copy link
Member

Looks like it's just how the VC++ library wants to implement it:
image

@Pentarctagon
Copy link
Member Author

The assert in wesnoth.cpp on line 1131 can just be removed then - no point in testing it on Windows. The logs directory is just the "logs" folder inside the userdata folder, so setting a custom userdata folder will move the logs too.

Out of curiosity, are you able to build 1.16 as well? I'm wondering if the stdout redirecting ever worked, since the lines 230-240 should be essentially what the Windows logging to file was doing for 1.16.

@Pentarctagon
Copy link
Member Author

Looks like Windows making the distinction between console vs gui applications is being a pain again - Windows GUI apps make _fileno(stderr) and _fileno(stdout) return -2 instead of the actual value. I've pushed a commit that hardcodes them as 1 and 2 instead, so please test again with that as well when you can.

@Wedge009
Copy link
Member

I pulled your changes, reverting to a clean build, no modifications.

No more assertion dialogue. Still zero-byte log file generated. Running with --no-log-to-file, I see 0-4 inclusive printed out before the normal log text. No mention of any assertion in the log output.

I'm afraid I'm not able to keep testing ad-hoc changes but I'll try to help with testing as I'm able.

@Pentarctagon
Copy link
Member Author

Pushed another attempt - if the previous code didn't work then I'm guessing this won't either, probably for the same reason. But maybe we'll get lucky and it's at least a lot simpler.

@Wedge009
Copy link
Member

Well... using your branch unmodified, I get 2 and 4 printed in the log file... and nothing else.

@Pentarctagon
Copy link
Member Author

Well, that's... different, at least. Pushed another attempt - ideally you should see 2 and 4 in the ".out.log" file and everything else in the regular ".log" file.

@Wedge009
Copy link
Member

Yes, 2 and 4 are in .out.log.

0, 1, 3, followed by normal log text are in .log.

@Pentarctagon
Copy link
Member Author

Alright, that's what I'm going to go with then, unless someone knows a way to get _dup2() to work with stdout/stderr for Windows GUI apps.

@Wedge009
Copy link
Member

There weren't any assertion messages after your changes. No dialogue, no printing in the log. I assume that's okay/expected? I haven't tried this at all on Linux.

@Wedge009
Copy link
Member

Oh, I just remembered your request to test non-ASCII characters. I ran wesnoth --userdata-dir=tést and the directory was created... but the logs directory remains empty. But then I tried wesnoth --userdata-dir=test and the logs directory there was also empty. So I don't know what's going on.

@Wedge009
Copy link
Member

Wedge009 commented Feb 15, 2024

I repeated the wesnoth --userdata-dir=test operation with the official Windows release for 1.17.24 and there are no logs generated either. So maybe not specific to your work here... in which case, should this be reported as an issue?

@Pentarctagon
Copy link
Member Author

Pentarctagon commented Feb 15, 2024

I can reproduce the issue on Linux as well - the build info reports the logs directory as it should (aka it uses the --userdata-dir value), but it's actually writing the log file to the old location. Well, I suppose I'll be fixing that here too, so this PR can be fully tested.

There weren't any assertion messages after your changes. No dialogue, no printing in the log. I assume that's okay/expected? I haven't tried this at all on Linux.

Yeah, I removed it since it wasn't relevant to testing on Windows.

@Wedge009
Copy link
Member

Wedge009 commented Feb 15, 2024

What does the build info dialog report the logs directory as being?

I checked that as well - it's reporting the logs directory corresponding with the userdata-dir input, just nothing gets written there. As I said, it's a currently-existing problem in 1.17. Testing 1.16.11 release, the log file gets written (in Windows at least, default log file writing wasn't in place for Linux).

...but it's actually writing the log file to the old location

Oh, yeah, so that's what happened. I thought it was just not writing anything at all.

@Pentarctagon
Copy link
Member Author

Looks like the issue is that the log file setup happens before the command line options are parsed. Not sure if it makes sense to move that earlier or not though.

@Wedge009
Copy link
Member

1.16 writes the log file in the correct location, though? Is the current set-up for 1.17 that much different? (I know you did a lot of work on the log redirection previously.)

@Pentarctagon
Copy link
Member Author

Pentarctagon commented Feb 15, 2024

For 1.16 it would initially write the log file to the Windows temp directory, then move it to the logs directory after the userdata folder is setup, wherever that ends up being. For 1.17 it just writes the log file directly to the logs directory.

@Pentarctagon
Copy link
Member Author

Alright, the log file not using a userdata folder provided via the command line option should be fixed.

@CelticMinstrel
Copy link
Member

I hate that there's a separate .out.log file…

@Pentarctagon
Copy link
Member Author

Yeah, it's dumb. Unfortunately, Windows makes it the only option I know of to work correctly.

@CelticMinstrel
Copy link
Member

CelticMinstrel commented Feb 16, 2024

If it's only needed on Windows, why do the other OSes need to suffer too?

(Unless I misread and it's only being done on Windows.)

@Pentarctagon
Copy link
Member Author

It's only being done for Windows. Other OSes will just use dup2(STDERR_FILENO, STDOUT_FILENO) to have both go to the same file.

@Wedge009
Copy link
Member

Specification of userdata-dir works, including log file output, now. Including non-ASCII characters in the specified directory. Only tested on Win10.

@Pentarctagon
Copy link
Member Author

Nice, glad that worked on the first attempt. Thanks for testing all this out!

@CelticMinstrel
Copy link
Member

It's only being done for Windows. Other OSes will just use dup2(STDERR_FILENO, STDOUT_FILENO) to have both go to the same file.

Okay… still bad, but less bad at least. I guess we can leave that as something to work out at some unspecified time in the future…

Apparently redirecting stdout/stderr also results in std::cout/std::cerr being redirected, but not the reverse. This is not compatible with using boost's tee.

Fixes wesnoth#8108
Fixes wesnoth#8255
src/log.hpp Outdated Show resolved Hide resolved
@CelticMinstrel
Copy link
Member

I'm wondering about the choice of name for .out.log. What actually goes in there, anyway?

@Pentarctagon
Copy link
Member Author

Pentarctagon commented Feb 16, 2024

Any output to stdout/std::cout would end up in the .out.log file. Nothing enabled by default in Wesnoth writes to either of those, though there's some stuff that's commented/#ifdef 0d out. So for regular players it'd end up empty all the time unless something in a dependency wrote to it for some reason.

@Wedge009
Copy link
Member

Tested on Win7 - logs working, redirecting to directory with non-ASCII characters also works. As a bonus, presumably because boost tee is no longer used, #8024 is fixed by this too.

@Pentarctagon
Copy link
Member Author

Nice - I'll merge this tomorrow if nothing else comes up then.

@stevecotton stevecotton merged commit 7a3a72e into wesnoth:master Feb 17, 2024
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Engine General game engine issues that do not fit in any other category.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Assertion when logging large essages on windows C output streams not redirected to logfile
5 participants