Set stdout to line buffering

[SirLynix]

Is your feature request related to a problem? Please describe.

When running xmake on GitHub actions on Linux and macOS, you have to wait a few minutes to see the log.

This is because on Linux/macOS stdout is fully buffered on Github by default.

Describe the solution you'd like

XMake could enable line buffering using setvbuf(stdout, NULL, _IONBF, 0); to flush on line feed.

Describe alternatives you've considered

I'm not sure if it can be done outside of xmake.

Additional context

No response

[waruqi]

I think it should be possible to control the stdout buffer size by creating a child process, like python -u.

[waruqi]

You can try this patch, but I'm not sure if it works. #2605

xmake update -s github:xmake-io/xmake#stdio

[SirLynix]

Unfortunately it doesn't work, same behavior as before (tested on Linux CI).

Isn't there some code missing there, since it tests for XMAKE_STDOUT_NBUF env? Or should I set it myself from CI?

[waruqi]

I improved it, try it again.

export XMAKE_STDOUT_NBUF=0

[SirLynix]

It seems to work, every line gets printed on Github CI directly.

However, this seems to disable completely stdout buffer, which could be problematic with verbose output. Couldn't we use _IOLBF (line buffering) by default? (it's a standard value).

setvbuf(stdout, tb_null, _IOLBF, BUFSIZ);

This is standard and can be used on Windows as well.

On Posix we can get a better value than BUFSIZE using

    struct stat stats;
    int fileno(FILE*);
    if(fstat(fileno(fp), &stats) == -1) { // POSIX only
        perror("fstat"); return EXIT_FAILURE;
    }
 
    printf("BUFSIZ is %d, but optimal block size is %ld\n", BUFSIZ, stats.st_blksize);
    if(setvbuf(fp, NULL, _IOFBF, stats.st_blksize) != 0) {
       perror("setvbuf failed"); // POSIX version sets errno
       return EXIT_FAILURE;
    }

[waruqi]

if(setvbuf(fp, NULL, _IOFBF, stats.st_blksize) != 0) {

? _IOFBF?

[SirLynix]

My mistake, I copy-pasted the code from the documentation. Should be _IOLBF for line-feed buffering

[waruqi]

try it again. export XMAKE_STDOUT_LINEFLUSH=y

[waruqi]

This is standard and can be used on Windows as well.

xmake do not use fwrite(stdout) on windows. It can be a little complicated to process. There is also an encoding conversion before writing to stdout, and a separate stream buffer

[SirLynix]

It works great. What would be the issue of enabling this by default?

This is standard and can be used on Windows as well.

xmake do not use fwrite(stdout) on windows. It can be a little complicated to process. There is also an encoding conversion before writing to stdout, and a separate stream buffer

Okay, it's not an issue on Windows CI anyway.

[waruqi]

I'm not sure if it has an impact on build performance or if there are any other side effects. For the time being it is better to control it through environment variables.

[SirLynix]

This is already the default behavior on most OS, and for reference this is done by Ninja as well:
https://github.com/ninja-build/ninja/blob/master/src/ninja.cc#L1518
https://github.com/ninja-build/ninja/blob/master/src/build_log.cc#L193

[waruqi]

ok, I have enabled it by default.

[SirLynix]

Thanks!