`std.builtin.subsystem` reports `.Windows` for console applications

[castholm]

Zig Version

0.15.1

Steps to Reproduce and Observed Behavior

const std = @import("std");

pub fn main() void {
    std.debug.print("{?}\n", .{std.builtin.subsystem});
}

PS C:\tmp> zig build-exe .\repro.zig
PS C:\tmp> .\repro.exe
.Windows
PS C:\tmp> objdump .\repro.exe -Pheader | sls Subsystem
Major Subsystem Version:        6
Minor Subsystem Version:        0
Subsystem:                      3

3 is IMAGE_SUBSYSTEM_WINDOWS_CUI, 2 is IMAGE_SUBSYSTEM_WINDOWS_GUI. You also know that it's not a GUI application by the fact that it prints to the console.

Expected Behavior

std.builtin.subsystem does not report incorrect information.

[castholm]

Areas of interest:

zig/src/link/Lld.zig

Lines 558 to 579 in 35f013d

	const resolved_subsystem: ?std.Target.SubSystem = blk: {
	if (coff.subsystem) |explicit| break :blk explicit;
	switch (target.os.tag) {
	.windows => {
	if (comp.zcu != null) {
	if (coff.lld_export_flags.dllmain_crt_startup or is_dyn_lib)
	break :blk null;
	if (coff.lld_export_flags.c_main or comp.config.is_test or
	coff.lld_export_flags.winmain_crt_startup or
	coff.lld_export_flags.wwinmain_crt_startup)
	{
	break :blk .Console;
	}
	if (coff.lld_export_flags.winmain or coff.lld_export_flags.wwinmain)
	break :blk .Windows;
	}
	},
	.uefi => break :blk .EfiApplication,
	else => {},
	}
	break :blk null;
	};

zig/src/codegen/llvm.zig

Lines 1632 to 1637 in 35f013d

	if (name.eqlSlice("main", ip)) flags.c_main = true;
	if (name.eqlSlice("WinMain", ip)) flags.winmain = true;
	if (name.eqlSlice("wWinMain", ip)) flags.wwinmain = true;
	if (name.eqlSlice("WinMainCRTStartup", ip)) flags.winmain_crt_startup = true;
	if (name.eqlSlice("wWinMainCRTStartup", ip)) flags.wwinmain_crt_startup = true;
	if (name.eqlSlice("DllMainCRTStartup", ip)) flags.dllmain_crt_startup = true;

But maybe std.builtin.subsystem should get removed since trying to detect the implicit subsystem like this is inherently flaky. It only has one single reference in std, which is for dealing with the nCmdShow [w]WinMain parameter.

[alexrp]

I would personally consider that std.builtin.subsystem detection code subpar. It's not actually asking the right question, namely whether these functions are exported, just an imperfect proxy for that. And the fact that it's trying to replicate linker logic that could get out of sync easily (and has) isn't great either...

Is there anything we can do about the nCmdShow usage?

Side note: Why is SubSystem in std.Target? It has no relation to anything in there... seems like it should be in std.builtin. (And also be fixed to follow Zig style in the process.)

[castholm]

You can quite trivially detect the subsystem at runtime by walking the PEB:

const std = @import("std");
const windows = std.os.windows;

pub extern "kernel32" fn AllocConsole() callconv(.winapi) windows.BOOL;

pub fn main() void {
    // Ensure there's a console window to print to
    _ = AllocConsole();

    std.debug.print("{?}\n", .{std.builtin.subsystem});

    const base: [*]const u8 = @ptrCast(std.os.windows.peb().ImageBaseAddress);
    // IMAGE_DOS_HEADER.e_lfanew
    const e_lfanew: *const windows.LONG = @ptrCast(@alignCast(base + 0x3c));
    // IMAGE_NT_HEADERS.OptionalHeader.Subsystem (same offset on both x86 and x64)
    const Subsystem: *const windows.USHORT = @ptrCast(@alignCast(base + @as(usize, @intCast(e_lfanew.*)) + 0x18 + 0x44));
    std.debug.print("{}\n", .{Subsystem.*});

    // Give the user enough time to actually read the console
    std.Thread.sleep(5 * std.time.ns_per_s);
}

(Side note: it would be nice if the PE header structures were defined in std.coff.)

[alexrp]

You can quite trivially detect the subsystem at runtime by walking the PEB:

That seems pretty reasonable. My vote would be for removing std.builtin.subsystem and @import("builtin").explicit_subsystem in favor of that.

[squeek502]

Should probably be considered a duplicate of #17808