Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorrect subsystem on windows with VS2022 #126

Open
octopoulos opened this issue Oct 11, 2023 · 18 comments
Open

Incorrect subsystem on windows with VS2022 #126

octopoulos opened this issue Oct 11, 2023 · 18 comments

Comments

@octopoulos
Copy link

Just following the basic guide:

> mkdir ~/scratch
> cd ~/scratch
> git clone https://github.com/floooh/sokol-samples
> cd sokol-samples
> ./fips set config sapp-d3d11-win64-vstudio-debug
> ./fips build

=> error LNK2019: unresolved external symbol main referenced in function "int __cdecl invoke_main(void)" (?invoke_main@@YAHXZ)

The reason is that the main hijack is not working, because it's being overridden by the project subsystem, which is set to Console (instead of NotSet).

    <Link>
      ...
      <SubSystem>Console</SubSystem>
    </Link>

If I change the SubSystem to Windows or NotSet then it works, but it would be nice if the tool could do it automatically, of if the hijack could override that setting.

@octopoulos
Copy link
Author

octopoulos commented Oct 11, 2023

Note that a solution might just be to always define main( whenever WinMain( is defined.
(in sokol_app.h)

@floooh
Copy link
Owner

floooh commented Oct 11, 2023

This is weird, the subsystem should be selected via fips_begin_app(... windowed) (requires WinMain) vs fips_begin_app(... cmdline) (requires main). Let me check if I have a CMakeLists error somewhere...

...also the CI should actually fail in that case...

@floooh
Copy link
Owner

floooh commented Oct 11, 2023

Hmm, I just tried the process from scratch and it works here.

Can you give a bit more context?

  • the output lines from cmake following === generating: sapp-d3d11-win64-vstudio-debug
  • the output lines around the actual error, e.g. which of the samples is failing
  • cmake and Visual Studio version
  • output of fips diag tools

@floooh
Copy link
Owner

floooh commented Oct 11, 2023

For instance on my system:

=== generating: sapp-d3d11-win64-vstudio-debug
config file: E:/scratch/sokol-samples/fips-files/configs/sapp-d3d11-win64-vstudio-debug.yml
Preset CMake variables:

  CMAKE_BUILD_TYPE="Debug"
  FIPS_AUTO_IMPORT="OFF"
  FIPS_CONFIG="sapp-d3d11-win64-vstudio-debug"
  FIPS_LOCAL_BUILD="OFF"
  SOKOL_USE_D3D11:BOOL="ON"
  USE_SOKOL_APP:BOOL="ON"

-- Building for: Visual Studio 17 2022
-- Selecting Windows SDK version 10.0.22000.0 to target Windows 6.2.9200.
-- The C compiler identification is MSVC 19.34.31935.0
-- The CXX compiler identification is MSVC 19.34.31935.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.34.31933/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.34.31933/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMAKE_BUILD_TYPE: Debug
FIPS_ROOT_DIR: E:/scratch/fips
FIPS_PROJECT_DIR: E:/scratch/sokol-samples
FIPS_BUILD_DIR: E:/scratch/fips-build
FIPS_DEPLOY_DIR: E:/scratch/fips-deploy
FIPS_PROJECT_NAME: sokol-samples
FIPS_AUTO_IMPORT: OFF
FIPS_CONFIG: sapp-d3d11-win64-vstudio-debug
FIPS_PROJECT_BUILD_DIR: E:/scratch/fips-build/sokol-samples/sapp-d3d11-win64-vstudio-debug
FIPS_PROJECT_DEPLOY_DIR: E:/scratch/fips-deploy/sokol-samples/sapp-d3d11-win64-vstudio-debug
Host system: Windows
CMAKE_CXX_COMPILER_ID: MSVC
FIPS_MSVC: 1
FIPS_PLATFORM: WIN64
FIPS_PLATFORM_NAME: win64
-- Found Python: C:/Users/floh/scoop/apps/python/current/python.exe (found version "3.12.0") found components: Interpreter
-- Configuring done (4.5s)
-- Generating done (0.5s)
-- Build files have been written to: E:/scratch/fips-build/sokol-samples/sapp-d3d11-win64-vstudio-debug
cmake --build . --config Debug --
MSBuild version 17.4.0+18d5aef85 for .NET Framework

cmake version 3.27.1

Microsoft Visual Studio Community 2022 (64-bit) - Current
Version 17.4.2

λ fips diag tools
=== tools:
git:    found
cmake:  found
cmake-gui:      found
make:   found
ninja:  found
javac:  found
java:   found
vscode: found
vscode-cpptools:        found
vscode-cmaketools:      found
vscode-codelldb:        found
wasm-dwarf-debugging:   OPTIONAL, NOT FOUND (https://marketplace.visualstudio.com/items?itemName=ms-vscode.wasm-dwarf-debugging)
clion:  OPTIONAL, NOT FOUND (used as IDE with clion configs)
http-server:    found
wasmtime:       OPTIONAL, NOT FOUND (required for running WASI executables)

@floooh
Copy link
Owner

floooh commented Oct 11, 2023

Also, when I open Visual Studio (via fips open), and look at the properties of the clear-sapp sample, the subsystem is set to Windows:

image

(none of the samples under sapp/ should actually use the Console subsystem)

...looking at the generated .vcxproj files, all targets are set to SubSystem Windows (which is correct but seems to be different on your side):

image

Something must be wrong about the cmake generation step I think...

@octopoulos
Copy link
Author

Ok redoing it from scratch:
C:> mkdir scratch
C:> cd scratch
C:\scratch> git clone https://github.com/floooh/sokol-samples
C:\scratch> cd sokol-samples
C:\scratch\sokol-samples> ./fips set config sapp-d3d11-win64-vstudio-debug
...
C:\scratch\sokol-samples> ./fips build
...

=== building: sapp-d3d11-win64-vstudio-debug
=== generating: sapp-d3d11-win64-vstudio-debug
config file: C:/scratch/sokol-samples/fips-files/configs/sapp-d3d11-win64-vstudio-debug.yml
Preset CMake variables:

  CMAKE_BUILD_TYPE="Debug"
  FIPS_AUTO_IMPORT="OFF"
  FIPS_CONFIG="sapp-d3d11-win64-vstudio-debug"
  FIPS_LOCAL_BUILD="OFF"
  SOKOL_USE_D3D11:BOOL="ON"
  USE_SOKOL_APP:BOOL="ON"

-- Building for: Visual Studio 17 2022
-- Selecting Windows SDK version 10.0.19041.0 to target Windows 10.0.19042.
-- The C compiler identification is MSVC 19.37.32824.0
-- The CXX compiler identification is MSVC 19.37.32824.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.37.32822/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.37.32822/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMAKE_BUILD_TYPE: Debug
FIPS_ROOT_DIR: C:/scratch/fips
FIPS_PROJECT_DIR: C:/scratch/sokol-samples
FIPS_BUILD_DIR: C:/scratch/fips-build
FIPS_DEPLOY_DIR: C:/scratch/fips-deploy
FIPS_PROJECT_NAME: sokol-samples
FIPS_AUTO_IMPORT: OFF
FIPS_CONFIG: sapp-d3d11-win64-vstudio-debug
FIPS_PROJECT_BUILD_DIR: C:/scratch/fips-build/sokol-samples/sapp-d3d11-win64-vstudio-debug
FIPS_PROJECT_DEPLOY_DIR: C:/scratch/fips-deploy/sokol-samples/sapp-d3d11-win64-vstudio-debug
Host system: Windows
CMAKE_CXX_COMPILER_ID: MSVC
FIPS_MSVC: 1
FIPS_PLATFORM:
FIPS_PLATFORM_NAME:
-- Found Python: C:/Code/Python38/python.exe (found version "3.8.1") found components: Interpreter
-- Configuring done
-- Generating done
-- Build files have been written to: C:/scratch/fips-build/sokol-samples/sapp-d3d11-win64-vstudio-debug
cmake --build . --config Debug --
MSBuild version 17.7.2+d6990bcfa for .NET Framework

All the samples fail, ex: first 2 errors:

  ## sokol-shdc: C:/scratch/sokol-samples/sapp/noentry-sapp.glsl C:/scratch/fips-build/sokol-samples/sapp-d3d11-win64-vstudio-debug/sapp/noentry-sapp.gls
  l.h {'slang': 'hlsl4', 'compiler': 'MSVC'}
  Building Custom Rule C:/scratch/sokol-samples/sapp/CMakeLists.txt
  Building Custom Rule C:/scratch/sokol-samples/libs/sokol/CMakeLists.txt
  sokol.c
  sokol.vcxproj -> C:\scratch\fips-build\sokol-samples\sapp-d3d11-win64-vstudio-debug\libs\sokol\Debug\sokol.lib
  Building Custom Rule C:/scratch/sokol-samples/sapp/CMakeLists.txt
  arraytex-sapp.c
MSVCRTD.lib(exe_main.obj) : error LNK2019: unresolved external symbol main referenced in function "int __cdecl invoke_main(void)" (?invoke_main@@YAHXZ) [
C:\scratch\fips-build\sokol-samples\sapp-d3d11-win64-vstudio-debug\sapp\arraytex-sapp.vcxproj]
C:\scratch\fips-deploy\sokol-samples\sapp-d3d11-win64-vstudio-debug\arraytex-sapp.exe : fatal error LNK1120: 1 unresolved externals [C:\scratch\fips-buil
d\sokol-samples\sapp-d3d11-win64-vstudio-debug\sapp\arraytex-sapp.vcxproj]
  Building Custom Rule C:/scratch/fips-imgui/CMakeLists.txt
  imgui.cpp
  imgui_demo.cpp
  imgui_draw.cpp
  imgui_tables.cpp
  imgui_widgets.cpp
  Generating Code...
  imgui.vcxproj -> C:\scratch\fips-build\sokol-samples\sapp-d3d11-win64-vstudio-debug\fips-imgui_imgui\Debug\imgui.lib
  Building Custom Rule C:/scratch/sokol-samples/libs/dbgui/CMakeLists.txt
  dbgui.cc
  dbgui.vcxproj -> C:\scratch\fips-build\sokol-samples\sapp-d3d11-win64-vstudio-debug\libs\dbgui\Debug\dbgui.lib
  Building Custom Rule C:/scratch/sokol-samples/sapp/CMakeLists.txt
  arraytex-sapp.c
MSVCRTD.lib(exe_main.obj) : error LNK2019: unresolved external symbol main referenced in function "int __cdecl invoke_main(void)" (?invoke_main@@YAHXZ) [
C:\scratch\fips-build\sokol-samples\sapp-d3d11-win64-vstudio-debug\sapp\arraytex-sapp-ui.vcxproj]
C:\scratch\fips-deploy\sokol-samples\sapp-d3d11-win64-vstudio-debug\arraytex-sapp-ui.exe : fatal error LNK1120: 1 unresolved externals [C:\scratch\fips-b
uild\sokol-samples\sapp-d3d11-win64-vstudio-debug\sapp\arraytex-sapp-ui.vcxproj]
  Building Custom Rule C:/scratch/sokol-samples/libs/basisu/CMakeLists.txt
  sokol_basisu.cpp

@octopoulos
Copy link
Author

octopoulos commented Oct 11, 2023

PS C:\scratch\sokol-samples> cmake --version
cmake version 3.25.0

=> Oh I see, it's not the latest one, I guess I can upgrade to 3.27, who knows maybe that will fix, I'll keep you updated.
Edit: I tried with cmake 3.27.7 ... same thing.

Microsoft Visual Studio Community 2022 (64-bit) - Current
Version 17.7.4

image

=> yours is showing stack reserve size, while mine shows nothing there. I'll check with the latest CMake. (edit: done)

By the way, I forgot to thank you for the prompt reply!

@octopoulos
Copy link
Author

After comparing your file and mine, I see that the main difference is this, you have:

FIPS_PLATFORM: WIN64
FIPS_PLATFORM_NAME: win64

and I have:

FIPS_PLATFORM:
FIPS_PLATFORM_NAME:

That's probably the issue, right?

@floooh
Copy link
Owner

floooh commented Oct 11, 2023

That's probably the issue, right?

Wow yeah, let me check how this is detected...

@floooh
Copy link
Owner

floooh commented Oct 11, 2023

Hmm, still weird. When FIPS_PLATFORM isn't set it means it's not including this windows.cmake file here:

https://github.com/floooh/fips/blob/master/cmake-toolchains/windows.cmake

That should happen here when FIPS_HOST_WINDOWS is set:

https://github.com/floooh/fips/blob/36ae6b60a5d6d8c6a5661f8cf56b474c28d9bdaa/cmake/fips.cmake#L164-L172

...but that is definitely set because the message Host system: Windows shows up in your log:

https://github.com/floooh/fips/blob/36ae6b60a5d6d8c6a5661f8cf56b474c28d9bdaa/cmake/fips.cmake#L115-L118

Your Visual Studio seems to be a bit more recent... currently updating.

One more thing, are you building on PowerShell? If yes can you try on a "vanilla" cmd.exe (e.g. also not a "Visual Studio Command Prompt"). It should still work, but these are the only other differences I can think of.

I'll continue investigating tomorrow, it's pretty late now :)

@floooh
Copy link
Owner

floooh commented Oct 11, 2023

Btw, can you also check this project:

https://github.com/floooh/pacman.c

This isn't using my fips cmake wrapper but a regular CMakeLists.txt instead. Build instructions are in the readme. If this also fails (which I don't expect), then it would be some incompatibility between cmake and the latest VS (unlikely though).

@floooh
Copy link
Owner

floooh commented Oct 11, 2023

Hmm, updated to VS 17.7.5 and it's still working here, both on cmd.exe and in Powershell.

Running out of ideas...

@floooh
Copy link
Owner

floooh commented Oct 11, 2023

Ok, one last thing before I hit the sack:

Can you add a couple of debug-messages to the fips directory that's been checked out next to sokol-samples:

...in fips/cmake/fips.cmake around line 164:

    if (NOT CMAKE_TOOLCHAIN_FILE)
        message("######### in NOT CMAKE_TOOLCHAIN_FILE")
        if (FIPS_HOST_WINDOWS)
            message("######### including windows.cmake")
            include(${FIPS_ROOT_DIR}/cmake-toolchains/windows.cmake)
        elseif (FIPS_HOST_OSX)
            message("######### including osx.cmake")
            include(${FIPS_ROOT_DIR}/cmake-toolchains/osx.cmake)
        else()
            message("######### including linux.cmake")
            include(${FIPS_ROOT_DIR}/cmake-toolchains/linux.cmake)
        endif()
    endif()

... then run fips gen in the sokol-samples directory. Here it looks like this:

FIPS_PROJECT_DEPLOY_DIR: E:/scratch/fips-deploy/sokol-samples/sapp-d3d11-win64-vstudio-debug
Host system: Windows
CMAKE_CXX_COMPILER_ID: MSVC
FIPS_MSVC: 1
######## in NOT CMAKE_TOOLCHAIN_FILE
######### including windows.cmake
FIPS_PLATFORM: WIN64
FIPS_PLATFORM_NAME: win64
-- Configuring done (0.5s)

I basically want to check if windows.cmake is actually included.

@floooh
Copy link
Owner

floooh commented Oct 11, 2023

PS: do you have an environment variable CMAKE_TOOLCHAIN_FILE set by any chance? Because this would definitely confuse this piece of code.

@floooh
Copy link
Owner

floooh commented Oct 11, 2023

Anyway logging out now :)

@octopoulos
Copy link
Author

Hi, just letting you know that I will check everything tomorrow, I just had a full day trip.
Note: I think the best way to fix would be to have CMakeLists.txt specify the subsystem:Windows.
So, expect some news tomorrow.

@octopoulos
Copy link
Author

octopoulos commented Oct 13, 2023

Pacman works fine.

message("######### in NOT CMAKE_TOOLCHAIN_FILE")

Bingo!

I edited the fips.cmake to remove the toolchain check, and then it compiles fine on windows. I also tried emscripten a few days ago and that worked fine too.

@floooh
Copy link
Owner

floooh commented Oct 14, 2023

message("######### in NOT CMAKE_TOOLCHAIN_FILE")

But this is entirely correct.

What's concerning is that there is no ######### including windows.cmake message (or is there?).

When the windows.cmake file is included, it should work because this should set the correct subsystems, otherwise such problems are expected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants