Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Publish Windows on Arm build #3452

Merged
merged 7 commits into from Oct 14, 2023
Merged

Conversation

hmartinez82
Copy link
Contributor

@hmartinez82 hmartinez82 commented Oct 9, 2023

Draft PR to discuss what's needed for creating the Windows on ARM installer. This requires #3450 to be merged first!

Description

The installer itself can be either x64 or x86 since what matters are the binary files it installs and they will be ARM64.
Here is what I was unable to find out:

  • I noticed that OpenSSL is copied from a Stellarium repository. Does Stellarium use OpenSSL directly or is it all done via QtNetwork? Would be ok to just copy the two DLLs from AppVeyor directly instead? AppVeyor does have OpenSSL 3.1 -> https://www.appveyor.com/docs/windows-images-software/#tools . Where do these (https://github.com/Stellarium/stellarium-data/releases/tag/qt-5.6) OpenSSL DLLs come from?

  • I ran qtdiag in an AppVeyor build. (See https://ci.appveyor.com/project/hernancmartinez/stellarium/builds/48227004#L68). I was trying to find out if AppVeyor's build of Qt was using OpenSSL or Schannel as its QtNetwork SSL backend. But here lies the issue: AppVeyor does not have OpenSSL compiled for Windows on ARM and I saw in qtdiag's output that it's using OpenSSL in its Qt's x64 build. If AppVeyor's build of Qt ARM64 is using OpenSSL, where are the OpenSSL binaries? (I can't run ARM64 qtdiag.exe in AppVeyor!) I asked them this at this post https://twitter.com/hmartinez82/status/1711263056549023835 . I don't know how to get in touch with AppVeyor and their Public Forum is not letting me create a new account! If someone here in the Stellarium project has a way to ask them that, please ask it for me!

  • Where is the copying of SpoutLibrary.dll to the output folder? In my x64 builds the CMake INSTALL target is not copying SpoutLibrary.dll to the bin folder even when ENABLE_SPOUT is enabled. When I try to run Steallarium.exe I get the SpoutLibrary.dll was not found error message until I manually copy it to the bin folder.

  • I never used Inno Setup before. So I may need some help, for instance, where are the installer scripts?

.appveyor.yml Outdated Show resolved Hide resolved
.appveyor.yml Outdated Show resolved Hide resolved
@github-actions github-actions bot added the has conflicts The pull request has conflicts label Oct 9, 2023
@github-actions
Copy link

github-actions bot commented Oct 9, 2023

This pull request has conflicts, please resolve those before we can evaluate the pull request.

@github-actions
Copy link

github-actions bot commented Oct 9, 2023

Conflicts have been resolved. A maintainer will review the pull request shortly.

@github-actions github-actions bot removed the has conflicts The pull request has conflicts label Oct 9, 2023
src/CMakeLists.txt Outdated Show resolved Hide resolved
@alex-w
Copy link
Member

alex-w commented Oct 9, 2023

I never used Inno Setup before. So I may need some help, for instance, where are the installer scripts?

it's generated by cmake (see skeleton in cmake directory) - I've added support arm64 packages for Windows into InnoSetup in master

@alex-w
Copy link
Member

alex-w commented Oct 9, 2023

I noticed that OpenSSL is copied from a Stellarium repository. Does Stellarium use OpenSSL directly or is it all done via QtNetwork? Would be ok to just copy the two DLLs from AppVeyor directly instead? AppVeyor does have OpenSSL 3.1 -> https://www.appveyor.com/docs/windows-images-software/#tools . Where do these (https://github.com/Stellarium/stellarium-data/releases/tag/qt-5.6) OpenSSL DLLs come from?

See https://slproweb.com/products/Win32OpenSSL.html

@hmartinez82
Copy link
Contributor Author

hmartinez82 commented Oct 9, 2023

I noticed that OpenSSL is copied from a Stellarium repository. Does Stellarium use OpenSSL directly or is it all done via QtNetwork? Would be ok to just copy the two DLLs from AppVeyor directly instead? AppVeyor does have OpenSSL 3.1 -> https://www.appveyor.com/docs/windows-images-software/#tools . Where do these (https://github.com/Stellarium/stellarium-data/releases/tag/qt-5.6) OpenSSL DLLs come from?

See https://slproweb.com/products/Win32OpenSSL.html

I wonder why they call it "VERY EXPERIMENTAL". OpenSSL was one of the first libraries to be ever compiled for Windows on ARM64 all the way back in 2018. I can build it locally, I mean those two DLLs.

But still. Does Stellarium use OpenSSL directly? Or all network communication is done via QtNetwork?

@alex-w
Copy link
Member

alex-w commented Oct 9, 2023

But still. Does Stellarium use OpenSSL directly? Or all network communication is done via QtNetwork?

via QtNetwork, but winqtdeploy do not attach OpenSSL libraries, so, we should packed it by hands

@hmartinez82
Copy link
Contributor Author

hmartinez82 commented Oct 9, 2023

But still. Does Stellarium use OpenSSL directly? Or all network communication is done via QtNetwork?

via QtNetwork, but winqtdeploy do not attach OpenSSL libraries, so, we should packed it by hands

Awesome, so now we need to find out if the Qt 6 for Win ARM-64 that AppVeyor has, uses OpenSSL or Schannel. If it uses Schannel then we don't have to copy the OpenSSL DLLs in the ARM64 build.

@alex-w
Copy link
Member

alex-w commented Oct 10, 2023

But still. Does Stellarium use OpenSSL directly? Or all network communication is done via QtNetwork?

via QtNetwork, but winqtdeploy do not attach OpenSSL libraries, so, we should packed it by hands

Awesome, so now we need to find out if the Qt 6 for Win ARM-64 that AppVeyor has, uses OpenSSL or Schannel. If it uses Schannel then we don't have to copy the OpenSSL DLLs in the ARM64 build.

I don't have a Windows/arm64 platform, so, I cannot install SLP packages (W/arm64) and prepare archive for including arm64 libraries into installer. Do you have an arm64 device?

@hmartinez82
Copy link
Contributor Author

hmartinez82 commented Oct 10, 2023

But still. Does Stellarium use OpenSSL directly? Or all network communication is done via QtNetwork?

via QtNetwork, but winqtdeploy do not attach OpenSSL libraries, so, we should packed it by hands

Awesome, so now we need to find out if the Qt 6 for Win ARM-64 that AppVeyor has, uses OpenSSL or Schannel. If it uses Schannel then we don't have to copy the OpenSSL DLLs in the ARM64 build.

I don't have a Windows/arm64 platform, so, I cannot install SLP packages (W/arm64) and prepare archive for including arm64 libraries into installer. Do you have an arm64 device?

Yes, I have a Windows Dev Kit and a personal laptop. Can you ask AppVeyor if their build of Qt for Windows ARM 64-bit uses OpenSSL or Schannel?

What's SLP ?

@alex-w
Copy link
Member

alex-w commented Oct 10, 2023

Yes, I have a Windows Dev Kit and a personal laptop. Can you ask AppVeyor if their build of Qt for Windows ARM 64-bit uses OpenSSL or Schannel?

According to https://www.appveyor.com/docs/windows-images-software/#tools AppVeyor don't have OpenSSL for arm64 devices

What's SLP ?

See https://slproweb.com/products/Win32OpenSSL.html

@hmartinez82
Copy link
Contributor Author

Yes, I have a Windows Dev Kit and a personal laptop. Can you ask AppVeyor if their build of Qt for Windows ARM 64-bit uses OpenSSL or Schannel?

According to https://www.appveyor.com/docs/windows-images-software/#tools AppVeyor don't have OpenSSL for arm64 devices

Correct. That's I have been saying all along! They don't have OpenSSL for arm64, so how can it be possible that their build of Qt for arm64 requires OpenSSL for arm64?I need to know if they used Schannel, not OpenSSL, for QtNetwork!

What's SLP ?

See https://slproweb.com/products/Win32OpenSSL.html

@alex-w
Copy link
Member

alex-w commented Oct 10, 2023

No OpenSSL for arm64 packages for first step. What about documentation for Qt6 for Windows/arm64?

@hmartinez82
Copy link
Contributor Author

Qt6 for Windows/arm64 says that you can use OpenSSL for Windows on ARM64. I already compiled it for ARM64 dozens of times, it's supported since 2018. The issue is knowing if AppVeyor compiled Qt with OpenSSL for Arm64 or they just used Schannel.

But you are right, I'll change the build script to build the installer without OpenSSL, let's see what happens

@hmartinez82
Copy link
Contributor Author

@alex-w All those screenshots of Stellarium on Windows ARM64 in my machine were without OpenSSL, because I compiled Qt with Schannel, not OpenSSL.

@hmartinez82 hmartinez82 marked this pull request as ready for review October 10, 2023 17:56
@hmartinez82
Copy link
Contributor Author

hmartinez82 commented Oct 10, 2023

I added a workaround for appveyor/ci#3892: https://github.com/hmartinez82/stellarium/blob/publish_arm64/util/qtpaths_workaround.bat

Question: Do we need libEGL.dll/libEGLv2.dll for Qt 6 builds? I see those files for in the installed folder

And: We don't need OpenSSL for the ARM64 build of Stellarium. AppVeyor didn't answer my question so I took the matter in my own hands. Their build of Qt uses Schannel instead of OpenSSL. QtNetwork was built with SChannel 🎉

The screenshot below shows the dependencies for Qt 6 Network dll, it is not linked against OpenSSL, but linked against the Schannel SSPI instead! TLS/SSL overview (Schannel SSP)
image

Updating star catalogs is downloading files.

@gzotti
Copy link
Member

gzotti commented Oct 10, 2023

Question: Do we need libEGL.dll/libEGLv2.dll for Qt 6 builds? I see those files for in the installed folder

No, these (and D3Dcompiler_47.dll) are necessary only for Qt5 when running ANGLE mode. A Qt6 build should include opengl32sw.dll to be able to run software OpenGL (Mesa) mode.

@alex-w alex-w added the infrastructure Infrastructure related issues label Oct 12, 2023
@alex-w alex-w added this to Needs triage in OS: Windows via automation Oct 12, 2023
@alex-w alex-w added this to Needs triage in Hardware: ARM via automation Oct 12, 2023
@alex-w alex-w added this to the 23.4 milestone Oct 12, 2023
@alex-w alex-w moved this from Needs triage to To do in Hardware: ARM Oct 14, 2023
@alex-w alex-w moved this from Needs triage to To do in OS: Windows Oct 14, 2023
@alex-w alex-w moved this from To do to In progress in Hardware: ARM Oct 14, 2023
@alex-w alex-w moved this from To do to In progress in OS: Windows Oct 14, 2023
@alex-w
Copy link
Member

alex-w commented Oct 14, 2023

I think we may skip mesa for Windows/arm64 at first step or we should prepare binaries for all Windows architectures

@hmartinez82
Copy link
Contributor Author

@gzotti @alex-w Sure thing. I'll try to work on that after this is merged. I just need to know does one build that opengl32sw.dll

@alex-w alex-w merged commit 3283c67 into Stellarium:master Oct 14, 2023
12 checks passed
OS: Windows automation moved this from In progress to Done Oct 14, 2023
Hardware: ARM automation moved this from In progress to Done Oct 14, 2023
@hmartinez82 hmartinez82 deleted the publish_arm64 branch October 14, 2023 04:42
@alex-w
Copy link
Member

alex-w commented Oct 14, 2023

@hmartinez82 please see #3414

@hmartinez82
Copy link
Contributor Author

hmartinez82 commented Oct 14, 2023

@alex-w @gzotti One important fact, but also a fun one.

  • There's no native OpenGL in Windows on ARM. There's only support for DirectX 11 or 12
    If it's an emulated app then Microsoft lets DirectX 9 apps run with a translation DLL that is only supported for emulated apps, if an app is native it must be DirectX 11 or 12
  • If the app must use real OpenGL then Microsoft provides, for free, this Compatibility Pack https://apps.microsoft.com/detail/9NQPSL29BFFF. In other words, we have to somehow let Windows on ARM users know that this Microsoft Store package must be installed.

The fun fact is that were talking about using Mesa's LLVMPipe to have a software renderer. So I was looking into that and found the page for Gallium D3D12 driver. I think that the "Compatibility Pack" that Microsoft provides is just that Gallium driver: https://docs.mesa3d.org/drivers/d3d12.html because I see this in the logs when running Stellarium on Windows on ARM (x64 or ARM64 builds):

Initialization StelMainView
Luminance textures are not supported
Maximum texture anisotropy: 16
Maximum 2D texture size: 16384
Detected: OpenGL 4.1
Driver version string: 4.1 (Core Profile) Mesa 23.1.0-devel (git-fbc637faa5)
GL vendor: Microsoft Corporation
GL renderer: D3D12 (Qualcomm(R) Adreno(TM) 8cx Gen 3)
GL Shading Language version: 4.10
MESA Version Number detected: 23.1
Mesa version is fine, we should not see a graphics problem.
GLSL Version Number detected: 4.1
GLSL version is fine, we should not see a graphics problem.
stel.OpenGLArray: Vertex Array Objects are supported
Detected a high resolution device! Device pixel ratio: 2

@hmartinez82
Copy link
Contributor Author

Given that all Windows on ARM devices that were launched since 2018 use Snapdragon CPUs with the same Adreno GPU family I think that this Compatibility Package will always work as the OpenGL implementation. But you are right, eventually we should provide opengl32sw.dll for Windows on ARM too.

@gzotti
Copy link
Member

gzotti commented Oct 14, 2023

If it is guaranteed that all Win/arm64 systems have the d3d12 compatibility, then requiring this compatibility pack should be enough. You may add another round of tests in the startup StelMainView::processOpenGLdiagnosticsAndWarnings() The opengl32sw.dll is for hardware without even d3d9 compatibility (for Qt5) or for rare cases like VMs on servers without proper GPU. In this case, also the --mesa-mode in CLIProcessor should be excluded for arm builds.

@alex-w alex-w added the state: published The fix has been published for testing in weekly binary package label Oct 23, 2023
@github-actions
Copy link

Hello @hmartinez82!

Please check the fresh version (development snapshot) of Stellarium:
https://github.com/Stellarium/stellarium-data/releases/tag/weekly-snapshot

@alex-w alex-w removed the state: published The fix has been published for testing in weekly binary package label Dec 23, 2023
Copy link

Hello @hmartinez82!

Please check the latest stable version of Stellarium:
https://github.com/Stellarium/stellarium/releases/latest

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
infrastructure Infrastructure related issues
Projects
Hardware: ARM
  
Done
OS: Windows
  
Done
Development

Successfully merging this pull request may close these issues.

None yet

3 participants