SWR AVX512 support

[oscarbg]

Phoronix found some difficulties to build AVX512 support so just asking? Perhaps fixed in 17.3 master branch?

[pal1000]

SWR AVX512 is only available in master branch.

I only publish stable releases made with Scons and Visual Studio, but nothing stops you from trying to build from a development branch / master.
Just clone mesa source code with git instead of downloading a stable release:
git clone --depth=100 --branch=master git://anongit.freedesktop.org/mesa/mesa mesa
All the other steps of build script documentation are unchanged.
Unlike Linux autotools where you can build with LLVM 6.0 SVN, Scons builds immediately break with new major LLVM version and they remain broken long after LLVM release. See Mesa bugs 100201 and 102318. This limits LLVM support to 4.0.1, so what I have is far different than Phoronix's environment. But I gave it a shot anyway, I'll post here the binaries if build succeeds.

[pal1000]

Build successful but no SWR AVX512. I am either missing a parameter or Scons build lacks AVX512 support.
Anyway it's too early to ask for this so I am closing this issue for now.

[pal1000]

Mesa 17.3 is soon to be released. It checks 2 defines to determine wherever to enable AVX512 or not as seen in this commit: https://cgit.freedesktop.org/mesa/mesa/commit/?h=17.3&id=f8a572cdf0cf7fb52348adc7862a7ffc612180ef. Unfortunately I don't have the knowledge on how to pass defines to Mesa Scons build system. This might not even be supported yet but I'll ask on mesa3d mailing lists.

Update: those 2 defines are actually for enabling/disabling of SSE and AVX optimizations applied to AVX512 build so they won't help.

[oscarbg]

would be nice if you can get AVX512 builds for Mesa 17.3..
thanks..

[pal1000]

I'm afraid I have bad news regarding swr AVX512 build. I looked at Mesa3D source code inside the Sconscript used by swr in both master and 17.3 branches and there is no define or environment import for it. This looks like it simply isn't there.
Then I looked at makefile.am and makefile.sources which are part of autools build and those have it. If you know how to build swr on linux you'd only have to append HAVE_SWR_SKX and HAVE_SWR_KNL build flags to the usual configure command. I also heard rumors about cross compilation to windows with mingw-w64 being broken which may even further confine AVX512 support to linux at the moment.

[pal1000]

I found the discussion and bug report regarding cross compilation with mingw and as an irony the driver that fails to build is swr: https://bugs.freedesktop.org/show_bug.cgi?id=101614

So to sum up:

• swr build with autotools needs mingw but it is broken as stated above;
• swr build with meson is not yet implemented, even master branch doesn't have it;
• swr build with scons lacks AVX512 support even on master branch as I pointed in my previous comment.

So the only configuration that should work is linux native build with autotools. It appears we'll have to wait even more as we completely lack upstream support at the moment.

[pal1000]

Sent an upstream feature request here:
https://bugs.freedesktop.org/show_bug.cgi?id=104166

[oscarbg]

Hi @pal1000 ,
just seen on mesa-dev "meson for SWR (on Linux) and Clover"
"https://lists.freedesktop.org/archives/mesa-dev/2017-December/179686.html"
which adds SWR support (albeit only Linux right now) to a new build generator (Meson) for Mesa which is getting mature fast..
seems SWR AVX512 is in but only for Linux also.. so hope Mesa gets Meson Windows SWR support soon (meson by itself supports generating Visual Studio projects so perhaps this Meson build files with little more work gets support)..
of course using Mingw will hit same bugs..
Finally don't know if may affect SWR AVX512 but seems Visual studio 2017 AVX512 was (is?) pretty buggy/not optimized right now.. see issue I opened here:
Mysticial/Flops#16
author reported bugs and note there have been some Visual Studio 2017 point releases (from 15.3 at the time to 15.6 preview right now) so perhaps Microsoft have fixed major ones..

BTW thanks for your 17.3 release..

[pal1000]

After a lot of work and refactoring I managed to hook basic Meson support. Unfortunately my first attempt seam to indicate that MSVC is not yet supported. I get the following error during build config phase:

Build command: meson.exe . .\build\windows-x86 --backend=vs2017 -Dplatforms=windows

The Meson build system
Version: 0.45.1
Source dir: C:\Software\DEVELO~1\projects\mesa\mesa
Build dir: C:\Software\DEVELO~1\projects\mesa\mesa\build\windows-x86
Build type: native build
Program python found: YES (C:\Software\Development\projects\mesa\py3\python.EXE)
Project name: mesa
Native C compiler: cl (msvc 19.13.26131.1)
Native C++ compiler: cl (msvc 19.13.26131.1)
Build machine cpu family: x86
Build machine cpu: x86

meson.build:349:6: ERROR: Problem encountered: Cannot build GLX support without X11 platform support and at least one OpenGL API

A full log can be found at C:\Software\DEVELO~1\projects\mesa\mesa\build\windows-x86\meson-logs\meson-log.txt

Microsoft Windows [Version 10.0.16299.371]
(c) 2017 Microsoft Corporation. All rights reserved.

C:\Software\DEVELO~1\projects\mesa\mesa>

@oscarbg If you want to check why it fails and debug it, make sure Python 3,x is either in PATH or is installed with Python launcher, then you can apply the patch called enablemeson from patches folder (git apply -v patches\enablemeson.patch) and finally run the build script.

The build script will attempt to build as you see above but then it opens an interactive command prompt to allow debugging.

The only build option I found relevant is -DGLX_USE_WINDOWSGL. Unfortunately if I use it, it is not recognized and Meson throws a warning.

[oscarbg]

Thanks for effort.. are you using Mesa 18.0 or Mesa-dev in testing?

[pal1000]

I tried with Mesa-dev, though probably the same thing happens with 18.0.

[pal1000]

This issue mostly depends on #7 .

[pal1000]

I asked on mesa-dev mailing list about Meson support with MSVC and I got this response:
https://lists.freedesktop.org/archives/mesa-dev/2018-April/193437.html

And considering Dylan Baker is on to it this will take a while considering he is also the release manager for Mesa 18.1.0. This will keep him away from Meson Windows support development on Mesa for 2 more weeks at least.

In a positive note I converted that enablemeson patch into a command line switch. It should be easier to attempt a Meson build this way. I also had the time to fix bugs that came out during Meson support implementation. The only thing left to do is to update the build script documentation as there are plenty of changes to make there. The Meson support is on hold until there is something upstream to work with. Right now there is nothing, not even on mesa master.

[oscarbg]

Thanks for all your effort..
can wait is not urgent..
I don't remember if asked before.. but will be easier to obtain SWR AVX512 Win builds if built via MingW GCC, Clang for Windows instead of MSVC?
just asking altough you aren't interested I may test other compilers if it's easier to build..

[pal1000]

I don't remember if asked before.. but will be easier to obtain SWR AVX512 Win builds if built via MingW > GCC, Clang for Windows instead of MSVC?
just asking although you aren't interested I may test other compilers if it's easier to build..

Mingw GCC can't build SWR at all regardless of what OS the build machine is running due to a GCC stack alignment bug with AVX that mostly affects Windows binaries. You can Google for info on it. As for Clang, it would be indeed interesting to see if it is possible.
On Windows, Clang runs an alternate toolset for MSVC, so you still need Visual Studio.

[pal1000]

Noticed a relevant patch on mesa-dev. Patchwork series: v1, v2. There is hope I get further with this feature request when Mesa 18.2 gets branched, until then we still have to wait for upstream support.

[oscarbg]

thanks..
I see some MSVC AVX512 SWR work..
seems these patches are already merged:
https://cgit.freedesktop.org/mesa/mesa/commit/?id=fb20ae0374425ae3aff2a50a498c7e2b428632a4

[pal1000]

Well, I got to the point where I can build Mesa3D with Meson with LLVM linking and SWR driver included using Dylan Baker's branch, unfortunately I have bad news. Enabling AVX512 support for SWR fails during build system configuration, it doesn't recognize SKX and KNL arguments which means no AVX512. Also even if you build only AVX and AVX2 you can't select SWR due to confusion in Meson build, basically GALLIUM_DRIVER=swr doesn't work for some reason.

[pal1000]

@oscarbg swr AVX512 can finally be built on Windows and it will most likely be part of Mesa 19.3 series.
I still have to add support for it in deployment tools and version information attachment script but that will be done before Mesa3D 19.3,0 release.

[oscarbg]

@pal1000 finally!! that's good news for today..

[oscarbg]

are you interested on doing a release of Mesa 19.3-RC1?

[pal1000]

Yes. but will take a while as I need to rebuild LLVM with latest Visual Studio update.
Also you should probably know swr AVX512 doesn't build on MSVC without an out-of-tree patch of which effectiveness I am not 100% sure so testing it is indeed welcome. See this issue thread for details.

[pal1000]

19.3.0-rc1 pre-release is out. Good luck.

[pal1000]

19.3..0 is built with experimental swrAVX512 support thanks to switch of MSVC binaries to Meson build and an out of tree patch. Official support is tracked here.