[Windows] Fixed build for OCC 7.6.0 and Boost 1.78, works with VS2022

[scottmudge]

This corrects the includes for the precompiled headers used on Windows to support OCC 7.6.0, and corrects a few issues presented by the new Boost v1.78 on Windows.

I have successfully compiled with the following libraries:

This software uses open source components whose copyright and other proprietary rights belong to their respective owners:

• Boost 1_78
• Coin3D 4.0.1
• Eigen3 3.4.0
• FreeType 2.11.1
• KDL
• libarea
• Open CASCADE Technology 7.6.0
• Point Cloud Library
• PyCXX 6.2.8
• Python 3.8.6+
• PySide 5.15.0
• Qt 5.15.2
• Salome SMESH
• Shiboken 5.15.0
• vtk 9.1.0
• Xerces-C 3.2.2
• Zipios++
• zlib 1.2.11

The following libraries were recompiled with VS2022:

• Boost
• FreeType 2
• OCC 7.6.0
• VTK 9.1.0

The others were re-used from the asm3 LibPack 12.5.5.

I can create an updated LibPack if desired. Some of the cmake files needed to be modified in the LibPack to ensure libraries and binaries were appropriately found with relative paths.

[realthunder]

It would be really helpful if you can provide an updated libpack for testing. BTW, can you please help compile gmsh as well? Although it's used by FEM as an external process, but since it uses OCCT, we still needs to distribute its dependencies.

[scottmudge]

Sure I can compile gmsh.

One thing to note is that I used the existing PCL version from 12.5.5. I'm not sure if it used VTK as a dependency or not when it was compiled, but since I updated VTK and the libraries may have changed, I may need to recompile PCL as well. I'll try compiling the cloud module to test whether or not I will need to.

[wsteffe]

Hi scottmudge, I think you should recompile also SMESH which depends on OCC.

[scottmudge]

Okay I will try to compile both.

@realthunder for GMSH, does it need to be the GUI version w/ dynamic linking (creating a gmsh DLL), or is the console version statically linked with no GMSH library sufficient?

[realthunder]

Console version gmsh is enough for FEM use.

I think SMESH source code is inside FreeCAD?

[scottmudge]

Alright I've compiled a console version. What example project would you recommend to verify its functionality?

And @wsteffe realthunder is correct, there is a FREECAD_USE_EXTERNAL_SMESH option for the FreeCAD cmake configuration. This allows you to use an external SMESH, but it is disabled by default, and so the bundled version is used.

[realthunder]

Try this tutorial. The first half uses gmsh for meshing.

[scottmudge]

Well I've compiled gmsh, I've installed the latest version from PyPi, gmsh works fine as a binary.

But I, for the life of me, cannot get the FEM module to load:

14:25:32  DLL load failed while importing Fem: The specified procedure could not be found.
14:26:06  Cannot create object 'Box_Mesh': (DLL load failed while importing Fem: The specified procedure could not be found.)
14:26:06  Cannot create object 'FemConstraintFixed': (DLL load failed while importing Fem: The specified procedure could not be found.)
14:26:06  Cannot create object 'FemConstraintForce': (DLL load failed while importing Fem: The specified procedure could not be found.)
14:26:06  Cannot create object 'Analysis': (DLL load failed while importing Fem: The specified procedure could not be found.)
14:26:06  Cannot create object 'CalculiXccxTools': (DLL load failed while importing Fem: The specified procedure could not be found.)
14:26:06  Cannot create object 'WarpVector': (DLL load failed while importing Fem: The specified procedure could not be found.)
14:26:06  Cannot create object 'Pipeline': (DLL load failed while importing Fem: The specified procedure could not be found.)
14:26:06  Cannot create object 'ResultMesh': (DLL load failed while importing Fem: The specified procedure could not be found.)
14:26:06  Cannot create object 'CCX_Results': (DLL load failed while importing Fem: The specified procedure could not be found.)
14:26:06  <PropShape> PropertyTopoShape.cpp(416): Pending recompute for generating element map: FemCalculixCantilever3D#Box
14:26:06  Traceback (most recent call last):
  File "C:\Program Files\FreeCAD\bin\Lib\site-packages\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "C:\Program Files\FreeCAD\Mod\Fem\femguiutils\migrate_gui.py", line 163, in exec_module
    return self.load_module(module)
  File "C:\Program Files\FreeCAD\Mod\Fem\femguiutils\migrate_gui.py", line 210, in load_module
    import femviewprovider.view_material_common
  File "C:\Program Files\FreeCAD\bin\Lib\site-packages\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "C:\Program Files\FreeCAD\Mod\Fem\femviewprovider\view_material_common.py", line 37, in <module>
    from . import view_base_femconstraint
  File "C:\Program Files\FreeCAD\bin\Lib\site-packages\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "C:\Program Files\FreeCAD\Mod\Fem\femviewprovider\view_base_femconstraint.py", line 35, in <module>
    from femviewprovider import view_base_femobject
  File "C:\Program Files\FreeCAD\bin\Lib\site-packages\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "C:\Program Files\FreeCAD\Mod\Fem\femviewprovider\view_base_femobject.py", line 38, in <module>
    import FemGui  # needed to display the icons in TreeView
  File "C:\Program Files\FreeCAD\bin\Lib\site-packages\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
<class 'ImportError'>: DLL load failed while importing FemGui: The specified procedure could not be found.

I've attached all manner of debugger I can think of, but it's failing to load some kind of symbol/method from a DLL. It doesn't tell me which it's failing to load, which is frustrating.. But I've copied every DLL I can possibly find that was generated by FreeCAD or its dependencies, and nothing is working.

In x64dbg, the ENTRY_POINT_NOT_FOUND status being emitted always seems to be associated with an address the symbol name ?Perform@BOPAlgo_Builder@@UEAAXXZ stored in register R12... but the symbol exists in the OCCT libraries compiled. So I don't know what it's complaining about.

Perhaps OCCT itself is trying to load another DLL, but it's not telling me which.

Overall quite frustrating...

[scottmudge]

I think I know the missing DLL, rebuilding to confirm.

[scottmudge]

Nevermind -- thought it may have been due to using the new LLVM OpenMP feature of MSVC, but turning that off made no difference.

[realthunder]

Have you used Dependency walker, before. There is an updated version here. You can open the fem DLL using the dependency walker. Check the missing dlls, and manually add search path to all those DLLs you can find. This'll help narrow down the problem.

[scottmudge]

Yep I ran Dependency Walker on Fem.pyd and everything else I could think of yesterday. It was the older version though (but it still appeared to work). It did not seem to find anything.

But sure enough, if I launch the Python interpreter shipped with FreeCAD and attempt to import Fem.pyd I get the same error.

Unfortunately whatever is being imported seems to be dynamically hooked in by Shiboken, so it might not be a normal PE/import-based import, leading to Dependency Walker not finding it.

I will try and analyze it more deeply in an attached Pycharm debugger later today, but my cursory attempts yesterday did not reveal much. Perhaps there is a verbose debug mode I can put python in?

[realthunder]

I don't think it has anything to do with Shiboken. Shiboken does seem to override Python import mechanism, presumably to do some hack on its own extension, but it will fall back to Python import (the original_import). Besides, the first few dll loading fail message is triggered by FreeCAD C++ code for loading the FEM extension dll.

Just a shot in the dark, are you sure you have copied your newly compiled OCCT bin folder to your FreeCAD bin folder? And it seems that you are running from C:\Program Files\FreeCAD\bin. Won't it be easier to run from the build directory, assuming you've already copied all the libpack runtimes there.

[scottmudge]

Oh I know it's not Shiboken causing the problem, but because the module is being dynamically imported (rather than set as a static dependency/PE import), apps like Dependency Walker will not correctly enumerate every actual runtime dependency. But Shiboken or not, that's just how Python works.

Yes I've copied everything from every "bin" folder for every dependency I recompiled (in addition to the normal LibPack bin folder), but still no dice.

After I step through with the debugger, I'll go through the Fem python module code and try and see what is being dynamically imported to make sure it's all there. When I attach a normal debugger to the python interpreter as it is importing function addresses with ntdll.dll, I keep seeing Perform@BOPAlgo_Builder@@UEAAXXZ in the stack right after it throws an exception for not being able to find a requested export.

Interestingly, when I look in TKBO.dll, I see a bunch of exported Perform@BOPAlgo_Builder functions, but none matching the exact PE string attempting to be imported:

And here is an import table from Netgen's nglib.dll:

To me it looks like it's attempting to import a version of

public: virtual void __cdecl BOPAlgo_Builder::Perform()

WITHOUT any required arguments, but the only one exported by TKBO.dll is:

public: virtual void __cdecl BOPAlgo_Builder::Perform(class Message_ProgressRange const & __ptr64).

So I'm wondering if I need to recompile Netgen/nglib.dll against the new version of OCCT? Or perhaps there's a bug in the OCCT code which isn't correctly exporting that function prototype in Windows? I'll need to investigate further.

[realthunder]

Right, that's the problem then. You definitely need to recompile negen.

[scottmudge]

Alright, I'll try that now.

[scottmudge]

Compiling Netgen is a bit of a pain because the CMake scripts for building it automatically download prebuilt binaries for an older version of the community edition of OCC from a separate repo, and there is no option to specify external prebuilt libraries. What an awful design.

I'm going to have to zip my own compiled versions of it, upload it to some server somewhere, and then specify the new URLs as download locations.

[realthunder]

You can check the conda netgen recipe for reference. The conda feedstock also contains a few patches you might want to check, e.g. this one.

[scottmudge]

Ah that helps!

[scottmudge]

Alright got it built, rebuilding FreeCAD now

[wsteffe]

Netgen is included in the gmsh sources and to build (together with gmsh) it is sufficient to set ON the cmake option ENABLE_NETGEN.

[scottmudge]

Indeed, however it does not produce the Python Netgen bindings, nor does it produce a dynamic library (or any library for that matter) from what I can tell, even when ENABLE_NETGEN is enabled. It appears to simply use the Netgen optimizer and links against it directly as compiled objects.

The FEM module appears to require Netgen as a library, so it was necessary to build it from source.

I got it built and the FEM module now loads, I'm just verifying everything works correctly now.

[wsteffe]

Ok @scottmudge, I see. Thanks for the clarification.

[scottmudge]

Alright so FEM loads and gmsh seems to work fine when used on its own, but when I try to create a mesh to use with FEM, I get a bunch of VTK errors out of the vtkOutputWindow:

ERROR: In C:\Development\VTK-9.1.0\Common\DataModel\vtkCellArray.cxx, line 551
vtkCellArray (000000009F9D6C40): Invalid location.

ERROR: In C:\Development\VTK-9.1.0\Common\DataModel\vtkCellArray.cxx, line 551
vtkCellArray (000000009F9D6C40): Invalid location.

The mesh also looks terrible, a bunch of disconnected triangles. However, if I mesh with gmsh alone (in Mesh workbench), the mesh looks fine.

Looking into it more deeply, it looks like a problem with SMESH/SALOME. It was recently patched for VTK 9.0, but perhaps there are more issues with 9.1? I might have to recompile a bunch of stuff for VTK 9.0 instead, unless someone can figure out how to patch this.

[scottmudge]

Alright I updated the FindSMESH.cmake and SetupSalomeSMESH.cmake scripts to appropriately find external SMESH libraries on Windows, and added precompiled version of SALOME 9.7.0 to the LibPack. The embedded version of SMESH is quite old - 7.7.1, so hopefully this will correct the VTK issues. Compiling again...

[wsteffe]

@scottmudge, I think you are making a great work building FC on VS2022.
But just to know, have you ever considered using Msys2/mingw-64 ?

I am using this platform to build my EmCAD code which also is based on QT5, OCCT, gmsh, Boost... In the past (several years ago) I tried a MS compiler but I found it much more difficult to handle. Then I discovered Msys2/mingw-64. It is much simpler. You may work in a Linux environment producing a code that runs well on windows.

[scottmudge]

Yes I've used MSYS2, CYGWIN, etc. in the past, which worked fine if their package managers actually had all of the required packages. But if something was missing which depended on a specific version of another library, which itself required a specific version of another, so and so on, you end up having to compile the entire ecosystem anyway.

I've also had a horrible time getting CMake to work right with MSYS2 or Cygwin. Invariably there is going to be some poorly written cmake script for some software library which doesn't understand how to interpret a POSIX/unix-style compiler on Windows.

I think I've almost got everything working now... had to revert to VTK-8.2. I don't think the code base is ready for 9.1, and I don't have the time to suss out the compatibility fixes to get it to work.

[scottmudge]

Okay, good news!

After recompiling everything with VTK 8.2, FEM seems to load fine.

The only remaining bug I've found is that when assigning a solid material in FEM (in the task view), every single material shows a young's modulus of 0.0, and I am unable to change them. In fact, most of the material properties are stuck at default values, and I cannot change them.

They show up correctly in the material editor, but they are not translated properly. Furthermore, they are not written out correctly to the CalculiX INP file either. All of the calculations don't look right (e.g., buckling examples), because none of the parameters are set right.

See video:

XXEeEt5tuZ.mp4

Do you know why that might be @realthunder ?

[scottmudge]

Here is the new libpack in its current state, used to compile the Fix_WindowsBuild_OCC760 branch:

https://github.com/scottmudge/FreeCAD/releases/download/libpack_occ760_win/FreeCADLibs_asm3_14122021_Exp_x64_VC17.tar.gz

[realthunder]

Just pushed two fix solving the fem unit problem. Please try again. There is still one self test failure (test_ccx_buckling_flexuralbuckling), which output the exact opposite force direction comparing to the sample output. Not sure what's the problem yet.

[scottmudge]

Thanks, I'll test now.

[scottmudge]

The UI/material task view seems to work fine now. I am getting some strange results when I run some of the test projects though, usually the buckling ones. It often produces Calculix_<name>_nan_results.

I'm not sure what the suffixed number is specifically indicating, but nan or NaN is something that makes me think something might be off.

This may be due to the issue you have noted though. Thanks for the fixes!

[realthunder]

@scottmudge It seems you've taken off your libpack. Can you put back again for me to download? Thanks!

[scottmudge]

Apologies, I had updated gmsh and the CalculiX solver, I will upload again now.

[scottmudge]

Here is the new pack: https://github.com/scottmudge/FreeCAD/releases/download/libpack_occ760_win/FreeCADLibs_asm3_22122021_Exp_x64.7z

[realthunder]

Thanks!

[realthunder]

@scottmudge I need a favor from you. Can you please rebuild netgen without the AVX2 option which is enabled by default. The necessary changes are described in this post. You can read up the thread for more context. In short, by default netgen build will generate code containing some extended CPU instruction set that's avaiable on your build machine. The resulting binary will crash on mahine with older CPU.

[scottmudge]

Sure. Is it the netgen libraries (nglib.dll, ngcore.dll) or netgen.exe?

I do not think netgen.exe is used, and in fact can be deleted from the dependency archive. It appears to be a vestige of a prior lib pack.

I did pull the latest netgen sources and rebuilt nglib.dll and ngcore.dll, without any AVX features enabled:

https://github.com/scottmudge/FreeCAD/releases/download/libpack_occ760_win/netgen_rebuilt.zip

Let me know if this works. It's possible the netgen cmake config is forcing AVX2 to be enabled without any exposed option to disable it. In which case I manually disable it.

[scottmudge]

Nevermind, I did not fully read your comment. You are right, the CMakeLists.txt for netgen does appear to auto-detect the local CPU features, and enables whatever the feature-set is on the build machine.

I've manually disabled the AVX detection, so I am rebuilding it now for generic x64 CPUs.

[scottmudge]

Alright so I had to disable the "Superbuild" cmake option in Netgen to get it to build against the already-built LibPack OCCT libraries. Otherwise it was completely rebuilding OCCT (statically) and inflating the netgen DLLs.

In-so-doing it also correctly built a new netgen.exe, so that now works as well.

FreeCAD launches and links against the new netgen libraries just fine, but it doesn't seem to generate meshes with my projects. However, even the old LibPack's Netgen (before I recompiled/updated things) did not work either.

In any case, the libraries and netgen.exe application should work now without AVX2/AVX.

I've updated the LibPack here:

https://github.com/scottmudge/FreeCAD/releases/download/libpack_occ760_win/FreeCADLibs_asm3_17012022_Exp_x64.7z

[realthunder]

Yes, it's working fine now. Thanks!

[realthunder]

Oh, BTW, how do you use the libpack? I am asking because I can't get it to work unless I create an empty file MANIFEST.db under the top directory of the libpack. Just curious how you get it to work without that special file.

[scottmudge]

MANIFEST.db? Strange, I do not have that file on my version.

What kind of cmake command are you running? Cmake version? Are you using VS2022 community?

[scottmudge]

Here is the cmake command I am using. Some of these flags are required, at least for me:

cmake .. -G "Visual Studio 17 2022" -DFREECAD_LIBPACK_DIR="C:\Development\3D\FreeCADLibs" -DBUILD_TEST=OFF -DRenderer_BUILD_STATIC_LIBRARY=ON -DCMAKE_BUILD_TYPE=Release -DFREECAD_USE_EXTERNAL_SMESH=OFF -DCMAKE_INSTALL_PREFIX="C:\Program Files\FreeCAD" -DFREECAD_USE_OCC_VARIANT="Official Version" -DFREECAD_USE_PCH=ON

Obviously change your directory and/or generator to meet your needs.

[scottmudge]

According to wmayer:

Now in order to identify your LibPack you only have to add the file MANIFEST.db to the root directory of the LibPack.

https://forum.freecadweb.org/viewtopic.php?style=1&p=295265

I'm not sure why I do not need this file, but that may explain it.

[scottmudge]

Here is the cmake script looking for this:

https://github.com/FreeCAD/FreeCAD/blob/master/cMake/FreeCAD_Helpers/FreeCADLibpackChecks.cmake

[donovaly]

Dear @scottmudge , we need a LibPack for the upcoming FreeCAD 0.20. In accordance with @realthunder , we created a new repository in FreeCAD: https://github.com/FreeCAD/FreeCAD-LibPack

I released your LibPack here: https://github.com/FreeCAD/FreeCAD-LibPack/releases/tag/2.0

We would be happy if you could help to create a LibPack for FreeCAD 0.20. I listed the requirements here: https://github.com/FreeCAD/FreeCAD-LibPack/blob/master/Requirements.txt

• I think we can start from the LibPack we used to provide FreeCAD 0.19: https://github.com/FreeCAD/FreeCAD-LibPack/releases/tag/1.0
• Then we need OCC 7.6.1 but with all the fixes collected by user BlobFish
• we need Qt 5.15.3

And that is it for now. After FreeCAD 0.20 was released we have time to work to get a newer Boost, Python 3.8.x etc.

thanks and regards on behalf of the FreeCAD team
Uwe

[scottmudge]

Hi Uwe,

Sure I can help on this project, I'm excited for 0.20.

I can migrate over my OCC fixes (most seemed to be central to precompiled header includes on Windows) and merge them with BlobFish's fixes.

Per the requirements, you are targeting Windows 7. I'll plan to use VS2019, and a Windows SDK low enough to support Windows 7. Microsoft is rapidly trying to deprecate and get rid of these SDKs to push people away from Windows 7, but I believe VS2019 still supports them.

To summarize, you need:

1. Recycle existing LibPack.
2. Update with OCC 7.6.1 (I may need to recompile a few lower dependencies to support this, but I'll re-use what I can from the existing LibPack).
3. Update Qt (precompiled).

[donovaly]

Hi Scott,

Great!

Your summary is right. We decided that FC 0.20 will be the lat one supporting Win7 and we still have a substantial amount of Win 7 users. (I know that especially smaller companies try to keep their OS as long as possible.)

I develop on Win 10 and to support Win 7, I did not need to do anything. I just have to assure that Python is 3.8.x and use MSVC 2019. The resulting binaries work for Win 7 users.

I provided this way the Windows binaries for FC 0.19.x. (I am also the release manager fro 0.19 and 0.20.)

[donovaly]

Where can/will you upload your work?
In case you have no suitable GitHub repository, I'll ask if I can give you write access to https://github.com/FreeCAD/FreeCAD-LibPack

[donovaly]

Hi Scott, we have now a LibPack read for FreeCAD 0.20:
https://github.com/FreeCAD/FreeCAD-LibPack/releases/tag/2.4