Compiling on Windows 10

[mrthiele]

I'm trying to compile on Windows 10 and running into an issue with HDF and was wondering if somebody can help.

• I've download the neural-fortran code via git, created the build directory, changed directory into the build directory and launched cmake .. -DSERIAL=1 and get the following:

-- Building for: Visual Studio 16 2019
-- Selecting Windows SDK version 10.0.19041.0 to target Windows 10.0.19044.
-- The C compiler identification is MSVC 19.29.30146.0
-- The Fortran compiler identification is Intel 2021.5.0.20211109
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Professional/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - done
-- Determine Intel Fortran Compiler Implicit Link Path
-- Determine Intel Fortran Compiler Implicit Link Path - done
-- Check for working Fortran compiler: C:/Program Files (x86)/Intel/oneAPI/compiler/2022.0.3/windows/bin/intel64/ifort.exe - skipped
-- Configuring build for serial execution
-- Configuring to build with -coarray=single
-- h5fortran 4.6.3  CMake 3.24.2
-- checking that C and Fortran compilers can link
-- checking that C and Fortran compilers can link - OK
-- Could NOT find HDF5 (missing: HDF5_C_LIBRARIES HDF5_links hdf5_prereqs Fortran HL) (Required is at least version "1.8.7")

• I do have HDF5 installed, but somehow it does not find it and I was unable to set a variable to make it find it. So I decided to simply rebuild as the the error message from the neural-fortran cmake says to do the following:

  HDF5 not found or working on the system.  First build HDF5 by:

    cmake -S scripts -B scripts/build -DCMAKE_INSTALL_PREFIX=~/mylibs
    cmake --build scripts/build

    Then build h5fortran:

    cmake -B build -DCMAKE_PREFIX_PATH=~/mylibs
    cmake --build build

My questions at this point are:

• Where to I save the extracted hdf5-1.12.2 directory I've downloaded and why is the cmake tool invoking a scripts directory when that directory is nowhere to be found.
• Is there a way to simply build the HDF5 externally (independently from neural-fortran) and use it when building the neural-fortran application?

Any help is much appreciated.

[milancurcic]

Thanks @mrthiele for trying neural-fortran. I don't have a direct and immediate answer for you due to my limited understanding of CMake and Windows.

As I understand it, h5fortran, which is a high-level interface to HDF5 and a dependency to neural-fortran, implements its own CMake HDF5 finder.

I think that you can set the path to your HDF5 build to the HDF5_ROOT environment variable. I think that's what CMake will use to attempt to find HDF5. At least that's how I specify my Intel build of HDF5 when I want to build neural-fortran with the Intel compiler, and it works. This is the first thing I suggest trying. I hope you know how to set an environment variable on Windows or Visual Studio, because I don't. :)

@scivision am I on the right track with the above? Anything you'd add to help @mrthiele build this?

Thanks!

[mrthiele]

@milancurcic Thanks Milan. I did try the HDF5_ROOT option (in Windows the command is set HDF_ROOT=<directory to the hdf5-1.12.2 directory but no luck.

I've been able to make some progress however:

I'm able to create to build directory in hdf5-directory:

cmake -S ..\..\hdf5-1.12.2 -B ..\..\hdf5-1.12.2\build -DCMAKE_INSTALL_PREFIX=~/mylibs
-- Selecting Windows SDK version 10.0.19041.0 to target Windows 10.0.19044.
-- Testing maximum decimal precision for C - 15;0
-- maximum decimal precision for C var - 15
-- ....All Warnings are enabled
-- Configuring done
-- Generating done
-- Build files have been written to: C:/Users/thiel/code/NNetwork/hdf5-1.12.2/build

And if I then change directory to the C:/Users/thiel/code/NNetwork/hdf5-1.12.2/build directory and execute

cmake --build build

it does something. But if I then return to the neural-fortran\build directory to continue to build, it complains that it can't find the HDF5 stuff.

I'm guessing I'm close, but struggling to understand where things have to sit relative to each other and possibly how to set environment variables so that the build works.

Thanks for all your help. Will try any other suggestions that may come in.

[milancurcic]

@mrthiele without further progress here, I encourage you to post your question on the Fortran Discourse, which has quite a few Windows and CMake experts.

[scivision]

The HDF5 install wasn't completed:

cmake --install C:/Users/thiel/code/NNetwork/hdf5-1.12.2/build --prefix ~/hdf5-1.12.2

Then tell CMake where your HDF5 library is via:

cmake -S neutral-fortran -B neural-fortran/build -DHDF5_ROOT=~/hdf5-1.12.2 --fresh
cmake --build neural-fortran/build

[scivision]

@milancurcic I could add to neural-fortran project a small script similar to h5fortran.cmake that would auto-build HDF5 if needed.

[scivision]

Example with Windows 11, Intel oneAPI 2022.3 (Intel Fortran version 2022.2), on latest commit of neural-fortran.

> cmake -Bbuild -DHDF5_ROOT=c:/lib_intel

-- buildng for: Ninja
-- The C compiler identification is IntelLLVM 2022.2.0 with MSVC-like command-line
-- The Fortran compiler identification is IntelLLVM 2022.2.0 with MSVC-like command-line
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files (x86)/Intel/oneAPI/compiler/latest/windows/bin/icx.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - done
-- Check for working Fortran compiler: C:/Program Files (x86)/Intel/oneAPI/compiler/latest/windows/bin/ifx.exe - skipped
-- Configuring build for parallel execution
-- h5fortran 4.6.3  CMake 3.24.2
-- checking that C and Fortran compilers can link
-- checking that C and Fortran compilers can link - OK
-- Looking for H5_HAVE_FILTER_SZIP
-- Looking for H5_HAVE_FILTER_SZIP - not found
-- Looking for H5_HAVE_FILTER_DEFLATE
-- Looking for H5_HAVE_FILTER_DEFLATE - found
-- Looking for H5_HAVE_PARALLEL
-- Looking for H5_HAVE_PARALLEL - found
-- Found MPI_C: C:/Program Files (x86)/Intel/oneAPI/mpi/latest/lib/release/impi.lib (found version "3.1")
-- Found MPI_Fortran: C:/Program Files (x86)/Intel/oneAPI/mpi/latest/lib/release/impi.lib (found version "3.1")
-- Found MPI: TRUE (found version "3.1") found components: C Fortran
-- Found ZLIB: C:/lib_intel/lib/zlibstatic.lib (found version "1.2.11")
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - not found
-- Found Threads: TRUE
-- Looking for H5Pset_fapl_mpio
-- Looking for H5Pset_fapl_mpio - found
-- Performing Test HDF5_C_links
-- Performing Test HDF5_C_links - Success
-- Performing Test HDF5_Fortran_links
-- Performing Test HDF5_Fortran_links - Success
-- Found HDF5: C:/lib_intel/lib/libhdf5_hl.lib;C:/lib_intel/lib/libhdf5.lib (found suitable version "1.12.2", minimum required is "1.8.7") found components: Fortran HL
-- Found MPI: TRUE (found version "3.1") found components: Fortran
-- Found MPI: TRUE (found version "3.1") found components: C Fortran
-- Found HDF5: C:/lib_intel/lib/libhdf5_hl.lib;C:/lib_intel/lib/libhdf5.lib (found version "1.12.2") found components: Fortran
-- Configuring done
-- Generating done
-- Build files have been written to: C:/neural-fortran/build

C:\neural-fortran>cmake --build build
[200/200] Linking Fortran executable bin\sine.exe

> ctest --test-dir build
Internal ctest changing into directory: C:/neural-fortran/build
Test project C:/neural-fortran/build
      Start 12: test_cnn_from_keras
      Start 11: test_dense_network_from_keras
      Start  8: test_dense_network
      Start 13: test_conv2d_network
 1/13 Test #13: test_conv2d_network ..............   Passed    1.06 sec
      Start  2: test_input3d_layer
 2/13 Test  #8: test_dense_network ...............   Passed    1.19 sec
      Start  1: test_input1d_layer
 3/13 Test  #2: test_input3d_layer ...............   Passed    0.75 sec
      Start  5: test_maxpool2d_layer
 4/13 Test  #1: test_input1d_layer ...............   Passed    0.73 sec
      Start  4: test_conv2d_layer
 5/13 Test #12: test_cnn_from_keras ..............   Passed    4.01 sec
      Start  3: test_dense_layer
 6/13 Test  #5: test_maxpool2d_layer .............   Passed    2.04 sec
      Start  6: test_flatten_layer
 7/13 Test  #4: test_conv2d_layer ................   Passed    2.15 sec
      Start  7: test_reshape_layer
 8/13 Test #11: test_dense_network_from_keras ....   Passed    4.22 sec
      Start  9: test_io_hdf5
 9/13 Test  #6: test_flatten_layer ...............   Passed    1.18 sec
10/13 Test  #3: test_dense_layer .................   Passed    1.25 sec
11/13 Test  #7: test_reshape_layer ...............   Passed    1.11 sec
12/13 Test  #9: test_io_hdf5 .....................   Passed    1.03 sec
      Start 10: test_keras_read_model
13/13 Test #10: test_keras_read_model ............   Passed    0.52 sec

100% tests passed, 0 tests failed out of 13

[milancurcic]

@milancurcic I could add to neural-fortran project a small script similar to h5fortran.cmake that would auto-build HDF5 if needed.

Michael, I think that'd be great, assuming the auto-build would run only if an existing build of HDF5 was not found or provided. Thank you!