Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CMake build environment with Visual Studio support #2684

Closed
wants to merge 34 commits into from

Conversation

neteler
Copy link
Member

@neteler neteler commented Dec 5, 2022

Copied over from #289 by @rkanavath:

This is a very BIG pr! that adds cmake support for GRASS GIS!.

_I have been working on and off mostly for fun!. Initial work started on grass-ci fork and then I recently rebased changes against proper upstream. _

This new CMake support comes with Visual Studio support (adding a bunch of ifdef, declspec etc..). This PR allows windows users to use native C compiler and not MINGW. PR itself does not add anything to break MINGW support.

  • Autoconf build still works as normal.
  • On Windows, /gisbase is portable. So you can build grass, zip, and distribute..
    There are still some missing bits required in cmake build such as man pages, locale, make install target.

Current code can build all grass that can be run from build directory. I had tagged with "WIP".

Reviews and feedback are welcome,


Additions on top:

Replaces #289

author Rashad Kanavath <mohammedrashadkm@gmail.com> 1497210739 +0200
committer Markus Neteler <neteler@gmail.com> 1669989265 +0100

parent 464ffca
author Rashad Kanavath <mohammedrashadkm@gmail.com> 1497210739 +0200
committer Markus Neteler <neteler@gmail.com> 1669989193 +0100

parent 464ffca
author Rashad Kanavath <mohammedrashadkm@gmail.com> 1497210739 +0200
committer Markus Neteler <neteler@gmail.com> 1669989189 +0100

parent 464ffca
author Rashad Kanavath <mohammedrashadkm@gmail.com> 1497210739 +0200
committer Markus Neteler <neteler@gmail.com> 1669989180 +0100

parent 464ffca
author Rashad Kanavath <mohammedrashadkm@gmail.com> 1497210739 +0200
committer Markus Neteler <neteler@gmail.com> 1669988019 +0100

parent 464ffca
author Rashad Kanavath <mohammedrashadkm@gmail.com> 1497210739 +0200
committer Markus Neteler <neteler@gmail.com> 1669987997 +0100

parent 464ffca
author Rashad Kanavath <mohammedrashadkm@gmail.com> 1497210739 +0200
committer Markus Neteler <neteler@gmail.com> 1669987899 +0100

parent 464ffca
author Rashad Kanavath <mohammedrashadkm@gmail.com> 1497210739 +0200
committer Markus Neteler <neteler@gmail.com> 1669986420 +0100

parent 464ffca
author Rashad Kanavath <mohammedrashadkm@gmail.com> 1497210739 +0200
committer Markus Neteler <neteler@gmail.com> 1669986410 +0100

parent 464ffca
author Rashad Kanavath <mohammedrashadkm@gmail.com> 1497210739 +0200
committer Markus Neteler <neteler@gmail.com> 1669985337 +0100

parent 464ffca
author Rashad Kanavath <mohammedrashadkm@gmail.com> 1497210739 +0200
committer Markus Neteler <neteler@gmail.com> 1669984062 +0100

parent 464ffca
author Rashad Kanavath <mohammedrashadkm@gmail.com> 1497210739 +0200
committer Markus Neteler <neteler@gmail.com> 1669984050 +0100

parent 464ffca
author Rashad Kanavath <mohammedrashadkm@gmail.com> 1497210739 +0200
committer Markus Neteler <neteler@gmail.com> 1669984037 +0100

parent 464ffca
author Rashad Kanavath <mohammedrashadkm@gmail.com> 1497210739 +0200
committer Markus Neteler <neteler@gmail.com> 1669984013 +0100

inital work on grass cmake build

wip: cmake fixes

add find iconv script

fix blas error wrong target added

link with dl lib on linux

disable some modules (wip)

search whatever is fftw's inlcude dir

driver lib depends on iconv

fftw in required package

build fftw modules

add python libs

test with py3

install sqlfiles to etc/sql

install lock, clean_temp, echo to ./etc/

install libs

link with dl lib

cleanup cmake code

fix python script & lib install dirs

fixed gisbase for grass startup script

install modules to bin

add PNG as dependency to r.out.png

install __init__.py

wrong install dir for python api

install with rpath to avoid LD_LIBRARY_PATH

whitespace fixes

MSVC: check for _WIN32 to use msvc and mingw32

MSVC: disable some programs temporarily

update cmake scripts to manage thirdparty libs

disable X11 on windows

update generation of config.h

fix list of cmake depends

msvc compile fixes

use INIFINITY rather than gcc only division by zero

ignore visual studio, cmake files

use INFINITY rather than divide by zero

use _WIN32 for mingw32 and msvc

add ssize_t for msvc

msvc add _USE_MATH_DEFINES an d export dll

keep check for long long int in cache

include driver/init.c when building display drivers

use _WIN32 to build msvc and mingw32

update cmake for lib/db

link with libm only on unix

msvc: skip chmod on windows msvc

use INFINITY rather than diivide by zero

add msvc specific headers: unistd.h, dirent.h

grass moved to git... So follow that in cmake.

fix check for HAVE_LONG_LONG

include math.h only on msvc. To be discussed

void* arithmetic (GCC extension) is not allowed in msvc

disable d.mon, d.font on msvc. TBD

missing O_ACCMODE on msvc

Add unistd.h on windows

Source copied from http://stackoverflow.com/a/826027

dbmi_base: add dirent.h and dirent.c for msvc

use macro INFINITY rather than GCC divide by zero

MSVC: fix missing strings.h

use math defines on msvc

update find package of fftw

added required defines into parent scope

update setting of HAVE_* defines. 1/n

use cmake target check if possible

add odbc target on windows

update values in generated config.h

reorganize cmake option defaults, cmake flags, macros

cmake c and cxx flags include _CRT_SECURE_NO_WARNINGS to avoid a lot msdn warnings
check_target macro is a helper to reduce number of lines in include/CMakeLists.txt

simlib: min, max are already available

Find which C standard library does not provide them
And if there is one, see if grass support that compiler/platform combination
To be discussed

CMake: reorganize cmake functions, macros, find_scripts (Like a Pro)

make_script_in_dir is renamed to build_script_in_subdir

I still don't the naming of macro is clear.
One without knowledge of CMake, AutoMake or any
other build scripting  experience must be able to deduct
what is the macro/functions.

Even if there will be a header comment with 'PURPOSE' field.

fix define used in lib/rst/interp_float

fix tools/timer build

add gettimeofday windowss implemenation from postgresql

CMake: reorganize cmake functions (2)

MSVC: copy UINT64CONST(x) from postgresql

WIP: disable pyc generation with g.parser on msvc

fix ccmath complex struct for msvc

take out GCC'sim and stick to C standard

copy external/ccmath/ccmath.h to include/grass/ccmath_grass.h

CMake: enable cmake export all symbols on for shared libs

CMake: build lib/python on MSVC

fix ctypesgen.cmake
reorg cmake functions used by python
add new cmake function to copy_python_files and compile

MSVC: include sys/time.h if available or time.h

CMake: remove debug trace

CMake: minor cleanup

CMake: make gui/images

CMake: fix run_grass of locale_strings and ctypesgen

CMake: link with postgres is optional

gid_t, uid_t, pid_t exists on *nix

use find_library_suffix to switch between debug and rest

avoid touch  .stamp files and depend on generated pyc file

CMake: update definiition of HAVE_PROJ_H

include winsock.h only on msvc

CMake: remove annoying logs

for fix TODO for g.version

CMake: depend on pyc file not on a new .stamp file

CMake: fix syntax error

CMake: fix sqlite include dir variable

CMake: detect version of proj4 before activating defines

MSVC: avoid pulling min max on windows

CMake: update list of enabled modules

TODO r.watershed, fix cmake proj4 library varname

Revert "simlib: min, max are already available"

This reverts commit ab2b961.

CMake: update proj library variables (remove suffix 4)

use macro INFINITY

MSVC: fix gisinit initialized flag

export initialized using dllexport when building.
export macros has been generated by CMake's GenerateExportHeader.
cmake calls this for all libraries in build_module function.

There is a change in initialized variable decl on msvc and rest of compilers.
This point has to be discussed with other devs

MSVC: fix min, max macro stuff

update TODO, list of modules not built by cmake

CMake: update find scripts to find debug then release

update list of options.
WITH_PYTHON addded to build python bindings (default is OFF)

CMake: zlib is check with cmake target

demolocation is configured in lib/init/CMakeLists.txt

CMake: generate grass.py and grass.sh scripts (build tree)

CMake: fix startup script generation (install tree)

fix generation of startup scripts

fix g.list building 1/2

MSVC: missing regex, use PCRE (wip)

lib/init: fix startup script on linux

fix input configure_file

activate building g.remove on msvc

add cmake messages for lib/init/

include sys/time.h if not on msvc.

As we don't include grass/config.h we cannot simply check against
HAVE_SYS_TIME_H

install proj data files

geotiff_csv only required in windows

check this with devs

wip: add compile defs via interface library

fix g.version, grass_gproj with proj6

create startup shell scripts in bin

fix startup script

MSVC: configure run_grass.bat, run_python.bat

generate html docs

fix copy_python_file (used in gui/wxpython, lib/python)

build all gui/wxpython modules

wrapper scripts to build html docs

skip html-description for g.parser

update msvc target properties

fix install directory for running from binary tree

install tools for buildtree and installtree

fix build docs using cmake POST_BUILD

update mkhtml.py for cmake

copy header to binary directory using add_custom_command

install extra files in lib/gis using post_build

add copy_header as depdendency to grass_datetime

first install tools directory

find cairo debug and then release libs

add POST_BUILD target for documentation only if WITH_DOCS

build gui/wxpython, fix html description generation

MSVC: uninitialized variable

CMake: update to work with autoconf and cmake

CMake: install et copy gui/images gui/icons

CMake: fix grass version date

CMake: fix building gui (python files, docs, html)

fix dist include dir name

build docs only if requested

fix typo

fix cmake syntax errors

cmake linux fixes

check for _WIN32 define to work with msvc

fix newline at end of file

WIP: update helper cmake scripts

update copy_header target

use gisbase as dist directory for build tree

cmake: fix build for db/drivers

ignore __pycache__ directory when scanning for .py files

wip: use a gisbase as dist directory

WIP: temporary fix for find_library output variable

CMake: move wxpython cmake codes to gui/wxpython

raise ScriptError

wip wip build docs

minor cleanup

install html docs for driver db

update building html docs (wip)

add missing dependencies for v.lrs

update building python modules (wip)

use target property to check if running python script (docs)

cleanup cmake helper functions (exe, libs, python, docs)

copy strings.h and unistd.h on msvc

python files (target) depend only grass.script if building docs

fix build docs for windows and linux (wip)

seperate list of g.gui.* modules

try to make generic build docs (wip)

build docs html for not win32 (wip)

fix cmake syntax error

update pgm extension for running html description

remove temp files after docs are finished

CMake: fix IN_LIST syntax

ficx cmake syntax error

fix again linux html description for python with a main script

wip: include from config build is breaking msvc

msvc: disable db drivers (wip)

add test.raster3d.lib into NO_HTML_DESCR_TARGETS

CMake: missing endif()

x extension on running html descr

copy r.in.wms directory to etc/

fix building py modules

add grass dll directory to path windows grass.bat

debug linux build failures

cmake missing endif

set main script file only for those selected modules

cmake: fix python docs for linux and windows

create scripts directory in gisbase

cmake debug message

add missing include

add cmake find scripts for liblas, netcdf, postgresql

update proj4 detection to support 4.x, 5.x , 6.x versions

update cmake functions to build grass modules

add proj4 version defines (support 5.x+)

add options for v.in.dwg and liblas modules

activate build of modules deactivated

detecting of new 3rdd party libraries

fix i.landsat.acca on msvc

add dll export macro for iostream, dspf, calc

fix r.terraflow on msvc

use _WIN32 rather than __MINGW32__ for msvc

use infinity macro to build on msvc

missing include on msvc

copy VERSIONUMBER and license to gisbase/etc

support for multiple proj4 version

update building lib/python (except ctypes)

use INFINITY macro to work with msvc

fix building gui/wxpython/xml

use approach with cmake env command for cross platform build

void* arithmetic is not allowed in msvc

To be discussed

copy __init__.py for python/grass/

fix wrong cmake varible used

copy init py to etc/python/grass/

db/drviers: odbc, sqlite, dbf,  ogr, postgresql

check for PQCmdTuples in postgresql

add defines to be posix conformat on msvc

bring in testing using ctest (wip)

improve proj4 detection

Signed-off-by: Rashad Kanavath <mohammedrashadkm@gmail.com>

keep proj4 version string in cache (very useful later)

find optional packages quietly

use PRIMARY_DEPENDS option in build_module

generate wxpython menu xml stuff after building all executables

ogsf and nviz depends on grass_raster

update gui/wxpython build

add missing modules to build

missing v.clip

install html file is exists

cmake cleanup

reorder dependencies of gui/wxpython modules

avoid breakage in autconf build

missing file copy

fix mkhtml doc building, exe, lib, python, gui

fix find scripts on windows

fix always out of date for custom targets

fix missing optional dependds to grass gis library

fix missing math.h include

wrong path used under cmake binary directory

keep autoconf build  conflict with cmake

.bat files must be in scripts/

fix installation of gui/images, gui/icons

Revert "install html file is exists"

This reverts commit 9e83f6f.

Apply suggestions from code review

trivial changes (comment style) applied

moved tools/ -> utils/

sync to main

sync to main

fix indentation

remove trailing white space

remove trailing white space

revert C related INFINITY/NAN changes (taken care of in OSGeo#2681)
@neteler neteler added the windows Microsoft Windows specific label Dec 5, 2022
@neteler neteler added this to the 8.3.0 milestone Dec 5, 2022
@neteler
Copy link
Member Author

neteler commented Dec 5, 2022

TODO: compare if anything is to be picked from PR "Cmake build2" in #348

@neteler neteler added the help wanted Extra attention is needed label Dec 5, 2022
@neteler neteler mentioned this pull request Dec 5, 2022
@neteler
Copy link
Member Author

neteler commented Dec 5, 2022

Congrats again to @rkanavath for his great work in preparing all the CMake files!

Current state: after some edits (in this PR) configuration seems to mostly work.

rm -rf build/ ; mkdir build
cd build/
cmake ..
-- The C compiler identification is GNU 12.2.1
-- The CXX compiler identification is GNU 12.2.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/lib64/ccache/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/lib64/ccache/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- GRASS_VERSION_NUMBER  =  '8.3.dev'
-- Found Git: /usr/bin/git (found version "2.38.1") 
-- Found FLEX: /usr/bin/flex (found version "2.6.4") 
-- Found BISON: /usr/bin/bison (found version "3.8.2") 
-- Found PROJ: /usr/lib64/libproj.so  
-- Found GDAL: /usr/lib64/libgdal.so  
-- Found ZLIB: /usr/lib64/libz.so (found version "1.2.11") 
-- Found PNG: /usr/lib64/libpng.so (found version "1.6.37") 
-- Found JPEG: /usr/lib64/libjpeg.so (found version "62") 
-- Found Freetype: /usr/lib64/libfreetype.so (found version "2.12.1") 
-- Found fftw3.h in /usr/include
-- Found FFTW: /usr/lib64/libfftw3.so;/usr/lib64/libfftw3_threads.so;/usr/lib64/libfftw3f.so;/usr/lib64/libfftw3f_threads.so  
-- Found Cairo: /usr/include/cairo;/usr/include  
-- Found X11: /usr/include   
-- Looking for XOpenDisplay in /usr/lib64/libX11.so;/usr/lib64/libXext.so
-- Looking for XOpenDisplay in /usr/lib64/libX11.so;/usr/lib64/libXext.so - found
-- Looking for gethostbyname
-- Looking for gethostbyname - found
-- Looking for connect
-- Looking for connect - found
-- Looking for remove
-- Looking for remove - found
-- Looking for shmat
-- Looking for shmat - found
-- Looking for IceConnectionNumber in ICE
-- Looking for IceConnectionNumber in ICE - found
-- Found TIFF: /usr/lib64/libtiff.so (found version "4.4.0")  
-- Performing Test ICONV_SECOND_ARGUMENT_IS_CONST
-- Performing Test ICONV_SECOND_ARGUMENT_IS_CONST - Failed
-- Found Iconv: /usr/lib64/libc.so
-- Found BZip2: /usr/lib64/libbz2.so (found version "1.0.8") 
-- Looking for BZ2_bzCompressInit
-- Looking for BZ2_bzCompressInit - found
-- Looking for sgemm_
-- Looking for sgemm_ - not found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
-- Looking for sgemm_
-- Looking for sgemm_ - found
-- Found BLAS: /usr/lib64/libflexiblas.so  
-- Looking for cheev_
-- Looking for cheev_ - found
-- Found LAPACK: /usr/lib64/libflexiblas.so;-lm;-ldl  
CMake Warning (dev) at /usr/share/cmake/Modules/FindOpenGL.cmake:315 (message):
  Policy CMP0072 is not set: FindOpenGL prefers GLVND by default when
  available.  Run "cmake --help-policy CMP0072" for policy details.  Use the
  cmake_policy command to set the policy and suppress this warning.

  FindOpenGL found both a legacy GL library:

    OPENGL_gl_LIBRARY: /usr/lib64/libGL.so

  and GLVND libraries for OpenGL and GLX:

    OPENGL_opengl_LIBRARY: /usr/lib64/libOpenGL.so
    OPENGL_glx_LIBRARY: /usr/lib64/libGLX.so

  OpenGL_GL_PREFERENCE has not been set to "GLVND" or "LEGACY", so for
  compatibility with CMake 3.10 and below the legacy GL library will be used.
Call Stack (most recent call first):
  thirdparty/CMakeLists.txt:112 (find_package)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Found OpenGL: /usr/lib64/libOpenGL.so   
-- Found PostgreSQL: /usr/lib64/libpq.so (found version "14.1")  
CMake Warning (dev) at /usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:438 (message):
  The package name passed to `find_package_handle_standard_args` (SQLITE)
  does not match the name of the calling package (SQLite).  This can lead to
  problems in calling code that expects `find_package` result variables
  (e.g., `_FOUND`) to follow a certain pattern.
Call Stack (most recent call first):
  cmake/find_scripts/FindSQLite.cmake:8 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
  thirdparty/CMakeLists.txt:126 (find_package)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Found SQLITE: /usr/lib64/libsqlite3.so  
-- Found GEOS: /usr/lib64/libgeos_c.so  
-- Found PythonInterp: /usr/bin/python (found version "3.10.8") 
-- Looking for limits.h
-- Looking for limits.h - found
-- Looking for termio.h
-- Looking for termio.h - found
-- Looking for termios.h
-- Looking for termios.h - found
-- Looking for unistd.h
-- Looking for unistd.h - found
-- Looking for values.h
-- Looking for values.h - found
-- Looking for sys/ioctl.h
-- Looking for sys/ioctl.h - found
-- Looking for sys/mtio.h
-- Looking for sys/mtio.h - found
-- Looking for sys/resource.h
-- Looking for sys/resource.h - found
-- Looking for sys/time.h
-- Looking for sys/time.h - found
-- Looking for time.h
-- Looking for time.h - found
-- Looking for sys/timeb.h
-- Looking for sys/timeb.h - found
-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for sys/utsname.h
-- Looking for sys/utsname.h - found
-- Looking for g2c.h
-- Looking for g2c.h - not found
-- Looking for f2c.h
-- Looking for f2c.h - not found
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for regex.h
-- Looking for regex.h - found
-- Looking for libintl.h
-- Looking for libintl.h - found
-- Looking for langinfo.h
-- Looking for langinfo.h - found
-- Looking for dbm.h
-- Looking for dbm.h - not found
-- Performing Test HAVE_PQCMDTUPLES - Success
-- Performing Test HAVE_LONG_LONG_INT
-- Performing Test HAVE_LONG_LONG_INT - Success
-- Performing Test TIME_WITH_SYS_TIME
-- Performing Test TIME_WITH_SYS_TIME - Success
-- Looking for ftime
-- Looking for ftime - found
-- Looking for gethostname
-- Looking for gethostname - found
-- Looking for gettimeofday
-- Looking for gettimeofday - found
-- Looking for lseek
-- Looking for lseek - found
-- Looking for time
-- Looking for time - found
-- Looking for uname
-- Looking for uname - found
-- Looking for seteuid
-- Looking for seteuid - found
-- Looking for setpriority
-- Looking for setpriority - found
-- Looking for setreuid
-- Looking for setreuid - found
-- Looking for setruid
-- Looking for setruid - not found
-- Looking for setpgrp
-- Looking for setpgrp - found
-- Looking for drand48
-- Looking for drand48 - found
-- Looking for nanosleep
-- Looking for nanosleep - found
-- Looking for asprintf
-- Looking for asprintf - found
-- Looking for putenv
-- Looking for putenv - found
-- Looking for setenv
-- Looking for setenv - found
-- Looking for socket
-- Looking for socket - found
-- Performing Test HAVE_PBUFFERS - Success
-- Performing Test HAVE_PIXMAPS - Success
-- Looking for fseeko
-- Looking for fseeko - found
-- Creating /home/mneteler/software/grass_main/build/include/grass/config.h
-- Performing Test COMPILER_HAS_HIDDEN_VISIBILITY
-- Performing Test COMPILER_HAS_HIDDEN_VISIBILITY - Success
-- Performing Test COMPILER_HAS_HIDDEN_INLINE_VISIBILITY
-- Performing Test COMPILER_HAS_HIDDEN_INLINE_VISIBILITY - Success
-- Performing Test COMPILER_HAS_DEPRECATED_ATTR
-- Performing Test COMPILER_HAS_DEPRECATED_ATTR - Success
-- GISBASE (install) : /usr/local
-- grass config directory (install): /home/mneteler/.grass8
-- GISBASE (build) : /home/mneteler/software/grass_main/build/gisbase
-- grass config directory (build): /home/mneteler/software/grass_main/build/gisbase/etc/config
-- env variables: /home/mneteler/software/grass_main/build/gisbase/etc/config/bashrc
-- Startup (build): /home/mneteler/software/grass_main/build/gisbase/grass.sh
-- gisrc (build): /home/mneteler/software/grass_main/build/gisbase/etc/config/rc
-- odbc disabled because ODBC is not available
[build_module] ADDING TEST r.info-test
-- r.in.lidar disabled because LIBLAS is not available
-- r3.in.lidar disabled because LIBLAS is not available
-- v.in.lidar disabled because LIBLAS is not available
-- v.lidar.correction disabled because LIBLAS is not available
-- v.lidar.edgedetection disabled because LIBLAS is not available
-- v.lidar.growing disabled because LIBLAS is not available
-- v.out.lidar disabled because LIBLAS is not available
-- Configuring done
-- Generating done
-- Build files have been written to: /home/mneteler/software/grass_main/build

TODO:

  • PDAL yet missing
  • compilation error:
make
[  0%] Copy /home/mneteler/software/grass_main/include/grass/arraystats.h to /home/mneteler/software/grass_main/build/include/grass/grass/arraystats.h
[  0%] Copy /home/mneteler/software/grass_main/include/grass/bitmap.h to /home/mneteler/software/grass_main/build/include/grass/grass/bitmap.h
[  0%] Copy /home/mneteler/software/grass_main/include/grass/blas.h to /home/mneteler/software/grass_main/build/include/grass/grass/blas.h
[  0%] Copy /home/mneteler/software/grass_main/include/grass/btree.h to /home/mneteler/software/grass_main/build/include/grass/grass/btree.h
[  0%] Copy /home/mneteler/software/grass_main/include/grass/calc.h to /home/mneteler/software/grass_main/build/include/grass/grass/calc.h
[  0%] Copy /home/mneteler/software/grass_main/include/grass/cdhc.h to /home/mneteler/software/grass_main/build/include/grass/grass/cdhc.h
[  0%] Copy /home/mneteler/software/grass_main/include/grass/cluster.h to /home/mneteler/software/grass_main/build/include/grass/grass/cluster.h
[  0%] Copy /home/mneteler/software/grass_main/include/grass/colors.h to /home/mneteler/software/grass_main/build/include/grass/grass/colors.h
[  0%] Copy /home/mneteler/software/grass_main/include/grass/datetime.h to /home/mneteler/software/grass_main/build/include/grass/grass/datetime.h
[  0%] Copy /home/mneteler/software/grass_main/include/grass/dbmi.h to /home/mneteler/software/grass_main/build/include/grass/grass/dbmi.h
...
[  2%] Copy /home/mneteler/software/grass_main/include/grass/vedit.h to /home/mneteler/software/grass_main/build/include/grass/grass/vedit.h
make[2]: *** No rule to make target 'include/LIB_PYTHON', needed by 'include/CMakeFiles/copy_header'.  Stop.
make[1]: *** [CMakeFiles/Makefile2:2461: include/CMakeFiles/copy_header.dir/all] Error 2
make: *** [Makefile:146: all] Error 2

Since I don't know anything about CMake I seek support to improve this PR. Edits welcome.

@nilason
Copy link
Contributor

nilason commented Dec 5, 2022

@neteler Great job to return this to life again!

neteler added a commit to neteler/grass that referenced this pull request Dec 6, 2022
@dnewcomb
Copy link
Contributor

dnewcomb commented Dec 6, 2022 via email

Copy link
Member

@echoix echoix left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I cannot add a suggestion to a non-existing file, but there is missing a CMakeLists.txt file at python/CMakeLists.txt, with contents to have cmake look in the subfolders' CMakeLists.txt, something like the following (that's what I used):

add_subdirectory(grass)
add_subdirectory(libgrass_interface_generator)

It doesn't fix everything, but at least it's not the same problem anymore. It is enough for tonight's testing

.github/workflows/cmake.yml Outdated Show resolved Hide resolved
.github/workflows/cmake.yml Outdated Show resolved Hide resolved
CMakeLists.txt Show resolved Hide resolved
CMakeLists.txt Outdated Show resolved Hide resolved
python/grass/CMakeLists.txt Outdated Show resolved Hide resolved
python/grass/CMakeLists.txt Show resolved Hide resolved
python/grass/CMakeLists.txt Show resolved Hide resolved
python/grass/CMakeLists.txt Outdated Show resolved Hide resolved
@echoix
Copy link
Member

echoix commented Jan 2, 2023

I'm still a CMake newbie, but I overall find it weird that the long commands calling a nested cmake call to copy every python files are really the cmake-est way to do that. I don't understand well enough what the targets are and how they can be used, but isn't it a better way? or the file(COPY), or even file(COPY_FILE), but it is only one by one.

I also struggle about understanding the implications of the GISBASE folder. Is it needed in the build tree, or could we consider that having the GISBASE could be made with the CMake commands used for the install concept?

And lastly, of the reasons I couldn't advance and continue suggesting fixes to unblock the issue, is that I have a hard time understanding/reading the makefiles. It's been a long time since I used them, but there are some pretty dense constructs that are really makefile specific. Like:

$(DSTDIR):
	$(MKDIR) $@

$(DSTDIR)/%: % | $(DSTDIR)
	$(INSTALL_DATA) $< $@

(from

$(DSTDIR):
$(MKDIR) $@
$(DSTDIR)/%: % | $(DSTDIR)
$(INSTALL_DATA) $< $@
)

Co-authored-by: Edouard Choinière <27212526+echoix@users.noreply.github.com>
.github/workflows/cmake.yml Outdated Show resolved Hide resolved
neteler and others added 2 commits January 2, 2023 09:48
Co-authored-by: Edouard Choinière <27212526+echoix@users.noreply.github.com>
@neteler
Copy link
Member Author

neteler commented Jan 4, 2023

First the "Conflicting files" need to be fixed... perhaps @nilason can support us here?

I have now manually ported the clang-format changes into this PR using meld, submitted in ea058ff (clang-format itself didn't yield any effect, for whatever reason).

Now the CI is running again.

@neteler
Copy link
Member Author

neteler commented Jan 4, 2023

Well, error: code should be clang-formatted [-Wclang-format-violations] still occur, see CI.
I have no clue how to convince my local clang-format tool to do the job. What I tried

  • cp .clang-format from main to ../
  • run clang-format on files in question
  • no changes happen

What's the trick? (this is relevant for most open PRs).

@nilason
Copy link
Contributor

nilason commented Jan 4, 2023

Well, error: code should be clang-formatted [-Wclang-format-violations] still occur, see CI. I have no clue how to convince my local clang-format tool to do the job. What I tried

  • cp .clang-format from main to ../
  • run clang-format on files in question
  • no changes happen

What's the trick? (this is relevant for most open PRs).

it is crucial to add the -i flag for inline modification: clang-format -i <file>.

@neteler
Copy link
Member Author

neteler commented Jan 4, 2023

Yes, this I did.

@nilason
Copy link
Contributor

nilason commented Jan 4, 2023

Yes, this I did.

... and no changes?

@nilason
Copy link
Contributor

nilason commented Jan 4, 2023

It is also important to cd to be within grass source directory, before running clang-format.

You could also try with --verbose:
eg. clang-format -i --verbose lib/manage/do_list.c.

If you wish, I can do the formatting and push here.

@neteler
Copy link
Member Author

neteler commented Jan 4, 2023

If you wish, I can do the formatting and push here.

Yes please.

(I'd suggest to have in future something similar to https://github.com/OSGeo/gdal/blob/master/scripts/clang-format.sh)

@nilason
Copy link
Contributor

nilason commented Jan 4, 2023

If you wish, I can do the formatting and push here.

Yes please.

Done.

(I'd suggest to have in future something similar to https://github.com/OSGeo/gdal/blob/master/scripts/clang-format.sh)

I did look at that and I think it would be a great to make a pre-commit hook with not only clang-format, but also black and flake8. In a similar fashion to https://github.com/OSGeo/gdal/blob/master/.pre-commit-config.yaml. However, that won't help with forwarding 'old' branches.

@nilason
Copy link
Contributor

nilason commented Jan 4, 2023

If you wish, I can do the formatting and push here.

Yes please.

Done.

(I'd suggest to have in future something similar to https://github.com/OSGeo/gdal/blob/master/scripts/clang-format.sh)

I did look at that and I think it would be a great to make a pre-commit hook with not only clang-format, but also black and flake8. In a similar fashion to https://github.com/OSGeo/gdal/blob/master/.pre-commit-config.yaml. However, that won't help with forwarding 'old' branches.

All the same, we need to replace https://github.com/OSGeo/grass/blob/main/utils/grass_indent.sh and https://github.com/OSGeo/grass/blob/main/utils/grass_indent_ALL.sh anyway.

neteler and others added 3 commits January 4, 2023 19:57
Co-authored-by: Edouard Choinière <27212526+echoix@users.noreply.github.com>
Co-authored-by: Edouard Choinière <27212526+echoix@users.noreply.github.com>
.github/workflows/pylint.yml Outdated Show resolved Hide resolved
- name: Configure
run: |
mkdir build
cmake ${CMAKE_OPTIONS} -S $GITHUB_WORKSPACE -B $GITHUB_WORKSPACE/build
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

... this still fails in the CI (using CMake 3.11.4). It still a more recent CMake version needed? (I cannot find any more the related comment by @echoix)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#2684 (comment)

The default version in Ubuntu 20.04 is 3.16.x, and GitHub runners already include 3.25.1.
That was a "new"-ish recommended syntax, but introduced in 3.13

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which version to be used, do you have any recommendation (e.g., we may try 3.13 - or later)?

@echoix
Copy link
Member

echoix commented Jan 4, 2023

I have a branch where I could continue working and debugging. The PR is a long way to be functional and useful. It is missing some steps, folders, and I was stuck yesterday at a problem where the headers in the include/grass aren't copied over to the required location, and thus a .c code cannot include <grass/gis.h>. Between CMake 3.9 when the work was started and more recent like 3.18 (today is 3.25.1), what is considered modern CMake has changed a bit, and can be cleaner and less hacky-er.

The situation now is that since most of the operations are done as a copy file operations, more like Makefiles, CMake is not really made aware of the dependencies of each target, what it provides, what should be private or public, etc., this prevents to order the compilation of targets. And it is quite hard to follow, along to be honest.

I tried to re-learn yesterday on how should the includes be defined, and got lost.

@wenzeslaus wenzeslaus modified the milestones: 8.3.0, 8.4.0 Feb 10, 2023
@marisn
Copy link
Contributor

marisn commented Jun 3, 2023

I tried to compile this PR on my Debian testing with cmake version 3.25.1 but it all failed as only a few .h files were copied to the include folder.

@hellik
Copy link
Member

hellik commented Jun 4, 2023

some (ugly hacky) windows testing:

MS Visual Studio 2022 Community version:

  • open an MS windows console: set PATH=%PATH%;C:\msys64\usr\bin;C:\OSGeo4W\bin;C:\msys64\mingw64\bin
  • in CMakeLists.txt add:
include_directories("C:/OSGeo4W/include")
target_link_libraries("C:/msys64/mingw64/bin")
target_link_libraries("C:/msys64/mingw64/lib")
target_link_libraries("C:/OSGeo4W/bin")
target_link_libraries("C:/OSGeo4W/lib")
In order to link to them from Visual C++, you will need to create .lib "import libraries" using the [lib.exe](https://msdn.microsoft.com/en-us/library/7ykb2k5f.aspx) program included with VC++. Run:

     lib /def:libfftw3-3.def
     lib /def:libfftw3f-3.def
     lib /def:libfftw3l-3.def
  • in grass-cmake_build\CMakeLists.txt add:
set(FFTW_INCLUDE_DIR "C:/msys64/mingw64/include")
set(FFTW_ROOT "D:/testcmake/fftw-3.3.5-dll64")
  • go to https://github.com/egpbos/findFFTW/blob/master/FindFFTW.cmake and replace find_scripts\FindFFTW.cmake with that downloaded FindFFTW.cmake

  • in the MS windows console do cd C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE and start devenv.exe and open the folder of source of this PR

then next dependency issue to solve with CAIRO

@hellik
Copy link
Member

hellik commented Jun 4, 2023

some preliminary findings for windows:

Compiling GRASS GIS on Windows depends on libraries from OSGeo4W and MSYS2, it is needed to configure CMake to find and link against those libraries.

e.g.

cmake_minimum_required(VERSION 3.12)
project(GRASSGIS VERSION 8.2.0)

# Add source files
set(SOURCES
    src/main.c
    src/other_file.c
    # Add more source files as needed
)

# Add header files
set(HEADERS
    include/your_header.h
    # Add more header files as needed
)

# Set include directories
include_directories(
    include
    ${GRASS_INCLUDE_DIRS}
    # Add more include directories as needed
)

# Set library directories
link_directories(
    ${OSGEO4W_ROOT}/lib
    ${MSYS2_ROOT}/mingw64/lib
    # Add more library directories as needed
)

# Create the executable
add_executable(grass_gis ${SOURCES} ${HEADERS})

# Set compiler flags and options
target_compile_options(grass_gis PRIVATE
    -Wall
    -Wextra
    # Add more compiler flags as needed
)

# Link libraries
target_link_libraries(grass_gis
    ${GRASS_LIBRARIES}
    # Add more libraries to link against as needed
    # For example: gdal, geos, proj, etc.
)

# Set build type (Debug or Release)
if(NOT CMAKE_BUILD_TYPE)
    set(CMAKE_BUILD_TYPE Release)
endif()
Added link_directories command to specify the library directories where OSGeo4W and MSYS2 libraries are located. Replace ${OSGEO4W_ROOT} and ${MSYS2_ROOT} with the actual paths to OSGeo4W and MSYS2 installations, respectively.

Updated the target_link_libraries command to link against the required libraries from OSGeo4W and MSYS2, such as GDAL, GEOS, Proj, etc. Make sure to add the necessary libraries to this section.

 Make sure that the necessary include directories are set correctly, either by using the ${GRASS_INCLUDE_DIRS} variable or by specifying the paths directly.

To define the ${OSGEO4W_ROOT} variable, there are a few options depending on your setup. Here are a couple of approaches you can take:

    Set the variable as an environment variable:

        Open a command prompt or terminal.

        Run the following command to set the environment variable:

        shell

    set OSGEO4W_ROOT=<path_to_osgeo4w_root_directory>

    Replace <path_to_osgeo4w_root_directory> with the actual path to your OSGeo4W root directory.

    Alternatively, set the environment variable through the Windows system properties:
        Open the Start menu and search for "Environment Variables."
        Select "Edit the system environment variables."
        In the System Properties window, click the "Environment Variables" button.
        Under "System variables," click the "New" button.
        Enter OSGEO4W_ROOT as the variable name and the path to your OSGeo4W root directory as the variable value.
        Click "OK" to save the changes.

Specify the variable directly in the CMake command:

    When running the cmake command, specify the OSGeo4W root directory directly using the -D option:

        cmake -DOSGEO4W_ROOT=<path_to_osgeo4w_root_directory> .

        Replace <path_to_osgeo4w_root_directory> with the actual path to the OSGeo4W root directory.
If compiling GRASS GIS project using Microsoft Visual Studio,  generating a Visual Studio solution file using CMake. A possible workflow for compiling GRASS GIS with Visual Studio may be:

    Open a command prompt or terminal.

    Navigate to the root directory of GRASS GIS source.

    Create a build directory (e.g., build) within the project directory:

mkdir build

Navigate to the build directory:

cd build

Generate the Visual Studio solution using CMake, specifying the generator as "Visual Studio" and the optional OSGEO4W_ROOT variable:

cmake -G "Visual Studio" -DOSGEO4W_ROOT=<path_to_osgeo4w_root_directory> ..

Replacing <path_to_osgeo4w_root_directory> with the actual path to the OSGeo4W root directory.

Opening the generated Visual Studio solution (GRASSGIS.sln) from the build directory in Microsoft Visual Studio.

In Visual Studio, building and compiling GRASS GIS by selecting the desired configuration (e.g., Debug or Release) and using the Build menu or toolbar buttons.
If CMake is not able to find the FFTW3 library even though the link directory is defined, there are a few potential solutions:

    Specifying the include directory: In addition to the link directory, providing the include directory where CMake can find the FFTW3 header files. Update your CMakeLists.txt to include the following line:

include_directories(${FFTW3_INCLUDE_DIR})

Make sure to replace ${FFTW3_INCLUDE_DIR} with the actual path to the FFTW3 include directory.

Set the FFTW3_ROOT variable: Explicitly set the FFTW3_ROOT variable in the CMake command to point to the FFTW3 root directory. For example:

cmake -G "Visual Studio" -DOSGEO4W_ROOT=<path_to_osgeo4w_root_directory> -DFFTW3_ROOT=<path_to_fftw3_root_directory> ..

Replacing <path_to_fftw3_root_directory> with the actual path to theFFTW3 root directory.

Specify the library name explicitly: If CMake is still unable to find the FFTW3 library, explicitly specify the library name in the target_link_libraries command. Updating the CMakeLists.txt to include the following line:

target_link_libraries(grass_gis ${FFTW3_LIBRARY})

Replacing ${FFTW3_LIBRARY} with the actual name of the FFTW3 library file (e.g., fftw3.lib, libfftw3.a, etc.).
if the issue persists with CMake not finding the FFTW3 library, try the following steps:

    Specify the library directory: In addition to the link directory, provide the full path to the directory where the FFTW3 library file is located using the link_directories command in your CMakeLists.txt. 

link_directories(${FFTW3_LIBRARY_DIR})

Replace ${FFTW3_LIBRARY_DIR} with the actual path to the directory containing the FFTW3 library file.

Specify the library name explicitly: If CMake is still unable to find the FFTW3 library, explicitly specify the library name in the target_link_libraries command. Update your CMakeLists.txt to include the following line:

target_link_libraries(grass_gis fftw3)

Replacing fftw3 with the actual name of the FFTW3 library file without the file extension.

Provide the full library path: Instead of using target_link_libraries, you can provide the full path to the FFTW3 library directly using the -l flag in the target_link_options command. Update your CMakeLists.txt to include the following line:

target_link_options(grass_gis PRIVATE -L${FFTW3_LIBRARY_DIR} -l${FFTW3_LIBRARY})

Replace ${FFTW3_LIBRARY_DIR} with the actual path to the directory containing the FFTW3 library file, and ${FFTW3_LIBRARY} with the actual name of the FFTW3 library file without the file extension.
In the FFTW3_ROOT variable, specify the root directory of the FFTW3 installation. This refers to the directory containing the bin, lib, and include directories of FFTW3.

When specifying the FFTW3_ROOT variable, providing the path to the root directory itself, not the bin or lib subdirectories.

For example, if your FFTW3 installation is located at C:/path/to/fftw3, then you would set the FFTW3_ROOT variable as follows:

cmake -G "Visual Studio" -DOSGEO4W_ROOT=<path_to_osgeo4w_root_directory> -DFFTW3_ROOT=C:/path/to/fftw3 ..

By providing the root directory, CMake will automatically search for the bin, lib, and include directories within the specified FFTW3_ROOT.

By setting the FFTW3_ROOT variable correctly, CMake should be able to locate the necessary libraries and headers within the specified root directory.
If CMake is unable to find the Cairo library:

    Specify the include directory: In addition to the link directory, the include directory where CMake can find the Cairo header files. Updating CMakeLists.txt to include the following line:

include_directories(${CAIRO_INCLUDE_DIRS})

replacing ${CAIRO_INCLUDE_DIRS} with the actual path to the Cairo include directory.

Specify the library directory: Using the link_directories command to specify the directory where the Cairo library file is located:

link_directories(${CAIRO_LIBRARY_DIRS})

Specify the library name explicitly: If CMake is still unable to find the Cairo library, explicitly specify the library name in the target_link_libraries command. Update your CMakeLists.txt to include the following line:

target_link_libraries(grass_gis cairo)

Replacing cairo with the actual name of the Cairo library file without the file extension.
When specifying the library name for Cairo in the target_link_libraries command, provide the name of the Cairo library file without the file extension. In the case of Cairo on Windows, the library file is typically named cairo.dll.

if the Cairo library file is named cairo.dll:

target_link_libraries(grass_gis cairo)

Te Cairo library file cairo.dll should be located in one of the directories specified in the link_directories command, or that the directory containing cairo.dll is included in the system's PATH environment variable.

@lbartoletti lbartoletti mentioned this pull request Jun 5, 2023
21 tasks
@HuidaeCho
Copy link
Member

I took the liberty to close this conflicting PR. We'll continue this effort in #3021.

@HuidaeCho HuidaeCho closed this Oct 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed windows Microsoft Windows specific
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

9 participants