Skip to content

/OpenTTD

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce CMake (and removing all other project-related code) #7270

Open
wants to merge 8 commits into
base: master
from

Conversation

Reviewers

@nielsmh nielsmh

@glx22 glx22

@LordAro LordAro

@DerDakon DerDakon

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
9 participants
@TrueBrain @nielsmh @glx22 @orudge @stormcone @janisozaur @PeterN @LordAro @DerDakon
@TrueBrain
Copy link
Member

TrueBrain commented Feb 23, 2019 •
edited

Fixes #6884
Fixes #7470

Stable release build
Nightly release build

This removes:

  • MSVC project files
  • Kdevelop generation files
  • configure / Makefile

It replaces them all with a single method: cmake. This should also remove any awk/vbs scripts we have around, meaning we will get a single method of doing stuff.

Currently tested with these 'IDEs':

  • VS2017 (yes, VS supports CMake)
  • VS2019 (yes, VS2019 supports CMake even better)
  • CMake, WSL Ubuntu 18.04

This is pretty much a Work In Progress, but I wanted to show others how it would look etc, so they can also test already a bit. Any feedback is welcome.

To test it out:

mkdir build
cd build
cmake ..
make

Or open the CMakeLists.txt in VS2017 / VS2019.

Things still to do:

  • Debian dependencies
  • rpm support
  • Linux install: menu entries
  • check if we are autodetecting everything
  • check all TODOs, and make sure they are fixed
  • validate this works on all current machines of all people that can compile OpenTTD now
  • validate this works on things like CLion, Visual Code Studio, Kdevelop, ..
  • rewrite Squirrel generation in CMake (done in a branch)
  • add the ability to make stable builds (NO_ASSERT, see MSVC changes for stable releases) via parameters
  • autodetect nforenum
  • autodetect grfcodec
  • add ICC support
  • add all the warning flags etc
  • add debug options
  • rewrite bundling code in CMake
  • rewrite baseset generation in CMake
  • rewrite regression in CMake
  • rewrite source.list in CMake
  • autodetect ICU
  • autodetect TiMiditity (libtimidity is no longer available on most OSes)
  • autodetect xdg-basedir
  • autodetect iconv
  • autodetect OSX stuff (Cocoa QuickDraw, Quartz, libraries, etc)
  • rewrite findversion in CMake (partially done here: LordAro@1ea407b)
    Currently it is calling findversion.sh, which only works on non-Windows. Porting it to CMake makes more sense (one single way of doing stuff).
  • add static builds (no static builds for now; we might reconsider later)
  • add CMake support for all CI targets
  • revisit dependencies for Dedicated Server (does it need SDL, Cocoa, ..)
  • fix CI for Linux targets
  • fix CI for Windows
  • fix CI for OSX
  • fix mingw
  • fix dedicated doesn't need a video driver
  • split CMakeLists.txt inside several files to increase readability
  • remove all old project files (including configure, Makefile, projects/, etc)
  • restore projects\os_versions.manifest usage in VS builds

(I am sure I am missing stuff .. but this is something to work on at least :D)

@TrueBrain TrueBrain force-pushed the TrueBrain:cmake branch from 2a97e01 to f6cd1ad Feb 23, 2019

@glx22

glx22 reviewed Feb 24, 2019
View changes

Show resolved Hide resolved CMakeLists.txt Outdated
@nielsmh

This comment has been minimized.

Sign in to view
Copy link
Contributor

nielsmh commented Feb 24, 2019

If CMake becomes the only (meta) build system, wouldn't it make sense to scrap the source.list file as well, and define that data via CMake syntax instead?

@TrueBrain TrueBrain force-pushed the TrueBrain:cmake branch from cfff2fc to a891bd6 Feb 24, 2019

@glx22

This comment has been minimized.

Sign in to view
Copy link
Contributor

glx22 commented Feb 24, 2019

For MinGW directmusic and xaudio2 needs to be detected. MinGW has directmusic headers, but doesn't have xaudio2.
@TrueBrain

This comment has been minimized.

Sign in to view
Copy link
Member Author

TrueBrain commented Feb 24, 2019

Added both @nielsmh and @glx22 points to the list at the top.
@orudge

This comment has been minimized.

Sign in to view
Copy link
Contributor

orudge commented Feb 24, 2019

For MinGW directmusic and xaudio2 needs to be detected. MinGW has directmusic headers, but doesn't have xaudio2.

Just to clarify, if you manually copy over an xaudio2 header file then mingw should be happy with it - I did test that when I wrote it.
@TrueBrain

TrueBrain reviewed Feb 24, 2019
View changes

Show resolved Hide resolved src/os/windows/ottdres.rc.in Outdated
@glx22

This comment has been minimized.

Sign in to view
Copy link
Contributor

glx22 commented Feb 26, 2019

I kept the manifest in projects, but it should probably move somewhere else
@glx22

This comment has been minimized.

Sign in to view
Copy link
Contributor

glx22 commented Feb 27, 2019 •
edited

For now we rely on cmake default CMAKE_CONFIGURATION_TYPES and CMAKE_XXXXX_FLAGS_<CONFIG>, but it should be better to manage that by ourselves
@TrueBrain

TrueBrain reviewed Feb 27, 2019
View changes

Show resolved Hide resolved src/os/windows/ottdres.rc.in Outdated
@TrueBrain

TrueBrain reviewed Feb 27, 2019
View changes

Show resolved Hide resolved CMakeLists.txt Outdated
@TrueBrain

This comment has been minimized.

Sign in to view
Copy link
Member Author

TrueBrain commented Feb 27, 2019

For now we rely on cmake default CMAKE_CONFIGURATION_TYPES and CMAKE_XXXXX_FLAGS_, but it should be better to manage that by ourselves

I disagree. The closer we stay to CMake, the easier our life will be. Let's try to do this, and only add exceptions if we really need to (and mark them as such). Otherwise we will have a config.lib in no time :D
@glx22

This comment has been minimized.

Sign in to view
Copy link
Contributor

glx22 commented Feb 27, 2019

Indeed something is wrong in the generated project file for lzo and lzma
AdditionalDependencies for Release:
D:\developpement\GitHub\vcpkg\installed\x64-windows-static\lib\libpng16.lib
D:\developpement\GitHub\vcpkg\installed\x64-windows-static\lib\zlib.lib
D:\developpement\GitHub\vcpkg\installed\x64-windows-static\debug\lib\lzma.lib
D:\developpement\GitHub\vcpkg\installed\x64-windows-static\debug\lib\lzo2.lib
D:\developpement\GitHub\vcpkg\installed\x64-windows-static\lib\freetype.lib
D:\developpement\GitHub\vcpkg\installed\x64-windows-static\lib\bz2.lib

AdditionalDependencies for Debug:
D:\developpement\GitHub\vcpkg\installed\x64-windows-static\debug\lib\libpng16d.lib
D:\developpement\GitHub\vcpkg\installed\x64-windows-static\debug\lib\zlibd.lib
D:\developpement\GitHub\vcpkg\installed\x64-windows-static\debug\lib\lzma.lib
D:\developpement\GitHub\vcpkg\installed\x64-windows-static\debug\lib\lzo2.lib
D:\developpement\GitHub\vcpkg\installed\x64-windows-static\debug\lib\freetyped.lib
D:\developpement\GitHub\vcpkg\installed\x64-windows-static\debug\lib\bz2d.lib
@glx22

This comment has been minimized.

Sign in to view
Copy link
Contributor

glx22 commented Mar 1, 2019

hmm that's weird, OSX cmake step now pass, lzo is found, compilation fails because it doesn't find an lzo header
@glx22

This comment has been minimized.

Sign in to view
Copy link
Contributor

glx22 commented Mar 1, 2019

Ok there's an issue with lzo detection, I can compile with MinGW by luck, as the detected

-- LZO_INCLUDE_DIRS:H:/msys64/mingw64/include/lzo

is covered by H:/msys64/mingw64/include from other libs.
But on OSX

-- LZO_INCLUDE_DIRS: /usr/local/Cellar/lzo/2.10/include/lzo

is alone

@TrueBrain TrueBrain force-pushed the TrueBrain:cmake branch 2 times, most recently from 6164e63 to a25c911 Mar 2, 2019

@glx22

This comment has been minimized.

Sign in to view
Copy link
Contributor

glx22 commented Mar 3, 2019

Stupid me failing in the commit message.

BTW I think test task for MacOS should probably be "make -C build test".

@TrueBrain TrueBrain force-pushed the TrueBrain:cmake branch 3 times, most recently from e5b7c05 to d71ac0a Mar 10, 2019

@nielsmh

nielsmh reviewed Mar 11, 2019
View changes

Show resolved Hide resolved cmake/AutodetectSSE.cmake Outdated
@glx22

This comment has been minimized.

Sign in to view
Copy link
Contributor

glx22 commented Mar 12, 2019

regression works with MinGW without converting the exe, but it fails with many

6: -  SetCommandDelay: (null : 0x00000000)
6: +  SetCommandDelay: (null : 0x0000000000000000)'

I guess the lovely regex was needed :)
@TrueBrain

This comment has been minimized.

Sign in to view
Copy link
Member Author

TrueBrain commented Mar 13, 2019

Yeah, but I refused to do that unreadable regex; so now I am just adding them one case at the time :D

@TrueBrain TrueBrain force-pushed the TrueBrain:cmake branch 5 times, most recently from 823f76d to 027a5b3 Mar 13, 2019

@glx22

This comment has been minimized.

Sign in to view
Copy link
Contributor

glx22 commented Apr 8, 2019

I think os/windows/openttd.manifest should be in "Add: introduce CMake for project management" and not in "Add: create bundles via CPack" as it's needed for compilation
@DerDakon

DerDakon reviewed Apr 9, 2019
View changes

Show resolved Hide resolved CMakeLists.txt Outdated
Show resolved Hide resolved CMakeLists.txt Outdated
Show resolved Hide resolved CMakeLists.txt
CMakeLists.txt
find_package(Threads REQUIRED)

include(AutodetectLibrary REQUIRED)
autodetect_library("zlib" ZLIB ZLIB zlib "" zlib.h z)

This comment has been minimized.

Sign in to view
Copy link
@DerDakon

DerDakon Apr 9, 2019

CMake has modules that care for libz, png and a few others, that already know all the header and library names. I would not duplicate it.

This comment has been minimized.

Sign in to view
Copy link
@TrueBrain

TrueBrain Apr 9, 2019

Author Member

Yes and no. It does for some libraries, it does not for others. To avoid making some library detection more special than others, we decided to just all make them equal. In reality, it will never fall back on pkg-config or manual detection for these anyway, so there is no real harm.

Mostly we noticed this with libraries that are supported in newer CMake, but not in older. There it became really blurry real quick. Given the wide range of systems we need to support, to me this feels like the most safe solution.

I am open for counter-arguments ;)

This comment has been minimized.

Sign in to view
Copy link
@LordAro

LordAro Apr 9, 2019

Member

AIUI, it is more common to provide your own Find* functions, if one doesn't exist in cmake itself

e.g. https://github.com/SFTtech/openage/tree/master/buildsystem/modules

This comment has been minimized.

Sign in to view
Copy link
@DerDakon

DerDakon Apr 13, 2019

I would go this route, too. This is easier to follow for everyone used to CMake, and you still can call your custom detect function in your own Find*.cmake if that really makes things easier.
Show resolved Hide resolved CMakeLists.txt Outdated
Show resolved Hide resolved CMakeLists.txt Outdated
Show resolved Hide resolved cmake/AutodetectEditbin.cmake Outdated
Show resolved Hide resolved cmake/AutodetectLibrary.cmake
Show resolved Hide resolved cmake/AutodetectLibrary.cmake
Show resolved Hide resolved cmake/AutodetectLibrary.cmake
@DerDakon

DerDakon reviewed Apr 9, 2019
View changes

Show resolved Hide resolved cmake/SourceList.cmake
@TrueBrain

This comment has been minimized.

Sign in to view
Copy link
Member Author

TrueBrain commented Apr 9, 2019

Wow, a lot to unpack here :) Tnx a lot @DerDakon! Sadly, GitHub only allows me to reply on one thing at the time, so this might get a bit spammy ;) But I seriously appreciate the review!
@DerDakon

DerDakon reviewed Apr 9, 2019
View changes

Show resolved Hide resolved cmake/PackageBundle.cmake

@TrueBrain TrueBrain force-pushed the TrueBrain:cmake branch 7 times, most recently from f3e4528 to 7984911 Apr 13, 2019

@TrueBrain TrueBrain marked this pull request as ready for review Apr 13, 2019

@TrueBrain TrueBrain force-pushed the TrueBrain:cmake branch from 7984911 to 1c5c3b4 Apr 13, 2019

@TrueBrain

This comment has been minimized.

Sign in to view
Copy link
Member Author

TrueBrain commented Apr 13, 2019

In my opinion, this is now ready to be merged. At the top are now two URLs that point to successful release builds; so nightlies etc will also just work after merging.

There are a few things left, on which I need help, all to do with packaging:

  • Linux 'install' can prepare menu entries, but I haven't figured out how yet.
  • Debian dependencies are wrong, and needs tuning.
  • RPM cannot be produced currently.
    For this I suggest to make a new ticket to track this, as they are only really important to fix before 1.10 is released.

Otherwise it is completely done, from what I can tell.

@TrueBrain TrueBrain force-pushed the TrueBrain:cmake branch 4 times, most recently from a500a12 to 2440a6e Apr 13, 2019

@PeterN

This comment has been minimized.

Sign in to view
Copy link
Member

PeterN commented Apr 15, 2019

Should the AI/GS compat files be copied into the appropriate directory automatically as well?
@glx22

This comment has been minimized.

Sign in to view
Copy link
Contributor

glx22 commented Apr 15, 2019

I think the current way to run is to do ../build/openttd from <repo>/bin directory. We set VS_DEBUGGER_WORKING_DIRECTORY property to <repo>/bin so MSVC debugger does that automatically.

BTW I noticed, when installing openttd from MSVC, that InstallAndPackage.cmake is not selective enough for bin/ai and bin/game. It copies all files/dir, even if they are not part of the clone.

TrueBrain added some commits Apr 7, 2019

@TrueBrain @glx22
Remove: all existing project files 
This prepares for the switch to CMake, which takes over all current
exisiting forms of project files.
4d944e1
@TrueBrain @glx22
Codechange: move regression outside of bin and make it work via CMake… 
… script

The tst_stationlist savegame had to be changed to start the correct
AI. In the old setup, all regression AIs had the same name, which
made it impossible to run both regressions in parallel. With the new
setup this is possible.

Although all files are available to run the regression, it won't
really work till CMake is introduced (which will happen in a few
commits from here)
e13dc54
@TrueBrain @glx22
Codechange: rework how grf and ob[msg] are generated 
For grfs, it now uses CMake scripts to do its job, and both grf
files are split into their own folder to make more clear what is
going on. Additionally, it no longer builds in-source (although the
resulting grf is copied back in the source folder).

For ob[msg] it now uses CMake scripts to generate the translation
files; the result is no longer stored in-source (but in the build
folder).

Although all files are available to create the GRFs and basesets, it
won't really work till CMake is introduced (which will happen in a
few commits from here)
0d526f7
@TrueBrain @glx22
Add: introduce CMake for project management 
CMake works on all our supported platforms, like MSVC, Mingw, GCC,
Clang, and many more. It allows for a single way of doing things,
so no longer we need shell scripts and vbs scripts to work on all
our supported platforms.

Additionally, CMake allows to generate project files for like MSVC,
KDevelop, etc.

This heavily reduces the lines of code we need to support multiple
platforms from a project perspective.

Addtiionally, this heavily improves our detection of libraries, etc.
0466882
@TrueBrain @glx22
Add: create bundles via CPack 
CPack works closely together with CMake to do the right thing in
terms of bundling (called 'package'). This generates all the
packaging we need, and some more.
097163a
@TrueBrain @glx22
Fix: [AzurePipelines] switch the CI / CD to CMake too 
This also means dropping Debian/jessie, as it has a CMake that is
too old (3.0), with no real path to upgrade.
7b693a7
@TrueBrain @glx22
Codechange: remove #ifdef from .cpp files to exclude features 
With CMake, these files are simply not compiled to start with.
b0e6bf0
@TrueBrain @glx22
Fix: update the README stating what changed with CMake
Loading status checks…
0a70a67

@glx22 glx22 force-pushed the TrueBrain:cmake branch from 2440a6e to 0a70a67 Apr 19, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.