[RDY] Revamp the build system

[jszakmeister]

This achieves several goals:

• Less reliance on scripts so we have better portability to Windows (though we still have a ways to go for proper Windows support). Luajit, luarocks, moonscript, and busted are all installed via CMake
  now.
• Trying to make use of pkg-config to get the correct libraries. The latest libuv is still broken in this regard, but we'll at least be in a position to use it.
• Allow the use of Ninja or make. The former runs faster in many environments, and automatically makes use of parallel builds.

This also allows for system installed dependencies--though not through the Makefile just yet--and adds support for FreeBSD. I also made the deps build of libuv and luajit build only the static versions.

Note: we no longer install moonrocks as we were only using install, and that was just adding a single parameter to the luarocks install command. Also, it bumps the minimum required CMake to 2.8.7 since that was the lowest version I had to test with. We could probably push it back further, if required, but it appears many distros have at least that version--RHEL being the notable exception, which appears to use 2.8.4.

Finally, I encountered a few issues along the way. CMake in FreeBSD was being built without SSL support. That has been fixed. Libuv didn't have the pkg-config description right, and now that's fixed in 0.11.22. I discovered a couple of CMake bugs that I still need to report. One being related to type size detection failing (because of a compilation error), but CMake failed to notice and left the define empty. The other being a way to shut off download progress notification when using ExternalProject_Add(). The former doesn't really affect most users, I just happened to run across it. The latter we could work-around by formulating our own file(DOWNLOAD ...) command and running it via ${CMAKE_COMMAND} -P.

I tested this on OS X (Mavericks and Snow Leopard), Debian 7.4, Ubuntu 12.04, and FreeBSD 10.

I believe this ultimately supersedes #198, though we may want to take some elements of #198 (moving the top-level Makefile, and the separate documentation and examples). It also fixes #245 and #232, address #202, #255, and I believe it makes progress toward #328.

One downside of using Ninja: we can't launch the legacy tests through it. It doesn't setup a pseudo-tty, and the tests require it. I do have another branch (https://github.com/jszakmeister/neovim/tree/revamp-vim-tests) that is exploring how to make this better.

[tarruda]

Looking good, thanks for this amazing job @jszakmeister

[pusewicz]

👍

[stefan991]

👍 , but there is a issue with the file downloading part on Travis. The downloads print more than 3000 lines to the log (e.g. https://travis-ci.org/neovim/neovim/jobs/20601268#L826).
Is there an easy way to disable this output?

[jszakmeister]

@stefan991 That was one of the bugs I mentioned:

The other being a way to shut off download progress notification when using ExternalProject_Add().

ExternalProject_Add() doesn't provide a way to shut-off the download progress. We can dance around it by calling out to our own cmake script to do the download and extraction. Looks like I should probably go ahead and do that.

[jszakmeister]

Ugh. I have something working to avoid the progress junk, but it's a lot uglier than I care for. :-( I want to spend a little more time with it. FWIW, I reported the issue to the CMake team.

[tarruda]

@jszakmeister I can see you rebased the commits. Whenever you think it's ready feel free to merge it yourself :)

[jszakmeister]

@tarruda Thanks. Done.

[tarruda]

The build system is looking much better now. Hopefully this will please everyone :)

Great job @jszakmeister, thanks

[mahkoh]

make clean && make cmake no longer works.

And why does make build nvim-test?

[jszakmeister]

"make clean" doesn't remove the build directory. Instead, it runs the clean target in cmake and cleans the legacy tests. If you'd like a separate target that will remove the build folder (without removing the bundled dependencies), I can do that.

I have a PR up to only build nvim-test when you run unit tests.

[lslah]

@jszakmeister

If you'd like a separate target that will remove the build folder (without removing the bundled dependencies), I can do that.

I think this would be a good idea. Currently when you add a new source file, you have to run make distclean && make cmake (or delete build/ manually) due to the cmake-blob-mechanism. make clean && make cmake felt more natural but I could live with another make-target as well.

[jszakmeister]

What if we change the cmake target to be something like this:

diff --git a/Makefile b/Makefile
index c20329a..c334ea7 100644
--- a/Makefile
+++ b/Makefile
@@ -32,7 +32,9 @@ all: nvim
 nvim: build/.ran-cmake deps
        +$(BUILD_TOOL) -C build

-cmake: | build/.ran-cmake
+cmake:
+       touch CMakeLists.txt
+       $(MAKE) build/.ran-cmake

 build/.ran-cmake: | deps
        mkdir -p build

That would force the cmake configure process to re-run. Or, we could list the source files instead of globbing, which would cause cmake to reconfigure with a bare make. I prefer the latter, but it can cause a few more conflicts along the way.

[lslah]

@jszakmeister 👍 I think that's a nice solution.

[jszakmeister]

@lslah which one? The diff or enumerating the source files? I assume you mean the proposed diff.

[lslah]

@jszakmeister Ah, sorry, somehow overlooked the second option. Yes, I've meant the diff in my last comment.
I've voted for listing the files explicitely in the first place as well, but we've decided to go the other way. It was discussed in #191. Not sure if we should start this discussion again.