Waf build system

[bjourne]

This PR is more of an idea and a proof of concept and not ready to be merged.

The suggestion is to replace Factors build system based on batch and shell scripts with waf: https://code.google.com/p/waf/ waf is a "real buildsystem" like autotools and both much faster and easier to maintain than hand-coded makefiles and scripts. The difference becomes more apparent when you add advanced build features, like making distribution tarballs, automated test running, window installer and installation targets. It's really hard writing all that by hand. Currently the wscript I've written only builds and installs, but I plan to add more features to it.

waf only requires Python and the script itself is included in the pr. So when you have applied it, you can build using:

python waf.py --prefix=/blah configure
python waf.py build
sudo python waf.py install

[bjourne]

I would love to get some feedback on this PR. :) It now builds a correct release for both Windows and Linux. On Windows, you get an MSI for your platform and on Linux "install" puts the files in the right place. I don't know how OSX works but it is very similar to Unix isn't it? Since it istalls cleanly it should be easy to build debs and rpms for various distros.

The waf addition doesn't require any major changes to Factor itself except for the addition of "install-prefix" which is needed for it to locate its standard library when it is installed. So the old build system could be kept along with waf. Not that I see the point because waf is awesome. :)

Oh, and the binary is renamed to "factor-lang" for Linux compatibility because the "factor" name is already taken by coreutils. But any other name could work too.

[mrjbq7]

Why change it from *.image which ignores the boot images also? (e.g. boot.unix-x86.64.image)

[mrjbq7]

Why are the boot images checked into git?

[mrjbq7]

Is there any way to determine the "install prefix" by the current location of the factor binary? This seems fragile.

[bjourne]

Install prefix allows running Factor from the build directory without installing it. Almost all unix programs encodes the installation prefix in the binary like that.

[mrjbq7]

Okay, I looked this over and I'm a little unsure why we want to add waf...

I can think of a few reasons maybe:

1. Because the build-support/factor.sh and build-support/factor.com don't share enough code
2. Because it is simpler or easy to maintain
3. Because this adds support for system-wide installations
4. Because we love python
5. Because waf is shorter than make :)

I'd like to understand the motivations a bit more -- is this intended to be a parallel build system? Is it supposed to replace ours? Is waf a reasonable dependency to add? I don't know any system that installs waf by default, whereas everyone has make and cc.

And most (least?) importantly, what kind of name is waf?!

Is it waf like "RAF" (the royal air force)? Or maybe "whiff" as in "swing and a miss"? Or "woof" like a dogs' bark? Or "waif", an abandoned child? So many options, they really need a pronunciation guide.

[bjourne]

Nice! So I'll try to explain why I like waf so much.

First you have make, or nmake, which just automates invocations to the compiler to build the binary. It's sufficient for small projects. But as the build process becomes more complicated you get more tasks you want to automate which are hard to code in makefiles. Like checking that the user has all required dependencies, handling build configure options nicely, making tarballs, installing, uninstalling, running test suites, uploading to ftp servers, building html documentation, building installers, tagging releases and so on.

For any complicated software with a complex build process like Factor has (with bootstrapping and self-compilig and all that), makefiles are not enough. Compiling is just one of the many tasks you want to automate.

To solve those problems you either roll your own scripts like those in build-support/ or you use an existing "meta build system", like waf. Scripts are good because they dont depend on anything extra, but its annoying to have to write one script for each platform and as the scripts get larger you're essentially writing your own build tool. The more features. robustness and polish you want to add, the more you gain by taking advantage of what a build tool gives you for free.

So because people was tired of all home-rolled unmaintainable build scripts, they created meta build systems like autotools, cmake, scons, waf and probably many more. The most popular of them is autotools which is used by almost all gnu software. However, it works poorly on Windows and you need to learn M4, which is a truly horrible laguage, to use it. The other tools are probably all good in their own ways. I don't have experience with them. Waf otoh, I know is really solid so I picked it. :)

The goal is to make the build and install process easy and clean on Linux. Because that makes it easy for distros to put Factor in their repositories. As it is now they can't package Factor because it runs from its source directory. That it makes it easy to create other useful build artifacts such as msi installers is imho icing on the cake.

The only added dependency is to python 2.4+. waf.py itself is included in the pr. I don't know where the name comes from. The author is german so maybe WAFür? or WAFfelnrezept?

[bjourne]

About the boot images. I checked in them (and changed .gitignore correspondingly) for two reasons.

First it is a big security hole to download and execute them during the build. Anyone who gets control of the factorcode.org domain, or can spoof the domain name for someone, can insert malicious code in the boot image and have it execute on builders machine.

Second, say some LInux distros decide to add Factor to their repositories. Then they want to build Factor from source using a known good image and not the latest one. Perhaps they add Factor version 0.99 but we continue to work on Factor in git and the new image might then break their build process.

It would be great to be able to build Factor without starting with a prebuilt image, but that's not possible is it?

[razetime]

is this still a thing? the link to waf here doesn't work, and i'm not able to find it in any ordinary search engine.

[catb0t]

is this still a thing? the link to waf here doesn't work, and i'm not able to find it in any ordinary search engine.

https://waf.io/

This PR has probably bitrotted pretty bad in the last 9 years, I think modern Waf might be somewhat different, and Waf is still maintained https://gitlab.com/ita1024/waf/activity

[mrjbq7]

Waf is cool, but not going to merge this in present form -- we need to get 0.99 released. Closing.