Parallelizing compilation of csources

[def-]

csources are built with build.sh, which only compiles each C file after each other, using only 1 CPU core.

I see two ways for parallelizing this:

• use jobs and wait (both POSIX) to build a little process pool in build.sh
• Or use simple POSIX conforming makefiles, like this:

CC=gcc
LINKER=gcc
COMP_FLAGS=-w -O3 -fno-strict-aliasing
LINK_FLAGS=-ldl -lm
binDir=bin

.POSIX:

%.o: %.c
    $(CC) $(COMP_FLAGS) -I.. -c $< -o $@

nim: nim.o stdlib_system.o testability.o commands.o stdlib_os.o stdlib_strutils.o stdlib_parseutils.o stdlib_times.o stdlib_posix.o msgs.o options.o lists.o stdlib_strtabs.o stdlib_hashes.o stdlib_osproc.o stdlib_streams.o stdlib_cpuinfo.o stdlib_linux.o stdlib_sets.o stdlib_math.o stdlib_tables.o ropes.o platform.o crc.o stdlib_sockets.o stdlib_unsigned.o nversion.o condsyms.o idents.o extccomp.o wordrecg.o nimblecmd.o stdlib_parseopt.o lexer.o nimlexbase.o llstream.o nimconf.o main.o ast.o stdlib_intsets.o idgen.o astalgo.o rodutils.o syntaxes.o parser.o pbraces.o filters.o renderer.o filter_tmpl.o rodread.o types.o trees.o stdlib_memfiles.o rodwrite.o passes.o magicsys.o nimsets.o bitsets.o importer.o lookups.o semdata.o treetab.o vmdef.o prettybase.o stdlib_lexbase.o sem.o semfold.o saturate.o procfind.o pragmas.o semtypinst.o sigmatch.o parampatterns.o pretty.o docgen.o docutils_rstast.o stdlib_json.o stdlib_unicode.o docutils_rst.o docutils_rstgen.o docutils_highlite.o stdlib_sequtils.o stdlib_algorithm.o sempass2.o guards.o stdlib_xmltree.o stdlib_macros.o stdlib_cgi.o stdlib_cookies.o typesrenderer.o transf.o cgmeth.o lambdalifting.o lowerings.o vm.o vmgen.o vmdeps.o evaltempl.o aliases.o patterns.o semmacrosanity.o semparallel.o cgen.o ccgutils.o cgendata.o ccgmerge.o jsgen.o passaux.o depends.o docgen2.o service.o modules.o
    $(LINKER) -o $@ $^ $(LINK_FLAGS)

.PHONY: clean

clean:
    rm -f *.o

Any opinions?

[reactormonk]

Looks good to me, although IIRC @Araq doesn't like Make.

[Araq]

Well since it's generated anyway, why not keep the build.sh and provide a makefile in addition to that? Seems to me the best of both worlds.

[Varriount]

Hrm, and since none of you *nix gurus have mentioned it, why not add parallelization support for the Windows batch file?

[def-]

@Varriount Because I don't know how to do that. Also, mingw comes with make, and Visual Studio with nmake, so makefiles might even work on Windows.

[oderwat]

There is also find /c_code -name '*.c' -type f -print0 | xargs -0 -P 4 ls -la to parallelize stuff without much effort. May be to fragile for this purpose though.

[def-]

@oderwat POSIX xargs doesn't have -P. I have a half-working solution for Makefiles, just have to fix them, clean up, and so on.

[def-]

This has been implemented.