Compiling in parallel

[michalmuskala]

Many of the plugins compile some files through make, that's why I'm opening the issue here, in the parent repository.

Should asdf detect if it can compile in parallel and set proper -jn flags for make? Is this a job for asdf, or should it assume user has proper MAKEFLAGS env variable set?
This can potentially significantly speed up compilation.

[HashNuke]

@michalmuskala If parallel is good, then let us make the asdf plugins use the parallel flag by default atleast for languages that are known to work well with it.

[michalmuskala]

I see that each plug-in has entirely separate code from the core manager, so each one would need to solve this on it's own.

The really tricky part is deciding on the number of concurrent jobs. A good rule is the number of cores, but the issue is there's no easy cross-platform way of determining it.
In order of how easy to use they are:

• On modern linux there should be nproc command
• On OSX you can use sysctl -n hw.ncpu
• On older linux (or some stripped down versions) you can go through /proc/cpuinfo: grep -c processor /proc/cpuinfo
• On Windows gnu make does not support concurrent jobs, so it doesn't matter.

[danhper]

I think we could set ASDF_CONCURRENCY before delegating the build to the plugin,
so that plugin writers do not have to do this check on their own.

On Windows gnu make does not support concurrent jobs, so it doesn't matter.

Does that means that make -j2 would break, or simply be noop?

[michalmuskala]

The GNU make documents it as being a noop on windows:

https://www.gnu.org/software/make/manual/html_node/Parallel.html

On MS-DOS, the ‘-j’ option has no effect, since that system doesn’t support multi-processing.

It talks about MS-DOS, so I'm not sure it applies to modern windowses as well.

It could probably detect it on a best effort basis and set to 1 otherwise. It will still be better than the current situation.

[danhper]

@michalmuskala

On MS-DOS, the ‘-j’ option has no effect, since that system doesn’t support multi-processing.
It talks about MS-DOS, so I'm not sure it applies to modern windowses as well.

Thanks.
I will try to check in a VM or something, but at least it should probably not emit any error.

It could probably detect it on a best effort basis and set to 1 otherwise. It will still be better than the current situation.

I agree. I will try to add this feature during the week.