build.d: Reduce the default number of jobs in cgroups

[tim-dlang]

CircleCI runs the tests on a server with many CPUs, but restricts the
number of CPUs and the memory using Linux cgroups. Using the total
number of CPUs as the default number of jobs can result in many
parallel DMD processes, which can consume too much memory. This can
result in random failures.

This commit tries to detect a reduced number of CPUs, so the number of
jobs can be decreased.

[dlang-bot]

Thanks for your pull request and interest in making D better, @tim-dlang! We are looking forward to reviewing it, and you should be hearing from a maintainer soon.
Please verify that your PR follows this checklist:

• My PR is fully covered with tests (you can see the coverage diff by visiting the details link of the codecov check)
• My PR is as minimal as possible (smaller, focused PRs are easier to review than big ones)
• I have provided a detailed rationale explaining my changes
• New or modified functions have Ddoc comments (with Params: and Returns:)

Please see CONTRIBUTING.md for more information.

---

If you have addressed all reviews or aren't sure how to proceed, don't hesitate to ping us with a simple comment.

Bugzilla references

Your PR doesn't reference any Bugzilla issue.

If your PR contains non-trivial changes, please reference a Bugzilla issue or create a manual changelog.

Testing this PR locally

If you don't have a local development environment setup, you can use Digger to test this PR:

dub run digger -- build "master + dmd#15504"

[tim-dlang]

From the CircleCI documentation:

Java, Erlang and any other languages that introspect the /proc directory for information about CPU count may require additional configuration to prevent them from slowing down when using the CircleCI resource class feature. Programs with this issue may request 32 CPU cores and run slower than they would when requesting one core. Users of languages with this issue should pin their CPU count to their guaranteed CPU resources.

The program build.d uses totalCPUs to determine the number of jobs. The log on CircleCI for this pull request shows 36. This could explain random failures like for dlang/dlang.org#3681, because the jobs can run out of memory.

[dkorpel]

This is an uninformative warning message. It should tell what it was looking for, what the consequence of the failure is, and suggest how to fix it.

[tim-dlang]

I now think using the cpu.shares file is not the best way. The factor 1024 is the default and used by CircleCI for one CPU, but other environments could use different values.
Maybe it would be better to set the number of jobs in the .circleci/run.sh files. The Makefiles would need to forward the parameter to build.d.

[Imperatorn]

Any idea why it reports 36 instead of 32?

[tim-dlang]

Any idea why it reports 36 instead of 32?

I don't know exactly. Maybe they had only servers with 32 CPU cores when the documentation was written. Later they could have added servers with 36 cores. If they have different servers, that could also further explain, why the tests only sometimes run out of memory.

[tim-dlang]

Closing in favour of #15799 and dlang/dlang.org#3724.