Compile kernel with retpoline GCC support

[clefru]

Retpoline is a spectre v2 mitigation technique. GCC 7.3.0 was released last week with support for that. The kernel being the most vulnerable and privileged part should be compiled with retpoline support.

Requirements:

• CONFIG_RETPOLINE enabled (it's the default)
• A compiler that supports -mindirect-branch=thunk-extern, see
  https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/commit/?h=v4.14.15&id=3a72bd4b60da338e66922e4f9eded174b3ad147d

Switching kernel compilation to GCC 7 should get us there. At the moment we use gcc 6.4.0.

[Baughn]

Do we have benchmarks on how much this costs, worst-case?

I mean, it should absolutely be enabled by default, but there are workloads for which Spectre mitigation is not absolutely required. If the cost is high enough then a configuration flag (with suitably severe wording) might be warranted.

[clefru]

@Baughn: Please see https://www.phoronix.com/scan.php?page=article&item=linux-retpoline-benchmarks&num=1 for performance numbers, thanks to the awesomely hardworking Michael Larabel. The change references in the PR above should be equivalent to switching from the "Retpoline" numbers to the "Retpoline + GCC" numbers. Yes, the impact is significant.

[clefru]

Another impact assessment by RedHat.

@Baughn: The compiler upgrade triggers retpoline, and that's a surprise performance regression yes. However users can disable it setting "RETPOLINE n" in their kernelPlatform.kernelExtraConfig. I would not expose more knobs and argue that this sticks to the tradition that no other performance relevant kernel knobs are exposed as extra knobs, see common-config.nix.

On the fact that this is a surprise regression, I'd say that I take that over a surprise vulnerability any day, and that with the mechanisms above versatile users can disable it. The only question is whether we should document CONFIG_RETPOLINE more extensively but also the nixos manual is not making any recommendations on kernel configurations, so I am not sure where to put such any remarks. Release notes for 18.03 seem most appropriate. I have no good suggestion for a 17.09 backport though.

[Baughn]

All right, fair enough.

I'm not going to argue it. If I were, however, than a counterargument might look like this: There is likely to be mitigations application-side as well as kernel-side, and having a single flag to disable all of them would be useful. On the flip side, I'm having trouble thinking of a single decent reason for doing so in e.g. Chrome.

[vcunat]

Done in 17.09-small channel; I expect others will follow soon.

[vcunat]

Channel unstable-small updated as well. The big ones move slower as usual. Enjoy!

[dotlambda]

I rebuilt my system from latest master and spectre-meltdown-checker still reports as vulnerable:

CVE-2017-5715 [branch target injection] aka 'Spectre Variant 2'
* Mitigated according to the /sys interface:  NO  (kernel confirms your system is vulnerable)
* Mitigation 1
  * Kernel is compiled with IBRS/IBPB support:  NO 
  * Currently enabled features
    * IBRS enabled for Kernel space:  NO 
    * IBRS enabled for User space:  NO 
    * IBPB enabled:  NO 
* Mitigation 2
  * Kernel compiled with retpoline option:  YES 
  * Kernel compiled with a retpoline-aware compiler:  NO  (kernel reports minimal retpoline compilation)
  * Retpoline enabled:  NO 
> STATUS:  VULNERABLE  (Vulnerable: Minimal generic ASM retpoline)

Also, gcc6 still seems to be in the kernel closure:

nix-store -qR (nix-instantiate -A linuxPackages_latest.kernel) | grep gcc
warning: you did not specify '--add-root'; the result might be removed by the garbage collector
/nix/store/lpfviz52hzvcxpnqnpcskdlnp4s5zhwg-bootstrap-gcc-wrapper.drv
/nix/store/bdwv7d0f21wd2zy1rdwiy01jhwdrvx1b-bootstrap-gcc-wrapper.drv
/nix/store/3cwx0g5ws3grg6s56njhnzz6hb1gf065-bootstrap-gcc-wrapper.drv
/nix/store/vmvkv585yrm51y6clh2lsk7ha4rn8b26-gcc-6.4.0.tar.xz.drv
/nix/store/56z78wrfrfp9s0h3gg5qbpqywx5mwcki-gcc-6.4.0.drv
/nix/store/xi0z1sm3grbsxxp6jik4dmlj68y3dd9k-gcc-wrapper-6.4.0.drv
/nix/store/zknz7g8vxkrmli3v1six4rxjlmyz6j83-gcc-wrapper-6.4.0.drv
/nix/store/blg28s3z496s2gmmi6466i8kysyl007j-gcc-7.3.0.tar.xz.drv
/nix/store/wklrlabfqc1r2fdf5m67x4n3nxak7j93-gcc-7.3.0.drv
/nix/store/f523zrkm0l0pgmxr2s8ih4ssaxnlb1bi-gcc-wrapper-7.3.0.drv
/nix/store/gccwz3nyglhnjgh1gk6ssri2cbbfb67r-libunistring-0.9.8.tar.gz.drv

I think this is because of

nixpkgs/pkgs/os-specific/linux/kernel/generic.nix

Line 108 in ae04052

make HOSTCC=${buildPackages.stdenv.cc.targetPrefix}gcc -C . O="$buildRoot" $kernelBaseConfig ARCH=$arch

[titanous]

I also see the same thing:

$ cat /sys/devices/system/cpu/vulnerabilities/spectre_v2
Vulnerable: Minimal generic ASM retpoline

[vcunat]

@dotlambda: thanks! The real problem is probably a few lines above what you pointed out. I didn't realize this difference from 17.09.

[vcunat]

Tested 17.09-small, to be sure:

$ cat /sys/devices/system/cpu/vulnerabilities/spectre_v2
Mitigation: Full generic retpoline

[vcunat]

Our cross-compilation fixes got into the way, as kernel needs two compilers in general, due to also running self-compiled stuff during the build.

If you need a fix immediately, use #34882 – I tested it this time :-)

[vcunat]

Re-tested the last version (as 17.09 above) and merged.