Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add latest version of e1000e module, built for grsec kernel #4024

Closed
zenmonkeykstop opened this issue Jan 9, 2019 · 12 comments
Closed

Add latest version of e1000e module, built for grsec kernel #4024

zenmonkeykstop opened this issue Jan 9, 2019 · 12 comments

Comments

@zenmonkeykstop
Copy link
Contributor

Description

Intel 7-series NUCs use the e1000e kernel driver for built-in NIC support under Linux. The version that ships with Ubuntu 14.04.5, and the version that's currently installed with SD's grsec kernels, is out-of-date and does not support the Ethernet chipset on new NUCs. Version 3.4.2.1, available as a source tarball on the Intel support site, does. The module shipped with the grsec kernel should be updated to this version if possible. This will allow for a wider range of NUC hardware to be added to the SecureDrop HCL.

More info on the build process for this driver is available at https://downloadcenter.intel.com/download/15817 - The build process has been confirmed to work for 14.04.5's most recent 4.4.0 kernel with recent updates, but has not been tested for the SD grsec kernel as it requires a linux-headers package.

User Stories

As an organization investigating SecureDrop, I want to have a choice of NUC hardware that's up-to-date and commercially available.

@eloquence
Copy link
Member

As part of the 1/9-/1/23 sprint, we've committed to a 4 person hour time-boxed investigation (likely led by @emkll and @zenmonkeykstop) to determine (and, if doable within the timebox, implement) the best way to ship/enable the driver with the next SecureDrop kernel release.

@eloquence eloquence moved this from Near Term Backlog to Sprint-Free Period - 1/2 to 1/9 in SecureDrop Team Board Jan 9, 2019
@eloquence eloquence added this to the 0.12.0 milestone Jan 9, 2019
@eloquence eloquence moved this from Current Sprint - 2/6-2/20 to Near Term Backlog in SecureDrop Team Board Feb 8, 2019
@redshiftzero redshiftzero modified the milestones: 0.12.0, 0.12.1 Feb 15, 2019
@redshiftzero
Copy link
Contributor

This won't get in for 0.12.0, bumping this off the milestone into a potential 0.12.1 point release milestone

@ageis
Copy link
Contributor

ageis commented Mar 1, 2019

I used to have e1000e cards for a few years before switching to igb and have built the driver many times (often complains about fPIC). FYI, the module that ships in the Linux kernel is typically a newer version than the source one can obtain from Intel.

@eloquence eloquence moved this from Near Term Backlog to Nominated for next sprint in SecureDrop Team Board Mar 5, 2019
@eloquence eloquence moved this from Nominated for next sprint to Current Sprint - 3/6-3/20 in SecureDrop Team Board Mar 6, 2019
@zenmonkeykstop
Copy link
Contributor Author

zenmonkeykstop commented Mar 7, 2019

Did some more tests. TL;DR: the e1000e module provided with the current 4.4.167-grsec kernel doesn't have an alias defined for the NIC in the NUC7i5BNH, despite being the same version as the e1000e module provided with the 4.4.0131-generic kernel in Ubuntu 16.04.6, which does. This is why it doesn't detect the NIC.

Device info

All testing done against an Intel NUC7i5BNH, with the latest available BIOS version (0072). NIC device info from lspci is as follows:

00:1f.6 Ethernet controller [0200]: Intel Corporation Ethernet Connection (4) I219-V [8086:15d8] (rev 21)
	DeviceName: Onboard - Ethernet
	Subsystem: Intel Corporation Ethernet Connection (4) I219-V [8086:2068]
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin A routed to IRQ 11
	Region 0: Memory at dfb00000 (32-bit, non-prefetchable) [size=128K]
	Capabilities: [c8] Power Management version 3
		Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=1 PME-
	Capabilities: [d0] MSI: Enable- Count=1/1 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000
	Capabilities: [e0] PCI Advanced Features
		AFCap: TP+ FLR+
		AFCtrl: FLR-
		AFStatus: TP-

Simple kernel testing

Tested the following kernels as installed, provided by @emkll, or retrieved from apt.freedom.press:

  • 4.4.0.131-generic: (default 16.04.5 kernel) - boots, e1000e module loaded, NIC detected and networking started successfully.
  • 4.4.167-grsec: kernel boots, but ip link show only lists loopback device. The e1000e module is not loaded automatically, and modprobe e1000e does not not detect the NIC.
  • 4.14.92-grsec: doesn't get past loading initial ramdisk...
  • 4.14.104-grsec: doesn't get past loading initial ramdisk...

Comparing e1000e modules

the output of modinfo e1000e for the 4.4.0-131 and 4.4.167-grsec kernels is attached, the module versions are the same (3.2.6-k) although the srcversion and vermagic values are different. Diffing the files, it appears that more PCI aliases are defined for the 4.4.0-131 version:

1c1
< filename:       /lib/modules/4.4.167-grsec/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko
---
> filename:       /lib/modules/4.4.0-131-generic/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko
6c6,11
< srcversion:     80BC444335228F44E92F55E
---
> srcversion:     EFFD6AC50F30C2893915E82
> alias:          pci:v00008086d000015D6sv*sd*bc*sc*i*
> alias:          pci:v00008086d000015E3sv*sd*bc*sc*i*
> alias:          pci:v00008086d000015D8sv*sd*bc*sc*i*
> alias:          pci:v00008086d000015D7sv*sd*bc*sc*i*
> alias:          pci:v00008086d000015B9sv*sd*bc*sc*i*
74c79
< vermagic:       4.4.167-grsec SMP mod_unload modversions KERNEXEC_BTS UDEREF RAP REFCOUNT CONSTIFY_PLUGIN STACKLEAK_PLUGIN GRSEC RANDSTRUCT_PLUGIN_611e53219759b6de2cf9c0d6413fbf884d21ec81c27a0c6173c7208142dd9aad
---
> vermagic:       4.4.0-131-generic SMP mod_unload modversions retpoline 

modinfo-vanilla.txt
modinfo-grsec.txt
One of said aliases corresponds to the PCI ID of the NUC's NIC (15d8). This would be the reason that the grsec version of the module doesn't detect the device.

Compiling from scratch

Pulling down the latest e1000e tarball (3.4.2.1) and building against the 4.4.167-grsec kernel headers completes successfully, but the resulting module causes a kernel panic when loaded via modprobe, with output starting with PAX: overwritten function pointer or return address detected. (If there's a straightforward way to get kernel panic output, please advise and I'll update the issue with the full output - tried kdump but it's not supported by the grsec kernel.)

adding the device via /sys/bus/.../new_id

I tried to manually attach the NIC's PCI device using echo -n "8086 15d8" > /sys/bus/pci/drivers/e1000e/new_id, but this command caused the system to hang.

Conclusion

I'd welcome ideas for further testing, but it looks like an upstream change is required to enable support for the NIC in the e1000e module build with the grsec kernel. Either that or the latest version of the module needs to be patched to avoid the kernel panic encountered as described above.

@eloquence eloquence modified the milestones: 0.12.1, 0.13.0 Mar 12, 2019
@eloquence
Copy link
Member

Updated milestones; this won't make it onto 0.12.1 per discussion today, but continues to be a high priority.

@conorsch
Copy link
Contributor

conorsch commented Mar 21, 2019

Reached out to the friendly folks upstream to confer about options for hardware support. In a nutshell, we were referred to these two patches:

We should rebuild the existing SD kernel with those two patches applied (in addition to the standard grsecurity patch, of course) and test the resulting packages on the 7th gen NUCs. Even with the patches applied, we will likely still be forced to disable RAP. That's unfortunate, but we can work with it.

In recognition of time constraints, let's disable RAP on the candidate build with the additional patches above verify hardware compat. If networking is still broken, additional research is required. If networking works, then we can discuss a timeboxed attempt of rebuilding with RAP enabled.

@zenmonkeykstop Care to take point on this one?

@conorsch
Copy link
Contributor

We should rebuild the existing SD kernel with those two patches applied

Before we do this, @zenmonkeykstop / @emkll can you confirm that we've already tested 4.14.x with RAP disabled? If that setup works, let's discuss that as an option to expand hardware support, rather than getting into the game of backporting compatibility patches upon request.

@zenmonkeykstop
Copy link
Contributor Author

4.14.x did not boot successfully at all, but those kernels did not have RAP disabled.

@zenmonkeykstop
Copy link
Contributor Author

Confirmed that a 4.4.177-grsec kernel with the patches above applied and RAP disabled works happily with the i219V chipset on the NUC7i5BNH.

@zenmonkeykstop
Copy link
Contributor Author

Also confirmed that the 4.4.177-grsec kernel with above patches and RAP enabled works happily with the same chipset.

@emkll emkll moved this from Current Sprint - 3/21-4/3 to Under Review in SecureDrop Team Board Mar 27, 2019
@eloquence eloquence modified the milestones: 0.13.0, 0.12.2 Mar 28, 2019
@eloquence
Copy link
Member

Ideally we want to land this in a point release in April, updated milestone accordingly.

@conorsch
Copy link
Contributor

conorsch commented Apr 2, 2019

Resolved by combination of:

We'll continue with hardware testing to confirm compatibility, and follow up with additional changes if necessary.

@conorsch conorsch closed this as completed Apr 2, 2019
SecureDrop Team Board automation moved this from Under Review to Done Apr 2, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Development

No branches or pull requests

5 participants