IMA & TPM load order broken, resulting in no hardware root of trust for Integrity Measurement Architecture

[lukehinds]

Describe the bug

The load order of IMA and a TPM device is incorrect , which results in IMA not seeing the TPM and activating a TPM-bypass.

As the TPM is bypassed, IMA cannot benefit from the hardware root of trust that is possible with a Trusted Platform Module.

The TPM should initialize first, and then IMA.

This is a key security feature for protecting run time environments.

To reproduce

Place a TPM chip onto GPIO (I used the Infineon Optiga™ SLB 9670 from letstrust.de)

Compile the Kernel with the below configs set to enable IMA, securityfs and the TPM

CONFIG_SECURITY=y
CONFIG_SECURITYFS=y
CONFIG_INTEGRITY=y
CONFIG_IMA=y
CONFIG_IMA_MEASURE_PCR_IDX=10
CONFIG_IMA_AUDIT=y
CONFIG_IMA_LSM_RULES=y
CONFIG_IMA_WRITE_POLICY=y
CONFIG_IMA_READ_POLICY=y
CONFIG_IMA_APPRAISE=y
CONFIG_IMA_APPRAISE_BOOTPARAM=y
CONFIG_HW_RANDOM_TPM=y
CONFIG_TCG_TPM=y
CONFIG_TCG_TIS_CORE=y
CONFIG_TCG_TIS=y
CONFIG_TCG_CRB=y

Put a basic IMA-policy in place:

cat /etc/ima/ima-policy 

# PROC_SUPER_MAGIC
dont_measure fsmagic=0x9fa0
# SYSFS_MAGIC
dont_measure fsmagic=0x62656572
# DEBUGFS_MAGIC
dont_measure fsmagic=0x64626720
# TMPFS_MAGIC
dont_measure fsmagic=0x01021994
# RAMFS_MAGIC
dont_measure fsmagic=0x858458f6
# SECURITYFS_MAGIC
dont_measure fsmagic=0x73636673
# MEASUREMENTS
measure func=BPRM_CHECK
measure func=FILE_MMAP mask=MAY_EXEC
measure func=MODULE_CHECK uid=0

reoot the pi, and run dmesg

pi@raspberrypi:~ $ dmesg | grep -i tpm
[    0.995705] ima: No TPM chip found, activating TPM-bypass! (rc=-19)
[    6.566532] tpm_tis_spi spi0.1: 2.0 TPM (device-id 0x1B, rev-id 22)
[    6.582596] tpm tpm0: A TPM error (256) occurred attempting the self test
[    6.582619] tpm tpm0: starting up the TPM manually

After some investigation I found the following mailing list discussion on the IMA list

One of the IMA developers was able to initialize the TPM prior to IMA, by reverting commit acddd39

And disabling the self test

-       rc = tpm2_start_selftest(chip, true);
+       rc = tpm2_start_selftest(chip, false);

This was from 2017 so a fresh RCA might play out differently.

Expected behaviour

The TPM should load first, which will result in IMA finding the TPM and using it for cryptographic extend operations in PCR 10.

Actual behaviour

The load order is wrong.

System

• Which model of Raspberry Pi?
  Raspberry Pi 3 Model B+

• Which OS and version (cat /etc/rpi-issue)?

Raspberry Pi reference 2019-07-10
Generated using pi-gen, https://github.com/RPi-Distro/pi-gen, 175dfb027ffabd4b8d5080097af0e51ed9a4a56c, stage2

• Which firmware version (vcgencmd version)?

Jul  9 2019 14:40:53 
Copyright (c) 2012 Broadcom
version 6c3fe3f096a93de3b34252ad98cdccadeb534be2 (clean) (release) (start)

• Which kernel version (uname -a)?

uname -a
Linux raspberrypi 4.19.79-v7+ #1 SMP PREEMPT Thu Oct 17 11:01:59 BST 2019 armv7l GNU/Linux

Additional context
I will continue to debug this, but truth be told I don't know the clock functionality in code very well, so if anyone thinks they can fix this, please do go ahead and if you want any tests run against a HW TPM, I am happy to help.

[pelwell]

1. How is the loading of the TPM and IMA drivers enabled?
2. Do you think Raspberry Pi behaves differently from other platforms? If so, how do they manage the driver loading?
3. I'm very skeptical about the connection between acddd39 and load order, unless it is simply a question of timing. What do you think is the mechanism?

This doesn't sound like a Raspberry Pi-specific problem. We haven't done any work in the area of TPM or IMA, so this is pure upstream code. If the system is relying on a specific driver load order without any attempt to guarantee it then failure is likely under some circumstances.

[lukehinds]

1. How is the loading of the TPM and IMA drivers enabled?

I am yet to find out and trying to read through the code. I will see if I can get others know the code better to comment.

2. Do you think Raspberry Pi behaves differently from other platforms? If so, how do they manage the driver loading?

I believe so, in that the right load order occurs (its a widely used feature in Linux distributions). Sorry I can't be of much value explaining how this happens, will again try and get someone who knows this area to comment while I get up to speed myself.

I'm very skeptical about the connection between acddd39 and load order, unless it is simply a question of timing. What do you think is the mechanism?

This is from mimi the IMA maintainer in the linux Kernel (the conversation is based on getting IMA working on a pi)

Last December we were able to initialize the TPM before IMA. It
required commit 29c7854 - "Register the clocks early during the boot
process, so that special/critical clocks can get enabled early
on in the boot process avoiding the risk of disabling a clock,
pll_divider or pll when a claiming driver fails to install
propperly - maybe it needs to defer."

We built a custom kernel with both commit 29c7854 and building in BCM2835_SPI (thanks Nayna).

Last she states:

The patch replaces the call to builtin_platform_driver(), with a call
to core_initcall().

https://github.com/raspberrypi/linux/blob/rpi-4.8.y/drivers/clk/bcm/clk-bcm2835.c

static int __init __bcm2835_clk_driver_init(void)
{
	return platform_driver_register(&bcm2835_clk_driver);
}
core_initcall(__bcm2835_clk_driver_init);

The conversation then concludes with:

So the bcm2835-clk calls rpi_firmware_get, which returns with NULL since the pdev has not been set yet,
as the rpi_firmware_probe gets called only later :/

That defers the probing of the bcm2835-clk, so it it not available for the bcm2835-spi which also gets defered, consequently making the tpm not available until ima is done :/

Sorry for the long delay! At the end of last year we were able to
boot the pi with IMA using the TPM.

On the rpi-4.8.y kernel with tip commit 061dccc ("BCM270X_DT: Add pi3-
disable-wifi overlay"), I had to disable the full TPM selftest in
order for the TPM to be initialized prior to IMA.

-       rc = tpm2_start_selftest(chip, true);
+       rc = tpm2_start_selftest(chip, false);

I now have it working on the rpi-4.9.y kernel with tip commit e80a8a5
("Merge remote-tracking branch 'stable/linux-4.9.y' into rpi-4.9.y")
as well. In addition to disabling the full selftest, I had to revert
commit b76c8d5 ("clk-bcm2835: Read max core clock from firmware") in
order for the TPM to initialize prior to IMA.

[lukehinds]

To add, the above quotes are from this thread: https://www.spinics.net/lists/linux-integrity/msg00051.html

[PeterHuewe]

How is the loading of the TPM and IMA drivers enabled?
The TPM driver is loaded by a suitable entry in the device tree.

I'm very skeptical about the connection between acddd39 and load order, unless it is simply a question of timing. What do you think is the mechanism?
By reverting acddd39 the order is working again. See also Mimi's reply:

Right, for rpi-4.14.y kernel, reverting commit acddd39 ("clk-bcm2835:
Read max core clock from firmware") allows the TPM to be initialized
prior to IMA, but is probably not the right solution.

The reason is that with acddd39 bcm2835_clk_probe returns -EPROBE_DEFER; if the firmware file is not yet available. So the initialization of the SPI clk is postponed until later in the boot phase (once firmware file becomes available) and then the TPM (which depends on SPI) gets enabled.
Which is unfortunately then way after IMA.
From a kernel interface point of view the dependencies of initcalls should be correct, but the -EPROBE_DEFER breaks it.

On other platforms the SPI clocks are usually initialized early not depending on a firmware file (like before acddd39).

Besided the thread linked by Luke, there is another thread on the same issue:
https://lore.kernel.org/linux-integrity/9cd0d399-2b11-779c-f767-660ea61721d9@moth.iki.fi/

We (Infineon) get the inquiry of enabling IMA on RPI quite often, as people love the rpi as a great kit for dipping into the world of (embedded) trusted computing.

[pelwell]

That's what I suspected - it works in most situations. but it sounds a bit fragile. There should be a hard dependency between IMA and TPM (for configurations where there is a TPM, obviously).

[mhecht]

It's not a Raspberry Pi specific issue. It seems that the behavior is depending on the plattforms and configurations. I didn't test in bcmxxxx but I work on Xilinx Zynq and ZynqMP. On Zynq (Cortex A9-MP) all works as expected with linux-stable 4.14.x and 4.19.x. The loading sequence is ok when drivers for ccf, spi and tpm are statically linked.
But I'm struggling to get the right load order on ZynqMP (ARM64). So TPM driver has been loaded much later than IMA because dependencies on SPI and CCF. I tried to adjust the initcalls but I didn't get a final solution right now.

[pelwell]

I'm watching this thread with interest, but at the moment I don't think it's a problem we can solve in anything but a hacky, makeshift way.

[AndreasFuchsTPM]

I've been watching. Cannot help much but wanted to express the importance to us...

[PeterHuewe]

Hmm,
I had a look at the initcall order
so by ordering it according to https://elixir.bootlin.com/linux/v5.3.6/source/include/linux/init.h the order on RPI is
clk-bcm2835.c is core_initcall(__bcm2835_clk_driver_init);
firmware/raspberrypi.c is subsys_initcall(rpi_firmware_init);
tpm_interface.c is subsys_initcall(tpm_init);
ima_main.c is late_initcall(init_ima);

The order of tpm first, then ima seems correct.
Of course I agree that ima should maybe look a second time whether there is a tpm available.

However the clock driver is called before its dependency of the firmware.
I think that is something that should be fixed for rpi in general.
Maybe the clock driver has to be split into a core part and a firmware dependent part?

By reversing the order clk-bcm2835 and firmware Markku was able to get it working
https://lore.kernel.org/linux-integrity/357e44f8-df31-48ec-d2f0-deabd0161fc0@moth.iki.fi/

diff --git a/drivers/clk/bcm/clk-bcm2835.c b/drivers/clk/bcm/clk-bcm2835.c
index d6caac9c3..7cdd597f1 100644
--- a/drivers/clk/bcm/clk-bcm2835.c
+++ b/drivers/clk/bcm/clk-bcm2835.c
@@ -2330,7 +2330,7 @@ static int __init __bcm2835_clk_driver_init(void)
  {
         return platform_driver_register(&bcm2835_clk_driver);
  }
-core_initcall(__bcm2835_clk_driver_init);
+subsys_initcall(__bcm2835_clk_driver_init);

  MODULE_AUTHOR("Eric Anholt <eric@anholt.net>");
  MODULE_DESCRIPTION("BCM2835 clock driver");
diff --git a/drivers/firmware/raspberrypi.c b/drivers/firmware/raspberrypi.c
index a82819a78..dfa362e1c 100644
--- a/drivers/firmware/raspberrypi.c
+++ b/drivers/firmware/raspberrypi.c
@@ -457,7 +457,7 @@ static int __init rpi_firmware_init(void)
  out1:
         return ret;
  }
-subsys_initcall(rpi_firmware_init);
+core_initcall(rpi_firmware_init);

  static void __init rpi_firmware_exit(void)
  {

[glassman81]

diff --git a/drivers/clk/bcm/clk-bcm2835.c b/drivers/clk/bcm/clk-bcm2835.c
index d6caac9c3..7cdd597f1 100644
--- a/drivers/clk/bcm/clk-bcm2835.c
+++ b/drivers/clk/bcm/clk-bcm2835.c
@@ -2330,7 +2330,7 @@ static int __init __bcm2835_clk_driver_init(void)
{
return platform_driver_register(&bcm2835_clk_driver);
}
-core_initcall(__bcm2835_clk_driver_init);
+subsys_initcall(__bcm2835_clk_driver_init);

MODULE_AUTHOR("Eric Anholt eric@anholt.net");
MODULE_DESCRIPTION("BCM2835 clock driver");
diff --git a/drivers/firmware/raspberrypi.c b/drivers/firmware/raspberrypi.c
index a82819a78..dfa362e1c 100644
--- a/drivers/firmware/raspberrypi.c
+++ b/drivers/firmware/raspberrypi.c
@@ -457,7 +457,7 @@ static int __init rpi_firmware_init(void)
out1:
return ret;
}
-subsys_initcall(rpi_firmware_init);
+core_initcall(rpi_firmware_init);

static void __init rpi_firmware_exit(void)
{

I'm glad you found that. I was about to post the same thing. I had to use the exact same hack, and it was hell finding it for me. I was also thinking that the Raspberry folks need to address, but in the meantime, we have this messy fix of playing with init calls.

FYI, I tested this on an aarch64 Ubuntu server installation for RPi 4, and I can confirm it works there too, but there's no reason it shouldn't.

[lukehinds]

I'm glad you found that. I was about to post the same thing. I had to use the exact same hack, and it was hell finding it for me. I was also thinking that the Raspberry folks need to address, but in the meantime, we have this messy fix of playing with init calls.

FYI, I tested this on an aarch64 Ubuntu server installation for RPi 4, and I can confirm it works there too, but there's no reason it shouldn't.

I was just about to post how it did not work for me, perhaps I need to double check.

Do you mind sharing your .config and what version of kernel you built against (master perhaps)?

[pelwell]

I'm open to a change like this (although it feels like a sticking plaster where something more fundamental is needed) provided it a) solves the problem, b) doesn't break anything else, and c) is acceptable upstream.

[lukehinds]

scrap my previous comment, it works:

sudo head -5 /sys/kernel/security/ima/ascii_runtime_measurements 
10 1d8d532d463c9f8c205d0df7787669a85f93e260 ima-ng sha1:0000000000000000000000000000000000000000 boot_aggregate
10 e338a254c18fa6b433e8929d2d99cd75f6d23779 ima-ng sha1:1129207b8da365c8eef91d27fb27c49581183e5c /lib/modules/4.19.79-v7+/kernel/net/ipv6/ipv6.ko
10 e7db6a5090068cf070a257ce27185c96654a9a47 ima-ng sha1:696b46b87d44fedbc081770f404ce983702806fd /lib/modules/4.19.79-v7+/kernel/net/netfilter/x_tables.ko
10 8a43424ee0c856e9f372bc37ce736e33d647c076 ima-ng sha1:5f0d0063db564cf009eac34464161a0f2c927b27 /lib/modules/4.19.79-v7+/kernel/net/ipv4/netfilter/ip_tables.ko
10 1edc441db35eebce7f4156182272efa0e732b886 ima-ng sha1:b07e26900585c0e1058a16e85f63e5dd05bda390 /lib/systemd/system-generators/systemd-bless-boot-generator

[glassman81]

scrap my previous comment, it works:

That's great! I guess there's no need to post my config then? The only other thing I may have done differently is make sure the SPI and TPM drivers were directly built into the kernel as opposed to being loaded as a module.

On an unrelated note, does anyone out there can know how to restrict the IMA policy to a point where PCR 10 has the same hash value upon subsequent reboots? That would be extremely useful.

[lukehinds]

scrap my previous comment, it works:

That's great! I guess there's no need to post my config then? The only other thing I may have done differently is make sure the SPI and TPM drivers were directly built into the kernel as opposed to being loaded as a module.

These were my steps:

https://gist.github.com/lukehinds/18785b3e1f625dba49c8bcac57cbe6c7

On an unrelated note, does anyone out there can know how to restrict the IMA policy to a point where PCR 10 has the same hash value upon subsequent reboots? That would be extremely useful.

To my knowledge, boot a completely unchanged system (which I don't think is even possible).

[pelwell]

Since a few of you seem happy editing kernel source code, can you put together a Pull Request with the necessary change? It will act as a focal point for the discussion of whether or not it is acceptable.

[lukehinds]

Since a few of you seem happy editing kernel source code, can you put together a Pull Request with the necessary change? It will act as a focal point for the discussion of whether or not it is acceptable.

On it's way.

[lukehinds]

hi @pelwell , there is not much discussion up yet for the pull request #3297 . do you know anyone I could tag who might be able to review, or even approve?

[JamesH65]

Awiting comment on #3297