Virtual Efuses do Not Work When Flash Encryption Previously Enabled (IDFGH-9580) #10929

sidwarkd · 2023-03-06T20:06:17Z

Answers checklist.

I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
I have searched the issue tracker for a similar issue and not found a similar issue.

IDF version.

v4.4.4

Operating System used.

Linux

How did you build your project?

Command line with idf.py

If you are using Windows, please specify command line type.

None

Development Kit.

ESP32 Pico DevKitM-2 v1.0

Power Supply used.

USB

What is the expected behavior?

If flash encryption was enabled prior to enabling virtual efuses I expect the application to run normally after enabling virtual efuses.

What is the actual behavior?

If flash encryption was enabled prior to enabling virtual efuses the bootloader appears to read complete garbage from the efuses and the device fails to boot. The documentation suggests that all flash reads are "transparently decrypted" when encryption is enabled but this does not appear to be happening.

Steps to reproduce.

Set up simple hello world project. Nothing special
Enable flash encryption
Allow the device to boot and encrypt all applicable partitions.
Enable virtual efuses
Boot device again. Device will read garbage from efuses and not boot.
Disable virtual efuses and reflash device with idf.py encrypted-flash monitor. Device will boot and run normally
Disable flash encryption (both in menuconfig as well as burning the FLASH_CRYPT_CNT fuse).
Re-enable virtual efuses
Flash device and run. Notice it will run normally with virtual efuses correctly loading.

Debug Logs.

ets Jul 29 2019 12:21:46

rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 271414342, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0140,len:11044
load:0x40078000,len:20828
load:0x40080400,len:3832
0x40080400: _init at ??:?

entry 0x4008069c
W (71) boot.esp32: eFuse virtual mode is enabled. If Secure boot or Flash encryption is enabled then it does not provide any security. FOR TESTING ONLY!
I (42) boot: ESP-IDF v4.4.4 2nd stage bootloader
I (42) boot: compile time 19:34:26
I (43) boot: chip revision: v3.0
I (45) boot.esp32: SPI Speed      : 80MHz
I (50) boot.esp32: SPI Mode       : DIO
I (55) boot.esp32: SPI Flash Size : 8MB
I (59) boot: Enabling RNG early entropy source...
I (65) boot: Partition Table:
I (68) boot: ## Label            Usage          Type ST Offset   Length
I (76) boot:  0 nvs              WiFi data        01 02 0000c000 00002000
I (83) boot:  1 nvs_custom       WiFi data        01 02 0000e000 00003000
I (90) boot:  2 otadata          OTA data         01 00 00011000 00002000
I (98) boot:  3 nvs_key          NVS keys         01 04 00013000 00001000
I (105) boot:  4 custom_key       WiFi data        01 02 00014000 0000a000
W (113) efuse: Loading virtual efuse blocks from flash
EFUSE_BLKx:
0) 0x4f22d9a1 0xeb46f811 0x5b013283 0x3b26ebb6 0xfdbfbc9a 0x6be293b5 0x653d821e 
1) 0x28347cbd 0x20dc5c62 0x0d4d9205 0x3bafef13 0x9ab0b0f4 0x56521c2e 0xae8c16ed 0xa2fa538d 
2) 0x933c2c83 0x6403b626 0x251e3498 0xc94049a9 0x933c2c83 0x6403b626 0x251e3498 0xc94049a9 
3) 0x9ed26466 0xbb0e112f 0x3a965276 0x55b25a3f 0x9ed26466 0xbb0e112f 0x3a965276 0x55b25a3f 

I (151) boot:  5 efuse_em         efuse            01 05 0001e000 00002000
I (159) boot:  6 ota_0            OTA app          00 10 00020000 00390000
I (166) boot:  7 ota_1            OTA app          00 11 003b0000 00390000
I (174) boot:  8 storage          Unknown data     01 81 00740000 00080000
I (182) boot:  9 coredump         Unknown data     01 03 007c0000 00010000
I (189) boot: End of partition table
I (193) boot: No factory image, trying OTA 0
I (198) esp_image: segment 0: paddr=00020020 vaddr=3f400020 size=09b30h ( 39728) map
I (219) esp_image: segment 1: paddr=00029b58 vaddr=3ffb0000 size=021b8h (  8632) load
I (223) esp_image: segment 2: paddr=0002bd18 vaddr=40080000 size=04300h ( 17152) load
I (232) esp_image: segment 3: paddr=00030020 vaddr=400d0020 size=14f68h ( 85864) map
I (261) esp_image: segment 4: paddr=00044f90 vaddr=40084300 size=05658h ( 22104) load
I (275) boot: Loaded app from partition at offset 0x20000
I (337) boot: Set actual ota_seq=1 in otadata[0]
I (337) boot: Checking flash encryption...
I (337) esp_image: segment 0: paddr=00001020 vaddr=3fff0140 size=02b24h ( 11044) 
I (346) esp_image: segment 1: paddr=00003b4c vaddr=40078000 size=0515ch ( 20828) 
I (355) esp_image: segment 2: paddr=00008cb0 vaddr=40080400 size=00ef8h (  3832) 
I (941) flash_encrypt: bootloader encrypted successfully
E (942) flash_encrypt: Failed to read partition table data - not plaintext?
E (944) boot: Flash encryption check failed (259).
E (949) boot: OTA app partition slot 0 is not bootable
E (955) esp_image: image at 0x3b0000 has invalid magic byte (nothing flashed here?)
E (963) boot: OTA app partition slot 1 is not bootable
E (969) boot: No bootable app partitions in the partition table
ets Jul 29 2019 12:21:46

rst:0x3 (SW_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 271414342, SPIWP:0xee

More Information.

As an additional comment it appears the documentation is wrong about flash encryption and the FLASH_CRYPT_CNT efuse. The documentation states "If even number of bits set (0, 2, 4, 6) - encrypt flash at boot time. If odd number of bits set (1, 3, 5, 7) - do not encrypt flash at boot time." However, when I have flash encryption disabled (confirmed with boot log) and I do an espefuse.py summary I get the following.

FLASH_CRYPT_CNT (BLOCK0):                          Flash encryption mode counter                      = 3 R/W (0b0000011)

There are an even number of bits set (two) and flash encryption is disabled. Which would also be true of the device from the factory with zero bits set. So it seems in reality it's the exact opposite of what the docs state. Can you confirm this?

The text was updated successfully, but these errors were encountered:

chipweinberger · 2023-03-06T22:07:38Z

yes this is a common issue.

you need to erase the flash sometime before enabling virtual fuses.

I personally think esp-idf should put a magic value in the virtual fuses partition, and assume all zeros or something sane if the magic value is not there.

sidwarkd · 2023-03-06T22:35:14Z

@chipweinberger Are you suggesting that if, in between my steps 3 and 4, I do a full flash erase that virtual fuses will work properly (when enabled in step 4) with flash encryption enabled? I want to clarify because you only get so many enable/disable cycles with flash encryption.

chipweinberger · 2023-03-06T23:01:38Z

you just need to make sure the virtual fuses partition is zero'd before you boot.

You can do it at any step before boot.

Note: if you reflash a different partition table youll need to erase the virtual fuses partition again.

sidwarkd · 2023-03-07T03:26:43Z

@chipweinberger this did not work for me. I verified the partition was fully erased, still experience this same issue with garbage efuse values.

KonstantinKondrashov · 2023-03-07T16:05:35Z

Hi @sidwarkd!

Set up simple hello world project. Nothing special

Enable flash encryption
Allow the device to boot and encrypt all applicable partitions.
Enable virtual efuses
...

Your chip is really encrypted at the (2) point. Why do you need to enable virt efuses?
When virt efuse mode is on then no real flash encryption happens.

"Enables/disables encryption at boot time. If even number of bits set (0, 2, 4, 6) - encrypt flash at boot time. If odd number of bits set (1, 3, 5, 7) - do not encrypt flash at boot time."https://docs.espressif.com/projects/esp-idf/en/latest/esp32/security/flash-encryption.html#relevant-efuses

You misunderstood it, it means that the bootloader during the first boot will take a look at FLASH_CRYPT_CNT and encrypt the images that require encryption if it is 0, 2, 4, 6, after encryption is done burn the next bit. So Flash encryption is on when it is 1 or 3 or 5 or 7.

Looks like efuse_em was encrypted too, but shouldn't be encrypted.
efuse_em should be erased before using otherwise the given data are used.

sidwarkd · 2023-03-07T20:19:35Z

@KonstantinKondrashov Thank you for the clarification on the number of bits set in FLASH_CRYPT_CNT. That makes sense.

I went back to my example and ran esptool.py erase_region to clear the efuse partition and it still will not boot with virtual efuses enabled with flash storage. It continues to read garbage on boot. Have you tried to reproduce the issue with the steps above?

As for efuses after encryption, there are lots of other things to test (Secure Boot) that you may want to not permanently burn efuses for. When you say "When virt efuse mode is on then no real flash encryption happens" what do you mean? From the Virtual Efuses section it states:

This option allows keeping eFuses after reboots (possible to test secure_boot and flash_encryption features with this option).

If you can't test flash encryption with virtual efuses then the documentation should be updated to clarify. Thanks again for your help and the clarification.

KonstantinKondrashov · 2023-03-16T08:42:40Z

@sidwarkd When CONFIG_EFUSE_VIRTUAL_KEEP_IN_FLASH=y then bootloader_flash_write() does not do any encryption operations -https://github.com/espressif/esp-idf/blob/master/components/bootloader_support/bootloader_flash/src/bootloader_flash.c#L95-L102.

This option allows keeping eFuses after reboots (possible to test secure_boot and flash_encryption features with this option).

Actually, we use this for testing here https://github.com/espressif/esp-idf/blob/master/examples/system/efuse/sdkconfig.ci.virt_sb_v2_and_fe.esp32. But not in that way as you.

I tried the way you suggested and it really shows garbage data. It is wired because the efuse_em partition should not be encrypted. I found a bug, https://github.com/espressif/esp-idf/blob/master/components/efuse/src/esp_efuse_utility.c#L470. The last arg must be false, not true. I will fix it on the mater and backport to 4.4 as well.

esp_err_t err = bootloader_flash_read(esp_efuse_flash_offset, &efuses_in_flash, sizeof(efuses_in_flash), false);

Sorry for the delay.

chipweinberger · 2023-03-16T09:54:44Z

Nice debugging Konstantin!

sidwarkd · 2023-03-16T22:07:07Z

@KonstantinKondrashov Thanks for digging in on this and finding the issue and fixing it. Great work!

I am still a little bit confused on the documentation side. If bootloader_flash_write() doesn't do any encryption operations then should the documentation be changed? If it's not doing any encryption then you really aren't testing flash encryption as the documentation suggests you are able to. Perhaps it should just say "possible to test secure_boot with this option".

Thanks again for your help.

KonstantinKondrashov · 2023-03-17T07:56:39Z

Flash encryption is a hardware feature, so it requires that eFuses (key and FLASH_CRYPT_CNT) should be burned fiscally otherwise FE does not work. In case when the VIRTUAL efuse mode is on then real eFuses (in regs) are not fiscally burned. Given that if real FE is not on then we should prevent the encryption in bootloader_flash_write(). Such tests we do on our CI. There are other tests that require that FE is already enabled for those tests we do not use the efuse virtual mode.
Back to your case, you enable FE first and then use the efuse virt mode, I can suggest you remove the blocking condition in the bootloader_flash_write() (note there are two the same func for bootloader and app) to reach what you want.

Probably it makes sense to extend that condition to check whether the FE is really enabled and do encryption then. That will fit your needs.

sidwarkd · 2023-03-17T14:51:18Z

@KonstantinKondrashov Sorry for the confusion. All of my needs are met with the fix you have provided. I was merely pointing out the documentation shouldn't suggest you can test flash encryption with virtual efuses when you cannot.

ermacv · 2023-03-20T07:46:20Z

I agree with @sidwarkd that this should be written in the docs because it is really confusing that there is no real flash encryption applied.

esp_efuse_utility_load_efuses_from_flash() read emul_efuse as an encrypted partition, but that is not correct, this partition was never encrypted. Need to read it as not encrypted partition. Fxed the case: If FE is already on then EFUSE VIRT mode can work with it. Closes espressif/esp-idf#10929

esp_efuse_utility_load_efuses_from_flash() read emul_efuse as an encrypted partition, but that is not correct, this partition was never encrypted. Need to read it as not encrypted partition. Fxed the case: If FE is already on then EFUSE VIRT mode can work with it. Closes #10929

sidwarkd added the Type: Bug bugs in IDF label Mar 6, 2023

espressif-bot added the Status: Opened Issue is new label Mar 6, 2023

github-actions bot changed the title ~~Virtual Efuses do Not Work When Flash Encryption Previously Enabled~~ Virtual Efuses do Not Work When Flash Encryption Previously Enabled (IDFGH-9580) Mar 6, 2023

KonstantinKondrashov self-assigned this Mar 7, 2023

espressif-bot assigned KonstantinKondrashov and unassigned KonstantinKondrashov Mar 8, 2023

espressif-bot added Status: In Progress Work is in progress and removed Status: Opened Issue is new labels Mar 16, 2023

espressif-bot added Status: Selected for Development Issue is selected for development Status: In Progress Work is in progress and removed Status: In Progress Work is in progress Status: Selected for Development Issue is selected for development labels Mar 16, 2023

espressif-bot closed this as completed in 88ef0d8 Apr 3, 2023

espressif-bot added Resolution: Done Issue is done internally Status: Done Issue is done internally labels Apr 3, 2023

espressif-bot removed the Status: In Progress Work is in progress label Apr 3, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Virtual Efuses do Not Work When Flash Encryption Previously Enabled (IDFGH-9580) #10929

Virtual Efuses do Not Work When Flash Encryption Previously Enabled (IDFGH-9580) #10929

sidwarkd commented Mar 6, 2023 •

edited

chipweinberger commented Mar 6, 2023 •

edited

sidwarkd commented Mar 6, 2023 •

edited

chipweinberger commented Mar 6, 2023 •

edited

sidwarkd commented Mar 7, 2023

KonstantinKondrashov commented Mar 7, 2023

sidwarkd commented Mar 7, 2023

KonstantinKondrashov commented Mar 16, 2023

chipweinberger commented Mar 16, 2023

sidwarkd commented Mar 16, 2023

KonstantinKondrashov commented Mar 17, 2023

sidwarkd commented Mar 17, 2023

ermacv commented Mar 20, 2023

Virtual Efuses do Not Work When Flash Encryption Previously Enabled (IDFGH-9580) #10929

Virtual Efuses do Not Work When Flash Encryption Previously Enabled (IDFGH-9580) #10929

Comments

sidwarkd commented Mar 6, 2023 • edited

Answers checklist.

IDF version.

Operating System used.

How did you build your project?

If you are using Windows, please specify command line type.

Development Kit.

Power Supply used.

What is the expected behavior?

What is the actual behavior?

Steps to reproduce.

Debug Logs.

More Information.

chipweinberger commented Mar 6, 2023 • edited

sidwarkd commented Mar 6, 2023 • edited

chipweinberger commented Mar 6, 2023 • edited

sidwarkd commented Mar 7, 2023

KonstantinKondrashov commented Mar 7, 2023

sidwarkd commented Mar 7, 2023

KonstantinKondrashov commented Mar 16, 2023

chipweinberger commented Mar 16, 2023

sidwarkd commented Mar 16, 2023

KonstantinKondrashov commented Mar 17, 2023

sidwarkd commented Mar 17, 2023

ermacv commented Mar 20, 2023

sidwarkd commented Mar 6, 2023 •

edited

chipweinberger commented Mar 6, 2023 •

edited

sidwarkd commented Mar 6, 2023 •

edited

chipweinberger commented Mar 6, 2023 •

edited