How to do 16 byte alignment in Secure boot Flash encryption

[horsemann07]

Hello everyone,

I was trying to run OTA in the secure boot and flash encrypted device in esp32. At first, it tried to update with encrypted .bin files and it got failed. I got some clarification, is that right now esp32 does not support pre-encrypted image from OTA Instead it takes a plain-text image and encrypts it using internal flash encrypted hardware.
So, I tried to do OTA using a plain text .bin file at but I got an error on that.

Log:

1361 55181 [OTA Agent Task] [prvIngestDataBlock] Received file block 312, size 4096
1362 55182 [OTA Agent Task] [prvIngestDataBlock] Remaining: 2
1363 55182 [OTA Agent Task] [prvOTAAgentTask] Called handler. Current State [WaitingForFileBlock] Event [ReceivedFileBlock] New state [WaitingForFileBlock] 
1364 55242 [OTA Agent Task] [prvIngestDataBlock] Received file block 313, size 4096
1365 55243 [OTA Agent Task] [prvIngestDataBlock] Remaining: 1
1366 55243 [OTA Agent Task] [prvOTAAgentTask] Called handler. Current State [WaitingForFileBlock] Event [ReceivedFileBlock] New state [WaitingForFileBlock] 
E (552870) esp_ota_ops: size should be 16byte aligned for flash encryption case
E (552870) ota_pal: Couldn't flash at the offset 1286144
I (552880) ota_pal: prvPAL_SetPlatformImageState, 3
W (552880) ota_pal: Set image as invalid!
I (552890) esp_ota_ops: aws_esp_ota_get_boot_flags: 1
W (552890) esp_ota_ops: otadata partition is invalid, factory/ota_0 is boot partition
E (552900) ota_pal: currently executing firmware not marked as valid, abort
1367 55283 [OTA Agent Task] [prvIngestDataBlock] Received file block 314, size 3364
1368 55284 [OTA Agent Task] [prvIngestDataBlock] Error (-1) writing file block
1369 55284 [OTA Agent Task] [prvIngestDataBlock] Remaining: 1
1370 55284 [OTA Agent Task] [prvStopRequestTimer] Stopping request timer.
1371 55284 [OTA Agent Task] [prvProcessDataMessage] Aborting due to IngestResult_t error -9
1372 55287 [OTA Agent Task] [prvUpdateJobStatus_Mqtt] Msg: {"status":"FAILED","statusDetails":{"reason":"0x27000000: 0xfffffff7"}}
1373 55287 [OTA Agent Task] [INFO ][MQTT][552870] (MQTT connection 0x3ffb40ec) MQTT PUBLISH operation queued.
1374 55287 [OTA Agent Task] [INFO ][MQTT][552870] (MQTT connection 0x3ffb40ec, PUBLISH operation 0x3ffdbd98) Waiting for operation completion.
1375 55303 [OTA Agent Task] [INFO ][MQTT][553030] (MQTT connection 0x3ffb40ec, PUBLISH operation 0x3ffdbd98) Wait complete with result SUCCESS.
1376 55303 [OTA Agent Task] [prvUpdateJobStatus_Mqtt] 'FAILED' to $aws/things/FW_246F2824E888/jobs/AFR_OTA-SecureBoot-7/update
1377 55303 [OTA Agent Task] Received eOTA_JobEvent_Fail callback from OTA Agent.
1378 55303 [OTA Agent Task] [prvOTAAgentTask] Called handler. Current State [WaitingForFileBlock] Event [ReceivedFileBlock] New state [WaitingForFileBlock] 
1379 55303 [OTA Agent Task] [prvOTA_Close] Context->0x0x3ffbf850
1380 55303 [OTA Agent Task] [prvOTAAgentTask] Called handler. Current State [WaitingForFileBlock] Event [CloseFile] New state [WaitingForJob] 
I (553690) ota_pal: prvPAL_SetPlatformImageState, 4
W (553690) ota_pal: Set image as aborted!
I (553690) esp_ota_ops: aws_esp_ota_get_boot_flags: 1
W (553700) esp_ota_ops: otadata partition is invalid, factory/ota_0 is boot partition
E (553700) ota_pal: currently executing firmware not marked as valid, abort
1381 55365 [OTA Agent Task] [prvParseJobDoc] Size of OTA_FileContext_t [64]
1382 55365 [OTA Agent Task] [prvParseJSONbyModel] parameter not present: execution
1383 55365 [OTA Agent Task] [prvParseJSONbyModel] parameter not present: jobId
1384 55365 [OTA Agent Task] [prvParseJSONbyModel] parameter not present: jobDocument
1385 55365 [OTA Agent Task] [prvParseJSONbyModel] parameter not present: afr_ota
1386 55365 [OTA Agent Task] [prvParseJSONbyModel] parameter not present: protocols
1387 55365 [OTA Agent Task] [prvParseJSONbyModel] parameter not present: files
1388 55365 [OTA Agent Task] [prvParseJSONbyModel] parameter not present: filepath
1389 55365 [OTA Agent Task] [prvParseJSONbyModel] parameter not present: filesize
1390 55365 [OTA Agent Task] [prvParseJSONbyModel] parameter not present: fileid
1391 55365 [OTA Agent Task] [prvParseJSONbyModel] parameter not present: certfile
1392 55365 [OTA Agent Task] [prvParseJSONbyModel] parameter not present: sig-sha256-ecdsa
1393 55365 [OTA Agent Task] [prvDefaultCustomJobCallback] Received Custom Job inside OTA Agent which is not supported.
1394 55365 [OTA Agent Task] [prvParseJobDoc] Ignoring job without ID.
1395 55365 [OTA Agent Task] [prvOTA_Close] Context->0x0x3fff6da0
1396 55367 [OTA Agent Task] [prvOTAAgentTask] Handler failed. Current State [WaitingForJob] Event  [ReceivedJobDocument] Error Code [603979776] 
1397 55402 [iot_thread] State: WaitingForFileBlock  Received: 318   Queued: 0   Processed: 0   Dropped: 0

partition table which I m using :

# Name,   Type, SubType, Offset,  Size, Flags
# Note: if you change the phy_init or app partition offset, make sure to change the offset in Kconfig.projbuild,,,,
nvs,      data, nvs,     0x14000,  0x6000,
otadata,  data, ota, ,  0x2000,
phy_init, data, phy,  ,  0x1000,
factory,0,    factory,,  1400K,
ota_0,0,    ota_0,   ,         1400K,
ota_1,0,    ota_1,   ,         200K,
file,  data,spiffs,     ,0x6000,

My .bin size files is 12,89,508 bytes

What does this mean?
E (552870) esp_ota_ops: size should be 16byte aligned for flash encryption case

[mingyue86010]

Hi @Raghav3107,

16byte aligned means the memory address of your data needs to be a multiple of 16. According to the error message that "size should be 16byte aligned", it requires your image size to be 16byte aligned. In your case your .bin size files is 12,89,508 bytes you may padding with some 0s at end to make the size to 12,89,510 bytes which is 16 byte aligned. I guess this requirement is comming from the underlying block encryption algorithm.

Hope it can help you.

Regards,
Ming

[horsemann07]

Hello @mingyue86010

In your case your .bin size files is 12,89,508 bytes you may padding with some 0s at end to make the size to 12,89,510 bytes which is 16 byte aligned.

I did not understand how should I pad with 0's to increase the size of two bytes.
Shall declare two char variable in code so it increases two byte or is something else I have to do?

Thanks!!

[mingyue86010]

Hi @Raghav3107,

For paddings you need check the tools that you generates your OTA image. Or you can check your linker to see how to do this.

From your log I also notice another potential problem that you probably need align the address as well. I found some docs from ESP website:

https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/system/ota.html --> esp_ota_write_with_offsetz() which require data should be 16 byte aligned.

Then further to their SPI flash doc:
https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/storage/spi_flash.html --> esp_flash_write_encrypted() which requires both address & length must be 16 byte aligned.

You have the error E (552870) ota_pal: Couldn't flash at the offset 1286144 which might be the address alignment issue. So try to get the image flash to a 16 bytes aliegned addresss or padding the size of image to 16 bytes aligned or both too see if it can resolve your problem.

Regards,
Ming

[horsemann07]

Hello @mingyue86010
Sorry for the late reply.

In your case your .bin size files is 12,89,508 bytes you may padding with some 0s at end to make the size to 12,89,510 bytes which is 16 byte aligned.

I divided 12,89,510 with 16 and I got 80594.375. So, I think it is also not 16 bytes aligned.

For paddings you need check the tools that you generates your OTA image.

Can you tell me where I will find the tools that you generate your OTA image?

Or you can check your linker to see how to do this.

I did not understand this. which linker you are talking about? Where I will find it?

If possible can you clarify in detail how I shall align the file to 16 bytes?
Is there any method to align the file to 16 bytes offset?

Thanks for your response.

[mingyue86010]

Hi @Raghav3107,

It looks it is some requirement by Espressif. Maybe it's better to put this question to Espressif forum https://www.esp32.com/viewforum.php?f=23 they may have more insight of this. The ESP32 secure bootloader has the requirement of 16 bytes image size and address alignment and they also have the requirement of the partition alignment as https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-guides/partition-tables.html#offset-size. With some investagation by myself:

1. The first error you got indicate you probably enabled the ESP flash encryption. And when the the OTA write the image to flash, the data passed to the write function didn't meet the alignment requirement (https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/system/ota.html). To padding an image size, my suggestion is to check the user guide of the linker you are using and revise your linker script. Usually the linker will provide you the command to file/pad a portion of memory also you can you use the linker to define where sections of data/code need to go.

2. The second error you got "otadata partition is invalid". I'd suggest you to check with your partition setting of ESP32 with https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/system/ota.html#ota-data-partition .

Hope those infomation can help you debugging these issue.

Thanks,

Ming Yue

[shubhamkulkarni97]

Hi @Raghav3107, could you share the steps you performed to generate OTA image?

From the error logs I think you have not enabled secure boot and flash encryption options in menuconfig while generating OTA image.

ESP-IDF build system takes care of alignment by padding the binary only when security features are enabled. This restriction of 16 byte alignment is not applicable when security features are disabled.

Thanks,
Shubham

[horsemann07]

Hello @shubhamkulkarni97
As ESP_Angus said on esp-forum
The ESP-IDF OTA mechanism relies on receiving the plaintext image via OTA, and it's then encrypted using the internal flash encryption hardware as it it is written to flash. This is why you see these errors, the firmware doesn't recognize the .bin file as a valid format.

So,I simply generated the OTA image bin file without enabling the secure boot and flash encryption. That time I got above error.

---

But Today I tried enabling the secure boot and flash encyption and build the project. After that I got two project bin file i.e aws_demos and aws_demos-unsigned.bin.
So, first I created the ota with aws_demos.bin file and got error

1277 26256 [OTA Agent Task] [prvOTAAgentTask] Called handler. Current State [WaitingForFileBlock] Event [RequestFileBlock] New state [WaitingForFileBlock] 
E (262961) esp_ota_ops: size should be 16byte aligned for flash encryption case
E (262961) ota_pal: Couldn't flash at the offset 1282048
I (262961) ota_pal: prvPAL_SetPlatformImageState, 3
W (262971) ota_pal: Set image as invalid!
I (262971) esp_ota_ops: aws_esp_ota_get_boot_flags: 1
W (262981) esp_ota_ops: otadata partition is invalid, factory/ota_0 is boot partition
E (262981) ota_pal: currently executing firmware not marked as valid, abort
1278 26292 [OTA Agent Task] [prvIngestDataBlock] Received file block 313, size 3300
1279 26292 [OTA Agent Task] [prvIngestDataBlock] Error (-1) writing file block
1280 26292 [OTA Agent Task] [prvIngestDataBlock] Remaining: 2
1281 26292 [OTA Agent Task] [prvStopRequestTimer] Stopping request timer.
1282 26292 [OTA Agent Task] [prvProcessDataMessage] Aborting due to IngestResult_t error -9
1283 26296 [OTA Agent Task] [prvUpdateJobStatus_Mqtt] Msg: {"status":"FAILED","statusDetails":{"reason":"0x27000000: 0xfffffff7"}}
1284 26296 [OTA Agent Task] [INFO ][MQTT][262960] (MQTT connection 0x3ffb5fc4) MQTT PUBLISH operation queued.
1285 26296 [OTA Agent Task] [INFO ][MQTT][262960] (MQTT connection 0x3ffb5fc4, PUBLISH operation 0x3ffdcbd0) Waiting for operation completion.
1286 26329 [OTA Agent Task] [INFO ][MQTT][263290] (MQTT connection 0x3ffb5fc4, PUBLISH operation 0x3ffdcbd0) Wait complete with result SUCCESS.
1287 26329 [OTA Agent Task] [prvUpdateJobStatus_Mqtt] 'FAILED' to $aws/things/FW_246F2824E888/jobs/AFR_OTA-sb3-test/update
1288 26329 [OTA Agent Task] Received eOTA_JobEvent_Fail callback from OTA Agent.
1289 26329 [OTA Agent Task] [prvOTAAgentTask] Called handler. Current State [WaitingForFileBlock] Event [ReceivedFileBlock] New state [WaitingForFileBlock] 
1290 26329 [OTA Agent Task] [prvIngestDataBlock] Received file block 312, size 4096
1291 26331 [OTA Agent Task] [prvIngestDataBlock] Remaining: 1
1292 26331 [OTA Agent Task] [prvOTAAgentTask] Called handler. Current State [WaitingForFileBlock] Event [ReceivedFileBlock] New state [WaitingForFileBlock] 
1293 26331 [OTA Agent Task] [prvOTA_Close] Context->0x0x3ffbf854
1294 26331 [OTA Agent Task] [prvOTAAgentTask] Called handler. Current State [WaitingForFileBlock] Event [CloseFile] New state [WaitingForJob] 
I (263701) ota_pal: prvPAL_SetPlatformImageState, 4
W (263701) ota_pal: Set image as aborted!
I (263701) esp_ota_ops: aws_esp_ota_get_boot_flags: 1
W (263701) esp_ota_ops: otadata partition is invalid, factory/ota_0 is boot partition
E (263711) ota_pal: currently executing firmware not marked as valid, abort

Then I create OTA job using aws_demos-unsigned.bin now this time OTA begin but after receiving the file I got the error

1285 28904 [OTA Agent Task] [prvIngestDataBlock] Received file block 312, size 4096
1286 28905 [OTA Agent Task] [prvIngestDataBlock] Remaining: 1
1287 28905 [OTA Agent Task] [prvOTAAgentTask] Called handler. Current State [WaitingForFileBlock] Event [ReceivedFileBlock] New state [WaitingForFileBlock] 
1288 28914 [OTA Agent Task] [prvIngestDataBlock] Received file block 313, size 3504
I (289179) ota_pal: No such certificate file: certs/device. Using aws_ota_codesigner_certificate.h.

1289 28915 [OTA Agent Task] [prvIngestDataBlock] Received final expected block of file.
1290 28915 [OTA Agent Task] [prvStopRequestTimer] Stopping request timer.
1291 29026 [OTA Agent Task] [prvIngestDataBlock] File receive complete and signature is valid.
1292 29026 [OTA Agent Task] [prvStopRequestTimer] Stopping request timer.
1293 29026 [OTA Agent Task] [prvUpdateJobStatus_Mqtt] Msg: {"status":"IN_PROGRESS","statusDetails":{"self_test":"ready","updatedBy":"0x2000000"}}
1294 29026 [OTA Agent Task] [INFO ][MQTT][290260] (MQTT connection 0x3ffb5f30) MQTT PUBLISH operation queued.
1295 29026 [OTA Agent Task] [INFO ][MQTT][290260] (MQTT connection 0x3ffb5f30, PUBLISH operation 0x3ffdc630) Waiting for operation completion.
1296 29041 [OTA Agent Task] [INFO ][MQTT][290410] (MQTT connection 0x3ffb5f30, PUBLISH operation 0x3ffdc630) Wait complete with result SUCCESS.
I (290449) boot_comm: chip revision: 1, min. application chip revision: 0
I (290449) esp_image: segment 0: paddr=0x00180020 vaddr=0x3f400020 size=0x3af54 (241492) map
1297 29042 [OTA Agent Task] [prvUpdateJobStatus_Mqtt] 'IN_PROGRESS' to $aws/things/FW_246F2824E888/jobs/AFR_OTA-sb-8/update
1298 29042 [OTA Agent Task] Received eOTA_JobEvent_Activate callback from OTA Agent.
I (290589) esp_image: segment 1: paddr=0x001baf7c vaddr=0x3ffbdb60 size=0x03330 ( 13104) 
I (290599) esp_image: segment 2: paddr=0x001be2b4 vaddr=0x40080000 size=0x00400 (  1024) 
0x40080000: _WindowOverflow4 at /home/horsemann/Desktop/WorkSpace/SecureBootTest/freertos_kernel/portable/ThirdParty/GCC/Xtensa_ESP32/xtensa_vectors.S:1685

I (290599) esp_image: segment 3: paddr=0x001be6bc vaddr=0x40080400 size=0x01954 (  6484) 
I (290609) esp_image: segment 4: paddr=0x001c0018 vaddr=0x400d0018 size=0xe45f0 (935408) map
0x400d0018: _flash_cache_start at ??:?

I (291079) esp_image: segment 5: paddr=0x002a4610 vaddr=0x40081d54 size=0x15774 ( 87924) 
0x40081d54: esp_timer_impl_set_alarm at /home/horsemann/Desktop/WorkSpace/SecureBootTest/vendors/espressif/esp-idf/components/esp32/esp_timer_esp32.c:340

I (291119) esp_image: Verifying image signature...
E (291249) esp_image: Secure boot signature verification failed
I (291249) esp_image: Calculating simple hash to check for corruption...
W (291849) esp_image: image valid, signature bad
E (291849) ota_pal: aws_esp_ota_end failed!

[horsemann07]

Hello @shubhamkulkarni97
Can you give me any update on this issue as soon as possible?

Thanks for the help.

[shubhamkulkarni97]

Hi @Raghav3107,

Typical OTA Update workflow is as follows:

• Upload unsigned image to S3 bucket
• AWS OTA Job with appropriate code signing profile will sign image
• AWS OTA Job sends image in small chunks over MQTT + TLS channel (possible that data is received here in out-of-order manner as well)
• AWS OTA Job document (sent initially to device) has corresponding signature of image in ASN1 encoded format

We write signature from step 4 to flash at the end of image (you can check here) and since security scheme (ECDSA + SHA256) is compatible with secure boot scheme in ESP32, it works well for validation as well.

You can follow these steps to correctly perform OTA Update:

• Upload unsigned image to S3 bucket
• Create a AWS code signing profile using the same key used for secure boot. Also create a certificate from secure boot key which will be used as code signing certificate
• Update code signing certificate in ota_demo_config.h

Hope this fixes your issue!

Thanks,
Shubham

[horsemann07]

Hello @shubhamkulkarni97

From the error logs I think you have not enabled secure boot and flash encryption options in menuconfig while generating OTA image.

Can you explain this? what I understand from this is I have to enable the secure boot and flash encryption and in menuconfig and build the program and generate the .bin file. Now this bin file I have to use the for the OTA. Am I right?

If I m right, by doing so i m getting the error "E (552870) esp_ota_ops: size should be 16byte aligned for flash encryption case" allignment issue.

Then I off the secure boot and flash encryption in menuconfig and build the program and i use that bin file for the OTA and surprisingly it partially worked means i did not any get issue at initially after downloading the bin file on esp and when esp try to test the ota image its getting crash. I don' t have any issue in program, program is working perfectly when I flash.

SecureBoot_OTA-PartialSuccess.txt

OTA-Error-27-8-2021.txt

[shubhamkulkarni97]

@Raghav3107,

Can you explain this? what I understand from this is I have to enable the secure boot and flash encryption and in menuconfig and build the program and generate the .bin file. Now this bin file I have to use the for the OTA. Am I right?

You are right. Once you enable secure boot and flash encryption on a device, you should keep it enabled in menuconfig for generation all images (for flashing and for OTA).

If I m right, by doing so i m getting the error "E (552870) esp_ota_ops: size should be 16byte aligned for flash encryption case" allignment issue.

You should always upload unsigned binary as OTA image (e.g. - aws_demos-unsigned.bin). You are observing this error because of uploading signed binary (e.g. - aws_demos.bin).

I'm not able to point the reason for crash, but most probably it occurs because you have generated OTA image with flash encryption and secure boot disabled.

[horsemann07]

Hello @shubhamkulkarni97 it worked.
Thanks for the help.
Before I was using the bin file instead of unsigned bin file.
Now everything is working fine.