Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

esp32: make use of wear levelling from IDF #4999

Open
wants to merge 1 commit into
base: master
from

Conversation

@c0d3z3r0
Copy link

commented Aug 11, 2019

This is a rework of loboris' wear leveling patch
micropython/micropython-esp32#126

Signed-off-by: Michael Niewöhner foss@mniewoehner.de

Tested and running productive on my device

esp32: make use of wear levelling from IDF
This is a rework of loboris' wear leveling patch
micropython/micropython-esp32#126

Signed-off-by: Michael Niewöhner <foss@mniewoehner.de>
@nevercast

This comment has been minimized.

Copy link
Contributor

commented Aug 13, 2019

May conflict with #4987
Should be checked.

@@ -26,9 +27,8 @@ def ioctl(self, op, arg):
return self.SEC_SIZE

size = esp.flash_size()
if size < 1024*1024:
if size < 512*1024:

This comment has been minimized.

Copy link
@nevercast

nevercast Aug 16, 2019

Contributor

Are you including the wear level size here? I'm not sure I see that.
#3576 (comment)
https://www.esp32.com/viewtopic.php?t=1897

This comment has been minimized.

Copy link
@c0d3z3r0

c0d3z3r0 Aug 20, 2019

Author

uhm... not really sure... but I guess this should be the minimum partition size, so this should be 528k... have to check this again, to be really sure

@dpgeorge dpgeorge added the port-esp32 label Aug 17, 2019

@c0d3z3r0

This comment has been minimized.

Copy link
Author

commented Aug 20, 2019

May conflict with #4987
Should be checked.

Yup and with #5027, #4854 and #3576 so we should wait for them; then we can simply drop the flashbdev patch from this PR

@dpgeorge

This comment has been minimized.

Copy link
Member

commented Aug 21, 2019

Thanks for the contribution.

But I guess this will no longer work if #5027 is merged. Because that patch removes calls to the esp.flash_read etc functions and instead uses a Partition object.

Do you know how to make wear levelling work with the IDF partition interface?

@nevercast

This comment has been minimized.

Copy link
Contributor

commented Aug 21, 2019

I see that wl_mount takes an esp_partition_t*

@nevercast

This comment has been minimized.

Copy link
Contributor

commented Aug 21, 2019

Actually, looking at the high level API, https://docs.espressif.com/projects/esp-idf/en/latest/api-reference/storage/wear-levelling.html#high-level-api-reference, it seems this is very much supported. Should be able to implement wear levelling as a Partition object that uses the high level API provided here as a middleware.

Edit:

WL doesn't explicitly mention anything about Flash Encryption. Its my assumption that it would be handled transparently if the partition table for a fatfs partition was tagged with encrypted. This would need to be verified.

Last thought:
I think I'm of the opinion that the vfs partitions, via the Partition class, should default to wear-levelling. I'm not sure I see a need otherwise; unless it breaks being able to read and write the fat partition via esptool / creating a fat image on the PC and pushing it to an ESP32.

It would be my preference that wear levelling just works and just happens whenever a read/write partition is mounted.

@c0d3z3r0

This comment has been minimized.

Copy link
Author

commented Aug 21, 2019

I see that wl_mount takes an esp_partition_t*

Yes, a partition must be "mounted as wearlevel partition".

Actually, looking at the high level API, https://docs.espressif.com/projects/esp-idf/en/latest/api-reference/storage/wear-levelling.html#high-level-api-reference, it seems this is very much supported. Should be able to implement wear levelling as a Partition object that uses the high level API provided here as a middleware.

The problem here is that we DO NOT use the highlevel API / espressif's vfs but instead build on our own fatfs and vfs implementation. This means no, we can not use that. The partitions must be mounted explicitely through the mid level API (wl*). @dpgeorge's code uses esp_partition_write which relies on spi_flash_write which does not do wear levelling natively.

Edit:

WL doesn't explicitly mention anything about Flash Encryption. Its my assumption that it would be handled transparently if the partition table for a fatfs partition was tagged with encrypted. This would need to be verified.

Wear levelling is completely transparent, as is encryption. You can mount an unencrypted or an encrypted partition as wearlevelling partition:

spi flash <- partition (encrypted or unencrypted) <- wear levelling <- <mapped blocks / partition> <- filesystem

Last thought:
I think I'm of the opinion that the vfs partitions, via the Partition class, should default to wear-levelling. I'm not sure I see a need otherwise;

unless it breaks being able to read and write the fat partition via esptool / creating a fat image on the PC and pushing it to an ESP32.

This is NOT possible with wear levelling currently as esptool writes directly to flash. You must create a wear levelling partition image and flash that. There's a tool for this: https://github.com/TobleMiner/mk_esp32fat

It would be my preference that wear levelling just works and just happens whenever a read/write partition is mounted.

The right way would be dropping our own vfs and fatfs and instead build on espressif's vfs and fatfs which both natively support encryption and wear levelling. This would even make it possible to easily use spiffs or anything else that is getting added to esp-idf in later versions.

@nevercast

This comment has been minimized.

Copy link
Contributor

commented Aug 21, 2019

Thanks for taking the time to write up the detailed reply, clears a lot of thoughts. 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.