upgrade to v2.1.4 #14

pilotak · 2019-09-01T10:21:37Z

Updates outdated alpha version to v2.1.4

pilotak · 2020-02-06T21:55:15Z

for some reason test fails with LFS2_READ_SIZE=64 and LFS2_CACHE_SIZE=64 all other are good, i noticed that in main repo there is no 64 byte test. So this is related to the source not to this PR. I have tested it physically on the board and it works fine.

Could somebody please review? maybe @geky or @0xc0170 ?

0xc0170 · 2020-02-11T09:47:03Z

@ARMmbed/mbed-os-storage Could you help please?

VeijoPesonen · 2020-02-11T11:35:14Z

I tried to run the tests against Mbed OS master(d3078a39b1). As you can see there were two failing test cases. As a next step I'll take a look on your changes.

Fetching

mbed-os/features/storage/filesystem$ git clone git@github.com:ARMmbed/mbed-littlefs.git littlefsv2
cd littlefsv2
mbed-os/features/storage/filesystem/littlefsv2$ git fetch origin pull/14/head:littlefsv2 && git checkout littlefsv2

Compilation

Patch required

diff --git i/tools/test_configs/QSPIFBlockDeviceAndHeapBlockDevice.json w/tools/test_configs/QSPIFBlockDeviceAndHeapBlockDevice.json
index 91dbefc5fa..58f3949d69 100644
--- i/tools/test_configs/QSPIFBlockDeviceAndHeapBlockDevice.json
+++ w/tools/test_configs/QSPIFBlockDeviceAndHeapBlockDevice.json
@@ -13,7 +13,7 @@
         "test-filesystem": {
             "help": "Used filesystem",
             "macro_name": "MBED_TEST_FILESYSTEM",
-            "value": "LittleFileSystem"
+            "value": "LittleFileSystem2"
         }
     }
 }

Compilation and testing

mbed test -m NRF52840_DK -n features-storage-filesystem-littlefsv2-* --app-config tools/test_configs/QSPIFBlockDeviceAndHeapBlockDevice.json

Test results

All results

mbedgt: test suite report:
| target              | platform_name | test suite                                                                             | result | elapsed_time (sec) | copy_method |
|---------------------|---------------|----------------------------------------------------------------------------------------|--------|--------------------|-------------|
| NRF52840_DK-GCC_ARM | NRF52840_DK   | features-storage-filesystem-littlefsv2-tests-filesystem-dirs                           | OK     | 55.56              | default     |
| NRF52840_DK-GCC_ARM | NRF52840_DK   | features-storage-filesystem-littlefsv2-tests-filesystem-files                          | OK     | 38.39              | default     |
| NRF52840_DK-GCC_ARM | NRF52840_DK   | features-storage-filesystem-littlefsv2-tests-filesystem-interspersed                   | OK     | 24.31              | default     |
| NRF52840_DK-GCC_ARM | NRF52840_DK   | features-storage-filesystem-littlefsv2-tests-filesystem-seek                           | OK     | 71.6               | default     |
| NRF52840_DK-GCC_ARM | NRF52840_DK   | features-storage-filesystem-littlefsv2-tests-filesystem_integration-format             | OK     | 28.82              | default     |
| NRF52840_DK-GCC_ARM | NRF52840_DK   | features-storage-filesystem-littlefsv2-tests-filesystem_recovery-resilience            | OK     | 51.7               | default     |
| NRF52840_DK-GCC_ARM | NRF52840_DK   | features-storage-filesystem-littlefsv2-tests-filesystem_recovery-resilience_functional | ERROR  | 36.24              | default     |
| NRF52840_DK-GCC_ARM | NRF52840_DK   | features-storage-filesystem-littlefsv2-tests-filesystem_recovery-wear_leveling         | FAIL   | 84.48              | default     |
| NRF52840_DK-GCC_ARM | NRF52840_DK   | features-storage-filesystem-littlefsv2-tests-filesystem_retarget-dirs                  | OK     | 58.58              | default     |
| NRF52840_DK-GCC_ARM | NRF52840_DK   | features-storage-filesystem-littlefsv2-tests-filesystem_retarget-files                 | OK     | 40.09              | default     |
| NRF52840_DK-GCC_ARM | NRF52840_DK   | features-storage-filesystem-littlefsv2-tests-filesystem_retarget-interspersed          | OK     | 23.5               | default     |
| NRF52840_DK-GCC_ARM | NRF52840_DK   | features-storage-filesystem-littlefsv2-tests-filesystem_retarget-seek                  | OK     | 82.22              | default     |
mbedgt: test suite results: 10 OK / 1 ERROR / 1 FAIL

Failures

| NRF52840_DK-GCC_ARM | NRF52840_DK   | features-storage-filesystem-littlefsv2-tests-filesystem_recovery-resilience_functional | ERROR  | 36.24              | default     |
| NRF52840_DK-GCC_ARM | NRF52840_DK   | features-storage-filesystem-littlefsv2-tests-filesystem_recovery-wear_leveling         | FAIL   | 84.48              | default     |

VeijoPesonen · 2020-02-11T12:55:28Z

LittleFileSystem2.cpp

@@ -24,8 +24,9 @@ namespace mbed {

 extern "C" uint32_t lfs2_crc(uint32_t crc, const void *buffer, size_t size)
 {
-    uint32_t initial_xor = crc;
-    MbedCRC<POLY_32BIT_REV_ANSI, 32> ct(initial_xor, 0x0, true, false);
+    uint32_t initial_xor = lfs2_rbit(crc);


@geky, @kjbracey-arm This part is like black magic for me so I trust you also take a look.

It was taken from https://github.com/ARMmbed/mbed-os/blob/master/features/storage/filesystem/littlefs/LittleFileSystem.cpp#L27

Which comes from this PR from @kjbracey-arm ARMmbed/mbed-os#11957

Given that it's the same as what's on mbed-os, I think it should be fine. The CRC didn't change v1->v2.

VeijoPesonen · 2020-02-11T13:29:09Z

To be certain that the test case failures were not somehow specific for the forementioned NRF52840_DK board (with QSPIF) I tried with K82F which is having a SPIF module. The issues remain.

mbedgt: test suite report:
| target       | platform_name | test suite                                                                             | result | elapsed_time (sec) | copy_method |
|--------------|---------------|----------------------------------------------------------------------------------------|--------|--------------------|-------------|
| K82F-GCC_ARM | K82F          | features-storage-filesystem-littlefsv2-tests-filesystem-dirs                           | OK     | 50.36              | default     |
| K82F-GCC_ARM | K82F          | features-storage-filesystem-littlefsv2-tests-filesystem-files                          | OK     | 32.8               | default     |
| K82F-GCC_ARM | K82F          | features-storage-filesystem-littlefsv2-tests-filesystem-interspersed                   | OK     | 26.26              | default     |
| K82F-GCC_ARM | K82F          | features-storage-filesystem-littlefsv2-tests-filesystem-seek                           | OK     | 71.85              | default     |
| K82F-GCC_ARM | K82F          | features-storage-filesystem-littlefsv2-tests-filesystem_integration-format             | OK     | 31.97              | default     |
| K82F-GCC_ARM | K82F          | features-storage-filesystem-littlefsv2-tests-filesystem_recovery-resilience            | OK     | 31.62              | default     |
| K82F-GCC_ARM | K82F          | features-storage-filesystem-littlefsv2-tests-filesystem_recovery-resilience_functional | ERROR  | 14.68              | default     |
| K82F-GCC_ARM | K82F          | features-storage-filesystem-littlefsv2-tests-filesystem_recovery-wear_leveling         | FAIL   | 53.96              | default     |
| K82F-GCC_ARM | K82F          | features-storage-filesystem-littlefsv2-tests-filesystem_retarget-dirs                  | OK     | 52.24              | default     |
| K82F-GCC_ARM | K82F          | features-storage-filesystem-littlefsv2-tests-filesystem_retarget-files                 | OK     | 34.17              | default     |
| K82F-GCC_ARM | K82F          | features-storage-filesystem-littlefsv2-tests-filesystem_retarget-interspersed          | OK     | 15.44              | default     |
| K82F-GCC_ARM | K82F          | features-storage-filesystem-littlefsv2-tests-filesystem_retarget-seek                  | OK     | 75.33              | default     |
mbedgt: test suite results: 10 OK / 1 ERROR / 1 FAIL
```

VeijoPesonen

Following test case failures need to be addressed.

| NRF52840_DK-GCC_ARM | NRF52840_DK   | features-storage-filesystem-littlefsv2-tests-filesystem_recovery-resilience_functional | ERROR  | 36.24              | default     |
| NRF52840_DK-GCC_ARM | NRF52840_DK   | features-storage-filesystem-littlefsv2-tests-filesystem_recovery-wear_leveling         | FAIL   | 84.48              | default     |

geky · 2020-02-11T16:21:08Z

Thanks for keeping this alive @pilotak, at this point I think we should bring this in if possible. Sorry for not getting the update script working in a timely manner.

I looked into the Travis tests, since those are fairly easy to run locally. There were two failures:

test_alloc fails with a false-positive. This isn't too surprising, test_alloc is a flimsy test that relies on the filesystem running out of blocks at an exact time. Changing the read/prog/cache sizes can change cause this to break.

I would just change the if statement in test_alloc.sh to skip the problematic tests for now:
```
- if [[ ! $MAKEFLAGS =~ "LFS_BLOCK_CYCLES" ]]
+ if [[ ! $MAKEFLAGS =~ "LFS_BLOCK_CYCLES" && ! $MAKEFLAGS =~ "LFS_READ_SIZE" ]]
```
test_relocations fails with a true-positive. This is the bug found in power loss reboot triggered internal assert littlefs-project/littlefs#343. Unfortunately it isn't going to be fixed until Rework test framework, fix a number of related bugs littlefs-project/littlefs#372. Though we may be able to cherry-pick littlefs-project/littlefs@f4b6a6b onto this branch.

The Mbed OS tests get a bit harder. These failures come from relocation errors. These are my fault since I did not run the Mbed OS tests on the relocation changes.

The relocation issues were the main motivation for littlefs-project/littlefs#372, which required some rather invasive fixes. So I'm not sure these can be fixed without waiting for littlefs-project/littlefs#372.

geky · 2020-02-11T16:24:19Z

It looks like these were all caused by the change to wear-leveling in v2.1. So I think all versions v2.1.0-v2.1.4 will show these failures.

geky · 2020-02-11T16:27:46Z

I wonder if it may be worth bringing in the CRC fix as a separate patch while waiting for v2.1.5?

pilotak · 2020-02-11T17:49:23Z

Thanks @geky for being onboard. I just like the code to be up to date, it would be shame if others can use v2 and mbedOS will use the old one. And because next major release in rising it's just in time.

pilotak · 2020-02-13T21:15:38Z

@VeijoPesonen i know you requested changes but what do I need to do? i only brought the update over to this repo including the test. It would be nice if somebody steps in and makes all the things required.

VeijoPesonen · 2020-02-14T13:28:55Z

I wonder if it may be worth bringing in the CRC fix as a separate patch while waiting for v2.1.5?

@geky, @pilotak, @SeppoTakalo fine by me.

VeijoPesonen

I would say this PR can be merged to the master branch of this repository. Once 2.1.5 is available - or we apply the forementioned patch - that can be brought in as a separate PR and then we can merge it to Mbed OS. Maybe after that exercise we should get rid of this repository and push changes directly to Mbed OS?

pilotak · 2020-02-21T11:38:36Z

that we be perfect

geky · 2020-04-01T18:05:08Z

Hi @pilotak, with the release of v2.2, I've gone ahead and created a new PR based off this:
#16

I should have cherry-picked all of your changes onto the v2.2 PR, let me know if you notice anything missing.

I noticed the addition of MBED_LFS2_READ_SIZE and MBED_LFS2_PROG_SIZE. I think they are a good idea but if we add the configs to mbed_lib.json we should also add the configs to LittleFileSystem2's constructor (in case multiple filesystem are instantiated in the system). Feel free to create another PR for that.

Unless you notice any problem, I'm going to go ahead and merge #16 in once it passes CI.

Thanks for creating this PR!

pilotak · 2020-04-03T07:29:58Z

@geky thanks for the update. I noticed that you have not updated the mbed_lib.json the debug there is obsolete and generated macros wil no nothing, it should be updated from mine. I realise that PROG and READ size is not perfect there, but it was there for V2-aplha so a kept it there

pilotak · 2020-04-03T07:32:34Z

actually whole file will generate macros which will not work, they have a MBED prefix which is not used by the lib

geky · 2020-04-06T15:28:54Z

Ah! Correct me if I'm wrong, but aren't the MBED_* macros used here?

mbed-littlefs/LittleFileSystem2.h

Lines 60 to 64 in 70aa179

    
           LittleFileSystem2(const char *name = NULL, mbed::BlockDevice *bd = NULL, 
        
                            lfs2_size_t block_size = MBED_LFS2_BLOCK_SIZE, 
        
                            uint32_t   block_cycles = MBED_LFS2_BLOCK_CYCLES, 
        
                            lfs2_size_t cache_size = MBED_LFS2_CACHE_SIZE, 
        
                            lfs2_size_t lookahead = MBED_LFS2_LOOKAHEAD_SIZE);

This way the config options slot into the C++ constructor and can be overwritten per class.

I noticed the READ/PROG config options were missing from here.

pilotak · 2020-04-08T11:13:38Z

@geky sorry i didn't spot that, my fault.

As you say LFS2_READ_SIZE and LFS2_PROG_SIZE are still not defined. Can you make a PR or do you want me to do it?

geky · 2020-04-09T09:39:06Z

Ah, if you think READ_SIZE/PROG_SIZE is valuable a PR would be helpful. I'd be happy to bring it in.

I'm not sure it's necessary though, since the LittleFileSystem class can get the read_size/prog_size from the BlockDevice class. This isn't the same in the C API since a callback for read_size/prog_size would have been overkill:

mbed-littlefs/LittleFileSystem2.cpp

Lines 183 to 184 in 70aa179

    
           _config.read_size       = bd->get_read_size(); 
        
           _config.prog_size       = bd->get_program_size();

One of the changes v1->v2 was to separate the limitations of the hardware (read_size/prog_size) from the cache allocation (cache_size). So now the read_size/prog_size serve a less important role for configuring littlefs.

pilotak · 2020-04-09T09:56:24Z

Thats fine, I'm only making sure it has everything set so it can go straight into OS so if you think it's not necessary i'm ok.

Now that everything is set can you please bring it to the OS so it's there ready for OS6 release?

geky · 2020-04-09T10:09:56Z

That part is out of my hands.

@VeijoPesonen, what are the next steps for this? v2.2 (master now) would be a great target for Mbed OS integration.

geky · 2020-04-09T10:11:21Z

@pilotak also thanks for all the help with bringing v2.x in here!

VeijoPesonen · 2020-04-09T12:36:02Z

PR ARMmbed/mbed-os#12783 to bring LittleFSv2 to Mbed OS created. Thanks @pilotak for your work.

v2.1.1

47d3796

0xc0170 mentioned this pull request Jan 14, 2020

Is mbedOS 6 going to support LittleFS2? ARMmbed/mbed-os#12250

Closed

pilotak added 2 commits February 6, 2020 17:33

mbed 5.15 CRC update

c64408d

v2.1.4

3defbe9

pilotak changed the title ~~upgrade to v2.1.1~~ upgrade to v2.1.4 Feb 6, 2020

pilotak added 8 commits February 6, 2020 18:08

travis test fix

eec00b1

test.py chmod

c950c89

scripts chmod & NO_INTRINSICS macro

f68bc61

chmod test_relocations.sh

5892861

library setting fix

3c9e5e5

missing library setting

c6c8ef1

travis new tests

6438cbd

request travis again

c4819de

pilotak requested review from geky and 0xc0170 February 7, 2020 07:04

VeijoPesonen reviewed Feb 11, 2020

View reviewed changes

VeijoPesonen suggested changes Feb 11, 2020

View reviewed changes

skip the problematic test

512a0b5

VeijoPesonen approved these changes Feb 21, 2020

View reviewed changes

geky mentioned this pull request Apr 1, 2020

Bring in v2.2 #16

Merged

pilotak closed this Apr 3, 2020

pilotak reopened this Apr 3, 2020

pilotak closed this Apr 9, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

upgrade to v2.1.4 #14

upgrade to v2.1.4 #14

pilotak commented Sep 1, 2019 •

edited

Loading

pilotak commented Feb 6, 2020 •

edited

Loading

0xc0170 commented Feb 11, 2020

VeijoPesonen commented Feb 11, 2020 •

edited

Loading

VeijoPesonen Feb 11, 2020

pilotak Feb 11, 2020

geky Feb 11, 2020

VeijoPesonen commented Feb 11, 2020

VeijoPesonen left a comment

geky commented Feb 11, 2020

geky commented Feb 11, 2020

geky commented Feb 11, 2020

pilotak commented Feb 11, 2020

pilotak commented Feb 13, 2020

VeijoPesonen commented Feb 14, 2020

VeijoPesonen left a comment

pilotak commented Feb 21, 2020

geky commented Apr 1, 2020

pilotak commented Apr 3, 2020

pilotak commented Apr 3, 2020

geky commented Apr 6, 2020 •

edited

Loading

pilotak commented Apr 8, 2020

geky commented Apr 9, 2020

pilotak commented Apr 9, 2020 •

edited

Loading

geky commented Apr 9, 2020

geky commented Apr 9, 2020

VeijoPesonen commented Apr 9, 2020 •

edited

Loading

upgrade to v2.1.4 #14

upgrade to v2.1.4 #14

Conversation

pilotak commented Sep 1, 2019 • edited Loading

pilotak commented Feb 6, 2020 • edited Loading

0xc0170 commented Feb 11, 2020

VeijoPesonen commented Feb 11, 2020 • edited Loading

Fetching

Compilation

Patch required

Compilation and testing

Test results

All results

Failures

VeijoPesonen Feb 11, 2020

Choose a reason for hiding this comment

pilotak Feb 11, 2020

Choose a reason for hiding this comment

geky Feb 11, 2020

Choose a reason for hiding this comment

VeijoPesonen commented Feb 11, 2020

VeijoPesonen left a comment

Choose a reason for hiding this comment

geky commented Feb 11, 2020

geky commented Feb 11, 2020

geky commented Feb 11, 2020

pilotak commented Feb 11, 2020

pilotak commented Feb 13, 2020

VeijoPesonen commented Feb 14, 2020

VeijoPesonen left a comment

Choose a reason for hiding this comment

pilotak commented Feb 21, 2020

geky commented Apr 1, 2020

pilotak commented Apr 3, 2020

pilotak commented Apr 3, 2020

geky commented Apr 6, 2020 • edited Loading

pilotak commented Apr 8, 2020

geky commented Apr 9, 2020

pilotak commented Apr 9, 2020 • edited Loading

geky commented Apr 9, 2020

geky commented Apr 9, 2020

VeijoPesonen commented Apr 9, 2020 • edited Loading

pilotak commented Sep 1, 2019 •

edited

Loading

pilotak commented Feb 6, 2020 •

edited

Loading

VeijoPesonen commented Feb 11, 2020 •

edited

Loading

geky commented Apr 6, 2020 •

edited

Loading

pilotak commented Apr 9, 2020 •

edited

Loading

VeijoPesonen commented Apr 9, 2020 •

edited

Loading