Implementation of files read/write on flash memory.

[fgaetani]

Given the lack of EEPROM memory in the NRF52840 SoC, it would be advisable to implement a library for save/write files within the flash memory of the microcontroller. Would it be possible to use the libraries already developed for the Adafruit Feather Express nRF52840 board?

Thank you!

[facchinm]

FlashIAP class and LittleFS (from mbed) are already available in the core; they provide a reliable way to write/read internal flash and a very convenient posix compliance.

For example, to use half of the internal flash space as storage, you can use something like this

#include "FlashIAPBlockDevice.h"
#include "FATFileSystem.h"

REDIRECT_STDOUT_TO(Serial)

static mbed::FATFileSystem fs("fs");
static FlashIAPBlockDevice bd(0x80000, 0x80000);

void return_error(int ret_val) {
  if (ret_val)
    printf("Failure. %d\r\n", ret_val);
  else
    printf("done.\r\n");
}

void errno_error(void* ret_val) {
  if (ret_val == NULL)
    printf(" Failure. %d \r\n", errno);
  else
    printf(" done.\r\n");
}

void setup() {

  Serial.begin(115200);
  while (!Serial);

  // put your setup code here, to run once:
  int error = 0;
  printf("Welcome to the filesystem example.\r\n"
         "Formatting a FAT, RAM-backed filesystem. ");
  error = mbed::FATFileSystem::format(&bd);
  return_error(error);

  printf("Mounting the filesystem on \"/fs\". ");
  error = fs.mount(&bd);
  return_error(error);

  printf("Opening a new file, numbers.txt.");
  FILE* fd = fopen("/fs/numbers.txt", "w");
  errno_error(fd);

  for (int i = 0; i < 20; i++) {
    printf("Writing decimal numbers to a file (%d/20)\r\n", i);
    fprintf(fd, "%d\r\n", i);
  }
  printf("Writing decimal numbers to a file (20/20) done.\r\n");

  printf("Closing file.");
  fclose(fd);
  printf(" done.\r\n");

  printf("Re-opening file read-only.");
  fd = fopen("/fs/numbers.txt", "r");
  errno_error(fd);

  printf("Dumping file to screen.\r\n");
  char buff[16] = {0};
  while (!feof(fd)) {
    int size = fread(&buff[0], 1, 15, fd);
    fwrite(&buff[0], 1, size, stdout);
  }
  printf("EOF.\r\n");

  printf("Closing file.");
  fclose(fd);
  printf(" done.\r\n");

  printf("Opening root directory.");
  DIR* dir = opendir("/fs/");
  errno_error(fd);

  struct dirent* de;
  printf("Printing all filenames:\r\n");
  while ((de = readdir(dir)) != NULL) {
    printf("  %s\r\n", &(de->d_name)[0]);
  }

  printf("Closing root directory. ");
  error = closedir(dir);
  return_error(error);
  printf("Filesystem Demo complete.\r\n");
}

void loop() {
  // put your main code here, to run repeatedly:

}

A convenient wrapper for this usage could be a good idea though 👍

[fgaetani]

Thank you so much for your support, I will try this code. However I would like to ask you if there is also a method to write byte-byte in the flash sectors, as was done with EEPROM.h on AVR microcontrollers.

Thank you!

[facchinm]

You can write byte by byte also without a filesystem but since rated flash writes are much less than an EEPROM's you "need" a wear-levelling filesystem that automatically relocates the data being written in a safe location.

[manchoz]

@fgaetani If you need to write/read byte-sized values (ie. parameters or configurations) from flash a better option could be using the KVStore API.

#include "KVStore.h"
#include "kvstore_global_api.h"

REDIRECT_STDOUT_TO(Serial)

#define EXAMPLE_KV_VALUE_LENGTH 64
#define EXAMPLE_KV_KEY_LENGTH 32
#define err_code(res) MBED_GET_ERROR_CODE(res)

void setup() {
    Serial.begin(115200);
    while (!Serial)
        ;
    // put your setup code here, to run once:
    char kv_value_in[EXAMPLE_KV_VALUE_LENGTH] = {"kvstore_dummy_value_hello_world"};
    char kv_key_in[EXAMPLE_KV_KEY_LENGTH] = {"/kv/dummy_key1"};
    char kv_key_out[EXAMPLE_KV_KEY_LENGTH] = {0};
    size_t actual_size = 0;

    /* key information container */
    kv_info_t info;

    /* kv store iterator */
    kv_iterator_t kvstore_it;

    int i_ind = 0;

    printf("--- Mbed OS KVStore static API example ---\n");

    int res = MBED_ERROR_NOT_READY;

    /* Start By Resetting the KV Storage */
    printf("kv_reset\n");
    res = kv_reset("/kv/");
    printf("kv_reset -> %d\n", err_code(res));

    /* Set First 'Dummy' Key/Value pair with unprotected clear value data */
    printf("kv_set first dummy key\n");
    res = kv_set(kv_key_in, kv_value_in, strlen(kv_value_in), 0);
    printf("kv_set -> %d\n", err_code(res));

    /* Read the KV Pair you've just set */
    /* Start by getting key's information */
    printf("kv_get_info of first key\n");
    res = kv_get_info(kv_key_in, &info);
    printf("kv_get_info -> %d\n", err_code(res));
    printf("kv_get_info key: %s\n", kv_key_in);
    printf("kv_get_info info - size: %u, flags: %lu\n", info.size, info.flags);

    /* Now that you know the data value size of this key,
     * allocate a buffer with matching size and get the value data */
    printf("kv_get first key\n");
    char *kv_first_value_out = new char[info.size + 1];
    memset(kv_first_value_out, 0, info.size + 1);
    res = kv_get(kv_key_in, kv_first_value_out, info.size, &actual_size);
    printf("kv_get -> %d\n", err_code(res));
    printf("kv_get key: %s\n", kv_key_in);
    printf("kv_get value: %s\n", kv_first_value_out);
    delete[] kv_first_value_out;

    /* Lets set some more 'Dummy' and 'Real' KV pairs */
    /* Set 'Dummy' Key2 */
    printf("kv_set second dummy key \n");
    res = kv_set("/kv/dummy_key2", "dummy_value2", strlen("dummy_value2"), 0);
    printf("kv_set -> %d\n", err_code(res));

    /* Set an authenticated-encrypted 'Dummy' key with Replay protection */
    printf("kv_set third key with Confidentiality and Replay Protection flags\n");
    res = kv_set("/kv/dummy_auth_enc_key", "auth_enc_value", strlen("auth_enc_value"),
                 KV_REQUIRE_CONFIDENTIALITY_FLAG | KV_REQUIRE_REPLAY_PROTECTION_FLAG);
    printf("kv_set -> %d\n", err_code(res));

    /* Set 2 non-dummy 'Real' KV pairs */
    /* Set 'Real' Key 1 */
    printf("kv_set Set 'Real' Key 1\n");
    res = kv_set("/kv/real_key1", "real_value1", strlen("real_value1"), 0);
    printf("kv_set -> %d\n", err_code(res));
    /* Set 'Real' Write-Once Key2 for a key that you do not want to be removed */
    printf("kv_set Set 'Real' Key 2 with flag write-once\n");
    res = kv_set("/kv/real_wo_key", "real_wo_value", strlen("real_wo_value"), KV_WRITE_ONCE_FLAG);
    printf("kv_set -> %d\n", err_code(res));

    /* Now lets remove all of the 'Dummy' Keys and remain with the 'Real' ones */
    printf("Removing 'Dummy' Keys\n");
    /* Iterate and remove Keys that start with prefix 'dummy' */
    res = kv_iterator_open(&kvstore_it, "dummy");
    memset(kv_key_out, 0, EXAMPLE_KV_KEY_LENGTH);
    while (kv_iterator_next(kvstore_it, kv_key_out, EXAMPLE_KV_KEY_LENGTH) != MBED_ERROR_ITEM_NOT_FOUND)
    {
        i_ind++;
        printf("%d) Removing %s\n", i_ind, kv_key_out);
        kv_remove(kv_key_out);
        memset(kv_key_out, 0, EXAMPLE_KV_KEY_LENGTH);
    }
    res = kv_iterator_close(kvstore_it);
    printf("Remaining with 'Real' Keys:\n");
    /* Iterate on all remaining Keys */
    res = kv_iterator_open(&kvstore_it, NULL);
    memset(kv_key_out, 0, EXAMPLE_KV_KEY_LENGTH);
    i_ind = 0;
    while (kv_iterator_next(kvstore_it, kv_key_out, EXAMPLE_KV_KEY_LENGTH) != MBED_ERROR_ITEM_NOT_FOUND)
    {
        i_ind++;
        printf("%d) %s\n", i_ind, kv_key_out);
        memset(kv_key_out, 0, EXAMPLE_KV_KEY_LENGTH);
    }
    res = kv_iterator_close(kvstore_it);

    /* Try to remove write-once Key - should fail */
    printf("kv_remove write-once file - should fail!\n");
    res = kv_remove("/kv/real_wo_key");
    printf("kv_remove -> %d\n", err_code(res));

    /* Finally, reset will format kvstore and remove All Keys (including write-once keys) */
    printf("kv_reset format kvstore (including write-once)\n");
    res = kv_reset("/kv/");
    printf("kv_reset -> %d\n", err_code(res));
}

void loop() {
    // put your main code here, to run repeatedly:
}

[fgaetani]

Thank you so much!

[marcel-bluestone]

@fgaetani : I'd also like to write a parameter (Device-name) to the flash of a Arduino nano 33 BLE. Could this KVStore API be used in the Arduino IDE as well? If so, is there maybe a short example on how to read / write a single parameter?

[Asad-OMN]

@facchinm
Hi Facchinm, I used your recommended code , I was trying to study your code and what is the function in each line and upload the code with minor modification to see what will happen
just I changed these lines

static FlashIAPBlockDevice bd(0x8000, 0x8000);

and I comment out these line

 // printf("Mounting the filesystem on \"/fs\". ");
 // error = fs.mount(&bd);
//  return_error(error);

#include "FlashIAPBlockDevice.h"
#include "FATFileSystem.h"

REDIRECT_STDOUT_TO(Serial)

static mbed::FATFileSystem fs("fs");
static FlashIAPBlockDevice bd(0x8000, 0x8000);   // origin  bd(0x80000, 0x80000); 

void return_error(int ret_val) {
  if (ret_val)
    printf("Failure. %d\r\n", ret_val);
  else
    printf("done.\r\n");
}

void errno_error(void* ret_val) {
  if (ret_val == NULL)
    printf(" Failure. %d \r\n", errno);
  else
    printf(" done.\r\n");
}

void setup() {

  Serial.begin(115200);
  while (!Serial);

  // put your setup code here, to run once:
  int error = 0;
  printf("Welcome to the filesystem example.\r\n"
         "Formatting a FAT, RAM-backed filesystem. ");
  error = mbed::FATFileSystem::format(&bd);
  return_error(error);
// comment out mount fs
//  printf("Mounting the filesystem on \"/fs\". ");
//  error = fs.mount(&bd);
//  return_error(error);

  printf("Opening a new file, numbers.txt.");
  FILE* fd = fopen("/fs/numbers.txt", "w");
  errno_error(fd);

  for (int i = 0; i < 20; i++) {
    printf("Writing decimal numbers to a file (%d/20)\r\n", i);
    fprintf(fd, "%d\r\n", i);
  }
  printf("Writing decimal numbers to a file (20/20) done.\r\n");

  printf("Closing file.");
  fclose(fd);
  printf(" done.\r\n");

  printf("Re-opening file read-only.");
  fd = fopen("/fs/numbers.txt", "r");
  errno_error(fd);

  printf("Dumping file to screen.\r\n");
  char buff[16] = {0};
  while (!feof(fd)) {
    int size = fread(&buff[0], 1, 15, fd);
    fwrite(&buff[0], 1, size, stdout);
  }
  printf("EOF.\r\n");

  printf("Closing file.");
  fclose(fd);
  printf(" done.\r\n");

  printf("Opening root directory.");
  DIR* dir = opendir("/fs/");
  errno_error(fd);

  struct dirent* de;
  printf("Printing all filenames:\r\n");
  while ((de = readdir(dir)) != NULL) {
    printf("  %s\r\n", &(de->d_name)[0]);
  }

  printf("Closing root directory. ");
  error = closedir(dir);
  return_error(error);
  printf("Filesystem Demo complete.\r\n");
}

void loop() {
  // put your main code here, to run repeatedly:

}

I lost the communication with nano 33 ble and I couldn't reset it. I used RST push-button twice but nothing happens. Did you thing that I damaged the internal flash? where the bootloader locate? is it inside the internal flash or in ROM?
and why I couldn't recover it? or how to recover it
could you please help me on this and thanks in advance for your time and effort

[facchinm]

Hi @Asad-OMN ,
unfortunately you damaged the bootloader since it's located in flash from 0x0 to 0x10000 and the partition you tried to create
(static FlashIAPBlockDevice bd(0x8000, 0x8000);) is in its middle.
To recover the board you need an SWD programmer with SWDIO and SWCLK connected as shown here, then just run Burn Bootloader from the IDE menu

[Asad-OMN]

@facchinm thanks Martino, I will be aware next time. I will look for the SWD programmer.
all the best sir.

[vikr53]

Hi @facchinm,
BASICALLY MY QUESTION IS: Can you use this for less than 0x80000 blocks of memory?
Thank you for your post! This was very helpful. I tried to run your code with the following modification:
static FlashIAPBlockDevice bd(0x90000, 0x70000)

since I am using the other portion of the flash for something else. I'm getting the following results with the associated error codes.

static mbed::FATFileSystem fs("fs");
static FlashIAPBlockDevice bd(0x90000, 0x70000);

Mounting the filesystem on "/fs". Failure. -22
Opening a new file, numbers.txt. Failure. 22
Writing decimal numbers to a file (0/20)
Writing decimal numbers to a file (1/20)
Writing decimal numbers to a file (2/20)
Writing decimal numbers to a file (3/20)
Writing decimal numbers to a file (4/20)
Writing decimal numbers to a file (5/20)
Writing decimal numbers to a file (6/20)
Writing decimal numbers to a file (7/20)
Writing decimal numbers to a file (8/20)
Writing decimal numbers to a file (9/20)
Writing decimal numbers to a file (10/20)
Writing decimal numbers to a file (11/20)
Writing decimal numbers to a file (12/20)
Writing decimal numbers to a file (13/20)
Writing decimal numbers to a file (14/20)
Writing decimal numbers to a file (15/20)
Writing decimal numbers to a file (16/20)
Writing decimal numbers to a file (17/20)
Writing decimal numbers to a file (18/20)
Writing decimal numbers to a file (19/20)
Writing decimal numbers to a file (20/20) done.
Closing file. done.
Re-opening file read-only. Failure. 22
Dumping file to screen.
EOF.
Closing file. done.

Would you know what I might be doing wrong? Is this the result of some kind of memory overlap?

[facchinm]

Hi @vikr53 ,
the problem is likely due to the underlying FAT filesystem that requires a minimum partition size that is quite big for the internal nRF52 flash.
If you just need a storage you can replace "FATFileSystem" with "LittleFileSystem" which allows a much broader choice of partition sizes.

[abu-rehan]

Hi @facchinm ,
I also want to use the internal flash. From your last reply, it seems, I should use LittleFileSystem instead of FATFileSystem with the code you shared in the first comment.
Can you please confirm?
I just want to store about 3000 bytes on Arduino Nano 33 BLE Sense. I am using Arduino IDE for uploading the application using USB. I want a safe solution so that there is no danger of damaging bootloader or mbed OS.