C3 + S3 Flash - CI test

[P-R-O-C-H-Y]

Add flash testing to the GitHub CI. Following needs to be taken care of:

1st round:

• Test files (sketch + test script)
• Add option for tests_build.sh to accept flash_mode parameter
• Add option to erase flash before uploading test to the chip
• Make sure to call the test for each chip for each flash mode

2nd round:

• Add Flash r/w performance test

[P-R-O-C-H-Y]

Adding @Ouss4, so we can discuss if needed here.

[lbernstone]

Maybe add a small r/w benchmark to keep an eye for any major issues that affect flash performance?

[P-R-O-C-H-Y]

@lbernstone Good idea, we can add some test for r/w performance. Adding that to the list.

[igrr]

Just 2c, there is a benchmarking test for flash performance in several different mode in esp-idf CI. The problem is that it is extremely unreliable:

• different flash chips of the same type may have varying erase times
• erase times get significantly longer over the lifetime of the part (in CI the flash chips get a lot of wear)

Given that Arduino-esp32 pretty much directly calls the underlying IDF flash related functions, i think the likelihood of a performance regression is low, while the effort to maintain such a performance test would be relatively high.

[me-no-dev]

The one thing I see worth testing as speed is to make sure that there is actually a difference between the flash modes. With the latest findings (and until we fix this in the lib-builder) it seems to me that there is a chance that flash modes actually do not correspond to what is being set and enabled by the bootloader. @igrr do you thing that such test makes sense and will work, regradles of the flash chip differences?