Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Writing files with SPIFFS no longer working for ESP-01 #4061

Closed
funtastic99 opened this issue Jan 1, 2018 · 162 comments
Closed

Writing files with SPIFFS no longer working for ESP-01 #4061

funtastic99 opened this issue Jan 1, 2018 · 162 comments

Comments

@funtastic99
Copy link

@funtastic99 funtastic99 commented Jan 1, 2018

Basic Infos

SPIFFS no longer works on a ESP-01.

Hardware

Hardware: ESP-01
Core Version: 2.3.0-rc2
also tried with stable

I reproduced this problem on several different ESP-01. So seems not to be hardware related

Description

Problem description
SPIFFS worked for a long time but all of a sudden it stoped working on a ESP-01.
Whenever i write a file its not existing or empty after next boot.
Using the same sketch on an Wemos D1 mini its works without problems. Actually im only facing this problem on a ESP-01.

Settings in IDE

Module: Generic ESP8266 Module
Flash Size: 1MB (64k SPIFFS), also tried with 512k (64k SPIFFS)
CPU Frequency: 80Mhz
Flash Mode: DIO
Flash Frequency: 40Mhz
Upload Using: SERIAL
Reset Method: ck

Sketch

#include <FS.h>

void setup()
{
Serial.begin(115200);

if (!SPIFFS.begin()) Serial.println("error while mounting filesystem!");

readFile();
writeFile();
readFile();

Serial.println("done");
}

void readFile()
{
Serial.println("reading");
File f = SPIFFS.open("/myFile.txt", "r");
if (!f) Serial.println("file not available");
else if (f.available()<=0) Serial.println("file exists but available <0");
else
{
String ssidString = f.readStringUntil('#');
Serial.print("read from file: ");
Serial.println(ssidString);
}
f.close();
}

void writeFile()
{
Serial.println("writing");
File f = SPIFFS.open("/myFile.txt", "w");
if (!f) Serial.println("File creation failed");
else
{
f.print("networkConfig");
f.print("#");
f.flush();
f.close();
}
}

void loop()
{
}

Debug Messages

messages here

  • Output of first run:
    reading
    file not available
    writing
    reading
    read from file: <--- here we see the problem already
    done

  • Output of second run:
    eading
    file not available
    writing
    reading
    read from file:
    done

  • Output of third run:
    reading
    read from file:
    writing
    File creation failed <---- check this!
    reading
    read from file:
    done

Any ideas?
thank you!

@funtastic99

This comment has been minimized.

Copy link
Author

@funtastic99 funtastic99 commented Jan 3, 2018

I found out some additional information that might be helpful.
The working ESP-01 report as flash_id:
Manufacturer: e0
Device: 4014

and the new ones that are not working:
Manufacturer: 85
Device: 6014

May the new flash chip is not supported by the arduino esp plugin yet ?

@ondabeach

This comment has been minimized.

Copy link

@ondabeach ondabeach commented Jan 5, 2018

I'm having the same problem.

Out of the first batch of 20 where the problem first showed up only 7 of them worked properly. All functions except for SPIFFS seem fine.

Out of the next batch of 20, 9 worked properly.

I just received another 2 batches of 20. One batch from a different seller on eBay and the other batch from a seller on AliExpress.

Out of the new batch from eBay only 2 work properly.

All 20 of the new batch from Aliexpress are faulty.

I checked the Chip ID's and I can confirm that Flash Chips with a manufacturer ID of 85 and Device ID of 6014 do not work. I can also confirm that the 2 that did work have the manufacturer ID of E0 and Chip ID of 4014.

I had a look at the flash chips with a magnifying glass and found the faulty ones were all branded FUYA and the 2 that have working SPIFFS are branded BergMicro.

I sure hope someone has a solution for this as it's a major problem for anyone who needs to store stuff like settings. EEPROM is an alternative but it's so clunky compared to SPIFFS and I really don't want to have to recode my sketches.

@funtastic99

This comment has been minimized.

Copy link
Author

@funtastic99 funtastic99 commented Jan 5, 2018

Its definitely a problem with writing SPIFFS. Reading works because i uploaded a flash image of a functional device with SPIFFS data present. But for reading the flash as anyway only mapped to memory while for writing a correct initializiation and timing is needed. I wonder why the booter itself has no problems writing the flash but SPIFFS has. I would have expected that finally the same functions are used to write the flash but it seems that the spiffs implementation is doing something strang and chip dependend.

@Pablo2048

This comment has been minimized.

Copy link

@Pablo2048 Pablo2048 commented Jan 5, 2018

So it seems like problem in SPIFFS and/or write implementation. Right?

@funtastic99

This comment has been minimized.

Copy link
Author

@funtastic99 funtastic99 commented Jan 5, 2018

yes seems like

@ondabeach

This comment has been minimized.

Copy link

@ondabeach ondabeach commented Jan 5, 2018

I found that the 'Upload Sketch Data' function of the IDE works. A sketch can then read the data but a sketch can't write to SPIFFS.

This will impact a lot of people, I've bought esp-01 modules witj these PUYA branded flash chips from 3 different suppliers now. Out of the 80 modules only 18 have worked as they had BergMicro flash chips onboard.

@weswitt

This comment has been minimized.

Copy link

@weswitt weswitt commented Jan 6, 2018

I'm seeing a similar, but different problem. My sketch does not find any files in the fs after uploading the spiffs image. I're tried uploading with esptool and the Arduino IDE, but I get the same result. The processor is an ESP01 on a Sonoff device. I've tried both the 512k64k and 1m64k configurations and neither work.

Is the issue I'm seeing related to this report? Am I possibly doing something wrong? What is the best way to diagnose this? I've added some code to the setup function that lists all files and it finds none.

@ondabeach

This comment has been minimized.

Copy link

@ondabeach ondabeach commented Jan 6, 2018

@weswitt is this a new board purchased recently?
Can you check the writing on the Flash chip, is it PUYA brand? Look for a number like P25Q80?.
Or you can load and run the following code to get the chip data. You're looking for the manufacturer ID of 85. If you have that flash chip then yes, you have the same problem as us.

//*************************************************
#include "FS.h"

void setup() {
Serial.begin(9600);

SPIFFS.begin();
FSInfo fs_info;
SPIFFS.info(fs_info);

float fileTotalKB = (float)fs_info.totalBytes / 1024.0;
float fileUsedKB = (float)fs_info.usedBytes / 1024.0;

float flashChipSize = (float)ESP.getFlashChipSize() / 1024.0 / 1024.0;
float realFlashChipSize = (float)ESP.getFlashChipRealSize() / 1024.0 / 1024.0;
float flashFreq = (float)ESP.getFlashChipSpeed() / 1000.0 / 1000.0;
FlashMode_t ideMode = ESP.getFlashChipMode();

Serial.printf("\n#####################\n");

Serial.printf("__________________________\n\n");
Serial.println("Firmware: ");
Serial.printf(" Chip Id: %08X\n", ESP.getChipId());
Serial.print(" Core version: "); Serial.println(ESP.getCoreVersion());
Serial.print(" SDK version: "); Serial.println(ESP.getSdkVersion());
Serial.print(" Boot version: "); Serial.println(ESP.getBootVersion());
Serial.print(" Boot mode: "); Serial.println(ESP.getBootMode());

Serial.printf("__________________________\n\n");

Serial.println("Flash chip information: ");
Serial.printf(" Flash chip Id: %08X (for example: Id=001640E0 Manuf=E0, Device=4016 (swap bytes))\n", ESP.getFlashChipId());
Serial.printf(" Sketch thinks Flash RAM is size: "); Serial.print(flashChipSize); Serial.println(" MB");
Serial.print(" Actual size based on chip Id: "); Serial.print(realFlashChipSize); Serial.println(" MB ... given by (2^( "Device" - 1) / 8 / 1024");
Serial.print(" Flash frequency: "); Serial.print(flashFreq); Serial.println(" MHz");
Serial.printf(" Flash write mode: %s\n", (ideMode == FM_QIO ? "QIO" : ideMode == FM_QOUT ? "QOUT" : ideMode == FM_DIO ? "DIO" : ideMode == FM_DOUT ? "DOUT" : "UNKNOWN"));

Serial.printf("__________________________\n\n");

Serial.println("File system (SPIFFS): ");
Serial.print(" Total KB: "); Serial.print(fileTotalKB); Serial.println(" KB");
Serial.print(" Used KB: "); Serial.print(fileUsedKB); Serial.println(" KB");
Serial.printf(" Block size: %lu\n", fs_info.blockSize);
Serial.printf(" Page size: %lu\n", fs_info.pageSize);
Serial.printf(" Maximum open files: %lu\n", fs_info.maxOpenFiles);
Serial.printf(" Maximum path length: %lu\n\n", fs_info.maxPathLength);

Dir dir = SPIFFS.openDir("/");
Serial.println("SPIFFS directory {/} :");
while (dir.next()) {
Serial.print(" "); Serial.println(dir.fileName());
Serial.println(" "); Serial.println(dir.fileSize());
}

Serial.printf("__________________________\n\n");

Serial.printf("CPU frequency: %u MHz\n\n", ESP.getCpuFreqMHz());
Serial.print("#####################");

// open file for reading
File f = SPIFFS.open("/login.txt", "r");
if (!f) {
Serial.println("file open failed");
} Serial.println("====== Reading from SPIFFS file =======");
// write 10 strings to file
for (int i=1; i<=10; i++){
String s=f.readStringUntil('\n');
Serial.print(i);
Serial.print(":");
Serial.println(s);
}
}

void loop() {
delay(2);
}
//********************************************

@devyte

This comment has been minimized.

Copy link
Collaborator

@devyte devyte commented Jan 6, 2018

Same issue as #3616 , tracking there.
Closing this one.

@devyte devyte closed this Jan 6, 2018
@weswitt

This comment has been minimized.

Copy link

@weswitt weswitt commented Jan 6, 2018

@ondabeach thanks for the assistance. below is the diagnostic output from the sketch you provided:

#####################


Firmware:
Chip Id: 0042A910
Core version: 2_4_0
SDK version: 2.1.0(deb1901)
Boot version: 31
Boot mode: 1


Flash chip information:
Flash chip Id: 001440E0 (for example: Id=001640E0 Manuf=E0, Device=4016 (swap bytes))
Sketch thinks Flash RAM is size: 0.50 MB
Actual size based on chip Id: 1.00 MB ... given by (2^( "Device" - 1) / 8 / 1024
Flash frequency: 40.00 MHz
Flash write mode: DIO


File system (SPIFFS):
Total KB: 51.72 KB
Used KB: 0.00 KB
Block size: 4096
Page size: 256
Maximum open files: 5
Maximum path length: 32

SPIFFS directory {/} :


CPU frequency: 80 MHz

#####################file open failed
====== Reading from SPIFFS file =======
1:
2:
3:
4:
5:
6:
7:
8:
9:
10:

@ondabeach

This comment has been minimized.

Copy link

@ondabeach ondabeach commented Jan 6, 2018

@weswitt The last sketch was just to get the flash chip data, the printing of 1~10 at the bottom was left in by mistake.

The chip ID retrieved shows that you have a flash chip that should work, it does for the OP and myself.
FYI, you have flash size set to 512k 64k but your board actually has 1MB. That isn't an issue as the actual size is bigger than your setting.

Try the following code, it should work on your board.

//**********************************************************
#include "FS.h"

String realSize = String(ESP.getFlashChipRealSize());
String ideSize = String(ESP.getFlashChipSize());
bool flashCorrectlyConfigured = realSize.equals(ideSize);

void setup() {
Serial.begin(9600);

if(flashCorrectlyConfigured){
SPIFFS.begin();
Serial.println("flash correctly configured, SPIFFS started, IDE size: " + ideSize + ", real size: " + realSize);
}
else Serial.println("flash incorrectly configured, SPIFFS cannot start, IDE size: " + ideSize + ", real size: " + realSize);

Serial.println("\nVery basic Spiffs example, writing 10 lines to SPIFFS filesystem, and then read them back");
SPIFFS.begin();

if (!SPIFFS.exists("/formatComplete.txt")) {
Serial.println("Please wait 30 secs for SPIFFS to be formatted");
SPIFFS.format();
//delay(30000);
Serial.println("Spiffs formatted");

File f = SPIFFS.open("/formatComplete.txt", "w");
if (!f) {
    Serial.println("file open failed");
} else {
    f.println("Format Complete");
}

} else {
Serial.println("SPIFFS is formatted. Moving along...");
}
}

void loop() {
// open file for writing
File f = SPIFFS.open("/f.txt", "w");
if (!f) {
Serial.println("file open failed");
}
Serial.println("====== Writing to SPIFFS file =========");
// write 10 strings to file
for (int i=1; i<=10; i++){
f.print("Millis() : ");
f.println(millis());
Serial.println(millis());
}

f.close();

// open file for reading
f = SPIFFS.open("/f.txt", "r");
if (!f) {
Serial.println("file open failed");
} Serial.println("====== Reading from SPIFFS file =======");
// write 10 strings to file
for (int i=1; i<=10; i++){
String s=f.readStringUntil('\n');
Serial.print(i);
Serial.print(":");
Serial.println(s);
}
// wait a few seconds before doing it all over again
delay(3000);

}
//*************************************************************************

@weswitt

This comment has been minimized.

Copy link

@weswitt weswitt commented Jan 7, 2018

Thanks @ondabeach. Running that code does confirm that writing to and reading from SPIFFS work fine on my device.

But this brings me back to my problem. What I'm seeing is that when I flash a SPIFFS to the device and then run a sketch the sketch sees zero files.

This happens if I use the Arduino IDE or esptool. This is the esptool command line that I'm using:

esptool.exe -vv -cd ck -cb 115200 -cp com3 -ca 0xEB000 -cf test-spiffs.bin

Does the tool used to flash the SPIFFS need to be up todate with the SPIFFS in the core? Maybe the tool is too old and is not writing a format that the core runtime understands?

@ondabeach

This comment has been minimized.

Copy link

@ondabeach ondabeach commented Jan 7, 2018

No problem @weswitt.

I'm not sure if esptool needs to.be updated. Much quicker and simpler to use the IDE as you're already in there rather than using command line methods.

We know your hardware is ok and that sketch can write and read SPIFFS so to look at your uploading files to SPIFFS issue:

No idea of your experience level, I'm just a hacker myself so we'll start with the basics.

Use "Show Sketch Folder" under "Sketch" menu.
Create a folder there called "data".
Create/copy your files there. (For testing just create a text file there called "Test.txt" with contents as follows:
Line 1
Line 2
Line 3
Line 4
Line 5
Line 6
Line 7
Line 8
Line 9
Line 10

Make sure you carriage return after the "10" or that line won't be read.
Select "Sketch Data Upload" from Tools menu. (Make sure serial monitor is not open and board is in upload mode)

Here's a tidied up version of the Get Flash Data/SPIFFS Read sketch.

//*************************************************************
#include "FS.h"

void setup() {
Serial.begin(9600);

SPIFFS.begin();
FSInfo fs_info;
SPIFFS.info(fs_info);

float fileTotalKB = (float)fs_info.totalBytes / 1024.0;
float fileUsedKB = (float)fs_info.usedBytes / 1024.0;

float flashChipSize = (float)ESP.getFlashChipSize() / 1024.0 / 1024.0;
float realFlashChipSize = (float)ESP.getFlashChipRealSize() / 1024.0 / 1024.0;
float flashFreq = (float)ESP.getFlashChipSpeed() / 1000.0 / 1000.0;
FlashMode_t ideMode = ESP.getFlashChipMode();

Serial.println("==========================================================");
Serial.println("Firmware: ");
Serial.printf(" Chip Id: %08X\n", ESP.getChipId());
Serial.print(" Core version: "); Serial.println(ESP.getCoreVersion());
Serial.print(" SDK version: "); Serial.println(ESP.getSdkVersion());
Serial.print(" Boot version: "); Serial.println(ESP.getBootVersion());
Serial.print(" Boot mode: "); Serial.println(ESP.getBootMode());

Serial.printf("__________________________\n\n");

Serial.println("Flash chip information: ");
Serial.printf(" Flash chip Id: %08X (for example: Id=001640E0 Manuf=E0, Device=4016 (swap bytes))\n", ESP.getFlashChipId());
Serial.printf(" Sketch thinks Flash RAM is size: "); Serial.print(flashChipSize); Serial.println(" MB");
Serial.print(" Actual size based on chip Id: "); Serial.print(realFlashChipSize); Serial.println(" MB ... given by (2^( "Device" - 1) / 8 / 1024");
Serial.print(" Flash frequency: "); Serial.print(flashFreq); Serial.println(" MHz");
Serial.printf(" Flash write mode: %s\n", (ideMode == FM_QIO ? "QIO" : ideMode == FM_QOUT ? "QOUT" : ideMode == FM_DIO ? "DIO" : ideMode == FM_DOUT ? "DOUT" : "UNKNOWN"));
Serial.printf(" CPU frequency: %u MHz\n\n", ESP.getCpuFreqMHz());
Serial.printf("__________________________\n\n");

Serial.println("File system (SPIFFS): ");
Serial.print(" Total KB: "); Serial.print(fileTotalKB); Serial.println(" KB");
Serial.print(" Used KB: "); Serial.print(fileUsedKB); Serial.println(" KB");
Serial.printf(" Block size: %lu\n", fs_info.blockSize);
Serial.printf(" Page size: %lu\n", fs_info.pageSize);
Serial.printf(" Maximum open files: %lu\n", fs_info.maxOpenFiles);
Serial.printf(" Maximum path length: %lu\n\n", fs_info.maxPathLength);
Serial.printf("__________________________\n\n");

Dir dir = SPIFFS.openDir("/");
Serial.println("SPIFFS directory {/} :");
while (dir.next()) {
Serial.print("File Name "); Serial.println(dir.fileName());
Serial.print("File Length "); Serial.println(dir.fileSize());
Serial.print("====== First 10 lines of file ("); Serial.print(dir.fileName()); Serial.println(") =======");
// open file for reading
File f = SPIFFS.open("/Test.txt", "r");
if (!f) {
Serial.println("file open failed");
}
else{
// read 10 strings from file
for (int i=1; i<=10; i++){
String s=f.readStringUntil('\n');
Serial.print(i);
Serial.print(" : ");
Serial.println(s);
}
}
Serial.println("==========================================================");
}

}

void loop() {
delay(3000);
}
//****************************************************************

@igrr igrr reopened this Jan 7, 2018
@ondabeach

This comment has been minimized.

Copy link

@ondabeach ondabeach commented Jan 7, 2018

Hi @igrr, here's the output with the PUYA chip:

phys_addr=962560
phys_size=65536
phys_page=256
phys_block=4096
Flash ID: 00146085
Flash size, calculated by ID: 1048576

Flash size, as set in IDE: 1048576
Formatting FS...
SPIFFSImpl: allocating 512+240+1400=2152 bytes
SPIFFSImpl: mounting fs @eb000, size=10000, block=1000, page=100
SPIFFSImpl: mount rc=0
Mounting FS...
SPIFFSImpl: mounting fs @eb000, size=10000, block=1000, page=100
SPIFFSImpl: mount rc=0
Testing read/write...
readFile
SPIFFSImpl::open: fd=-10002 path=f.txt openMode=0 accessMode=1 err=-10002
33: Failed to open file
appendFile
File size=0
f.write returned 7
SPIFFS_close: fd=1
readFile
File size=7
SPIFFS_read rc=-10012
File contents:


SPIFFS_close: fd=1
appendFile
File size=7
f.write returned 7
SPIFFS_close: fd=1
readFile
File size=7
SPIFFS_read rc=-10012
File contents:


SPIFFS_close: fd=2

@ondabeach

This comment has been minimized.

Copy link

@ondabeach ondabeach commented Jan 7, 2018

@igrr For comparison, this is the output from a BergMicro chip on one of my last 2 modules that can use SPIFFS (hint hint ;)

phys_addr=962560
phys_size=65536
phys_page=256
phys_block=4096
Flash ID: 001440E0
Flash size, calculated by ID: 1048576

Flash size, as set in IDE: 1048576
Formatting FS...
SPIFFSImpl: allocating 512+240+1400=2152 bytes
SPIFFSImpl: mounting fs @eb000, size=10000, block=1000, page=100
SPIFFSImpl: mount rc=-10025
Mounting FS...
SPIFFSImpl: mounting fs @eb000, size=10000, block=1000, page=100
SPIFFSImpl: mount rc=0
Testing read/write...
readFile
SPIFFSImpl::open: fd=-10002 path=f.txt openMode=0 accessMode=1 err=-10002
33: Failed to open file
appendFile
File size=0
f.write returned 7
SPIFFS_close: fd=1
readFile
File size=7
File contents:
foobar


SPIFFS_close: fd=1
appendFile
File size=7
f.write returned 7
SPIFFS_close: fd=1
readFile
File size=14
File contents:
foobar
foobar


SPIFFS_close: fd=1

@igrr

This comment has been minimized.

Copy link
Member

@igrr igrr commented Jan 7, 2018

@ondabeach thanks for these logs! What happens if you run the test on the PUYA module, setting flash configuration in IDE to, say, 512K/64K?

@ondabeach

This comment has been minimized.

Copy link

@ondabeach ondabeach commented Jan 7, 2018

@igrr pretty much the same result Ivan. Thanks for your help with this issue. Whatever you need, just let me know.

phys_addr=438272
phys_size=65536
phys_page=256
phys_block=4096
Flash ID: 00146085
Flash size, calculated by ID: 1048576

Flash size, as set in IDE: 524288
Formatting FS...
SPIFFSImpl: allocating 512+240+1400=2152 bytes
SPIFFSImpl: mounting fs @6b000, size=10000, block=1000, page=100
SPIFFSImpl: mount rc=0
Mounting FS...
SPIFFSImpl: mounting fs @6b000, size=10000, block=1000, page=100
SPIFFSImpl: mount rc=0
Testing read/write...
readFile
SPIFFSImpl::open: fd=-10002 path=f.txt openMode=0 accessMode=1 err=-10002
33: Failed to open file
appendFile
File size=0
f.write returned 7
SPIFFS_close: fd=1
readFile
File size=7
SPIFFS_read rc=-10012
File contents:


SPIFFS_close: fd=1
appendFile
File size=7
f.write returned 7
SPIFFS_close: fd=1
readFile
File size=7
SPIFFS_read rc=-10012
File contents:


SPIFFS_close: fd=2

@igrr

This comment has been minimized.

Copy link
Member

@igrr igrr commented Jan 7, 2018

Hmm, that doesn't give any clue, to be honest. Okay, let's try another kind of a test:

https://gist.github.com/igrr/964b0d139ce63fcee07c8c1f1facb24d

Same as before, please have debug level set to CORE. No need to run on the "working" board, just on the PUYA one.

@funtastic99

This comment has been minimized.

Copy link
Author

@funtastic99 funtastic99 commented Jan 7, 2018

Hi igrr,
i am also still facing this problem but as a workaround now switched to eeprom.
I also loaded your test sketch. Here is the output:

scandone
state: 0 -> 2 (b0)
state: 2 -> 3 (0)
state: 3 -> 5 (10)
add 0
aid 12
cnt

connected with CAROLAN, channel 6
dhcp client start...
phys_addr=962560
phys_size=65536
phys_page=256
phys_block=4096
Flash ID: 00146085
Flash size, calculated by ID: 1048576

Flash size, as set in IDE: 1048576
Testing write...
Test data partition @ 0xeb000
dst=0 src=0 len=0
dst=0 src=0 len=4
dst=0 src=0 len=16
dst=0 src=0 len=64
dst=0 src=0 len=1
dst=0 src=1 len=1
dst=1 src=0 len=1
dst=1 src=1 len=1
dst=1 src=1 len=2
dst=1 src=1 len=3
dst=1 src=1 len=4
dst=1 src=1 len=5
dst=3 src=2 len=5
dst=4 src=4 len=60
dst=59 src=0 len=5
dst=60 src=0 len=4
dst=60 src=0 len=3
dst=60 src=0 len=2
dst=63 src=0 len=1
dst=64 src=0 len=0
dst=59 src=59 len=5
dst=60 src=60 len=4
dst=60 src=60 len=3
dst=60 src=60 len=2
dst=63 src=63 len=1
dst=64 src=64 len=0
Testing write/read...
Seed=0 Start=235 Count=16
235: erase write ip:192.168.178.107,mask:255.255.255.0,gw:192.168.178.1
read done
236: erase write read done
237: erase write read done
238: erase write read done
pm open,type:2 0
239: erase write read done
240: erase write read done
241: erase write read done
242: erase write read done
243: erase write read done
244: erase write read done
245: erase write read done
246: erase write read done
247: erase write read done
248: erase write read done
249: erase write read done
250: erase write bcn_timout,ap_probe_send_start
read done
Seed=1 Start=235 Count=16
235: erase write ap_probe_send over, rest wifi status to disassoc
state: 5 -> 0 (1)
rm 0
pm close 7
read done
236: erase write scandone
state: 0 -> 2 (b0)
state: 2 -> 3 (0)
state: 3 -> 5 (10)
add 0
aid 12
cnt

connected with CAROLAN, channel 6
dhcp client start...
read done
237: erase write read done
238: erase write read done
239: erase write read done
240: erase write read done
241: erase write pm open,type:2 0
read done
242: erase write read done
243: erase write read done
244: erase write read done
245: erase write read done
246: erase write read done
247: erase write read done
248: erase write read done
249: erase write read done
250: erase write read done
Seed=42 Start=235 Count=16
235: erase write read done
236: erase write read done
237: erase write read done
238: erase write read done
239: erase write read done
240: erase write read done
241: erase write read done
242: erase write read done
243: erase write read done
244: erase write read done
245: erase write read done
246: erase write read done
247: erase write read done
248: erase write read done
249: erase write read done
250: erase write read done
Formatting FS...
SPIFFSImpl: allocating 512+240+1400=2152 bytes
SPIFFSImpl: mounting fs @eb000, size=10000, block=1000, page=100
SPIFFSImpl: mount rc=-10025
Mounting FS...
SPIFFSImpl: mounting fs @eb000, size=10000, block=1000, page=100
SPIFFSImpl: mount rc=0
Testing read/write...
readFile
SPIFFSImpl::open: fd=-10002 path=f.txt openMode=0 accessMode=1 err=-10002
36: Failed to open file
appendFile
File size=0
f.write returned 7
SPIFFS_close: fd=1
readFile
File size=7
File contents:
foobar


SPIFFS_close: fd=1
appendFile
File size=7
f.write returned 7
SPIFFS_close: fd=1
readFile
File size=14
File contents:
foobar
foobar


SPIFFS_close: fd=1

@igrr

This comment has been minimized.

Copy link
Member

@igrr igrr commented Jan 7, 2018

@funtastic99 Your output is what i would describe as "normal". Perhaps the issues you and @ondabeach are seeing are not the same after all...

@funtastic99

This comment has been minimized.

Copy link
Author

@funtastic99 funtastic99 commented Jan 7, 2018

@igrr The strange thing is that ondebeach an me both have spiffs problems with flash id Manufacturer: 85, Device: 6014. Maybe its causing different effects.
But still the initial testprogramm at the beginning of this ticket is not showing the expected output after writing. Any idea why ?
Thank!

@ondabeach

This comment has been minimized.

Copy link

@ondabeach ondabeach commented Jan 7, 2018

@igrr Ok, exact same settings as last test, same board, nothing moved or changed.

phys_addr=962560
phys_size=65536
phys_page=256
phys_block=4096
Flash ID: 00146085
Flash size, calculated by ID: 1048576

Flash size, as set in IDE: 1048576
Testing write...
Test data partition @ 0xeb000
dst=0 src=0 len=0
dst=0 src=0 len=4
dst=0 src=0 len=16
dst=0 src=0 len=64
dst=0 src=0 len=1
dst=0 src=1 len=1
dst=1 src=0 len=1
dst=1 src=1 len=1
dst=1 src=1 len=2
dst=1 src=1 len=3
dst=1 src=1 len=4
dst=1 src=1 len=5
dst=3 src=2 len=5
dst=4 src=4 len=60
dst=59 src=0 len=5
dst=60 src=0 len=4
dst=60 src=0 len=3
dst=60 src=0 len=2
dst=63 src=0 len=1
dst=64 src=0 len=0
dst=59 src=59 len=5
dst=60 src=60 len=4
dst=60 src=60 len=3
dst=60 src=60 len=2
dst=63 src=63 len=1
dst=64 src=64 len=0
Testing write/read...
Seed=0 Start=235 Count=16
235: erase write read done
236: erase write read done
237: erase write read done
238: erase write read done
239: erase write read done
240: erase write read done
241: erase write read done
242: erase write read done
243: erase write read done
244: erase write read done
245: erase write read done
246: erase write read done
247: erase write read done
248: erase write read done
249: erase write read done
250: erase write read done
Seed=1 Start=235 Count=16
235: erase write read done
236: erase write read done
237: erase write read done
238: erase write read done
239: erase write read done
240: erase write read done
241: erase write read done
242: erase write read done
243: erase write read done
244: erase write read done
245: erase write read done
246: erase write read done
247: erase write read done
248: erase write read done
249: erase write read done
250: erase write read done
Seed=42 Start=235 Count=16
235: erase write read done
236: erase write read done
237: erase write read done
238: erase write read done
239: erase write read done
240: erase write read done
241: erase write read done
242: erase write read done
243: erase write read done
244: erase write read done
245: erase write read done
246: erase write read done
247: erase write read done
248: erase write read done
249: erase write read done
250: erase write read done
Formatting FS...
SPIFFSImpl: allocating 512+240+1400=2152 bytes
SPIFFSImpl: mounting fs @eb000, size=10000, block=1000, page=100
SPIFFSImpl: mount rc=-10025
Mounting FS...
SPIFFSImpl: mounting fs @eb000, size=10000, block=1000, page=100
SPIFFSImpl: mount rc=0
Testing read/write...
readFile
SPIFFSImpl::open: fd=-10002 path=f.txt openMode=0 accessMode=1 err=-10002
36: Failed to open file
appendFile
File size=0
f.write returned 7
SPIFFS_close: fd=1
readFile
File size=7
SPIFFS_read rc=-10012
File contents:


SPIFFS_close: fd=1
appendFile
File size=7
f.write returned 7
SPIFFS_close: fd=1
readFile
File size=7
SPIFFS_read rc=-10012
File contents:


SPIFFS_close: fd=2

@igrr

This comment has been minimized.

Copy link
Member

@igrr igrr commented Jan 7, 2018

Sorry, not getting any closer...

Please try replacing Esp.cpp in your esp8266 core directory with the one attached here, run same test.

Esp.zip

@ondabeach

This comment has been minimized.

Copy link

@ondabeach ondabeach commented Jan 7, 2018

@igrr esp.cpp replaced, IDE restarted and all settings the same.

phys_addr=438272
phys_size=65536
phys_page=256
phys_block=4096
Flash ID: 00146085
Flash size, calculated by ID: 1048576

Flash size, as set in IDE: 524288
Testing write...
Test data partition @ 0x6b000
dst=0 src=0 len=0
dst=0 src=0 len=4
dst=0 src=0 len=16
dst=0 src=0 len=64
dst=0 src=0 len=1
dst=0 src=1 len=1
dst=1 src=0 len=1
dst=1 src=1 len=1
dst=1 src=1 len=2
dst=1 src=1 len=3
dst=1 src=1 len=4
dst=1 src=1 len=5
dst=3 src=2 len=5
dst=4 src=4 len=60
dst=59 src=0 len=5
dst=60 src=0 len=4
dst=60 src=0 len=3
dst=60 src=0 len=2
dst=63 src=0 len=1
dst=64 src=0 len=0
dst=59 src=59 len=5
dst=60 src=60 len=4
dst=60 src=60 len=3
dst=60 src=60 len=2
dst=63 src=63 len=1
dst=64 src=64 len=0
Testing write/read...
Seed=0 Start=107 Count=16
107: erase write Flash verify error @0x6b000, orig=0xffffffff, data=0x00000000, read=0x00000000
read done
108: erase write Flash verify error @0x6c000, orig=0xffffffff, data=0x00000000, read=0x00000000
read done
109: erase write Flash verify error @0x6d000, orig=0xffffffff, data=0x00000000, read=0x00000000
read done
110: erase write Flash verify error @0x6e000, orig=0xffffffff, data=0x00000000, read=0x00000000
read done
111: erase write Flash verify error @0x6f000, orig=0xffffffff, data=0x00000000, read=0x00000000
read done
112: erase write Flash verify error @0x70000, orig=0xffffffff, data=0x00000000, read=0x00000000
read done
113: erase write Flash verify error @0x71000, orig=0xffffffff, data=0x00000000, read=0x00000000
read done
114: erase write Flash verify error @0x72000, orig=0xffffffff, data=0x00000000, read=0x00000000
read done
115: erase write Flash verify error @0x73000, orig=0xffffffff, data=0x00000000, read=0x00000000
read done
116: erase write Flash verify error @0x74000, orig=0xffffffff, data=0x00000000, read=0x00000000
read done
117: erase write Flash verify error @0x75000, orig=0xffffffff, data=0x00000000, read=0x00000000
read done
118: erase write Flash verify error @0x76000, orig=0xffffffff, data=0x00000000, read=0x00000000
read done
119: erase write Flash verify error @0x77000, orig=0xffffffff, data=0x00000000, read=0x00000000
read done
120: erase write Flash verify error @0x78000, orig=0xffffffff, data=0x00000000, read=0x00000000
read done
121: erase write Flash verify error @0x79000, orig=0xffffffff, data=0x00000000, read=0x00000000
read done
122: erase write Flash verify error @0x7a000, orig=0xffffffff, data=0x00000000, read=0x00000000
read done
Seed=1 Start=107 Count=16
107: erase write read done
108: erase write read done
109: erase write read done
110: erase write read done
111: erase write read done
112: erase write read done
113: erase write read done
114: erase write read done
115: erase write read done
116: erase write read done
117: erase write read done
118: erase write read done
119: erase write read done
120: erase write read done
121: erase write read done
122: erase write read done
Seed=42 Start=107 Count=16
107: erase write read done
108: erase write read done
109: erase write read done
110: erase write read done
111: erase write read done
112: erase write read done
113: erase write read done
114: erase write read done
115: erase write read done
116: erase write read done
117: erase write read done
118: erase write read done
119: erase write read done
120: erase write read done
121: erase write read done
122: erase write read done
Formatting FS...
SPIFFSImpl: allocating 512+240+1400=2152 bytes
SPIFFSImpl: mounting fs @6b000, size=10000, block=1000, page=100
SPIFFSImpl: mount rc=-10025
Flash verify error @0x6b0fc, orig=0xffff, data=0xffff0429, read=0xfff70429
Flash verify error @0x6c0fc, orig=0xffff, data=0xffff0429, read=0xffff0429
Flash verify error @0x6d0fc, orig=0xffff, data=0xffff0429, read=0xffdf0429
Flash verify error @0x6e0fc, orig=0xffff, data=0xffff0429, read=0xffff0429
Flash verify error @0x6f0fc, orig=0xffff, data=0xffff0429, read=0xefff0429
Flash verify error @0x700fc, orig=0xffff, data=0xffff0429, read=0xffff0429
Flash verify error @0x710fc, orig=0xffff, data=0xffff0429, read=0xfbff0429
Flash verify error @0x720fc, orig=0xffff, data=0xffff0429, read=0x7fff0429
Flash verify error @0x730fc, orig=0xffff, data=0xffff0429, read=0x7fff0429
Flash verify error @0x740fc, orig=0xffff, data=0xffff0429, read=0xffff0429
Flash verify error @0x750fc, orig=0xffff, data=0xffff0429, read=0xefff0429
Flash verify error @0x760fc, orig=0xffff, data=0xffff0429, read=0xffff0429
Flash verify error @0x770fc, orig=0xffff, data=0xffff0429, read=0xfff70429
Flash verify error @0x780fc, orig=0xffff, data=0xffff0429, read=0xffff0429
Flash verify error @0x790fc, orig=0xffff, data=0xffff0429, read=0xfff70429
Flash verify error @0x7a0fc, orig=0xffff, data=0xffff0429, read=0xbfff0429
Mounting FS...
SPIFFSImpl: mounting fs @6b000, size=10000, block=1000, page=100
SPIFFSImpl: mount rc=0
Testing read/write...
readFile
SPIFFSImpl::open: fd=-10002 path=f.txt openMode=0 accessMode=1 err=-10002
36: Failed to open file
appendFile
Flash verify error @0x6b105, orig=0xffffffff, data=0x00000000, read=0x00000000
Flash verify error @0x6b106, orig=0xffffffff, data=0x00000000, read=0x00000000
Flash verify error @0x6b107, orig=0xffffffff, data=0x00000000, read=0x00000000
Flash verify error @0x6b108, orig=0xffffffff, data=0x00000000, read=0x00000000
Flash verify error @0x6b109, orig=0xffffffff, data=0x00000000, read=0x00000000
Flash verify error @0x6b10a, orig=0xffffffff, data=0x00000000, read=0x00000000
File size=0
f.write returned 7
Flash verify error @0x6b000, orig=0xffff8001, data=0x0001ffff, read=0x0001f77f
Flash verify error @0x6b204, orig=0xfffffffc, data=0x6f6f66ff, read=0x6f6f66ff
Flash verify error @0x6b105, orig=0x00, data=0x00000000, read=0x00000000
Flash verify error @0x6b106, orig=0x00, data=0x00000000, read=0x00000000
Flash verify error @0x6b107, orig=0x00, data=0x00000000, read=0x00000000
Flash verify error @0x6b108, orig=0x00, data=0x00000000, read=0x00000000
Flash verify error @0x6b109, orig=0x00, data=0x00000000, read=0x00000000
Flash verify error @0x6b10a, orig=0x00, data=0x00000000, read=0x00000000
SPIFFS_close: fd=1
readFile
File size=7
SPIFFS_read rc=-10012
File contents:


SPIFFS_close: fd=1
appendFile
File size=7
f.write returned 7
SPIFFS_close: fd=1
readFile
File size=7
SPIFFS_read rc=-10012
File contents:


SPIFFS_close: fd=2

@ondabeach

This comment has been minimized.

Copy link

@ondabeach ondabeach commented Jan 7, 2018

@igrr I have a second machine as a backup. I made everything the same as far as I can see, including the new Esp.cpp. The only difference I'm aware of is that on the original machine that gave the Flash Verify errors in the output The esp8266com/esp8266 folder was updated to v2.4.0 using GitGui whereas on this machine it was done through the Boards Manager and the URL in IDE preferences.

I can't write to SPIFFS via a sketch on either machine but the esp8266 Sketch Data Upload plugin can.

phys_addr=438272
phys_size=65536
phys_page=256
phys_block=4096
Flash ID: 00146085
Flash size, calculated by ID: 1048576

Flash size, as set in IDE: 524288
Testing write...
Test data partition @ 0x6b000
dst=0 src=0 len=0
dst=0 src=0 len=4
dst=0 src=0 len=16
dst=0 src=0 len=64
dst=0 src=0 len=1
dst=0 src=1 len=1
dst=1 src=0 len=1
dst=1 src=1 len=1
dst=1 src=1 len=2
dst=1 src=1 len=3
dst=1 src=1 len=4
dst=1 src=1 len=5
dst=3 src=2 len=5
dst=4 src=4 len=60
dst=59 src=0 len=5
dst=60 src=0 len=4
dst=60 src=0 len=3
dst=60 src=0 len=2
dst=63 src=0 len=1
dst=64 src=0 len=0
dst=59 src=59 len=5
dst=60 src=60 len=4
dst=60 src=60 len=3
dst=60 src=60 len=2
dst=63 src=63 len=1
dst=64 src=64 len=0
Testing write/read...
Seed=0 Start=107 Count=16
107: erase write read done
108: erase write read done
109: erase write read done
110: erase write read done
111: erase write read done
112: erase write read done
113: erase write read done
114: erase write read done
115: erase write read done
116: erase write read done
117: erase write read done
118: erase write read done
119: erase write read done
120: erase write read done
121: erase write read done
122: erase write read done
Seed=1 Start=107 Count=16
107: erase write read done
108: erase write read done
109: erase write read done
110: erase write read done
111: erase write read done
112: erase write read done
113: erase write read done
114: erase write read done
115: erase write read done
116: erase write read done
117: erase write read done
118: erase write read done
119: erase write read done
120: erase write read done
121: erase write read done
122: erase write read done
Seed=42 Start=107 Count=16
107: erase write read done
108: erase write read done
109: erase write read done
110: erase write read done
111: erase write read done
112: erase write read done
113: erase write read done
114: erase write read done
115: erase write read done
116: erase write read done
117: erase write read done
118: erase write read done
119: erase write read done
120: erase write read done
121: erase write read done
122: erase write read done
Formatting FS...
SPIFFSImpl: allocating 512+240+1400=2152 bytes
SPIFFSImpl: mounting fs @6b000, size=10000, block=1000, page=100
SPIFFSImpl: mount rc=-10025
Mounting FS...
SPIFFSImpl: mounting fs @6b000, size=10000, block=1000, page=100
SPIFFSImpl: mount rc=0
Testing read/write...
readFile
SPIFFSImpl::open: fd=-10002 path=f.txt openMode=0 accessMode=1 err=-10002
36: Failed to open file
appendFile
File size=0
f.write returned 7
SPIFFS_close: fd=1
readFile
File size=7
SPIFFS_read rc=-10012
File contents:


SPIFFS_close: fd=1
appendFile
File size=7
f.write returned 7
SPIFFS_close: fd=1
readFile
File size=7
SPIFFS_read rc=-10012
File contents:


SPIFFS_close: fd=2

@igrr

This comment has been minimized.

Copy link
Member

@igrr igrr commented Jan 7, 2018

Thanks @ondabeach , these logs with "flash verify error" mean that we are indeed seeing some hardware dependent issue.
I can't explain why you are not seeing these with the other machine, my only guess is that you are copying Esp.cpp file into one core installation, but the IDE is using some other one.

I have updated the test: https://gist.github.com/igrr/964b0d139ce63fcee07c8c1f1facb24d and Esp.zip, please give it another try. If the log ends up being long, could you please paste it into gist.github.com and post a link here?
Suggest doing this on the machine where you got Flash verify errors in the previous test.

@ondabeach

This comment has been minimized.

Copy link

@ondabeach ondabeach commented Jan 8, 2018

@igrr No longer than the others I think.

phys_addr=438272
phys_size=65536
phys_page=256
phys_block=4096
Flash ID: 00146085
Flash size, calculated by ID: 1048576

Flash size, as set in IDE: 524288
Testing flash write bits...
Mask=0xffffffff
Mask=0xfffffffe
Mask=0xfffffffd
Mask=0xfffffffb
Mask=0xfffffff7
Mask=0xffffffef
Mask=0xffffffdf
Mask=0xffffffbf
Mask=0xffffff7f
Mask=0xfffffeff
Mask=0xfffffdff
Mask=0xfffffbff
Mask=0xfffff7ff
Mask=0xffffefff
Mask=0xffffdfff
Mask=0xffffbfff
Mask=0xffff7fff
Mask=0xfffeffff
Mask=0xfffdffff
Mask=0xfffbffff
Mask=0xfff7ffff
Mask=0xffefffff
Mask=0xffdfffff
Mask=0xffbfffff
Mask=0xff7fffff
Mask=0xfeffffff
Mask=0xfdffffff
Mask=0xfbffffff
Mask=0xf7ffffff
Mask=0xefffffff
Mask=0xdfffffff
Mask=0xbfffffff
Mask=0x7fffffff
Testing flash random write...
Seed=0 Start=107 Count=16
Seed=1 Start=107 Count=16
Seed=2 Start=107 Count=16
Seed=3 Start=107 Count=16
Formatting FS...
SPIFFSImpl: allocating 512+240+1400=2152 bytes
SPIFFSImpl: mounting fs @6b000, size=10000, block=1000, page=100
SPIFFSImpl: mount rc=-10025
Flash write verify error addr=0x6b0fc, orig=0x0000ffff, data=0xffff0429, read=0xfff70429
Flash write verify error addr=0x6c0fc, orig=0x0000ffff, data=0xffff0429, read=0xffff0429
Flash write verify error addr=0x6d0fc, orig=0x0000ffff, data=0xffff0429, read=0xffdf0429
Flash write verify error addr=0x6e0fc, orig=0x0000ffff, data=0xffff0429, read=0xffef0429
Flash write verify error addr=0x6f0fc, orig=0x0000ffff, data=0xffff0429, read=0xefff0429
Flash write verify error addr=0x700fc, orig=0x0000ffff, data=0xffff0429, read=0xffff0429
Flash write verify error addr=0x710fc, orig=0x0000ffff, data=0xffff0429, read=0xffff0429
Flash write verify error addr=0x720fc, orig=0x0000ffff, data=0xffff0429, read=0x7fff0429
Flash write verify error addr=0x730fc, orig=0x0000ffff, data=0xffff0429, read=0x7fff0429
Flash write verify error addr=0x740fc, orig=0x0000ffff, data=0xffff0429, read=0xffff0429
Flash write verify error addr=0x750fc, orig=0x0000ffff, data=0xffff0429, read=0xefdf0429
Flash write verify error addr=0x760fc, orig=0x0000ffff, data=0xffff0429, read=0xffff0429
Flash write verify error addr=0x770fc, orig=0x0000ffff, data=0xffff0429, read=0xff770429
Flash write verify error addr=0x780fc, orig=0x0000ffff, data=0xffff0429, read=0xffff0429
Flash write verify error addr=0x790fc, orig=0x0000ffff, data=0xffff0429, read=0xffd70429
Flash write verify error addr=0x7a0fc, orig=0x0000ffff, data=0xffff0429, read=0xbfff0429
Mounting FS...
SPIFFSImpl: mounting fs @6b000, size=10000, block=1000, page=100
Flash read verify (#0) error addr=0x710fc, data=0xffbf0429, check=0xffff0429
Flash read verify (#1) error addr=0x710fc, data=0xffbf0429, check=0xffff0429
Flash read verify (#2) error addr=0x710fc, data=0xffbf0429, check=0xffff0429
Flash read verify (#0) error addr=0x710fc, data=0xffbf0429, check=0xffff0429
Flash read verify (#1) error addr=0x710fc, data=0xffbf0429, check=0xffff0429
Flash read verify (#2) error addr=0x710fc, data=0xffbf0429, check=0xffff0429
Flash read verify (#0) error addr=0x720fc, data=0x7eff0429, check=0x7fff0429
Flash read verify (#1) error addr=0x720fc, data=0x7eff0429, check=0x7fff0429
Flash read verify (#2) error addr=0x720fc, data=0x7eff0429, check=0x7fff0429
Flash read verify (#0) error addr=0x790fc, data=0xffd70429, check=0xffdf0429
Flash read verify (#2) error addr=0x790fc, data=0xffd70429, check=0xffdf0429
Flash read verify (#0) error addr=0x720fc, data=0x7eff0429, check=0x7fff0429
Flash read verify (#1) error addr=0x720fc, data=0x7eff0429, check=0x7fff0429
Flash read verify (#2) error addr=0x720fc, data=0x7eff0429, check=0x7fff0429
SPIFFSImpl: mount rc=0
Testing read/write...
readFile
SPIFFSImpl::open: fd=-10002 path=f.txt openMode=0 accessMode=1 err=-10002
37: Failed to open file
appendFile
File size=0
f.write returned 7
Flash write verify error addr=0x6b000, orig=0xffff8001, data=0x0001ffff, read=0x0001f57f
Flash write verify error addr=0x6b204, orig=0xfffffffc, data=0x6f6f66ff, read=0x6f6f66ff
SPIFFS_close: fd=1
readFile
File size=7
SPIFFS_read rc=-10012
File contents:


SPIFFS_close: fd=1
appendFile
File size=7
f.write returned 7
SPIFFS_close: fd=1
readFile
File size=7
SPIFFS_read rc=-10012
File contents:


SPIFFS_close: fd=2

@kevcavein

This comment has been minimized.

Copy link

@kevcavein kevcavein commented Sep 19, 2018

Thank you @igrr. Your patch worked for me. I had trouble until I caught the comments later on about adding #define FLASH_QUIRK_WRITE_0_TO_1 and especially the part about uploading filesystem from IDE, to fix the previous possibly incorrect state of the filesystem on the device.

May I know what file should I upload to filesystem???? I almost pull all my hair off trying to figure out how to solve the spiff problem. I can't store my blynk id using wifimanager.

@DarrenSchwartz

This comment has been minimized.

Copy link

@DarrenSchwartz DarrenSchwartz commented Sep 19, 2018

Take this file Esp.zip from @mrlightsman's post above. Replace it with the esp.cpp file in your arduino IDE program folder. I am using the portable version so it looks like this: C:\arduino-1.8.6\portable\packages\esp8266\hardware\esp8266\2.4.2\cores\esp8266\Esp.cpp. Just look for the 'packages' folder in your install directory and follow it down. Once the esp.cpp file is replaced, restart the IDE and flash your sketch using the "Tools -> Erase Flash -> All Flash Contents" option. After it's done flashing, go to "Tools -> ESP8266 Sketch Data Upload". This will upload all the files in the "data" subfolder of your sketch folder.

@mcpicoli

This comment has been minimized.

Copy link

@mcpicoli mcpicoli commented Oct 5, 2018

I can confirm that @uzi18 v2 patch works nicely (thanks to everyone involved in creating it!). Using Arduino IDE, on some "Sonoff Basic" modules with PUYA chips.

However, I have some tips to give:

  • Make SURE you're patching the correct Esp.h/Esp.cpp files. Lost some time patching the wrong ones.
  • Make SURE you've cleaned any precompiled/cached versions. They're usually stored on (Windows) "<system_drive>/Users/<name_of_user>/AppData/Local/Temp/arduino_*". Kill them all and then restart the IDE. Also lost some significant amount of time before fixing this.
  • Make sure that you select (at least for the first upload) "Erase ALL Flash" under "Tools" -> "Erase Flash".
  • Clean the SPIFFS area using your favorite ESP8266 Sketch Data Uploader (answer "yes" to the question "do you want to upload a clean SPIFFS image?"). Your SPIFFS image MAY contain files already, but it must be pristine, that is, NOTHING may have been written to it after is was uploaded.
  • Having your sketch (after patch) write anything to an "unclean" SPIFFS (one that was "touched" by any kind of write from a pre-patch sketch) doesn't make it "clean".
  • Any kind of write from a pre-patch sketch to any SPIFFS image will make it unclean. Patching and reuploading your sketch will NOT work. You've got to reupload your SPIFFS image and then upload your sketch after patching before the old one still in flash writes anything to SPIFFS. I solved this by adding a large temporary delay to my sketch, giving me time to put it again in flash mode after SPIFFS image upload and before actual old sketch execution.
@PaulBortnikov

This comment has been minimized.

Copy link

@PaulBortnikov PaulBortnikov commented Oct 8, 2018

Edit file Esp.cpp:

 bool EspClass::flashWrite(uint32_t offset, uint32_t *data, size_t size) {
    static uint32_t flash_chip_id = 0;
        uint32_t *read_buf=NULL;
    if (flash_chip_id == 0)
        flash_chip_id = getFlashChipId();
    ets_isr_mask(FLASH_INT_MASK);
    int rc;
    uint32_t* ptr = data;
    if ((flash_chip_id & 0x000000ff) == 0x85) { // 0x146085 PUYA
        read_buf=new uint32_t [SPI_FLASH_SEC_SIZE / 4];
        if(!read_buf) return false;
        rc = spi_flash_read(offset, read_buf, size);
        if (rc != 0) {
            delete read_buf;
            ets_isr_unmask(FLASH_INT_MASK);
            return false;
        }
        for (size_t i = 0; i < size / 4; ++i) {
            read_buf[i] &= data[i];
        }
        ptr = read_buf;
    }
    rc = spi_flash_write(offset, ptr, size);
        if(read_buf!=NULL) delete read_buf;
    ets_isr_unmask(FLASH_INT_MASK);
    return rc == 0;
}
@uzi18

This comment has been minimized.

Copy link

@uzi18 uzi18 commented Nov 7, 2018

@ALL updated puya patch v3 used now in ESPEasy project:
https://github.com/letscontrolit/ESPEasy/blob/mega/patches/puya_v3.patch#L8

@maxim-kukushkin

This comment has been minimized.

Copy link

@maxim-kukushkin maxim-kukushkin commented Dec 1, 2018

The described fix (flashWrite function) worked for me as well with Sonoff S26, which also uses PUYA flash. Any plans to perform the change in mainline?

@mrWheel

This comment has been minimized.

Copy link

@mrWheel mrWheel commented Dec 13, 2018

Any plans to perform the change in mainline?
Is this patch part of 2.5.0-Beta1?

@devyte

This comment has been minimized.

Copy link
Collaborator

@devyte devyte commented Dec 13, 2018

It is not in -beta1. A variation of the patch used in ESPEasy will be used in the core, but it's not targeted to a release yet.

@TD-er

This comment has been minimized.

Copy link
Contributor

@TD-er TD-er commented Dec 13, 2018

@devyte Just curious, what will be different?

@mrWheel

This comment has been minimized.

Copy link

@mrWheel mrWheel commented Dec 13, 2018

Ok, so all the people that use the ESP-01 with (almost all nowadays) a PUYA chip cannot upgrade to a new version in the foreseeable future ?!
If I may ask: What's holding them up?

@maxim-kukushkin

This comment has been minimized.

Copy link

@maxim-kukushkin maxim-kukushkin commented Dec 13, 2018

Not only ESP-01 but many other new devices. Sonoff S26 is one of the examples

@devyte

This comment has been minimized.

Copy link
Collaborator

@devyte devyte commented Dec 13, 2018

@devyte Just curious, what will be different?

The patch:

  1. uses a static 4KB buffer, which means everyone using the patch has 4KB less heap, including those users who do not have a puya chip. Instead, a dynamically allocated buffer should be used.
  2. evaluates the first if condition every single call of the function, although only the first one after boot actually does anything. Instead of a private static, a global could be used that is set elsewhere only once at bootup, and then if condition is no longer needed. Alternatively, The ESPClass could get a new data member (i.e.: instead of the global flag) which is set on construction, and a method to return its value.
  3. calculates the flash_chip_id condition in the second if every single call of the function, which makes no sense, because it can't ever change at runtime. Instead, the condition should be fully calculated once and the result stored in a flag, which should then be checked.
  4. detects the condition based on flash_chip_id, which isn't very robust, as has been shown with the flash chip size. Instead, a general purpose detection approach should be implemented that transparently covers other potential brands and manufacturers that have the same quirk, should they appear.
    Example: on first bootup after flashing, choose a sector, e.g.: within the update area of the flash layout., preferably one that is unlikely to be used. Do a write test and read back sequence known to behave differently with puya chips. If puya is detected, set a quirk flag somewhere. On all successive bootups, detect that the quirk flag is set, so don't do the test again. Use the quirk flag in the write function to decide whether the additional logic is done or not.

Given the above, the patch won't be accepted as-is into our core. There have been several internal discussions on exactly what to do, but given that the patch is available, there have been, and still are, higher priorities, e.g.: figuring out how to ease IRAM usage.

There have been a lot of comments from many users in this discussion, and I see the patch is even being applied elsewhere, but I still don't see a PR here. Instead of wondering what will happen, I invite those involved to make a PR with the proposed patch, and implement at least points 1-3 above on top of it. If you come up with a good solution for point 4, even better.

@maxim-kukushkin

This comment has been minimized.

Copy link

@maxim-kukushkin maxim-kukushkin commented Dec 13, 2018

While do understand that the expected behavior in open source is to some up with a proper patch for an issue, I tend to disagree with this point:
"given that the patch is available, there have been, and still are, higher priorities"

No. You've just indicated that the patch is a really basic workaround, apparently not suitable for any production use. People only use it as PoC because it makes their devices work at least somehow, comparing to not working at all

So, if there's any prioritization done, pease consider this issue as not having any workaround at all, as apparently there's no candidate for a merge

Current firmware literally doesn't work on recent chips. Also, comparing to other issues, at least some investigation has been done by the community, identifying the root cause and a PoC fix

@mrWheel

This comment has been minimized.

Copy link

@mrWheel mrWheel commented Dec 13, 2018

Totally agree with maxim-kukushkin

@earlephilhower

This comment has been minimized.

Copy link
Collaborator

@earlephilhower earlephilhower commented Dec 13, 2018

@maxim-kukushkin and @mrWheel everyone else on the thread: you obviously care very much about the Puya patch.

Please, one of you take what @devyte has suggested and wrap it into a PR so we can advance.

@mrWheel

This comment has been minimized.

Copy link

@mrWheel mrWheel commented Dec 13, 2018

@maxim-kukushkin & @devyte
I would if only I new how ...
But yes, your right: I try to make software for devices with ESP-01’s and thats not without problems!

@TD-er

This comment has been minimized.

Copy link
Contributor

@TD-er TD-er commented Dec 13, 2018

@devyte Sounds like very legitimate points.
I didn't have a look at it myself, but wasting 4k of heap is reason enough to not apply it like it is now.
When is the deadline for 2.5.0? ;)

@devyte

This comment has been minimized.

Copy link
Collaborator

@devyte devyte commented Dec 14, 2018

Given the length of this thread, I'm closing in favor of #5493 . Resolution will be handled there.

@devyte devyte closed this Dec 14, 2018
radimkeseg added a commit to radimkeseg/esp8266-DS18B20-DeepSleep that referenced this issue Mar 20, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.