Trying to get SD card information using `SDFS.info64` function leads to a crash. #8822

BrokeStudio · 2023-01-23T13:27:07Z

Basic Infos

This issue complies with the issue POLICY doc.
I have read the documentation at readthedocs and the issue is not addressed there.
I have tested that the issue is present in current master branch (aka latest git).
I have searched the issue tracker for a similar issue.
If there is a stack dump, I have decoded it.
I have filled out all fields below.

Platform

Hardware: ESP-12E
Core Version: 3.1.1 (using platform = espressif8266@4.1.0 in platformio.ini file)
Development Env: Platformio (with -DDEBUG_ESP_CORE -DDEBUG_ESP_PORT=Serial build flags)
Operating System: Windows

Settings in IDE

Module: Generic ESP8266 Module
Flash Mode: qio (I guess)
Flash Size: 4MB
lwip Variant: v2 Lower Memory
Reset Method: nodemcu
Flash Frequency: 26MHz
CPU Frequency: 80MHz
Upload Using: Serial
Upload Speed: 921600

Problem Description

Trying to get SD card information using SDFS.info64 function leads to a crash.

It works fine using Core version 3.0.2 (using platform = espressif8266@4.0.1 in platformio.ini file).

MCVE Sketch

#include <Arduino.h>
#include <LittleFS.h>
#include <SD.h>

// source: https://github.com/yoursunny/PriUint64/blob/main/src/PriUint64.h
#include "PriUint64.h"

void setup() {
  Serial.begin(115200);
  Serial.println("\nSetup...");

  if (!SD.begin(D0)) {
    Serial.println("Initialization failed!");
  }
  else
  {
    Serial.println("Initialization successful!");
    FSInfo64 sd_info;
    SDFS.info64(sd_info);
    Serial.print(F("File system total space: "));
    Serial.print(PriUint64<DEC>(sd_info.totalBytes));
    Serial.println();

    Serial.print(F("File system used space: "));
    Serial.print(PriUint64<DEC>(sd_info.usedBytes));
    Serial.print(F(" ("));
    Serial.print(PriUint64<DEC>(round((sd_info.usedBytes * 100) / sd_info.totalBytes)));
    Serial.println(F("%)"));

    Serial.print(F("File system free space: "));
    Serial.print(PriUint64<DEC>(sd_info.totalBytes - sd_info.usedBytes));
    Serial.print(F(" ("));
    Serial.print(PriUint64<DEC>(round(((sd_info.totalBytes - sd_info.usedBytes) * 100) / sd_info.totalBytes)));
    Serial.println(F("%)"));
  }
}

void loop() {
}

Debug Messages

SDK:2.2.2-dev(38a443e)/Core:3.1.1=30101000/lwIP:STABLE-2_1_3_RELEASE/glue:1.2-65-g06164fb/BearSSL:b024386

Setup...

Initialization successful!

--------------- CUT HERE FOR EXCEPTION DECODER ---------------

Soft WDT reset

>>>stack>>>

ctx: cont
sp: 3ffffcb0 end: 3fffffd0 offset: 01a0
3ffffe50:  00000040 3ffefab8 00000000 402091c4  
3ffffe60:  00000040 00000000 00000010 00000001  
3ffffe70:  000001c0 3ffef938 00000000 402091a2
3ffffe80:  00000040 00000000 00000010 00000001
3ffffe90:  00000000 00000cc1 3ba5e353 3ffefb20
3ffffea0:  00000200 3ffef8f8 00000dec 40207b9c
3ffffeb0:  00000200 3ffef8f8 00000dec 40207390
3ffffec0:  3ffef8f8 00003181 3ffefb20 00000001
3ffffed0:  3ffef8f8 00000000 3ffefb20 40207a3d  
3ffffee0:  00003180 00000002 3ffefb20 00003182
3ffffef0:  00003181 00000002 3ffef8e8 40207c6f
3fffff00:  3ffef8c4 00198aa2 00042979 402070f9
3fffff10:  3ffef8e8 00000100 3ffe885d 40204204
3fffff20:  3ffe8d00 fffffffc feefeffe 00000000
3fffff30:  00000001 001db420 3fffff60 4020195c
3fffff40:  3fffdad0 0000001a 3ffeeac0 3ffeeb2c
3fffff50:  3fffdad0 3ffeea94 3ffeeac0 402011a8
3fffff60:  6d080000 00000000 003d0900 feefeffe
3fffff70:  00004000 00000000 000003e7 000000ff
3fffff80:  feefeffe feefeffe feefeffe feefeffe
3fffff90:  feefeffe feefeffe feefeffe feefeffe
3fffffa0:  feefeffe feefeffe feefeffe 3ffeeb2c
3fffffb0:  3fffdad0 00000000 3ffeeb18 402032fc  
3fffffc0:  feefeffe feefeffe 3fffdab0 40101431
<<<stack<<<

0x402091c4 in SPIClass::transferBytes(unsigned char const*, unsigned char*, unsigned int) at ??:?
0x402091a2 in SPIClass::transferBytes(unsigned char const*, unsigned char*, unsigned int) at ??:?
0x40207b9c in SdSpiArduinoDriver::receive(unsigned char*, unsigned int) at ??:?
0x40207390 in SharedSpiCard::readData(unsigned char*, unsigned int) at ??:?
0x40207a3d in DedicatedSpiCard::readSectors(unsigned int, unsigned char*, unsigned int) at ??:?
0x40207c6f in FsCache::prepare(unsigned int, unsigned char) at ??:?
0x402070f9 in FatPartition::freeClusterCount() at ??:?
0x40204204 in uart_write at ??:?
0x4020195c in sdfs::SDFSImpl::info64(fs::FSInfo64&) at ??:?
0x402011a8 in setup at ??:?
0x402032fc in loop_wrapper() at core_esp8266_main.cpp:?
0x40101431 in cont_wrapper at ??:?


--------------- CUT HERE FOR EXCEPTION DECODER ---------------

 ets Jan  8 2013,rst cause:2, boot mode:(3,6)

The text was updated successfully, but these errors were encountered:

mcspr · 2023-01-23T15:34:26Z

if (!SD.begin(D0)) {

Looks like an accident that this worked at all

Arduino/libraries/SD/src/SD.h

Lines 35 to 38 in ead5f94

    
           bool begin(uint8_t csPin, uint32_t cfg = SPI_HALF_SPEED) { 
        
               SDFS.setConfig(SDFSConfig(csPin, cfg)); 
        
               return (boolean)SDFS.begin(); 
        
           }

Arduino/libraries/SDFS/src/SDFS.h

Lines 137 to 143 in ead5f94

    
              bool setConfig(const fs::FSConfig &cfg) override 
        
              { 
        
                  if ((cfg._type != SDFSConfig::FSId) || _mounted) { 
        
                      DEBUGV("SDFS::setConfig: invalid config or already mounted\n"); 
        
                      return false; 
        
                  } 
        
           _cfg = *static_cast<const SDFSConfig *>(&cfg);

Unless I am missing some other issue... ~~we can't exactly store pointer to a temporary storage, and should fix the SD class~~
edit: it is a copy... still, something is definitely wrong in the config part

I don't see any more WDTs after replacing SD.begin(D0) with

SDFSConfig config;
config.setCSPin(D0);
SDFS.setConfig(config);
SDFS.begin();

mcspr · 2023-01-23T15:57:13Z

...and the only difference seems to be

SD.begin(D0) is using SPI_HALF_SPEED
default-initialized SDFSConfig is `SD_SCK_MHZ(10)

Arduino/libraries/SDFS/src/SDFS.h

Line 47 in cf24024

SDFSConfig(uint8_t csPin = 4, uint32_t spi = SD_SCK_MHZ(10)) : FSConfig(FSId, false), _csPin(csPin), _part(0), _spiSettings(spi) { }

BrokeStudio · 2023-01-23T17:04:27Z

I got the same issue with this piece of code:

  SDFSConfig config;
  config.setCSPin(D0);
  SDFS.setConfig(config);

  if (!SDFS.begin()) {
    Serial.println("Initialization failed!");
  }
  else
  {
    Serial.println("Initialization successful!");
    FSInfo64 sd_info;
    SDFS.info64(sd_info);
    Serial.print(F("File system total space: "));
    Serial.print(PriUint64<DEC>(sd_info.totalBytes));
    Serial.println();
  }

Output:

SDK:2.2.2-dev(38a443e)/Core:3.1.1=30101000/lwIP:STABLE-2_1_3_RELEASE/glue:1.2-65-g06164fb/BearSSL:b024386

Setup...

Initialization successful!

--------------- CUT HERE FOR EXCEPTION DECODER ---------------

Soft WDT reset

>>>stack>>>

ctx: cont
sp: 3ffffce0 end: 3fffffd0 offset: 01a0
3ffffe80:  00000000 00000cc1 ab439581 000ebfee
3ffffe90:  3ffef8c0 00000009 3ffefae8 3ffefae8
3ffffea0:  00000200 3ffef8c0 00000ded 40207246  
3ffffeb0:  3ffef8c0 00003794 3ffefae8 00000001  
3ffffec0:  3ffef8c0 00000000 3ffefae8 40207925
3ffffed0:  00003793 00000002 3ffefae8 00003795
3ffffee0:  00003794 00000002 3ffef8b0 40207b57
3ffffef0:  3ffef88c 00168122 000732f9 40206fe1
3fffff00:  3ffef8b0 00000100 3ffe885d 40204104
3fffff10:  3ffe8cdc fffffffc feefeffe 00000000  
3fffff20:  00000001 001db420 3fffff50 40201948
3fffff30:  3fffdad0 0000001a 3ffeea90 3ffeeafc
3fffff40:  3fffdad0 3ffeea64 3ffeea90 40201194
3fffff50:  6d080000 00000000 feefeffe feefeffe
3fffff60:  00004000 00000000 000003e7 000000ff
3fffff70:  feefeffe feefeffe feefeffe feefeffe
3fffff80:  53444653 00001000 00989680 feefeffe
3fffff90:  feefeffe feefeffe feefeffe feefeffe
3fffffa0:  feefeffe feefeffe feefeffe 3ffeeafc
3fffffb0:  3fffdad0 00000000 3ffeeae8 4020324c
3fffffc0:  feefeffe feefeffe 3fffdab0 4010110d
<<<stack<<<

0x40207246 in SharedSpiCard::readData(unsigned char*, unsigned int) at ??:?
0x40207925 in DedicatedSpiCard::readSectors(unsigned int, unsigned char*, unsigned int) at ??:?
0x40207b57 in FsCache::prepare(unsigned int, unsigned char) at ??:?
0x40206fe1 in FatPartition::freeClusterCount() at ??:?
0x40204104 in uart_write at ??:?
0x40201948 in sdfs::SDFSImpl::info64(fs::FSInfo64&) at ??:?
0x40201194 in setup at ??:?
0x4020324c in loop_wrapper() at core_esp8266_main.cpp:?
0x4010110d in cont_wrapper at ??:?

mcspr · 2023-01-25T09:31:50Z

As I understood our SPI logic, this just happens when we fail the transaction / don't receive a reply.
Meaning, SPI communication just doesn't work or was not initialized properly :/

Have you tried adjusting speeds to a different MHZ values? Only thing that comes to mind, otherwise I am not really sure of where the problem might be based on the traces.
(Unless we hit some kind of issue with SDFAT implementation after updating to 2.1.1)

BrokeStudio · 2023-01-31T13:52:49Z

I tried different settings in the platformio.ini file.
Hopefully it'll help a bit.
Changing the speed didn't change anything, 39MHz seems to be the maximum. Above this, SDFS.begin() will fail.

SDFS.info64() works fine

platform = espressif8266@3.1.0
=> SDK:2.2.2-dev(38a443e)/Core:3.0.1=30001000/lwIP:STABLE-2_1_2_RELEASE/glue:1.2-48-g7421258/BearSSL:c0b69df

platform = espressif8266@3.1.0
platform_packages = framework-arduinoespressif8266 @ 3.30002.0
=> SDK:2.2.2-dev(38a443e)/Core:3.0.2=30002000/lwIP:STABLE-2_1_2_RELEASE/glue:1.2-48-g7421258/BearSSL:6105635

platform = espressif8266@3.2.0
platform_packages = framework-arduinoespressif8266 @ 3.30002.0
=> SDK:2.2.2-dev(38a443e)/Core:3.0.2=30002000/lwIP:STABLE-2_1_2_RELEASE/glue:1.2-48-g7421258/BearSSL:6105635

platform = espressif8266@4.0.0
platform_packages = framework-arduinoespressif8266 @ 3.30002.0
=> SDK:2.2.2-dev(38a443e)/Core:3.0.2=30002000/lwIP:STABLE-2_1_2_RELEASE/glue:1.2-48-g7421258/BearSSL:6105635

platform = espressif8266@4.0.1
platform_packages = framework-arduinoespressif8266 @ 3.30002.0
=> SDK:2.2.2-dev(38a443e)/Core:3.0.2=30002000/lwIP:STABLE-2_1_2_RELEASE/glue:1.2-48-g7421258/BearSSL:6105635

platform = espressif8266@4.1.0
platform_packages = framework-arduinoespressif8266 @ 3.30002.0
=> SDK:2.2.2-dev(38a443e)/Core:3.0.2=30002000/lwIP:STABLE-2_1_2_RELEASE/glue:1.2-48-g7421258/BearSSL:6105635

SDFS.info64() crashes

platform = espressif8266@3.1.0
platform_packages = framework-arduinoespressif8266 @ 3.30101.0
=> SDK:2.2.2-dev(38a443e)/Core:3.1.1=30101000/lwIP:STABLE-2_1_3_RELEASE/glue:1.2-65-g06164fb/BearSSL:b024386

platform = espressif8266@3.2.0
#platform_packages = framework-arduinoespressif8266 @ 3.30101.0
=> SDK:2.2.2-dev(38a443e)/Core:3.1.1=30101000/lwIP:STABLE-2_1_3_RELEASE/glue:1.2-65-g06164fb/BearSSL:b024386

platform = espressif8266@4.0.0
#platform_packages = framework-arduinoespressif8266 @ 3.30101.0
=> SDK:2.2.2-dev(38a443e)/Core:3.1.1=30101000/lwIP:STABLE-2_1_3_RELEASE/glue:1.2-65-g06164fb/BearSSL:b024386

platform = espressif8266@4.0.1
platform_packages = framework-arduinoespressif8266 @ 3.30101.0
=> SDK:2.2.2-dev(38a443e)/Core:3.1.1=30101000/lwIP:STABLE-2_1_3_RELEASE/glue:1.2-65-g06164fb/BearSSL:b024386

platform = espressif8266@4.1.0
platform_packages = framework-arduinoespressif8266 @ 3.30101.0
=> SDK:2.2.2-dev(38a443e)/Core:3.1.1=30101000/lwIP:STABLE-2_1_3_RELEASE/glue:1.2-65-g06164fb/BearSSL:b024386

platform = espressif8266@4.1.0
platform_packages = platformio/framework-arduinoespressif8266 @ https://github.com/esp8266/Arduino.git
=> SDK:2.2.2-dev(38a443e)/Core:3.1.0-dev=30100000/lwIP:STABLE-2_1_2_RELEASE/glue:1.2-48-g7421258/BearSSL:6105635

mcspr · 2023-01-31T19:20:07Z

Another guess related to speed - how big is your SD card?

Since we know the trace above shows us stalling around here

0x402070f9 in FatPartition::freeClusterCount() at ??:?

https://github.com/greiman/SdFat/blob/628effa1c277e6dc1fef461b4b755ad300735248/src/FatLib/FatPartition.cpp#L354-L380
What about adding yield() right at the start of the loop?

edit: pio run with --verbose would show the library path, which is likely somewhere in ~/.platformio/packages/framework-arduinoespressif8266/libraries/Esp8266SdFat/; depends on the version used

mcspr · 2023-02-01T17:43:15Z

2.0.2 used sdfat::SdSpiCard::isTimedOut which in turn did the yield()ing through the SysCall helper class, not yet sure why the updated version removed it (or forgot to call there) :/
https://github.com/greiman/SdFat/blob/1535ac2b0332c22da26ca876fd2a04641dffadb4/src/SdCard/SdSpiCard.cpp#L450

Comparing with some random 4GB HC SD, at least some explanation for my results

10MHZ takes around 1.8s to perform that specific loop, that happens to stay within WDT limit (including initial setup)
Default value takes around 3s, which tips over

SdFat 2.0.2+ doesn't call yield(), make sure we don't trigger SWDT fix esp8266#8822

SdFat 2.0.2+ doesn't call yield(), make sure we don't trigger SWDT fix #8822

mcspr mentioned this issue Feb 2, 2023

Workaround for timeout yield() removal earlephilhower/ESP8266SdFat#11

Merged

mcspr added a commit to mcspr/esp8266-Arduino that referenced this issue Feb 2, 2023

Update SdFat fork

f291f6d

SdFat 2.0.2+ doesn't call yield(), make sure we don't trigger SWDT fix esp8266#8822

mcspr mentioned this issue Feb 2, 2023

Fix SdFat long operations triggering watchdog #8844

Merged

earlephilhower closed this as completed in #8844 Feb 3, 2023

earlephilhower pushed a commit that referenced this issue Feb 3, 2023

Update SdFat fork (#8844)

badb407

SdFat 2.0.2+ doesn't call yield(), make sure we don't trigger SWDT fix #8822

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Trying to get SD card information using `SDFS.info64` function leads to a crash. #8822

Trying to get SD card information using `SDFS.info64` function leads to a crash. #8822

BrokeStudio commented Jan 23, 2023 •

edited

mcspr commented Jan 23, 2023 •

edited

mcspr commented Jan 23, 2023

BrokeStudio commented Jan 23, 2023

mcspr commented Jan 25, 2023

BrokeStudio commented Jan 31, 2023

mcspr commented Jan 31, 2023 •

edited

mcspr commented Feb 1, 2023 •

edited

Trying to get SD card information using SDFS.info64 function leads to a crash. #8822

Trying to get SD card information using SDFS.info64 function leads to a crash. #8822

Comments

BrokeStudio commented Jan 23, 2023 • edited

Basic Infos

Platform

Settings in IDE

Problem Description

MCVE Sketch

Debug Messages

mcspr commented Jan 23, 2023 • edited

mcspr commented Jan 23, 2023

BrokeStudio commented Jan 23, 2023

mcspr commented Jan 25, 2023

BrokeStudio commented Jan 31, 2023

mcspr commented Jan 31, 2023 • edited

mcspr commented Feb 1, 2023 • edited

Trying to get SD card information using `SDFS.info64` function leads to a crash. #8822

Trying to get SD card information using `SDFS.info64` function leads to a crash. #8822

BrokeStudio commented Jan 23, 2023 •

edited

mcspr commented Jan 23, 2023 •

edited

mcspr commented Jan 31, 2023 •

edited

mcspr commented Feb 1, 2023 •

edited