SD_MMC not using daylight saving time on file times ??? #6786

jameszah · 2022-05-22T06:58:55Z

Board

ESP32-S on AI-Thinker ESP32-CAM

Device Description

normal esp32-cam with camera and wifi turned on

Hardware Configuration

no

Version

v2.0.3

IDE Name

Arduino ide 1.8.19

Operating System

Windows 10

Flash frequency

80MHz

PSRAM enabled

yes

Upload speed

115200

Description

After correctly fetching the time over wifi, and creating a file on the sd card, the file date from file.getLastWrite() shows 1 hour ahead of time. When the sd is removed and put in a computer the date shows correct - 1 hour behind the date that getLastWrite shows.

This worked before the time-change on 2.02. Since the time changed in March, and the 2.03 just arrived last week or so, ... I don't know if both factors are relevant.

Would it be in the FS library?

pictures show a esp32 hosted file directory, and the same files on windows 10 "date modified".

Sketch

here is the code that just gets the date on file and prints it out.  There is my 3600 second work around.

 // sd_mmc not recognizing daylight savings time ???? jz may 21 2022 - seemed to work before dst

      //time_t t= file.getLastWrite();
      time_t t= file.getLastWrite() - 3600;
      struct tm * tmstruct = localtime(&t);
      
      char ccz[100];
      sprintf(ccz, " %d-%02d-%02d %02d:%02d:%02d\n",(tmstruct->tm_year)+1900,( tmstruct->tm_mon)+1, tmstruct->tm_mday,tmstruct->tm_hour , tmstruct->tm_min, tmstruct->tm_sec);

Debug Message

none

Other Steps to Reproduce

none

I have checked existing issues, online documentation and the Troubleshooting Guide

I confirm I have checked existing issues, online documentation and Troubleshooting guide.

The text was updated successfully, but these errors were encountered:

SuGlider · 2022-05-22T13:00:02Z

@P-R-O-C-H-Y, PTAL. Thanks.

lbernstone · 2022-05-22T15:45:57Z

This sounds like a localization issue. How are you setting your time? Why do you subtract 3600 from the time?
SimpleTime example
A list of the timezones

jameszah · 2022-05-22T21:36:17Z

I have fetched the correct internet time, and printed it -- here is the code and the serial monitor.
But then I create a file, and it would normally have a time 1 hour forward, but for that 3600 second kludge.

So the ctime and localtime give me correct time, but when I create the file it shows correct time after I move the sd card to windows computer, but when I access it from getLastWrite(), then it it 1 hour too high.

I am MST7MDT,M3.2.0,M11.1.0, so utc-7 in winter, and utc-6 current, and utc is about 21:09 and I get 15:09 about 3:09 pm which is correct, but the file desklens2.999.txt and desklens2.001.avi created just after the reboot (3:10 by then), need that 3600 second adjustment to show the correct time, so getLastWrite() is giving me time of utc-5 or 4:09 if I hadn't subtracted 3600 seconds.

    configTime(0, 0, "pool.ntp.org");
    char tzchar[60];
    
    TIMEZONE.toCharArray(tzchar, TIMEZONE.length()+1);        
    Serial.println(TIMEZONE);
    Serial.printf("Char >%s<\n",tzchar);
    setenv("TZ", tzchar, 1);  
    tzset();

    time(&now);

    while (now < 15) {        // try for 15 seconds to get the time, then give up 
      delay(1000);
      Serial.print("o");
      time(&now);
    }

    Serial.print("Local time: "); Serial.print(ctime(&now));

..MST7MDT,M3.2.0,M11.1.0
Char >MST7MDT,M3.2.0,M11.1.0<
Local time: Sun May 22 15:09:54 2022

      time_t t= file.getLastWrite() - 3600;
      struct tm * tmstruct = localtime(&t);

jameszah · 2022-05-26T07:16:28Z

Here is a bit of code to demonstrate the problem.

It sets the time to MST7MDT,M3.2.0,M11.1.0, in

zero-time of Jan 1 1970
10:10 oct 10, 2010
10:10 oct 10, 2021

... and the creates a file, closes it, opens it and checks the last write.

getlastwrite says dec 31, 1980 5 PM ???
getlastwrite says 11:10 oct 10, 2010
getlastwrite says 11:10 oct 10, 2021

Over on Windows, the sd card shows the correct time for all the files.
The 2010 and 2021 are 1 hour ahead on getlastwrite, and the 1970 file wrong - out of bounds maybe.

If you switch timezone to GMT -- no daylight savings -- then it works correctly (other than the 1970 case which shows 1980). (screeshots at the bottom)

#include <SD_MMC.h>
File logfile;

static esp_err_t init_sdcard()
{

  int succ = SD_MMC.begin("/sdcard", true);
  if (succ) {
    Serial.printf("SD_MMC Begin: %d\n", succ);
    uint8_t cardType = SD_MMC.cardType();
    Serial.print("SD_MMC Card Type: ");
    if (cardType == CARD_MMC) {
      Serial.println("MMC");
    } else if (cardType == CARD_SD) {
      Serial.println("SDSC");
    } else if (cardType == CARD_SDHC) {
      Serial.println("SDHC");
    } else {
      Serial.println("UNKNOWN");
    }

    uint64_t cardSize = SD_MMC.cardSize() / (1024 * 1024);
    Serial.printf("SD_MMC Card Size: %lluMB\n", cardSize);

  } else {
    Serial.printf("Failed to mount SD card VFAT filesystem. \n");
  }

  return ESP_OK;
}


void setup() {


  Serial.begin(115200);
  Serial.println("\n\n---");

  //configTime(0, 0, "pool.ntp.org");

  Serial.println("Mount the sd card");
  init_sdcard();

  time_t now = time(&now);
  Serial.print("Local time: "); Serial.print(ctime(&now));

//////////////////////////////////////////////

  Serial.println("set time to mountain");
  char tzchar[60];
  String TIMEZONE = "MST7MDT,M3.2.0,M11.1.0";
  TIMEZONE.toCharArray(tzchar, TIMEZONE.length() + 1);
  Serial.println(TIMEZONE);
  Serial.printf("Char >%s<\n", tzchar);
  setenv("TZ", tzchar, 1);  // mountain time zone from #define at top
  tzset();

  now = time(&now);
  Serial.print("Local time: "); Serial.print(ctime(&now));

  time_t tnow = time(&tnow);
  struct tm * tmstruct = localtime(&tnow);
  Serial.printf("  Local: %d-%02d-%02d %02d:%02d:%02d\n", (tmstruct->tm_year) + 1900, ( tmstruct->tm_mon) + 1, tmstruct->tm_mday, tmstruct->tm_hour , tmstruct->tm_min, tmstruct->tm_sec);

  char logname[50];
  sprintf(logname, "/test1970.txt");

  Serial.println("Open file test1970.txt");
  logfile = SD_MMC.open(logname, FILE_WRITE);

  logfile.printf("Hello\n");

  Serial.println("Get time of logfile");
  time_t t = logfile.getLastWrite();  
  tmstruct = localtime(&t);
  Serial.printf("  LAST WRITE: %d-%02d-%02d %02d:%02d:%02d\n", (tmstruct->tm_year) + 1900, ( tmstruct->tm_mon) + 1, tmstruct->tm_mday, tmstruct->tm_hour , tmstruct->tm_min, tmstruct->tm_sec);

  Serial.println("Close file");
  logfile.close();

  Serial.println("Open file test1970.txt");
  logfile = SD_MMC.open(logname, FILE_WRITE);

  Serial.println("Get time of logfile");
  t = logfile.getLastWrite();  
  tmstruct = localtime(&t);
  Serial.printf("  LAST WRITE: %d-%02d-%02d %02d:%02d:%02d\n", (tmstruct->tm_year) + 1900, ( tmstruct->tm_mon) + 1, tmstruct->tm_mday, tmstruct->tm_hour , tmstruct->tm_min, tmstruct->tm_sec);

  Serial.println("Close file");
  logfile.close();

  //////////////////////////////////////////////////

  Serial.println("\nSet time to 10:10 oct 10, 2010");

  struct timeval tv;
  tv.tv_sec = 1286727010;
  tv.tv_usec = 0;
  settimeofday(&tv, NULL);

  now = time(&now);
  Serial.print("Local time: "); Serial.print(ctime(&now));

  Serial.println("reset the time zone");

  setenv("TZ", tzchar, 1);  // mountain time zone from #define at top
  tzset();


  now = time(&now);
  Serial.print("Local time: "); Serial.print(ctime(&now));

  sprintf(logname, "/test2010.txt");
  Serial.println("Open file test2010.txt");
  logfile = SD_MMC.open(logname, FILE_WRITE);

  logfile.printf("Hello\n");

  Serial.println("Get time of logfile");
  t = logfile.getLastWrite();  
  tmstruct = localtime(&t);
  Serial.printf("  LAST WRITE: %d-%02d-%02d %02d:%02d:%02d\n", (tmstruct->tm_year) + 1900, ( tmstruct->tm_mon) + 1, tmstruct->tm_mday, tmstruct->tm_hour , tmstruct->tm_min, tmstruct->tm_sec);

  Serial.println("Close file");
  logfile.close();

  Serial.println("Open file test2010.txt");
  logfile = SD_MMC.open(logname, FILE_WRITE);

  t = logfile.getLastWrite();  
  tmstruct = localtime(&t);
  Serial.printf("  LAST WRITE: %d-%02d-%02d %02d:%02d:%02d\n", (tmstruct->tm_year) + 1900, ( tmstruct->tm_mon) + 1, tmstruct->tm_mday, tmstruct->tm_hour , tmstruct->tm_min, tmstruct->tm_sec);

  Serial.println("Close file");
  logfile.close();

  //////////////////////////////////////////////////////////////////////

  Serial.println("\nSet time to 10:10 oct 10, 2021");

  //struct timeval tv;
  tv.tv_sec = 1633882210;
  tv.tv_usec = 0;
  settimeofday(&tv, NULL);

  now = time(&now);
  Serial.print("Local time: "); Serial.print(ctime(&now));

  Serial.println("reset the time zone");

  setenv("TZ", tzchar, 1);  // mountain time zone from #define at top
  tzset();


  now = time(&now);
  Serial.print("Local time: "); Serial.print(ctime(&now));

  sprintf(logname, "/test2021.txt");
  Serial.println("Open file test2021.txt");
  logfile = SD_MMC.open(logname, FILE_WRITE);

  logfile.printf("Hello\n");

  Serial.println("Get time of logfile");
  t = logfile.getLastWrite();  
  tmstruct = localtime(&t);
  Serial.printf("  LAST WRITE: %d-%02d-%02d %02d:%02d:%02d\n", (tmstruct->tm_year) + 1900, ( tmstruct->tm_mon) + 1, tmstruct->tm_mday, tmstruct->tm_hour , tmstruct->tm_min, tmstruct->tm_sec);

  Serial.println("Close file");
  logfile.close();

  Serial.println("Open file test2021.txt");
  logfile = SD_MMC.open(logname, FILE_WRITE);

  t = logfile.getLastWrite();  
  tmstruct = localtime(&t);
  Serial.printf("  LAST WRITE: %d-%02d-%02d %02d:%02d:%02d\n", (tmstruct->tm_year) + 1900, ( tmstruct->tm_mon) + 1, tmstruct->tm_mday, tmstruct->tm_hour , tmstruct->tm_min, tmstruct->tm_sec);

  Serial.println("Close file");
  logfile.close();


}

void loop() {

}

jameszah · 2022-05-26T15:26:40Z

I forget to say that it does work with standard time (not daylight savings) - this is jan 10, 2021 10:10 (unixtime 1610298610), and the system time on the esp32 is the same as the time written on the file.

lbernstone · 2022-05-26T19:49:31Z

Can you please test to isolate whether this is a file issue or locale issue. The following code works correctly for me. Post a variation of it that compiles and shows your issue.

#include <WiFi.h>
#include <LittleFS.h>
void setup() {
  Serial.begin(115200);
  WiFi.begin("myssid", "mypasswd");
  WiFi.waitForConnectResult();
  configTime(0, 0, "pool.ntp.org");
  setenv("TZ", "MST7MDT,M3.2.0,M11.1.0", 1);
  tzset();
  while (time(NULL) < 1E6) delay(1000);
  Serial.printf("systime: %d\n", time(NULL));
  LittleFS.begin(true);
  File tfile = LittleFS.open("/test", FILE_WRITE);
  tfile.print("hello");
  tfile.flush();
  Serial.printf("filetime: %d\n", tfile.getLastWrite());
}
void loop() {delay(-1);}

jameszah · 2022-05-26T19:56:22Z

I'll try that this evening, but I am using sd_mmc rather than Littlefs - might be an issue???

lbernstone · 2022-05-26T20:22:45Z

Please post a variation of it that compiles and shows your issue.

jameszah · 2022-05-27T06:43:33Z

I think it must be the code for getLastWrite in sd_mmc. sd_mmc seems to get the correct system time when the file is created, but it doesn't handle the daylight savings time correctly when reading the file time.

Thanks for your interest in the topic.

A little interesting that the time is not 3600 seconds wrong, but 3599 seconds wrong.

#include <WiFi.h>
#include <LittleFS.h>
#include <SD_MMC.h>
void setup() {

  Serial.begin(115200);
  WiFi.begin("myssid", "mypasswd");
  
  WiFi.waitForConnectResult();
  configTime(0, 0, "pool.ntp.org");
  setenv("TZ", "MST7MDT,M3.2.0,M11.1.0", 1);
  tzset();
  while (time(NULL) < 1E6) delay(1000);

  Serial.printf("\n\nsystime: %d\n", time(NULL));
  time_t now = time(&now);  Serial.println(ctime(&now));

  if (!LittleFS.begin(true)){
    Serial.println("LITTLEFS Mount Failed");
  }
  File tfile = LittleFS.open("/test", FILE_WRITE);
  tfile.print("hello");
  tfile.flush();
  Serial.printf("littlefs filetime: %d\n", tfile.getLastWrite());
  now = tfile.getLastWrite();
  Serial.println(ctime(&now));
  

  int succ = SD_MMC.begin("/sdcard", true);
  if (succ) {
    Serial.printf("SD_MMC Begin: %d\n", succ);
  } else {
    Serial.printf("Failed to mount SD card VFAT filesystem. \n");
  }
  File sd1 = SD_MMC.open("/sd1.txt", FILE_WRITE);
  sd1.print("hello");
  sd1.flush();
  Serial.printf("sd1 filetime: %d\n", sd1.getLastWrite());
  now = sd1.getLastWrite();
  Serial.println(ctime(&now));
  sd1.close();
  
  ////////////////////////////////////////////

  Serial.println("\nSet time to 10:10 jan 10, 2021");

  struct timeval tv;
  tv.tv_sec = 1610298610; //1633882210; this is oct 2021
  tv.tv_usec = 0;
  settimeofday(&tv, NULL);
  Serial.printf("\n\nsystime: %d\n", time(NULL));

  now = time(&now);  Serial.print(ctime(&now));

  File tfile2 = LittleFS.open("/test2", FILE_WRITE);
  tfile2.print("hello");
  tfile2.flush();
  Serial.printf("littlefs filetime: %d\n", tfile2.getLastWrite());
  now = tfile2.getLastWrite();
  Serial.println(ctime(&now));
  
  int succ2 = SD_MMC.begin("/sdcard", true);
  if (succ2) {
    Serial.printf("SD_MMC Begin: %d\n", succ);
  } else {
    Serial.printf("Failed to mount SD card VFAT filesystem. \n");
  }
  File sd2 = SD_MMC.open("/sd2.txt", FILE_WRITE);
  sd2.print("hello");
  sd2.flush();
  Serial.printf("sd2 filetime: %d\n", sd2.getLastWrite());
  now = sd2.getLastWrite();
  Serial.println(ctime(&now));
  sd2.close();

}
void loop() {
  delay(-1);
}

lbernstone · 2022-05-27T08:51:43Z

Funky. That's upstream in fatfs. Reproducible with FFat as well. I've opened an issue in ESP-IDF.

espressif/esp-idf#9039

mktime function uses tm_isdst member as an indicator whether the time stamp is expected to be in daylight saving time (1) or not (0). FAT filesystem uses local time as mtime, so no information about DST is available from the filesystem. According to mktime documentation, tm_isdst can be set to -1, in which case the C library will try to determine if DST was or wasn't in effect at that time, and will set UTC time accordingly. Note that the conversion from UTC to local time and then back to UTC (time_t -> localtime_r -> FAT timestamp -> mktime -> time_t) does not always recover the same UTC time. In particular, the local time in the hour before DST comes into effect can be interpreted as "before DST" or "after DST", which would correspond to different UTC values. In this case which option the C library chooses is undefined. Closes #9039 Originally reported in espressif/arduino-esp32#6786

P-R-O-C-H-Y · 2022-08-03T12:33:51Z

Hi @jameszah, can you retest it with Arduino-esp32 2.0.4 version? Thanks

I got this output, which looks correct. Tested with last sketch you posted.

systime: 1659529699
Wed Aug  3 06:28:19 2022

littlefs filetime: 1659529699
Wed Aug  3 06:28:19 2022

SD_MMC Begin: 1
sd1 filetime: 1659529698
Wed Aug  3 06:28:18 2022


Set time to 10:10 jan 10, 2021

systime: 1610298610
Sun Jan 10 10:10:10 2021

littlefs filetime: 1610298610
Sun Jan 10 10:10:10 2021

SD_MMC Begin: 1
sd2 filetime: 1610298610
Sun Jan 10 10:10:10 2021

jameszah · 2022-08-04T06:24:49Z

Yep, its correct here too.
There is the 1 second issue that sd is behind littlefs and system time -- but that's okay with me! 😄

Thanks

P-R-O-C-H-Y · 2022-08-04T10:06:23Z

As it was mentioned in esp-idf issue, FATFS keeps time at 2-second precision. So there can be this 0-2 seconds difference 😄.
Closing this one as done :) Thanks for testing 👍

mktime function uses tm_isdst member as an indicator whether the time stamp is expected to be in daylight saving time (1) or not (0). FAT filesystem uses local time as mtime, so no information about DST is available from the filesystem. According to mktime documentation, tm_isdst can be set to -1, in which case the C library will try to determine if DST was or wasn't in effect at that time, and will set UTC time accordingly. Note that the conversion from UTC to local time and then back to UTC (time_t -> localtime_r -> FAT timestamp -> mktime -> time_t) does not always recover the same UTC time. In particular, the local time in the hour before DST comes into effect can be interpreted as "before DST" or "after DST", which would correspond to different UTC values. In this case which option the C library chooses is undefined. Closes #9039 Originally reported in espressif/arduino-esp32#6786

jameszah added the Status: Awaiting triage Issue is waiting for triage label May 22, 2022

SuGlider assigned P-R-O-C-H-Y May 22, 2022

VojtechBartoska added Area: Libraries Issue is related to Library support. Area: ESP-IDF related ESP-IDF related issues Status: Pending and removed Status: Awaiting triage Issue is waiting for triage labels May 30, 2022

P-R-O-C-H-Y added Resolution: Awaiting response Waiting for response of author and removed Status: Pending labels Aug 3, 2022

P-R-O-C-H-Y closed this as completed Aug 4, 2022

P-R-O-C-H-Y added Status: Solved and removed Resolution: Awaiting response Waiting for response of author labels Aug 4, 2022

jameszah mentioned this issue Nov 29, 2023

Timezone jameszah/ESP32-CAM-Video-Recorder-junior#33

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

SD_MMC not using daylight saving time on file times ??? #6786

SD_MMC not using daylight saving time on file times ??? #6786

jameszah commented May 22, 2022

SuGlider commented May 22, 2022

lbernstone commented May 22, 2022

jameszah commented May 22, 2022

jameszah commented May 26, 2022

jameszah commented May 26, 2022

lbernstone commented May 26, 2022 •

edited

Loading

jameszah commented May 26, 2022

lbernstone commented May 26, 2022

jameszah commented May 27, 2022

lbernstone commented May 27, 2022

P-R-O-C-H-Y commented Aug 3, 2022 •

edited

Loading

jameszah commented Aug 4, 2022

P-R-O-C-H-Y commented Aug 4, 2022

SD_MMC not using daylight saving time on file times ??? #6786

SD_MMC not using daylight saving time on file times ??? #6786

Comments

jameszah commented May 22, 2022

Board

Device Description

Hardware Configuration

Version

IDE Name

Operating System

Flash frequency

PSRAM enabled

Upload speed

Description

Sketch

Debug Message

Other Steps to Reproduce

I have checked existing issues, online documentation and the Troubleshooting Guide

SuGlider commented May 22, 2022

lbernstone commented May 22, 2022

jameszah commented May 22, 2022

jameszah commented May 26, 2022

jameszah commented May 26, 2022

lbernstone commented May 26, 2022 • edited Loading

jameszah commented May 26, 2022

lbernstone commented May 26, 2022

jameszah commented May 27, 2022

lbernstone commented May 27, 2022

P-R-O-C-H-Y commented Aug 3, 2022 • edited Loading

jameszah commented Aug 4, 2022

P-R-O-C-H-Y commented Aug 4, 2022

lbernstone commented May 26, 2022 •

edited

Loading

P-R-O-C-H-Y commented Aug 3, 2022 •

edited

Loading