New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ardunio SD.h lib faster? #2

Closed
mgk opened this Issue Jan 15, 2016 · 8 comments

Comments

Projects
None yet
3 participants
@mgk

mgk commented Jan 15, 2016

More of a mystery than a bug, reporting on the off chance it is of interet.

I'm able to run the OpenNext.ino example at SPI_HALF_SPEED but at SPI_FULL_SPEED I get

SD errorCode: 0X4,0X33

I understand there are a lot of possible causes for this, in particular long paths in the SPI circuit. The reason I'm reporting this is that I can connect the same circuit at full speed using the CardInfo.ino example and the Arduino SD.h lib.

I found my way to the beta from greiman/SdFat#16

(Looks like among other things, the beta configures the SPI layer to use automatically, which is nice. 👍)

Using https://github.com/greiman/SdFat with #define SD_SPI_CONFIGURATION 1 I'm able to run on the M0 Pro, but as with SdFat-beta, only at half speed.

I've run with each s/w config a dozen times and the results are consistent.

The mystery is how the standard Arduino library could be faster than SdFat.

Environment

  • Board: Arduino M0 Pro
  • SD card reader connected by hw SPI pins, no other peripherals
  • no breadboard, jumper wires about 15cm long
  • Arduino.org IDE 1.7.8
@greiman

This comment has been minimized.

Show comment
Hide comment
@greiman

greiman Jan 15, 2016

Owner

Arduino SD.h runs at half speed. Also SD.h is just a wrapper for a 2009 version of SdFat.

boolean SDClass::begin(uint8_t csPin) {
  /*

    Performs the initialisation required by the sdfatlib library.

    Return true if initialization succeeds, false otherwise.

   */
  return card.init(SPI_HALF_SPEED, csPin) &&
         volume.init(card) &&
         root.openRoot(volume);
}
Owner

greiman commented Jan 15, 2016

Arduino SD.h runs at half speed. Also SD.h is just a wrapper for a 2009 version of SdFat.

boolean SDClass::begin(uint8_t csPin) {
  /*

    Performs the initialisation required by the sdfatlib library.

    Return true if initialization succeeds, false otherwise.

   */
  return card.init(SPI_HALF_SPEED, csPin) &&
         volume.init(card) &&
         root.openRoot(volume);
}

@greiman greiman closed this Jan 15, 2016

@mgk

This comment has been minimized.

Show comment
Hide comment
@mgk

mgk Jan 15, 2016

Yes, but the CardInfo example uses the wrapped SdFat lib directly, yes?

When I change:

https://github.com/arduino-org/Arduino/blob/ide-org-1.6.1.x/libraries/SD/examples/CardInfo/CardInfo.ino#L55

to

if (!card.init(SPI_FULL_SPEED, chipSelect)) {

the CardInfo example sketch works (presumably at full speed).

So I guess the mystery is why the wrapped 2009 SdFat lib is able to run at full speed when the modern ones are not.

mgk commented Jan 15, 2016

Yes, but the CardInfo example uses the wrapped SdFat lib directly, yes?

When I change:

https://github.com/arduino-org/Arduino/blob/ide-org-1.6.1.x/libraries/SD/examples/CardInfo/CardInfo.ino#L55

to

if (!card.init(SPI_FULL_SPEED, chipSelect)) {

the CardInfo example sketch works (presumably at full speed).

So I guess the mystery is why the wrapped 2009 SdFat lib is able to run at full speed when the modern ones are not.

@greiman

This comment has been minimized.

Show comment
Hide comment
@greiman

greiman Jan 15, 2016

Owner

The new SdFat is able to run much faster than the 2009 version.

The problem is an SPI error. That suggests a hardware problem with the new SdFat-beta having a different access pattern to the SPI bus.

Strange since the M0 Pro uses slow programmed I/O. While the SPI clock may be fast there will be a lot of dead time between bytes.

I have been optimizing SdFat-beta for new faster systems with DMA like the Particle Photon.

On the Due at full speed I get over 4 MB/sec read/write. What do you get with the bench example?

File size 20 MB
Buffer size 32768 bytes
Starting write test, please wait.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
4150.43,20699,7318,7887

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
4326.51,9505,7035,7571

I don't have a M0 Pro so I can't connect my scope and see why SD status is getting garbled over the SPI bus.

You might try enabling CRC to check for transmission errors. Edit SdFatConfig.h and set this

#define USE_SD_CRC 2

See if this catches more errors.

Owner

greiman commented Jan 15, 2016

The new SdFat is able to run much faster than the 2009 version.

The problem is an SPI error. That suggests a hardware problem with the new SdFat-beta having a different access pattern to the SPI bus.

Strange since the M0 Pro uses slow programmed I/O. While the SPI clock may be fast there will be a lot of dead time between bytes.

I have been optimizing SdFat-beta for new faster systems with DMA like the Particle Photon.

On the Due at full speed I get over 4 MB/sec read/write. What do you get with the bench example?

File size 20 MB
Buffer size 32768 bytes
Starting write test, please wait.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
4150.43,20699,7318,7887

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
4326.51,9505,7035,7571

I don't have a M0 Pro so I can't connect my scope and see why SD status is getting garbled over the SPI bus.

You might try enabling CRC to check for transmission errors. Edit SdFatConfig.h and set this

#define USE_SD_CRC 2

See if this catches more errors.

@mgk

This comment has been minimized.

Show comment
Hide comment
@mgk

mgk Jan 16, 2016

Ouch, yeah I'm only getting 140 KB/sec.

#define USE_SD_CRC 2 did not print additional errors.

Card Info

Card is class 10 SanDisk Extreme.

Type is FAT32
Card size: 15.93 GB (GB = 1E9 bytes)

Manufacturer ID: 0X3
OEM ID: SD
Product: SU16G
Version: 8.0
Serial number: ...
Manufacturing date: 4/2013

5 MB file / 512 B buffer tests
File size 5 MB
Buffer size 512 bytes
Starting write test, please wait.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
140.61,30692,3182,3637
142.36,37502,3131,3592
141.97,55195,3141,3602
142.41,43009,3134,3591
142.15,60967,3139,3598
142.54,30061,3138,3588
139.97,85977,3103,3654
139.80,85114,3176,3658
140.14,83855,3179,3649
139.78,142024,3138,3659

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
237.66,4292,2052,2151
237.68,4289,2055,2151
237.70,4291,2055,2151
237.70,4289,2051,2151
237.68,4288,2052,2151

20 MB file / 16KB buffer tests

File size 20 MB
Buffer size 16384 bytes

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
285.16,77936,56682,57445
284.94,131935,56676,57491
284.81,129107,56679,57515

speed,max,min,avg
KB/Sec,usec,usec,usec
297.05,57289,55043,55153
297.05,57290,55043,55153
297.05,57287,55042,55153

mgk commented Jan 16, 2016

Ouch, yeah I'm only getting 140 KB/sec.

#define USE_SD_CRC 2 did not print additional errors.

Card Info

Card is class 10 SanDisk Extreme.

Type is FAT32
Card size: 15.93 GB (GB = 1E9 bytes)

Manufacturer ID: 0X3
OEM ID: SD
Product: SU16G
Version: 8.0
Serial number: ...
Manufacturing date: 4/2013

5 MB file / 512 B buffer tests
File size 5 MB
Buffer size 512 bytes
Starting write test, please wait.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
140.61,30692,3182,3637
142.36,37502,3131,3592
141.97,55195,3141,3602
142.41,43009,3134,3591
142.15,60967,3139,3598
142.54,30061,3138,3588
139.97,85977,3103,3654
139.80,85114,3176,3658
140.14,83855,3179,3649
139.78,142024,3138,3659

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
237.66,4292,2052,2151
237.68,4289,2055,2151
237.70,4291,2055,2151
237.70,4289,2051,2151
237.68,4288,2052,2151

20 MB file / 16KB buffer tests

File size 20 MB
Buffer size 16384 bytes

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
285.16,77936,56682,57445
284.94,131935,56676,57491
284.81,129107,56679,57515

speed,max,min,avg
KB/Sec,usec,usec,usec
297.05,57289,55043,55153
297.05,57290,55043,55153
297.05,57287,55042,55153

@greiman

This comment has been minimized.

Show comment
Hide comment
@greiman

greiman Jan 16, 2016

Owner

Wow that's less than a Uno.

Here is an Uno with a 32GB SanDisk Extreme.

Type is FAT32
Card size: 31.91 GB (GB = 1E9 bytes)

Manufacturer ID: 0X3
OEM ID: SD
Product: SE32G
Version: 8.0
Serial number: 0X838DE929
Manufacturing date: 9/2015

File size 5 MB
Buffer size 512 bytes
Starting write test, please wait.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
319.06,75704,1368,1596

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
501.72,1996,988,1014

Too bad the M0 Pro has a such a slow SPI driver

I am using SdFat on many other systems.

I bought a $19 Particle Photon:

https://store.particle.io/

I has 30 MHz SPI and I get over 3 MB/sec read/write with the stock SPI driver. It is a sweet WiFi development board.

I get over 7 MB/sec with this STM32 board using SDIO instead of SPI.

https://developer.mbed.org/platforms/Seeed-Arch-Max/

Owner

greiman commented Jan 16, 2016

Wow that's less than a Uno.

Here is an Uno with a 32GB SanDisk Extreme.

Type is FAT32
Card size: 31.91 GB (GB = 1E9 bytes)

Manufacturer ID: 0X3
OEM ID: SD
Product: SE32G
Version: 8.0
Serial number: 0X838DE929
Manufacturing date: 9/2015

File size 5 MB
Buffer size 512 bytes
Starting write test, please wait.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
319.06,75704,1368,1596

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
501.72,1996,988,1014

Too bad the M0 Pro has a such a slow SPI driver

I am using SdFat on many other systems.

I bought a $19 Particle Photon:

https://store.particle.io/

I has 30 MHz SPI and I get over 3 MB/sec read/write with the stock SPI driver. It is a sweet WiFi development board.

I get over 7 MB/sec with this STM32 board using SDIO instead of SPI.

https://developer.mbed.org/platforms/Seeed-Arch-Max/

@mgk

This comment has been minimized.

Show comment
Hide comment
@mgk

mgk Jan 30, 2016

Thanks for all the background (and for SdFat itself!). Yeah the photon looks great. I'm using the M0 Pro as a dev board for a project that will embed a SAMD21.

Just fyi when I load the Arduino Zero bootloader and run with the arduino.cc libs it's even slower (100KB/s vs 140KB/s). Others have run into this too: https://forum.arduino.cc/index.php?topic=358160.30
using the patch from that thread and SdFat-beta I get 130KB/s with the arduino.cc libs.

For our project it will have to do, until a DMA SPI driver is available for the SAMD duino's.

mgk commented Jan 30, 2016

Thanks for all the background (and for SdFat itself!). Yeah the photon looks great. I'm using the M0 Pro as a dev board for a project that will embed a SAMD21.

Just fyi when I load the Arduino Zero bootloader and run with the arduino.cc libs it's even slower (100KB/s vs 140KB/s). Others have run into this too: https://forum.arduino.cc/index.php?topic=358160.30
using the patch from that thread and SdFat-beta I get 130KB/s with the arduino.cc libs.

For our project it will have to do, until a DMA SPI driver is available for the SAMD duino's.

@comiati

This comment has been minimized.

Show comment
Hide comment
@comiati

comiati Aug 11, 2017

Hi,
Can you share the library for SD with bugs fixed?
I use Arduino M0 with SD.h (standard library) and I have many problems, working just with some type of SD cards......and sometimes even the righit SD don't working (same SD sometime working perfect)
Best regards

comiati commented Aug 11, 2017

Hi,
Can you share the library for SD with bugs fixed?
I use Arduino M0 with SD.h (standard library) and I have many problems, working just with some type of SD cards......and sometimes even the righit SD don't working (same SD sometime working perfect)
Best regards

@greiman

This comment has been minimized.

Show comment
Hide comment
@greiman

greiman Aug 11, 2017

Owner

I am not planning on doing a custom driver for M0.

Also I don't support the old SD.h library that I wrote 2009. SD.h is maintained by the Arduino company.

Owner

greiman commented Aug 11, 2017

I am not planning on doing a custom driver for M0.

Also I don't support the old SD.h library that I wrote 2009. SD.h is maintained by the Arduino company.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment