New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mkspiffs #11

Open
hallard opened this Issue Dec 7, 2015 · 2 comments

Comments

Projects
None yet
2 participants
@hallard

hallard commented Dec 7, 2015

@igrr
I asked on arduino-esp8266 how to make custom SPIFFS Size (see).
I followed Markus kind reply and all seems to be fine for me (new linker script/boards.txt entries) until I'm really trying to generate my new spiffs image for 1M/160KB SPIFFS target

1st point when creating SPIFFS from Arduino IDE if there is a problem (full FS) we do not see error on IDE, looks like IDE says all is fine, resulting truncated files on SPIFFS (makes sense since we're full) but took me some time to understand why my web app wasn't working anymore because I did not had any error message from IDE;about SPIFFS error -)

So I tried to generate SPIFFS from command line and here something I don't understand, it says :
SPIFFS_write error(-10001): File system is full. but calculation looks like there is enough place, here the screenshot

image

Total size is 124331 bytes, in worst case with 4096 block has we've got 6 files it could add 6*4096 = 24576 bytes so 148907 bytes which is less than 163840 defined on FS size ?

Do I'm doing something wrong ?

I've tested with 256K SPIFFS and here sketch result from HTTP query to my web app, as you can see total used returned by SPIFFS is 127006 so far below my 160K (163840) FS passed to mkspiffs

image

@igrr

This comment has been minimized.

Show comment
Hide comment
@igrr

igrr Dec 7, 2015

Owner

Doesn't look like you are doing anything wrong. But since SPIFFS_write tells that FS is full, it probably really is. Perhaps you can try setting block size to 8192? This way you will have less space wasted. I think we are using 8k blocks for the 256k FS option.
As for the actual calculations (i.e. what do 'used' and 'free' mean), I'm not sure I can help you.
If you can build mkspiffs from source, you may try adding some SPIFFS_vis calls to see how the FS gets filled as you are adding files to it.

Owner

igrr commented Dec 7, 2015

Doesn't look like you are doing anything wrong. But since SPIFFS_write tells that FS is full, it probably really is. Perhaps you can try setting block size to 8192? This way you will have less space wasted. I think we are using 8k blocks for the 256k FS option.
As for the actual calculations (i.e. what do 'used' and 'free' mean), I'm not sure I can help you.
If you can build mkspiffs from source, you may try adding some SPIFFS_vis calls to see how the FS gets filled as you are adding files to it.

@hallard

This comment has been minimized.

Show comment
Hide comment
@hallard

hallard Dec 7, 2015

@igrr
thanks for this help, I increased SPIFF size until it works, and worked with 192KB but that is so strange, because I've got this on my definition of boards.txt

wifinfo.menu.ESPModule.ESP07192=ESP07 (1M/192K SPIFFS)
wifinfo.menu.ESPModule.ESP07192.build.board=ESP8266_ESP07
wifinfo.menu.ESPModule.ESP07192.build.flash_size=1M
wifinfo.menu.ESPModule.ESP07192.build.flash_ld=eagle.flash.1m192.ld
wifinfo.menu.ESPModule.ESP07192.build.spiffs_start=0xCB000
wifinfo.menu.ESPModule.ESP07192.build.spiffs_end=0xFB000
wifinfo.menu.ESPModule.ESP07192.build.spiffs_blocksize=4096
wifinfo.menu.ESPModule.ESP07192.upload.maximum_size=827376

and of course eagle.flash.1m192.ld

/* Flash Split for 1M chips */
/* sketch 807KB */
/* spiffs 192KB */
/* eeprom 20KB */

MEMORY
{
  dport0_0_seg :                        org = 0x3FF00000, len = 0x10
  dram0_0_seg :                         org = 0x3FFE8000, len = 0x14000
  iram1_0_seg :                         org = 0x40100000, len = 0x8000
  irom0_0_seg :                         org = 0x40201010, len = 0xc9ff0
}
PROVIDE ( _SPIFFS_start = 0x402CB000 );
PROVIDE ( _SPIFFS_end = 0x402FB000 );
PROVIDE ( _SPIFFS_page = 0x100 );
PROVIDE ( _SPIFFS_block = 0x1000 );
INCLUDE "../ld/eagle.app.v6.common.ld"

What does this mean, linker script tells SPIFF size is 0x402FB000 - 0x402CB000 = 0x3000 = 196608 bytes, that seems correct.

boards.txt says SPIFFS size if 0xFB000 - 0xCB000 = 0x3000 = 196608 bytes, that seems correct and consistent, so let's flash all of this (sketch + SPIFFS) from Arduino IDE and see what does my web interface says
image
seems I'm using 124KB of 169KB

Ok let's get raw json values to check
image
ok, that's correct I'm using 127006 bytes on 173441 (at least this is what is telling my SPIFFS api)

Let's now check raw SPIFFS generated by IDE with mkspiffs

build7562102176516660972.tmp #  mkspiffs -b 4096 -p 256 -i Wifinfo.spiffs.bin -d 5 -s 196608
Debug output enabled
   0 d/dddddddd/d/d/    era_cnt: 18
   1 d/dddddddd/d/d/    era_cnt: 19
   2 /dddd/dddddddid    era_cnt: 49
   3 d/dddddddd/d/d/    era_cnt: 20
   4 /ddddddd/ddd/d/    era_cnt: 50
   5 d/ddd/dddddid/d    era_cnt: 21
   6 /ddddddd/d/d/d/    era_cnt: 22
   7 d/dddd/ddddddd/    era_cnt: 51
   8 d/dddddddd/d/d/    era_cnt: 23
   9 d/ddddddd/ddd/d    era_cnt: 52
  10 d/dddddddd/d/d/    era_cnt: 24
  11 d/dddddddd/d/d/    era_cnt: 25
  12 d/dddddddd/d/d/    era_cnt: 26
  13 d/dddddddd/d/d/    era_cnt: 27
  14 d/dddd/dddddddi    era_cnt: 53
  15 d/dddddddd/d/dd    era_cnt: 28
  16 d/dddddddiddd/d    era_cnt: 54
  17 /dddd/dddd/d/d/    era_cnt: 29
  18 d/dddddddd/d/d/    era_cnt: 30
  19 /ddddddddddd/d/    era_cnt: 55
  20 d/dddddddd/d/d/    era_cnt: 31
  21 d/ddddddddd/d/d    era_cnt: 56
  22 d/dddddddd/d/d/    era_cnt: 32
  23 /ddddddd/ddd/d/    era_cnt: 57
  24 d/dddddddd/d/d/    era_cnt: 33
  25 d/dddddddd/d/d/    era_cnt: 34
  26 d/ddd/dddddddd/    era_cnt: 58
  27 d/dddddddd/d/d/    era_cnt: 35
  28 d/ddd/dddddddd/    era_cnt: 59
  29 d/dddddddd/d/d/    era_cnt: 36
  30 d/dddiddddddddi    era_cnt: 60
  31 d/ddd/ddddd/d/d    era_cnt: 37
  32 /dddddddd/d/d/d    era_cnt: 38
  33 i______________    era_cnt: 61
  34 /dddddddd/d/d/d    era_cnt: 39
  35 _______________    era_cnt: 62
  36 /dddddddd/d/d/d    era_cnt: 40
  37 /ddddddd/ddd/d/    era_cnt: 41
  38 _______________    era_cnt: 63
  39 d/ddddddd/ddd/d    era_cnt: 42
  40 _______________    era_cnt: 64
  41 /ddddddd/dddd/d    era_cnt: 43
  42 d/dddddddd/d/d/    era_cnt: 17
  43 /dddd/ddddddd/d    era_cnt: 44
  44 /dddd/ddddddd/d    era_cnt: 47
  45 /ddddddd/ddd/d/    era_cnt: 45
  46 d/dddd/ddddddd/    era_cnt: 48
  47 d/ddddddd/ddd/d    era_cnt: 46
era_cnt_max: 65
last_errno:  0
blocks:      48
free_blocks: 3
page_alloc:  506
page_delet:  155
used:        127006 of 173441
total: 173441
used: 127006
build7562102176516660972.tmp #

Same thing, all numbers seems consistent, but hey, wait, should my FS should be equal to 196608 bytes ?
Well I think there is on all SPIFFS FS files/dir entries and all FS management so this is why I think all of this takes 23167 bytes (196608 - 173441) good to know, Approx 12%, So on my 192K FS I have 46K free.
This does not solve why when I set my FS to 160KB (so 32K less than the actual that have 46K free) SPIFFS is not able to generate the FS

All all this seems really look likes kind of voodoo on my side ;-)

hallard commented Dec 7, 2015

@igrr
thanks for this help, I increased SPIFF size until it works, and worked with 192KB but that is so strange, because I've got this on my definition of boards.txt

wifinfo.menu.ESPModule.ESP07192=ESP07 (1M/192K SPIFFS)
wifinfo.menu.ESPModule.ESP07192.build.board=ESP8266_ESP07
wifinfo.menu.ESPModule.ESP07192.build.flash_size=1M
wifinfo.menu.ESPModule.ESP07192.build.flash_ld=eagle.flash.1m192.ld
wifinfo.menu.ESPModule.ESP07192.build.spiffs_start=0xCB000
wifinfo.menu.ESPModule.ESP07192.build.spiffs_end=0xFB000
wifinfo.menu.ESPModule.ESP07192.build.spiffs_blocksize=4096
wifinfo.menu.ESPModule.ESP07192.upload.maximum_size=827376

and of course eagle.flash.1m192.ld

/* Flash Split for 1M chips */
/* sketch 807KB */
/* spiffs 192KB */
/* eeprom 20KB */

MEMORY
{
  dport0_0_seg :                        org = 0x3FF00000, len = 0x10
  dram0_0_seg :                         org = 0x3FFE8000, len = 0x14000
  iram1_0_seg :                         org = 0x40100000, len = 0x8000
  irom0_0_seg :                         org = 0x40201010, len = 0xc9ff0
}
PROVIDE ( _SPIFFS_start = 0x402CB000 );
PROVIDE ( _SPIFFS_end = 0x402FB000 );
PROVIDE ( _SPIFFS_page = 0x100 );
PROVIDE ( _SPIFFS_block = 0x1000 );
INCLUDE "../ld/eagle.app.v6.common.ld"

What does this mean, linker script tells SPIFF size is 0x402FB000 - 0x402CB000 = 0x3000 = 196608 bytes, that seems correct.

boards.txt says SPIFFS size if 0xFB000 - 0xCB000 = 0x3000 = 196608 bytes, that seems correct and consistent, so let's flash all of this (sketch + SPIFFS) from Arduino IDE and see what does my web interface says
image
seems I'm using 124KB of 169KB

Ok let's get raw json values to check
image
ok, that's correct I'm using 127006 bytes on 173441 (at least this is what is telling my SPIFFS api)

Let's now check raw SPIFFS generated by IDE with mkspiffs

build7562102176516660972.tmp #  mkspiffs -b 4096 -p 256 -i Wifinfo.spiffs.bin -d 5 -s 196608
Debug output enabled
   0 d/dddddddd/d/d/    era_cnt: 18
   1 d/dddddddd/d/d/    era_cnt: 19
   2 /dddd/dddddddid    era_cnt: 49
   3 d/dddddddd/d/d/    era_cnt: 20
   4 /ddddddd/ddd/d/    era_cnt: 50
   5 d/ddd/dddddid/d    era_cnt: 21
   6 /ddddddd/d/d/d/    era_cnt: 22
   7 d/dddd/ddddddd/    era_cnt: 51
   8 d/dddddddd/d/d/    era_cnt: 23
   9 d/ddddddd/ddd/d    era_cnt: 52
  10 d/dddddddd/d/d/    era_cnt: 24
  11 d/dddddddd/d/d/    era_cnt: 25
  12 d/dddddddd/d/d/    era_cnt: 26
  13 d/dddddddd/d/d/    era_cnt: 27
  14 d/dddd/dddddddi    era_cnt: 53
  15 d/dddddddd/d/dd    era_cnt: 28
  16 d/dddddddiddd/d    era_cnt: 54
  17 /dddd/dddd/d/d/    era_cnt: 29
  18 d/dddddddd/d/d/    era_cnt: 30
  19 /ddddddddddd/d/    era_cnt: 55
  20 d/dddddddd/d/d/    era_cnt: 31
  21 d/ddddddddd/d/d    era_cnt: 56
  22 d/dddddddd/d/d/    era_cnt: 32
  23 /ddddddd/ddd/d/    era_cnt: 57
  24 d/dddddddd/d/d/    era_cnt: 33
  25 d/dddddddd/d/d/    era_cnt: 34
  26 d/ddd/dddddddd/    era_cnt: 58
  27 d/dddddddd/d/d/    era_cnt: 35
  28 d/ddd/dddddddd/    era_cnt: 59
  29 d/dddddddd/d/d/    era_cnt: 36
  30 d/dddiddddddddi    era_cnt: 60
  31 d/ddd/ddddd/d/d    era_cnt: 37
  32 /dddddddd/d/d/d    era_cnt: 38
  33 i______________    era_cnt: 61
  34 /dddddddd/d/d/d    era_cnt: 39
  35 _______________    era_cnt: 62
  36 /dddddddd/d/d/d    era_cnt: 40
  37 /ddddddd/ddd/d/    era_cnt: 41
  38 _______________    era_cnt: 63
  39 d/ddddddd/ddd/d    era_cnt: 42
  40 _______________    era_cnt: 64
  41 /ddddddd/dddd/d    era_cnt: 43
  42 d/dddddddd/d/d/    era_cnt: 17
  43 /dddd/ddddddd/d    era_cnt: 44
  44 /dddd/ddddddd/d    era_cnt: 47
  45 /ddddddd/ddd/d/    era_cnt: 45
  46 d/dddd/ddddddd/    era_cnt: 48
  47 d/ddddddd/ddd/d    era_cnt: 46
era_cnt_max: 65
last_errno:  0
blocks:      48
free_blocks: 3
page_alloc:  506
page_delet:  155
used:        127006 of 173441
total: 173441
used: 127006
build7562102176516660972.tmp #

Same thing, all numbers seems consistent, but hey, wait, should my FS should be equal to 196608 bytes ?
Well I think there is on all SPIFFS FS files/dir entries and all FS management so this is why I think all of this takes 23167 bytes (196608 - 173441) good to know, Approx 12%, So on my 192K FS I have 46K free.
This does not solve why when I set my FS to 160KB (so 32K less than the actual that have 46K free) SPIFFS is not able to generate the FS

All all this seems really look likes kind of voodoo on my side ;-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment