Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

esp8266: Support up to 16 MB of flash #2335

Open
Ratfink opened this issue Aug 21, 2016 · 44 comments
Open

esp8266: Support up to 16 MB of flash #2335

Ratfink opened this issue Aug 21, 2016 · 44 comments

Comments

@Ratfink
Copy link

@Ratfink Ratfink commented Aug 21, 2016

The ESP8266 supports up to 16 megabytes of SPI flash (source, page 12). NodeMCU has supported this much flash for about two months now, and I notice that MicroPython does not. I don't know all the nuances of the ESP8266, but from what I understand there are three things that should all encode the same size: the flash chip's ID, the SpiFlashChip structure from the SDK, and the bootloader size ID. They might not all agree about the flash chip's size, so we try to make them all agree in flashbdev.py. It looks like in the similar routine from NodeMCU, they use size IDs 8 and 9 for 8 MB and 16 MB, respectively. Setting these IDs in MicroPython seems to make mpy think that it has only 512 kB of flash, and I'm not sure why. Ignoring the bootloader size ID and only calculating flash size from the flash chip's ID seems to work, but somehow that seems like the wrong thing to do.

Long story short, I want to add support for 8 and 16 MB of flash to MicroPython, but I don't know what the right way to do it is. Any advice would be greatly appreciated.

@dpgeorge
Copy link
Member

@dpgeorge dpgeorge commented Aug 22, 2016

@Ratfink
Copy link
Author

@Ratfink Ratfink commented Aug 22, 2016

I tried making the suggested change:

diff --git a/esp8266/modules/flashbdev.py b/esp8266/modules/flashbdev.py
index 98b0208..284fa1a 100644
--- a/esp8266/modules/flashbdev.py
+++ b/esp8266/modules/flashbdev.py
@@ -37,6 +37,10 @@ def set_bl_flash_size(real_size):
         code = 3
     elif real_size == 4096*1024:
         code = 4
+    elif real_size == 8192*1024:
+        code = 5
+    elif real_size == 16384*1024:
+        code = 6
     else:
         code = 2
     buf = bytearray(4096)
@@ -50,7 +54,7 @@ def set_bl_flash_size(real_size):
 size = esp.flash_id() >> 16
 # Check that it looks like realistic power of 2 for flash sizes
 # commonly used with esp8266
-if 22 >= size >= 18:
+if 24 >= size >= 18:
     size = 1 << size
     if size != esp.flash_size():
         import machine

It results in a boot loop, printing this repeatedly:

l��r��  yr�'b����x�p�lp�������p�lll��r��        yr�'b��l����x��p�������p��cl`䈌������rn������x���b���b����ߌ��b��bl`�>r�8�l`#��r�����8����r�����8�����lrr��9l�Ğ����br�b�������b�#�n��1�n��`ll������l��l�����;�n��Β��b���b�>��ܒ��b�|lrrr�n�����|~����       6��������b����xbl`��l�#4 ets_task(40100390, 3, 3fff6300, 4)
Bootloader Flash size appear to have been set incorrectly, trying to fix
bcn 0
del if1
usl

 ets Jan  8 2013,rst cause:2, boot mode:(3,6)

load 0x40100000, len 30860, room 16 
tail 12
chksum 0x17
ho 0 tail 12 room 4
load 0x3ffe8000, len 1060, room 12 
tail 8
chksum 0xd7
load 0x3ffe8430, len 3000, room 0 
tail 8
chksum 0xb1
csum 0xb1

If I set it to use code 8 for 8 MB and 9 for 16 MB, like NodeMCU does, I get a similar boot loop, but with more stuff being printed each time it boots:

�����������r������r�l�������������8������
���l����l���l����l��������l�����������܌���rl��r��       yr�'b�����x�p�lp�������p�lll��r��       yr�'b��l
����8��p�������p��cl`��������rn������8���b���b�����ߌ��b��bl`�>r�x��l`#��r��#3 ets_task(40100390, 3, 3fff
6300, 4)
Fatal exception 9(LoadStoreAlignmentCause):
epc1=0x40250a35, epc2=0x00000000, epc3=0x00000000, excvaddr=0x0000001b, depc=0x00000000

 ets Jan  8 2013,rst cause:2, boot mode:(3,6)

load 0x40100000, len 30860, room 16
tail 12
chksum 0x17
ho 0 tail 12 room 4
load 0x3ffe8000, len 1060, room 12
tail 8
chksum 0xd7
load 0x3ffe8430, len 3000, room 0
tail 8
chksum 0xb1
csum 0xb1
system param error
47 00 00 00 08 00 00 00 0c d0 27 40 46 06 00 00 5e 0e 00 00 76 12 00 00 94 d0 27 40 fe 0f 00 00
88 d0 27 40 6e 0f 00 00 7c d0 27 40 7e 0f 00 00 70 d0 27 40 be 13 00 00 64 d0 27 40 ce 13 00 00
58 d0 27 40 96 08 00 00 4c d0 27 40 04 26 27 40 02 00 01 00 68 1a 27 40 04 26 27 40 04 00 02 00 
d4 1a 27 40 04 26 27 40 04 00 02 00 18 1b 27 40 04 26 27 40 02 00 01 00 4c 1b 27 40 04 26 27 40 
02 00 01 00 90 1b 27 40 04 26 27 40 02 00 01 00 b4 1b 27 40 04 26 27 40 02 00 01 00 d8 1b 27 40 
23 25 64 20 65 74 73 5f 74 61 73 6b 28 25 70 2c 20 25 64 2c 20 25 70 2c 20 25 64 29 0a 00 74 61 
73 6b 20 70 72 69 6f 20 6f 75 74 20 6f 66 20 72 61 6e 67 65 3a 20 25 64 0a 00 65 74 73 5f 70 6f 
73 74 3a 20 74 61 73 6b 20 25 64 20 71 75 65 75 65 20 66 75 6c 6c 0a 00 65 74 73 5f 61 6c 74 5f 
74 61 73 6b 3a 20 65 74 73 5f 72 75 6e 00 23 20 41 50 41 31 30 32 20 64 72 69 76 65 72 20 66 6f 
72 20 4d 69 63 72 6f 50 79 74 68 6f 6e 20 6f 6e 20 45 53 50 38 32 36 36 0a 23 20 4d 49 54 20 6c 
69 63 65 6e 73 65 3b 20 43 6f 70 79 72 69 67 68 74 20 28 63 29 20 32 30 31 36 20 52 6f 62 65 72 
74 20 46 6f 73 73 2c 20 44 61 6e 69 65 6c 20 42 75 73 63 68 0a 0a 66 72 6f 6d 20 65 73 70 20 69 
6d 70 6f 72 74 20 61 70 61 31 30 32 5f 77 72 69 74 65 0a 0a 63 6c 61 73 73 20 41 50 41 31 30 32 
3a 0a 20 20 20 20 64 65 66 20 5f 5f 69 6e 69 74 5f 5f 28 73 65 6c 66 2c 20 63 6c 6f 63 6b 5f 70 
69 6e 2c 20 64 61 74 61 5f 70 69 6e 2c 20 6e 29 3a 0a 20 20 20 20 20 20 20 20 73 65 6c 66 2e 63 
6c 6f 63 6b 5f 70 69 6e 20 3d 20 63 6c 6f 63 6b 5f 70 69 6e 0a 20 20 20 20 20 20 20 20 73 65 6c 
66 2e 64 61 74 61 5f 70 69 6e 20 3d 20 64 61 74 61 5f 70 69 6e 0a 20 20 20 20 20 20 20 20 73 65 
6c 66 2e 6e 20 3d 20 6e 0a 20 20 20 20 20 20 20 20 73 65 6c 66 2e 62 75 66 20 3d 20 62 79 74 65 
61 72 72 61 79 28 6e 20 2a 20 34 29 0a 0a 20 20 20 20 20 20 20 20 73 65 6c 66 2e 63 6c 6f 63 6b 
5f 70 69 6e 2e 69 6e 69 74 28 63 6c 6f 63 6b 5f 70 69 6e 2e 4f 55 54 29 0a 20 20 20 20 20 20 20 
20 73 65 6c 66 2e 64 61 74 61 5f 70 69 6e 2e 69 6e 69 74 28 64 61 74 61 5f 70 69 6e 2e 4f 55 54 
29 0a 0a 20 20 20 20 64 65 66 20 5f 5f 73 65 74 69 74 65 6d 5f 5f 28 73 65 6c 66 2c 20 69 6e 64 
65 78 2c 20 76 61 6c 29 3a 0a 20 20 20 20 20 20 20 20 72 2c 20 67 2c 20 62 2c 20 62 72 69 67 68 
74 6e 65 73 73 20 3d 20 76 61 6c 0a 20 20 20 20 20 20 20 20 73 65 6c 66 2e 62 75 66 5b 69 6e 64 
65 78 20 2a 20 34 5d 20 3d 20 72 0a 20 20 20 20 20 20 20 20 73 65 6c 66 2e 62 75 66 5b 69 6e 64 
65 78 20 2a 20 34 20 2b 20 31 5d 20 3d 20 67 0a 20 20 20 20 20 20 20 20 73 65 6c 66 2e 62 75 66 
5b 69 6e 64 65 78 20 2a 20 34 20 2b 20 32 5d 20 3d 20 62 0a 20 20 20 20 20 20 20 20 73 65 6c 66 
2e 62 75 66 5b 69 6e 64 65 78 20 2a 20 34 20 2b 20 33 5d 20 3d 20 62 72 69 67 68 74 6e 65 73 73 
0a 0a 20 20 20 20 64 65 66 20 5f 5f 67 65 74 69 74 65 6d 5f 5f 28 73 65 6c 66 2c 20 69 6e 64 65 
78 29 3a 0a 20 20 20 20 20 20 20 20 69 20 3d 20 69 6e 64 65 78 20 2a 20 34 0a 20 20 20 20 20 20 
20 20 72 65 74 75 72 6e 20 73 65 6c 66 2e 62 75 66 5b 69 5d 2c 20 73 65 6c 66 2e 62 75 66 5b 69 
20 2b 20 31 5d 2c 20 73 65 6c 66 2e 62 75 66 5b 69 20 2b 20 32 5d 2c 20 73 65 6c 66 2e 62 75 66 
5b 69 20 2b 20 33 5d 0a 0a 20 20 20 20 64 65 66 20 77 72 69 74 65 28 73 65 6c 66 29 3a 0a 20 20 
20 20 20 20 20 20 61 70 61 31 30 32 5f 77 72 69 74 65 28 73 65 6c 66 2e 63 6c 6f 63 6b 5f 70 69 
6e 2c 20 73 65 6c 66 2e 64 61 74 61 5f 70 69 6e 2c 20 73 65 6c 66 2e 62 75 66 29 0a 00 23 20 4e 
65 6f 50 69 78 65 6c 20 64 72 69 76 65 72 20 66 6f 72 20 4d 69 63 72 6f 50 79 74 68 6f 6e 20 6f 
6e 20 45 53 
70 69 6e 29 0a 0a 20 20 20 20 64 65 66 20 72 65 61 64 62 79 74 65 28 73 65 6c 66 29 3a 0a 20 20 
20 20 20 20 20 20 72 65 74 75 72 6e 20 5f 6f 77 2e 72 65 61 64 62 79 74 65 28 73 65 6c 66 2e 70 
69 6e 29 0a 0a 20 20 20 20 64 65 66 20 72 65 61 64 28 73 65 6c 66 2c 20 63 6f 75 6e 74 29 3a 0a 
20 20 20 20 20 20 20 20 62 75 66 20 3d 20 62 79 74 65 61 72 72 61 79 28 63 6f 75 6e 74 29 0a 20 
20 20 20 20 20 20 20 66 6f 72 20 69 20 69 6e 20 72 61 6e 67 65 28 63 6f 75 6e 74 29 3a 0a 20 20 
20 20 20 20 20 20 20 20 20 20 62 75 66 5b 69 5d 20 3d 20 5f 6f 77 2e 72 65 61 64 62 79 74 65 28 
73 65 6c 66 2e 70 69 6e 29 0a 20 20 20 20 20 20 20 20 72 65 74 75 72 6e 20 62 75 66 0a 0a 20 20 
20 20 64 65 66 20 77 72 69 74 65 62 69 74 28 73 65 6c 66 2c 20 76 61 6c 75 65 29 3a 0a 20 20 20 
20 20 20 20 20 72 65 74 75 72 6e 20 5f 6f 77 2e 77 72 69 74 65 62 69 74 28 73 65 6c 66 2e 70 69 
6e 2c 20 76 61 6c 75 65 29 0a 0a 20 20 20 20 64 65 66 20 77 72 69 74 65 62 79 74 65 28 73 65 6c 
66 2c 20 76 61 6c 75 65 29 3a 0a 20 20 20 20 20 20 20 20 72 65 74 75 72 6e 20 5f 6f 77 2e 77 72 
69 74 65 62 79 74 65 28 73 65 6c 66 2e 70 69 6e 2c 20 76 61 6c 75 65 29 0a 0a 20 20 20 20 64 65 
66 20 77 72 69 74 65 28 73 65 6c 66 2c 20 62 75 66 29 3a 0a 20 20 20 20 20 20 20 20 66 6f 72 20 
62 20 69 6e 20 62 75 66 3a 0a 20 20 20 20 20 20 20 20 20 20 20 20 5f 6f 77 2e 77 72 69 74 65 62 
79 74 65 28 73 65 6c 66 2e 70 69 6e 2c 20 62 29 0a 0a 20 20 20 20 64 65 66 20 73 65 6c 65 63 74 
5f 72 6f 6d 28 73 65 6c 66 2c 20 72 6f 6d 29 3a 0a 20 20 20 20 20 20 20 20 73 65 6c 66 2e 72 65 
73 65 74 28 29 0a 20 20 20 20 20 20 20 20 73 65 6c 66 2e 77 72 69 74 65 62 79 74 65 28 4d 41 54 
43 48 5f 52 4f 4d 29 0a 20 20 20 20 20 20 20 20 73 65 6c 66 2e 77 72 69 74 65 28 72 6f 6d 29 0a 
0a 20 20 20 20 64 65 66 20 73 63 61 6e 28 73 65 6c 66 29 3a 0a 20 20 20 20 20 20 20 20 64 65 76 
69 63 65 73 20 3d 20 5b 5d 0a 20 20 20 20 20 20 20 20 64 69 66 66 20 3d 20 36 35 0a 20 20 20 20 
20 20 20 20 72 6f 6d 20 3d 20 46 61 6c 73 65 0a 20 20 20 20 20 20 20 20 66 6f 72 20 69 20 69 6e 
20 72 61 6e 67 65 28 30 78 66 66 29 3a 0a 20 20 20 20 20 20 20 20 20 20 20 20 72 6f 6d 2c 20 64 
69 66 66 20 3d 20 73 65 6c 66 2e 5f 73 65 61 72 63 68 5f 72 6f 6d 28 72 6f 6d 2c 20 64 69 66 66 
29 0a 20 20 20 20 20 20 20 20 20 20 20 20 69 66 20 72 6f 6d 3a 0a 20 20 20 20 20 20 20 20 20 20 
20 20 20 20 20 20 64 65 76 69 63 65 73 20 2b 3d 20 5b 72 6f 6d 5d 0a 20 20 20 20 20 20 20 20 20 
20 20 20 69 66 20 64 69 66 66 20 3d 3d 20 30 3a 0a 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 
20 62 72 65 61 6b 0a 20 20 20 20 20 20 20 20 72 65 74 75 72 6e 20 64 65 76 69 63 65 73 0a 0a 20 
20 20 20 64 65 66 20 5f 73 65 61 72 63 68 5f 72 6f 6d 28 73 65 6c 66 2c 20 6c 5f 72 6f 6d 2c 20 
64 69 66 66 29 3a 0a 20 20 20 20 20 20 20 20 69 66 20 6e 6f 74 20 73 65 6c 66 2e 72 65 73 65 74 
28 29 3a 0a 20 20 20 20 20 20 20 20 20 20 20 20 72 65 74 75 72 6e 20 4e 6f 6e 65 2c 20 30 0a 20 
20 20 20 20 20 20 20 73 65 6c 66 2e 77 72 69 74 65 62 79 74 65 28 53 45 41 52 43 48 5f 52 4f 4d
29 0a 20 20 20 20 20 20 20 20 69 66 20 6e 6f 74 20 6c 5f 72 6f 6d 3a 0a 20 20 20 20 20 20 20 20
20 20 20 20 6c 5f 72 6f 6d 20 3d 20 62 79 74 65 61 72 72 61 79 28 38 29 0a 20 20 20 20 20 20 20
20 72 6f 6d 20 3d 20 62 79 74 65 61 72 72 61 79 28 38 29 0a 20 20 20 20 20 20 20 20 6e 65 78 74
5f 64 69 66 66 20 3d 20 30 0a 20 20 20 20 20 20 20 20 69 20 3d 20 36 34 0a 20 20 20 20 20 20 20
20 66 6f 72 20 62 79 74 65 20 69 6e 20 72 61 6e 67 65 28 38 29 3a

All that hex decodes to the following, which is mainly portions of esp8266/scripts/apa102.py, esp8266/scripts/neopixel.py, and esp8266/scripts/onewire.py:

G
Ð'@F�^v��Ð'@þÐ'@n|Ð'@~pÐ'@¾�dÐ'@Î�XÐ'LÐ'@�&'@��h�'@�&'@��Ô�'@�&'@����'@�&'@��L�'@�&'@����'@�&'@��´�'@�&'@��Ø�'@#%d ets_task(%p, %d, %p, %d)
task prio out of range: %d
ets_post: task %d queue full
ets_alt_task: ets_run# APA102 driver for MicroPython on ESP8266
# MIT license; Copyright (c) 2016 Robert Foss, Daniel Busch

from esp import apa102_write

class APA102:
    def __init__(self, clock_pin, data_pin, n):
        self.clock_pin = clock_pin
        self.data_pin = data_pin
        self.n = n
        self.buf = bytearray(n * 4)

        self.clock_pin.init(clock_pin.OUT)
        self.data_pin.init(data_pin.OUT)

    def __setitem__(self, index, val):
        r, g, b, brightness = val
        self.buf[index * 4] = r
        self.buf[index * 4 + 1] = g
        self.buf[index * 4 + 2] = b
        self.buf[index * 4 + 3] = brightness

    def __getitem__(self, index):
        i = index * 4
        return self.buf[i], self.buf[i + 1], self.buf[i + 2], self.buf[i + 3]

    def write(self):
        apa102_write(self.clock_pin, self.data_pin, self.buf)
# NeoPixel driver for MicroPython on ESpin)

    def readbyte(self):
        return _ow.readbyte(self.pin)

    def read(self, count):
        buf = bytearray(count)
        for i in range(count):
            buf[i] = _ow.readbyte(self.pin)
        return buf

    def writebit(self, value):
        return _ow.writebit(self.pin, value)

    def writebyte(self, value):
        return _ow.writebyte(self.pin, value)

    def write(self, buf):
        for b in buf:
            _ow.writebyte(self.pin, b)

    def select_rom(self, rom):
        self.reset()
        self.writebyte(MATCH_ROM)
        self.write(rom)

    def scan(self):
        devices = []
        diff = 65
        rom = False
        for i in range(0xff):
            rom, diff = self._search_rom(rom, diff)
            if rom:
                devices += [rom]
            if diff == 0:
                break
        return devices

    def _search_rom(self, l_rom, diff):
        if not self.reset():
            return None, 0
        self.writebyte(SEARCH_ROM)
        if not l_rom:
            l_rom = bytearray(8)
        rom = bytearray(8)
        next_diff = 0
        i = 64
        for byte in range(8):```
@dpgeorge
Copy link
Member

@dpgeorge dpgeorge commented Aug 25, 2016

Try erasing the flash entirely before deploying the image. It looks like the ESP SDK is finding the wifi/system parameters in the wrong location.

@Ratfink
Copy link
Author

@Ratfink Ratfink commented Aug 25, 2016

I erased the flash before deploying the image every time above. It does seem like it's reading the system parameters from the wrong location. From this pull request from NodeMCU, it looks like the ESP SDK doesn't really support more than 4 MB of flash even though the chip itself does. Maybe it would be better to not worry about supporting this much flash until Espressif themselves do.

@dpgeorge
Copy link
Member

@dpgeorge dpgeorge commented Aug 26, 2016

it looks like the ESP SDK doesn't really support more than 4 MB of flash even though the chip itself does. Maybe it would be better to not worry about supporting this much flash until Espressif themselves do.

Yes. We can revisit this issue when upgrading to the latest ESP SDK, which has a new way of locating and storing the persistent wifi/config parameters.

@peterhinch
Copy link
Contributor

@peterhinch peterhinch commented Aug 31, 2016

Is that kosher? It doesn't appear on http://www.wemos.cc/

@mcauser
Copy link
Contributor

@mcauser mcauser commented Aug 31, 2016

Yeah it's legit. Was only released a few hrs ago. It'll be on their website soon, I guess.

@deshipu
Copy link
Contributor

@deshipu deshipu commented Oct 19, 2016

So now that we use the new sdk, I tried this again, but I'm still getting a boot loop.

l��|��rrnb��l�b�lb쌜���lb�lrlll��|��rrnb��ll��b�b쌜��b��lrl�l��|��rrnb��l��b�b쌜��lb�lbl����n�r��n|�llll`��r�l�l�l`��r�l�l�l`��r�l���ll`rl��rl���b��b�bbr�rb��n�nn�l��l�l��ll������l�n����bll�rp���bl�brlrlr�n�����n����brb��b쎜�l쌌�pbl`��l�#4 ets_task(40100164, 3, 3fff6388, 4)
Bootloader Flash size appear to have been set incorrectly, trying to fix
bcn 0
del if1
usl

 ets Jan  8 2013,rst cause:2, boot mode:(3,6)

load 0x40100000, len 32020, room 16 
tail 4
chksum 0x18
load 0x3ffe8000, len 1084, room 4 
tail 8
chksum 0xd6
load 0x3ffe8440, len 3000, room 0 
tail 8
chksum 0xac
csum 0xac
@robert-hh
Copy link
Contributor

@robert-hh robert-hh commented Jan 5, 2017

I looked a little bit around on this topic and found this link: http://www.packom.org/esp8266/16mb/flash/eeprom/2016/10/14/esp8266-16mbyte-flash_handling.html
It also point to the ESP SDK documentation and uses a patch for the flashchip data structure, which embedded in ROM and has the flash size hard coded. espressif seems to have built a patch mechanism into it's code for these purposes. I did not dig further, but in addition to the changes to the python code, that seems to be required.

@deshipu
Copy link
Contributor

@deshipu deshipu commented Feb 24, 2017

There is some hope, the next version of (non OS) SDK should support it: espressif/esptool#123 (comment)

@RockyTV
Copy link

@RockyTV RockyTV commented Oct 28, 2017

Any update on this?

@ghost
Copy link

@ghost ghost commented Nov 9, 2017

Yes any updates regarding this? It's been a while now. No hope I guess.

@BoKKeR
Copy link

@BoKKeR BoKKeR commented Dec 10, 2017

Any update? The SDK seems to support it

@whatnick
Copy link

@whatnick whatnick commented Jan 25, 2018

I installed v2.1.0 of esp-open-sdk . Compiled upy and still cannot load firmware with 16MB flash set, works with 4MB just fine.

@robert-hh
Copy link
Contributor

@robert-hh robert-hh commented Jan 25, 2018

What is returned by esp.flash_size()?

@whatnick
Copy link

@whatnick whatnick commented Jan 25, 2018

@dpgeorge
Copy link
Member

@dpgeorge dpgeorge commented Feb 2, 2018

Thanks to @whatnick I now have a 16M board and was able to work on this issue. I managed to get uPy working with the full 16M but it required 3 things:

  1. Update ESP SDK to v2.1.0 or later.
  2. Update esptool.py to a recent version that supports 16M flash.
  3. After erasing the flash (make erase), manually flash the esp_init_data_default.bin file (found in the ESP SDK release) to the location 0xffc000. This can be done using the command: esptool.py --port /dev/ttyUSB0 write_flash --verify 0xffc000 esp_init_data_default.bin

Then it seems to work and esp.flash_size() returns 16M. Also the filesystem uses up to the end of the flash.

For flash sizes up to 4Mbytes, uPy can automatically write the esp_init_data_default.bin data to the correct location in the flash. But for 16M flash chips this automatic step seems to fail because of 2 reasons: 1) the flash size is incorrectly reported as 4M; 2) SPIRead and SPIWrite can't seem to access beyond 4Mbytes. My guess is that the full range of 16Mbytes of flash requires special handling and is "unlocked" during the Espressif boot code, which happens too late for the uPy automatic writing part.

So it might be that the only way to support 16M of flash is to require the user to manually flash the esp_init_data_default.bin file (of course we can provide a convenient "make" target to do this automatically). I'm not sure how Arduino/NodeMCU deal with this.

@whatnick
Copy link

@whatnick whatnick commented Feb 2, 2018

@Proteus86
Copy link

@Proteus86 Proteus86 commented Feb 8, 2018

esptool.py --port /dev/ttyUSB0 --baud 115200 write_flash --flash_size=4MB 0 esp8266-20171101-v1.9.3.bin

Work fine .

@whatnick
Copy link

@whatnick whatnick commented Feb 9, 2018

@keijsvog
Copy link

@keijsvog keijsvog commented Feb 11, 2018

Hi @dpgeorge,

Can you describe the procedure step-by-step for my D1 Mini Pro? I first followed your steps:

  1. updates SDK to 2.1.0
  2. updated ESPtool
  3. Erase flash by esptool.py --port /dev/cu.SLAB_USBtoUART erase_flash
  4. Executed: esptool.py --port /dev/cu.SLAB_USBtoUART write_flash --verify 0xffc000 esp_init_data_default.bin

Next I flashed Micropython:
esptool.py --port /dev/cu.SLAB_USBtoUART --baud 460800 write_flash -fm dio -fs 16MB 0 esp8266-20171101-v1.9.3.bin
Unfortunately this does not work because the board keeps resetting. What am I doing wrong?

Another question: when I download the newest SDK 2.2.0, there is a esp_init_data_default_v05.bin and a esp_init_data_default_v08.bin, which on to use to flash instep 4?

Thanks in advance.

@dpgeorge
Copy link
Member

@dpgeorge dpgeorge commented Feb 13, 2018

Unfortunately this does not work because the board keeps resetting. What am I doing wrong?

@keijsvog did you rebuild MicroPython and use that firmware (rather than one from https://micropython.org/download)? The point is that you need to build the ESP8266 firmware using the new SDK; see ports/esp8266/README.md for more details about building it.

there is a esp_init_data_default_v05.bin and a esp_init_data_default_v08.bin, which on to use to flash instep 4?

I'm not sure. Try one and if it doesn't work then try the other :)

@keijsvog
Copy link

@keijsvog keijsvog commented Feb 14, 2018

@dpgeorge,

Thanks it worked! Short description for those how struggle with the same challenge:

First of all you should have a ESP8266 board with 16MB memory. I have the Wemos D1 Pro. The description below work on a Mac.

  1. Start with upgrading the SEPTools: pip install esptool --upgrade

  2. Compile the esp-open-sdk by following the instructions This will download during make the 2.1.0 SDK (at least at time of writing these instructions) You will need the file esp_init_data_default.bin later on (the file is located in the directory esp-open-sdk/ESP8266_NONOS_SDK-2.1.0-18-g61248df/bin/) For your convenience, copy the file to a separate directory.

  3. Compile the MicroPython port to ESP8266 by following the instructions

  4. You will end up with a file called firmware-combined.bin which you can also copy to the directory already containing esp_init_data_default.bin. Change to this directory

  5. First erase the ESP8266: esptool.py --port /dev/cu.SLAB_USBtoUART --baud 460800 erase_flash

  6. Flash both the micropython firmware and the esp_init_data_default.bin: esptool.py --port /dev/cu.SLAB_USBtoUART --baud 460800 write_flash -fm dio -fs 16MB 0 firmware-combined.bin 0xffc000 esp_init_data_default.bin

  7. Open a serial terminal to your ESP8288 (e.g. with the serial monitor included in the Arduino IDE) and reset the ESP8288

  8. Type into the serial terminal following commands:

>>> import esp
>>> esp.flash_size()

I get following result:
16777216

So now we can use the full 16MB of the board!

@ghost
Copy link

@ghost ghost commented Feb 15, 2018

Do you think it is worth to create a paragraph in 'ports/esp8266/README.md' pointing to this thread and specifically the answer above regarding the chips with more than 4MB flash? There's a lot of them out there, a paragraph like that would be quite helpful I would imagine.

@mcauser
Copy link
Contributor

@mcauser mcauser commented Feb 22, 2018

I pip upgraded my esptool to the latest version, however, I was still getting errors trying to connect at baud 460800 and using -fs 16MB. Turns out, the older esptool 1.x in my esp-open-sdk volume was being called.

In my ~./bash_profile:
export PATH=/Volumes/case-sensitive/esp-open-sdk/xtensa-lx106-elf/bin:$PATH

$ which esptool.py
/Volumes/case-sensitive/esp-open-sdk/xtensa-lx106-elf/bin/esptool.py

Commented out the export PATH, opened a new terminal. Ran esptool write_flash, success!!

>>> esp.flash_size()
1677721

🍻

@alanjds
Copy link

@alanjds alanjds commented Feb 27, 2018

@keijsvog Can you please share your firmware-combined.bin? Then the ones that want not to go thru the process just to see if it works can try it.

@keijsvog
Copy link

@keijsvog keijsvog commented Feb 27, 2018

@alanjds,

Attached is a zip file with the two files needed: firmware-combined.bin and esp_init_data_default.bin

I hope this works for you.
MicroPython-Firmware-esp8266.zip

@alanjds
Copy link

@alanjds alanjds commented Feb 27, 2018

Thanks a lot.

I had not worked. I will try to use some old binaries reported to be working, even if on 4MB. I had not yet made to work on Mini Pro on any version.

Update: It worked. Forgot to set the --baud 115200 on the terminal client

@alanjds
Copy link

@alanjds alanjds commented Mar 1, 2018

Looks like the 4MB version worked. I got baud-rate issues to test some days ago.

Will try your binary again later..

@flashypepo
Copy link

@flashypepo flashypepo commented Mar 1, 2018

@keijsvog: thank you for the two binairies! It works on a WeMOS D1 mini Pro with 16MB after following your instructions. The esptools.py I used is version 2.3.1. Will try to build the binary firmware-combined.bin later.

@alanjds
Copy link

@alanjds alanjds commented Mar 1, 2018

Now it worked! I had some mismatch on the baud rate of the serial terminal connection :S

Worked:

$ picocom /dev/tty.SLAB_USBtoUART --baud 115200
@whatnick
Copy link

@whatnick whatnick commented May 30, 2018

Success here. Thanks a lot for the firmware build.

@devxpy
Copy link

@devxpy devxpy commented Sep 11, 2018

Here are the upto date firmaware images.

wemos d1 mini pro.zip

@matthiasheng
Copy link

@matthiasheng matthiasheng commented Nov 16, 2018

Thanks all for the hard work! I got 16MB works fine here!

@bcsrepo
Copy link

@bcsrepo bcsrepo commented Jan 13, 2019

@devxpy - what's the difference between the firmware you have and the one from @keijsvog ? Not sure which to use?

@gretel
Copy link

@gretel gretel commented May 9, 2019

thanks folks 😻just flashed a wemos d1 mini pro to support 16MB of flash.

$ esptool.py version
esptool.py v2.6
2.6
$ esptool.py --port ${PORT} --after no_reset --baud 460800 erase_flash
# ./esp-open-sdk/ESP8266_NONOS_SDK-2.1.0-18-g61248df/bin/esp_init_data_default.bin
$ esptool.py --port ${PORT} write_flash 0xffc000 esp_init_data_default.bin
# do not autodetect, force 16MB flash
$ esptool.py --port ${PORT} --baud 460800 write_flash -fm dio -fs 16MB 0x00000 firmware-combined.bin
import esp
esp.flash_size()
16777216
@MarSoft
Copy link

@MarSoft MarSoft commented Jul 24, 2019

Right now it works if I just flash the latest firmware version from downloads and esp_init_data_default.bin from https://github.com/espressif/esp8266_at/tree/master/bin - first erase flash, then flash them both, just like @gretel suggested in the comment above.

@flashypepo
Copy link

@flashypepo flashypepo commented Jul 24, 2019

Also confirmed. It works just like @gretel suggested. Thanks.
Used esptool.py version 2.5.1 - Lolin D1 mini Pro v2.0.0 - esp.flash_size() -> 16MB!

@xlee2000
Copy link

@xlee2000 xlee2000 commented Jul 30, 2019

Thanks very much.

@reillybeacom
Copy link

@reillybeacom reillybeacom commented Aug 11, 2019

I would also like to confirm this new strategy works and I have been waiting for something like this "proper" solution for a long time! Thanks @gretel !

It is still slightly confusing that the answer is spread over the last couple of comments so I would like to combine them here:

Download esp_init_data_default.bin from here (click "View raw"):

https://github.com/espressif/ESP8266_AT/blob/master/bin/esp_init_data_default.bin

Download the latest version of MicroPython for 8266 from here:

http://micropython.org/download#esp8266

Erase the flash:

 esptool.py --port ${PORT} --after no_reset --baud 460800 erase_flash

Flash esp_init_data_default.bin:

esptool.py --port ${PORT} write_flash 0xffc000 esp_init_data_default.bin

Flash to the 16mb ESP8266 module (eg. Lolin D1 Mini Pro) for your version of esp8266-20190529-v1.11.bin:

esptool.py --port ${PORT} --baud 460800 write_flash -fm dio -fs 16MB 0x00000 esp8266-20190529-v1.11.bin

Can anyone explain what esp_init_data_default.bin is and does? Any downsides?

Can anyone suggest a good place in the documentation for this? (So as to maybe close this issue with a PR.) eg. https://github.com/micropython/micropython/blob/master/docs/esp8266/tutorial/intro.rst

Thanks

@MJRuffDevelop
Copy link

@MJRuffDevelop MJRuffDevelop commented Oct 8, 2019

Been looking at this myself and I cannot for the life of me, get it to work.

@robert-hh
Copy link
Contributor

@robert-hh robert-hh commented Oct 8, 2019

Did you flash both the firmware image and esp_init_data_default.bin?
Can you flash the device in 4MB mode, by specifying the flash size, option: -fs 4MB
If you, you may have have to try other flash modes.

@mycognosist
Copy link

@mycognosist mycognosist commented Dec 10, 2019

A slight variation of the approach suggested by @gretel :

I received an error on the write_flash command for esp_init_data_default.bin, since esptool.py was auto-detecting the flash size as 4MB. I had to specify the -fs 16MB flag in the first flash command to achieve success:

esptool.py --port /dev/ttyUSB0 write_flash 0xffc000 esp_init_data_default.bin -fs 16MB

Thanks to all.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
You can’t perform that action at this time.