inflate failure - MZ_ERR: -3, inflate status: -2

[playmiel]

[190037][I][flashz.cpp:298] flash_cb(): [FLASHZ] flashed 32768 bytes
[190099][D][flashz.cpp:171] inflate_block_to_cb(): [FLASHZ] CB - idx:1671168, head:1065363604, dbgn:0, dend:0, ddatalen:32768, avin:1085, tin:1018615, tout:1671168, fin:0
[190359][I][flashz.cpp:298] flash_cb(): [FLASHZ] flashed 32768 bytes
[190375][W][flashz.cpp:144] inflate_block_to_cb(): [FLASHZ] inflate failure - MZ_ERR: -3, inflate status: -2
[190376][E][flashz.cpp:263] writez(): [FLASHZ] Inflate ERROR: -3
[190380][W][flashz-http.cpp:163] file_upload(): [FZ-HTTP] OTA failed in progress: No Error
I don't know why I have this error always in the same place with a compressed file sent to asyncwebserver either with the command

pigz -9kzc firmware.bin | curl -v http://192.168.0.165/update -F file=@-

or with my code :

function sendFile(file, url) {
  var reader = new FileReader();
  reader.onload = function(e) {
    var data = new Uint8Array(e.target.result);
    // Compress only if the file is firmware and not already compressed
// Compress only if the file is firmware and not already compressed
if (url.includes("firmware") && !file.name.endsWith('.zz')) {
  var compressedData = pako.deflate(data);
  var decompressedData = pako.inflate(compressedData);

  // Les données décompressées doivent être identiques aux données originales
  console.log('Les données sont identiques:', data.toString() === decompressedData.toString());
  
  data = compressedData; // Utiliser deflate pour la compression zlib
}

    var blob = new Blob([data], {type: 'application/zlib'}); 
    var formData = new FormData();
    formData.append('file', blob, file.name.endsWith('.zz') ? file.name : file.name + '.zz'); // Changez l'extension du fichier pour .zz
    formData.append("img", url.includes("spiffs") ? "fs" : "firmware");

    fetch("/update", { 
      method: "POST",
      body: formData
    })
    .then((response) => {
      if (!response.ok) {
        throw new Error("Erreur réseau lors de l'envoi du fichier.");
      }
      return response.text();
    })
    .then((data) => {
      console.log(data);
      document.getElementById("message_" + (url.includes("spiffs") ? "spiffs" : "firmware")).innerText = data;
    })
    .catch((error) => {
      console.error("Erreur lors de l'envoi du fichier :", error);
      document.getElementById("message_" + (url.includes("spiffs") ? "spiffs" : "firmware")).innerText =
        "Erreur lors de l'envoi du fichier "+ (url.includes("spiffs") ? "spiffs" : "firmware");
    });
  };
  reader.readAsArrayBuffer(file);
}

[vortigont]

what's the size of firmware.bin?
Looks like it's the last chunk and you probably miss to signal final flag in the call FlashZ::getInstance().writez(data, len, final)

Check how it's done in FlashZhttp
https://github.com/vortigont/esp32-flashz/blob/05e2667652efe4c77db7488f07225821aafdc8df/src/flashz-http.cpp#LL162C12-L162C58

[playmiel]

The size of the compressed file is 964kb and I don't really understand why it is missing

final

because I use FlashZhttp::handle_ota_form
To receive the file

[vortigont]

pls give exact file size, you did not mentioned you used FlashZhttp::handle_ota_form.
This status inflate failure - MZ_ERR: -3, inflate status: -2 indicates data stream error, more specific -2 is TINFL_STATUS_ADLER32_MISMATCH
so need to find out what is wrong with it. Either the end of stream is given wrong when there is more data or vise versa. Or it is something else that corrupts data stream.

[playmiel]

I did a test:

[97803][D][flashz.cpp:171] inflate_block_to_cb(): [FLASHZ] CB - idx:884736, head:1065363604, dbgn:0, dend:0, ddatalen:32768, avin:619, tin:478705, tout:884736, fin:0
[98066][I][flashz.cpp:298] flash_cb(): [FLASHZ] has flashed 32768 bytes
[98131][D][flashz.cpp:171] inflate_block_to_cb(): [FLASHZ] CB - idx:917504, head:1065363604, dbgn:0, dend:0, ddatalen:32768, avin:0, tin:503804, tout:917504, fin:0
heap size: 106856
[98222][I][flashz.cpp:298] flash_cb(): [FLASHZ] has flashed 32768 bytes
[98228][W][flashz.cpp:144] inflate_block_to_cb(): [FLASHZ] inflation failure - MZ_ERR: -3, inflation status: -1
[98228][E][flashz.cpp:263] writez(): [FLASHZ] Inflate error: -3

the exact size of the compressed file (firmware.bin.zz) is 977 Kb (1 001 263 bytes)

[vortigont]

this one is different
avin:0, tin:503804 shows that previous chunk ended inflation filling the buffer to the top, which is really uncommon unless raw data contains large blocks of void. And it was in the middle of the transfer (about 500k). Then next chunk failed.
Have you tried it several times and it always fails at the same position CB - idx:917504?
Have you tried uploading uncompressed image to see if it goes well? A faulty board maybe?

[playmiel]

-Have you tried several times and it always fails at the same position CB - idx:917504 ?

yes it always fails at the same place with the same file (it changes depending on the browser)

-Have you tried to download an uncompressed image to see if everything is fine?

yes with an uncompressed file everything is fine

-A defective card perhaps?

indeed I tested on another card (same reference) it works well so I do not know where the problem comes from on the first card (it is perhaps maybe a corruption of the memory)

[vortigont]

oh, that was interesting, thanks for testing and confirmation! Will keep that case in mind for possible defective chips/boards.
Agreed, most probable cause is defected portions of memory. Not sure if any memtest-like tools exist for esp32, could have been useful.

[playmiel]

finally I found some differences and there's a difference, one that doesn't work is an ESP32-Ethernet-Kit V1.1 and the second is ESP32-Ethernet-Kit V1.2 which work.
In the Getting Started Guide, it states that difference between two is :

• Correct the placement of the GPIO pin number marking on the board silkscreen next to the DIP switch.

• The values of C1, C2, C42 and C43 are updated to 20 pF. For further information, please refer to the ESP32-Ethernet-Kit V1.2 (A) Ethernet board diagram.

• Replace ESP32-WROVER-B for ESP32-WROVER-E.

Replace ESP32-WROVER-B for ESP32-WROVER-E

This modification may indicate that ESP32-WROVER-B is maybe the source of the problem.
I have 2 cards with ESP32-WROVER-E that works.

[vortigont]

BTW, thanks for the reference js code for compressing uploaded files on the fly. Think this could be useful example implementation.
If you care to prepare a PR with a small notes for it, I would gladly include it this repo!

[playmiel]

ok I'm going to open a PR for that with part of my code and I wanted to open another PR for another feature that allows to encrypt the firmware with a key to add a security layer (like esp32 FOTA) that I implemented on my side and that I think could be interesting.

[vortigont]

that would be great, thanks!