Memory allocation error reading new message #83
Comments
This seems to imply you are receiving very long messages. Is this the case? There was a recent change that may have affected the ability to receive long messages. I have pushed an update reverting this change. Please could you re-test and report your results. |
|
The message sizes are reported next to the topic names, I am not sure if 20k would be considered large (possibly). I took a copy of the new module and used that. As you can see on 2 consecutive runs the error repeats but on the first very quickly, the second runs a bit more. Once the error is hit, the app continues without any activity and after a while it restarts the connection and msgs flow again .. until the next memory error. thanks Checking WiFi integrity. ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
|
|
On further thought I realised that this version would not fix the problem and I will revert the change. In principle the MQTT protocol is designed to handle huge messages. However there is a practical limit on a micocontroller running MicroPython. This is because whatever approach is used for the read, the message must be assembled into a single array of bytes. This requires a block of contiguous RAM. In practice available RAM becomes fragmented, so even with your 119360 bytes free it's extremely unlikely that you'll be able to allocate a buffer of 33460 bytes. You need to use much smaller messages. |
|
Hi
It is missing the _as_write method
neil
Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows
From: Peter ***@***.***>
Sent: 26 July 2022 16:40
To: ***@***.***>
Cc: ***@***.***>; ***@***.***>
Subject: Re: [peterhinch/micropython-mqtt] Memory allocation error reading new message (Issue #83)
Closed #83<#83> as completed.
—
Reply to this email directly, view it on GitHub<#83 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AJ4QC4F6IYAZVC4UYRADMZTVWABHBANCNFSM54VNKRAA>.
You are receiving this because you authored the thread.Message ID: ***@***.***>
|
Apologies. Now fixed. |
Certain new messages are causing a memory error as reported below. The error seems to point to the _as_read function, maybe when the code allocates the buffer or data for a new message, i assume with the aync nature of the code meaning there can be multiple message instances at the same time momentarily overwhelming the memory capacity of the pico. Would it be possible within the async nature of the code to add a check before the allocation to ensure there is enough memory before trying to allocate and what would be the best way of handling a lack of memory without mucking up the whole process ? txs for a great app
RAM free 125104 alloc 40976
Topic: "MM/metofficecot1/MMQTRelay" MessageLen: "6511" Retained: False
Topic: "MM/RSSFeed2/MMQTRelay" MessageLen: "15067" Retained: False
Topic: "MM/metofficecot/MMQTRelay" MessageLen: "6510" Retained: False
Topic: "MM/RSSFeed2/MMQTRelay" MessageLen: "2944" Retained: False
Topic: "MM/RSSFeed2/MMQTRelay" MessageLen: "1299" Retained: False
Topic: "MM/MMMTwitter/MMQTRelay" MessageLen: "13841" Retained: False
Topic: "MM/MMFP5/MMQTRelay" MessageLen: "6413" Retained: False
Topic: "MM/MMMTwitter/MMQTRelay" MessageLen: "11205" Retained: False
Task exception wasn't retrieved
future: coro= <generator object '_handle_msg' at 20015220>
Traceback (most recent call last):
File "uasyncio/core.py", line 1, in run_until_complete
File "/lib/mqtt_as.py", line 581, in _handle_msg
File "/lib/mqtt_as.py", line 466, in wait_msg
File "/lib/mqtt_as.py", line 171, in _as_read
MemoryError: memory allocation failed, allocating 33268 bytes
RAM free 125296 alloc 40784
The text was updated successfully, but these errors were encountered: