Possible memory leak in urquests.get and others.

[emacstheviking]

Hi, I just answered a Reddit issue where somebody has some very trivial code in a loop, and it runs out of memory after a while, the reddit page is here:

https://www.reddit.com/r/raspberrypipico/comments/12v8k0a/oserror_errno_12_enomem/?sort=new

I walked through the code and on line 93 of urequests.py, there is an allocation taking place in wrap_socket(), assigned to s and then not seemingly released after s.close() has been called. I have only looked at the code for aobut half an hour so it's entirely possible the deallocation is done somewhere else but wrap_socket returns a handle to the memory, that's s, and there seems to be no corresponding call to free it.

https://github.com/micropython/micropython-lib/blob/master/python-ecosys/urequests/urequests.py

[GM-Script-Writer-62850]

looking at the thread why do you assume this is a https connection wrap_socket would only be used if we were using https

the example code posted uses example.com, without a protocol specified

[emacstheviking]

I had failed to notice that. It's not my code that has the problem, but on closer inspection that's correct. In which case I apologise for wasting your time with it. Where does that memory get freed then ?

It is of course entirely possible that the code posted has other mismtached allocate/deallocate pairings the author is unaware of.

Thanks for your time, as an intermittent user of micropython myself, I greatly appreciate the amount of effort put into it.

[GM-Script-Writer-62850]

i found this from the same reddit thread, not saying this is not a issue, but it may not be the issue you are looking for, seems like something worth checking

[emacstheviking]

For many years I was an embedded systems designer / developer and memory allocation is one of those things that can bite you eventually in a long running process! All the best with it.

[GM-Script-Writer-62850]

turns out the OP was using https

[emacstheviking]

Wow, so I might have been right after all?! There's always a first time for everything. Can I get that in writing to show the wife next time we have an argument please.

[GM-Script-Writer-62850]

if you pay the postage ;) BTW it will look bad if you pay for international shipping just to prove to your wife you MAY have been right, i would suggest just letting her win it makes things easier

[emacstheviking]

if you pay the postage ;) BTW it will look bad if you pay for international shipping just to prove to your wife you MAY have been right, i would suggest just letting her win it makes things easier

Yeah but, being a software developer means I deal in logic and truth and false being recognised...sigh...yeah, after 30 years you'd think I'd have learned. :D

[GM-Script-Writer-62850]

So why close the report? we know OP used https now

could email you a pdf

here is your boolean logic:

if correct or wife_said_so and not finical_ruin: 
  print("OK")
elif wife_said_so:
  print("I see your point that this is a bad idea cause of",X)
else:
  print("your wrong")

[emacstheviking]

So why close the report? we know OP used https now

could email you a pdf

here is your boolean logic:

if correct or wife_said_so and not finical_ruin: 
  print("OK")
elif wife_said_so:
  print("I see your point that this is a bad idea cause of",X)
else:
  print("your wrong")

Didn't realise I'd closed it, my bad, I mean why destroy valuable ammo for the future ?

[ned-pcs]

The SSL socket's allocated memory is freed after the stream is closed (via an ioctl), then the ioctl is passed to the wrapped socket, which also frees its allocated memory.

See modssl_axtls.c, modssl_mbedtls.c and modsocket.c for details.

[matssigge]

I don't know if this is the same issue, but I'm getting a very clear memory leak when connecting via HTTPS and the server returns an invalid status line. I'm seeing the exception thrown on line 153 of python-ecosys/requests/requests/init.py - HTTP error: BadStatusLine:\n[]. I'm losing about 25k each time this happens. There is no memory leak when using HTTP.

I'm running on a Pimoroni Badger 2040W, i.e. a Raspberry Pi Pico. The code below reproduces the problem very simply when run on the Badger. I've used it connected via cable and run through Thonny, so I can see the console output.

# memory_leak.py
import time
import network
from urequests import get
import gc

URL = "https://<dev-machine-ip>:8443/broken"
REQUEST_TIMEOUT = 10
SSID = "<ssid>"
PSK = "<wifi-password>"


def connect(ssid, psk):
    print(f"Trying to connect to {SSID}...")

    wlan = network.WLAN(network.STA_IF)
    wlan.active(True)
    wlan.connect(ssid, psk)

    retries = 20
    while retries > 0:
        if wlan.isconnected():
            ip = wlan.ifconfig()[0]
            print(f"Connected to {SSID} with IP {ip}!")
            return ip
        print(f"Waiting for connection... ({retries} retries left)")
        time.sleep(1)
        retries -= 1
    return None


def request():
    response = None
    try:
        print("GET ", URL)
        response = get(URL, headers={"User-Agent": "Badger2040"}, timeout=REQUEST_TIMEOUT)
        print("Response status code:", response.status_code)
        print("Response content:", response.text)
    except Exception as e:
        print("An error occurred:", e)
    finally:
        if response:
            response.close()


def main():
    if not connect(SSID, PSK):
        print("No network connection. Bailing!")
        exit(1)

    while True:
        request()

        free_before = gc.mem_free()
        gc.collect()
        free_after = gc.mem_free()
        print(f"GC: {free_before} -> {free_after} bytes free")

        time.sleep(1)


main()

For the broken server, I ran the following:

# main.py
import http.server
import ssl
import socketserver
import os

# --- Configuration ---
PORT = 8443
CERT_FILE = "server.crt"  # Path to your SSL certificate file
KEY_FILE = "server.key"   # Path to your SSL private key file
BROKEN_PATH = "/broken"   # The endpoint that will return a broken response

# --- Custom Request Handler ---
class BrokenHandler(http.server.SimpleHTTPRequestHandler):
    def do_GET(self):
        if self.path == BROKEN_PATH:
            print(f"[{self.client_address[0]}:{self.client_address[1]}] "
                  f"Serving intentionally broken response for {self.path}")
            # Intentionally send a blank line instead of a proper status line
            # This will likely cause the client to hang or error out.
            self.wfile.write(b"\r\n") # Just a blank line
        else:
            print(f"[{self.client_address[0]}:{self.client_address[1]}] "
                  f"Serving normal response for {self.path}")
            super().do_GET()

def run_server():
    if not os.path.exists(CERT_FILE) or not os.path.exists(KEY_FILE):
        print(f"Error: SSL certificate '{CERT_FILE}' or key '{KEY_FILE}' not found.")
        print("Please generate them using the instructions below before running the server.")
        return

    with socketserver.TCPServer(("", PORT), BrokenHandler) as httpd:
        print(f"Serving HTTPS on port {PORT}")
        print(f"Normal content served from: https://localhost:{PORT}/")
        print(f"Broken endpoint: https://localhost:{PORT}{BROKEN_PATH}")
        print(f"To stop the server, press Ctrl+C")

        try:
            httpd.socket = ssl.wrap_socket(
                httpd.socket,
                keyfile=KEY_FILE,
                certfile=CERT_FILE,
                server_side=True
            )
            httpd.serve_forever()
        except ssl.SSLError as e:
            print(f"SSL Error: {e}")
            print("Ensure your certificate and key files are valid and correctly formatted.")
        except Exception as e:
            print(f"An unexpected error occurred: {e}")

if __name__ == "__main__":
    run_server()

Before running, I ran this in the same folder as main.py:

openssl req -x509 -newkey rsa:2048 -nodes -keyout server.key -out server.crt -days 365 -subj "/CN=localhost"

I will report this to the Pimoroni folks as well.