Significant slowdown with large payloads

[saviorand]

Issue raised by the author of https://github.com/alainrollejr/mocodes

Currently, the performance rapidly deteriorates with the increase in packet size and request/response payloads. Compared to Python frameworks like Flask, Lightbug seems to become slower the larger the size of content that's being sent over the network.

Describe the solution you'd like
Optimize performance by removing unnecessary allocations and string to byte conversions. Benchmark to make sure we can see the improvements

[alainrollejr]

I wrote the following simple test client for this purpose:

import requests
import time

npacket = 1000 # nr of packets to send in for loop



# URL of the server
url = "http://0.0.0.0:8080"

# Send the data as a POST request to the server
# response = requests.post(url, data=data)
headers = {'Content-Type': 'application/octet-stream'}

nbyte = 100 

for i in range(4):

    nbyte = 10*nbyte
    data = bytes([0x0A] * nbyte)


    tic = time.perf_counter()
    for i in range(npacket):
        response = requests.post(url, data=data, headers=headers)
        try:
            # Get the response body as bytes
            response_bytes = response.content

        except Exception as e:
            print("Error parsing server response:", e)

    toc = time.perf_counter()

    dt = toc-tic
    packet_rate = npacket/dt
    bit_rate = packet_rate*nbyte*8

    print("=======================")
    print(f"packet size {nbyte} Bytes:")
    print("=========================")
    print(f"Sent and received {npacket} packets in {toc - tic:0.4f} seconds")
    print(f"Packet rate {packet_rate/1000:.2f} kilo packets/s")
    print(f"Bit rate {bit_rate/1e6:.1f}  Mbps")

I also created a reference test server for comparison with lightbug:

from fastapi import FastAPI, Request
from fastapi.responses import Response

app = FastAPI()

@app.post("/")
async def echo(request: Request):
    data = await request.body()
    return Response(content=data, media_type="application/octet-stream")

if __name__ == "__main__":
    import uvicorn
    uvicorn.run(app, host="0.0.0.0", port=8080)

Finally the Mojo test server code making use of lightbug is the following:

from lightbug_http import *

from sys import is_defined

@value
struct MyPrinter(HTTPService):
    fn func(self, req: HTTPRequest) raises -> HTTPResponse:
        var body = req.body_raw

        return OK(body)

fn main() raises:
    var server = SysServer()
    var handler = MyPrinter()
    server.listen_and_serve("0.0.0.0:8080", handler)

[alainrollejr]

with the above test client and server I get the following output on my system:

=======================
packet size 1000 Bytes:
=========================
Sent and received 1000 packets in 0.5271 seconds
Packet rate 1.9 kilo packets/s
Bit rate 15.2  Mbps
=======================
packet size 10000 Bytes:
=========================
Sent and received 1000 packets in 0.5160 seconds
Packet rate 1.9 kilo packets/s
Bit rate 155.1  Mbps
=======================
packet size 100000 Bytes:
=========================
Sent and received 1000 packets in 0.5882 seconds
Packet rate 1.7 kilo packets/s
Bit rate 1360.0  Mbps
=======================
packet size 1000000 Bytes:
=========================
Sent and received 1000 packets in 1.3013 seconds
Packet rate 0.8 kilo packets/s
Bit rate 6147.8  Mbps

Whereas with the current version of the lightbug server I get:

=======================
packet size 1000 Bytes:
=========================
Sent and received 1000 packets in 0.5089 seconds
Packet rate 1.96 kilo packets/s
Bit rate 15.7  Mbps
=======================
packet size 10000 Bytes:
=========================
Sent and received 1000 packets in 0.6919 seconds
Packet rate 1.45 kilo packets/s
Bit rate 115.6  Mbps
=======================
packet size 100000 Bytes:
=========================
Sent and received 1000 packets in 3.1594 seconds
Packet rate 0.32 kilo packets/s
Bit rate 253.2  Mbps
=======================
packet size 1000000 Bytes:
=========================
Sent and received 1000 packets in 56.5228 seconds
Packet rate 0.02 kilo packets/s
Bit rate 141.5  Mbps

Obviously the absolute numbers don't really matter but one can see that the packet rate with lightbug immediately drops as packet size increases by factor 10, whereas with the fastAPI we are able to maintain more or less the packet rate except for the largest packet size at the end of the test

[saviorand]

@alainrollejr the issue with a mismatch in content-length and the content received should be fixed now. About the crashing issue, I think this happens now on large payloads because the logic we use for parsing request first line, headers and body does not account for that. See here the exact part in the code that causes this: #50 (comment)
Basically currently we make several calls to read(), each one receives to a fixed size buffer. If you increase this buffer the crash seems to be fixed. So the issue is probably with trying to find a request line /headers in the wrong part of the payload (e.g. in the part of the body that only contains the zero bytes). So this needs to be refactored. Would appreciate any help!

[saviorand]

Looks better now, somewhat slower on small payloads but better on large payloads:

=======================
packet size 1280 Bytes:
=========================
Sent and received 1000 packets in 1.3073 seconds
Packet rate 0.76 kilo packets/s
Bit rate 7.8  Mbps
=======================
packet size 12800 Bytes:
=========================
Sent and received 1000 packets in 1.6074 seconds
Packet rate 0.62 kilo packets/s
Bit rate 63.7  Mbps
=======================
packet size 128000 Bytes:
=========================
Sent and received 1000 packets in 1.5311 seconds
Packet rate 0.65 kilo packets/s
Bit rate 668.8  Mbps
=======================
packet size 1280000 Bytes:
=========================
Sent and received 1000 packets in 1.5267 seconds
Packet rate 0.66 kilo packets/s
Bit rate 6707.3  Mbps

Or using SysServer(tcp_keep_alive=True) and your script:

=======================
packet size 1000 Bytes:
=========================
Sent and received 1000 packets in 0.8721 seconds
Packet rate 1.15 kilo packets/s
Bit rate 9.2  Mbps
=======================
packet size 10000 Bytes:
=========================
Sent and received 1000 packets in 1.2002 seconds
Packet rate 0.83 kilo packets/s
Bit rate 66.7  Mbps
=======================
packet size 100000 Bytes:
=========================
Sent and received 1000 packets in 1.1176 seconds
Packet rate 0.89 kilo packets/s
Bit rate 715.8  Mbps

I'm using your handler and this adjusted Python script:

import requests
import time

npacket = 1000 # nr of packets to send in for loop

# URL of the server
url = "http://localhost:8080"

# Send the data as a POST request to the server
# response = requests.post(url, data=data)
headers = {'Content-Type': 'application/octet-stream'}

nbyte = 128

# for i in range(4):
for i in range(4):
    nbyte = 10*nbyte
    data = bytes([0x0A] * nbyte)


    tic = time.perf_counter()
    for i in range(npacket):
        # print( f"packet {i}")
        response = requests.post(url, data=data, headers=headers)
        try:
            # Get the response body as bytes
            response_bytes = response.content

        except Exception as e:
            print("Error parsing server response:", e)

    toc = time.perf_counter()

    dt = toc-tic
    packet_rate = npacket/dt
    bit_rate = packet_rate*nbyte*8

    print("=======================")
    print(f"packet size {nbyte} Bytes:")
    print("=========================")
    print(f"Sent and received {npacket} packets in {toc - tic:0.4f} seconds")
    print(f"Packet rate {packet_rate/1000:.2f} kilo packets/s")
    print(f"Bit rate {bit_rate/1e6:.1f}  Mbps")