Analyzing HTTP Server overhead on Android/Termux vs Native Linux: 15% vs 10% performance drop

[sunhighy-oss]

Hi everyone,

I've been benchmarking the HTTP API overhead of local LLM servers across different environments using Gemma4-4B, and found a significant discrepancy in how the HTTP layer impacts performance on mobile vs desktop.

The Data:

• Native C++ execution: Mobile (4.6 t/s) vs Legacy x86 Linux (4.7 t/s).
• Via HTTP API: Mobile drops to 3.9 t/s (-15%), while x86 drops to 4.19 t/s (-10%).

My Analysis:
On a minimal Linux (XFCE) environment, the HTTP server (likely using httplib or similar) operates with near-zero overhead.
However, on Android (Termux), the HTTP request has to traverse:

1. Android's complex network stack.
2. JNI boundaries (if interacting with Java services).
3. Binder IPC overhead.
4. Sudden GC (Garbage Collection) pauses in the background, which interrupt the inference thread.

Discussion Points:

1. Has anyone measured the overhead of using Unix Domain Sockets instead of TCP/IP localhost for local API calls on Android? It might bypass the heavy network stack.
2. Would switching to gRPC or Protobuf instead of JSON/REST significantly reduce the serialization/deserialization CPU overhead on the ARM cores?

I actually produced an 8-minute deep-dive video analyzing this exact HTTP degradation, comparing the Android Termux stack vs bare-metal Linux. It includes system call traces and overhead pie charts. Happy to share it if the community finds this topic useful!