I did this
I observed that the Alt-Svc cache in libcurl allows for persistent, unbounded memory accumulation because it lacks a global limit on the number of entries and employs a very narrow eviction policy.
To reproduce this, I generated an Alt-Svc cache file with 50,000,000 unique source origins:
# gen_altsvc.py
with open("large_altsvc.cache", "w") as f:
f.write("# Alt-Svc cache\n")
for i in range(50000000):
# Unique source host for every entry
f.write(f'h1 step{i}.com 80 h2 alt{i}.com 443 "20300101 00:00:00" 0 0\n')
Then I ran curl using this cache:
/usr/bin/time -v curl -sL --alt-svc large_altsvc.cache https://google.com -o /dev/null
Observation:
- Memory Usage: Memory usage scales linearly with the number of entries in the file.
- Narrow Flush: In
lib/altsvc.c:altsvc_flush(), entries are only evicted if the ALPN ID, hostname, and port match exactly. Consequently, a series of redirects can populate the cache with thousands of entries for different ports or protocols on the same host, or across many different hosts, and they will never be pruned unless they expire and are specifically looked up.
- Unbounded Load:
altsvc_load() reads every valid line from the cache file into the in-memory linked list (asi->list) without any global cap.
I expected the following
I expected libcurl to enforce a reasonable global limit on the number of Alt-Svc entries maintained in memory and stored on disk (e.g., 1000 or 5000 entries).
When the cache reaches this limit:
altsvc_load() should stop importing new entries.Curl_altsvc_parse() should implement an LRU (Least Recently Used) or similar eviction strategy to prevent the list from growing indefinitely when encountering new origins.
While Alt-Svc is intended to be persistent, the current implementation allows a malformed or excessively large cache file to become a "slow-burn" resource exhaustion vector, significantly increasing the memory footprint of any application using CURLOPT_ALTSVC.
curl/libcurl version
curl 8.15.0
curl 8.20.0-DEV
operating system
Fedora Linux 43
I did this
I observed that the Alt-Svc cache in libcurl allows for persistent, unbounded memory accumulation because it lacks a global limit on the number of entries and employs a very narrow eviction policy.
To reproduce this, I generated an Alt-Svc cache file with 50,000,000 unique source origins:
Then I ran curl using this cache:
/usr/bin/time -v curl -sL --alt-svc large_altsvc.cache https://google.com -o /dev/nullObservation:
lib/altsvc.c:altsvc_flush(), entries are only evicted if the ALPN ID, hostname, and port match exactly. Consequently, a series of redirects can populate the cache with thousands of entries for different ports or protocols on the same host, or across many different hosts, and they will never be pruned unless they expire and are specifically looked up.altsvc_load()reads every valid line from the cache file into the in-memory linked list (asi->list) without any global cap.I expected the following
I expected libcurl to enforce a reasonable global limit on the number of Alt-Svc entries maintained in memory and stored on disk (e.g., 1000 or 5000 entries).
When the cache reaches this limit:
altsvc_load()should stop importing new entries.Curl_altsvc_parse()should implement an LRU (Least Recently Used) or similar eviction strategy to prevent the list from growing indefinitely when encountering new origins.While Alt-Svc is intended to be persistent, the current implementation allows a malformed or excessively large cache file to become a "slow-burn" resource exhaustion vector, significantly increasing the memory footprint of any application using CURLOPT_ALTSVC.
curl/libcurl version
curl 8.15.0
curl 8.20.0-DEV
operating system
Fedora Linux 43