zone loading over HTTP appears to have buffer issues

[fhriley]

I have several ad block lists configured in unbound as RPZs loading over HTTP. I am having issues where unbound randomly thinks the downloaded data is corrupt. I am including a test case. There is a simple server serving the RPZ files and a client script that downloads the files and verifies the downloaded data matches the rpz files. Then I run unbound with that server and get errors like this:

[1608142238] unbound[294254:0] notice: init module 0: respip
[1608142238] unbound[294254:0] notice: init module 1: validator
[1608142238] unbound[294254:0] notice: init module 2: iterator
[1608142238] unbound[294254:0] info: start of service (unbound 1.13.0).
[1608142238] unbound[294254:0] error: 127.0.0.1/216a6303ecf3f9a3891e7f2fc4002f55.rpz parse failure RR[20]: Syntax error, could not parse the RR's TTL in 'great-hookuponline2.'
[1608142238] unbound[294254:0] error: 127.0.0.1/47d30336688a9bec9703650fbe9556ca.rpz parse failure RR[26]: Syntax error, could not parse the RR's type in 'searchresultsguide.com CNA'
[1608142238] unbound[294254:0] error: 127.0.0.1/0b1b956e3bd1438e6b20504044bcb868.rpz parse failure RR[26]: Syntax error, could not parse the RR's TTL in 'googleplaygiftcardcodegene'
[1608142238] unbound[294254:0] error: http download 127.0.0.1/73a8892eeac8dcf9941d8bf912a1844e.rpz does not contain a zonefile, but got ''
[1608142238] unbound[294254:0] error: 127.0.0.1/35a07c327a1291d2cb1606a00547f392.rpz parse failure RR[14]: Syntax error, could not parse the RR's TTL in 'aksdk-images.a'
[1608142238] unbound[294254:0] error: 127.0.0.1/3cc28501fe335c8f7a4f7943a73de792.rpz parse failure RR[15]: Syntax error, could not parse the RR's TTL in 'zzxlukodff.com '
[1608142238] unbound[294254:0] error: 127.0.0.1/70ae2b9395370fa89788a4a0520cbd1a.rpz parse failure RR[15]: Syntax error, could not parse the RR's TTL in 'zzxlukodff.com '
[1608142238] unbound[294254:0] error: 127.0.0.1/bdf5dd5f2a86ddb2cb238d6f50f4112f.rpz parse failure RR[5]: Syntax error, could not parse the RR's TTL in 'ads.c'

Note that if I use a very small chunk size in the server, such as 128, unbound loads the RPZs properly. Once you start increasing the chunk size, unbound gets progressively worse at loading the RPZs.

Build the test case:

unzip unbound_bug.zip
cd unbound_bug
make build

Run the server:

make server

In another shell, run the client to verify the server is working:

make client

Run unbound and it will print errors while loading the RPZs:

make unbound

unbound_bug.zip

[wcawijngaards]

Thanks for the detailed reproducer case. Fixed the issue it was a failure in the http (not https) chunked reader where it would lose the boundary between http chunks, causing the data to get wrongly read in.