You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Compressor and decompressor window_bits set to 15 leads to periodically truncated data #32221
I'm trying to transparently compress response data between services on my (Istio) mesh.
To do this, I have a compressor filter on destination services and a decompressor on the source services.
I discovered that setting window_bits to 15 on both of them results in periodically truncated data. This script loops through the same call and you can see how the data was truncated for one of them:
This does not happen if i set the compressor window bits to 14 or lower.
Hey sorry for the delay, as i've been coming back to this in bits and bobs.
No errors in the envoy logs, everything looks "normal" at debug level.
I've tried all sorts of different configurations in case it was some other weird interaction with things we're doing and haven't found anything at all. The only thing that "fixes" it is a window_size of <= 14.
Not sure where else to go with it to help you debug!
Attached some debug logs from our non prod in case you can spot anything, first request worked, second was truncated. debug.txt
Just realised i was logging from the compressor side there, and it's the decompressor side that would likely show any errors. Will have another stab tomorrow.
This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or "no stalebot" or other activity occurs. Thank you for your contributions.
This issue has been automatically closed because it has not had activity in the last 37 days. If this issue is still valid, please ping a maintainer and ask them to label it as "help wanted" or "no stalebot". Thank you for your contributions.
@Stono we just ran into a problem that is similar to this. Most gzipped requests are fine, but then some are truncated. We have our window_bits set to 15 as well. Did you manage to grab those decompressor logs?
Strange... I'm also confused how this would happen. We applied the compressor globally using an EnvoyFilter and had no issue for hundreds of our endpoints, but one endpoint gave back truncated responses. Removing the filter solved the issue, but now we're back to uncompressed responses.
Activity
alyssawilk commentedon Feb 6, 2024
cc @KBaichoo @mattklein123
KBaichoo commentedon Feb 7, 2024
Hmm, from the https://www.zlib.net/manual.html it seems like those parameters should be ok.
Do you have any Envoy stats, or logs that occur on decompression showing any errors with your reproduction?
Stono commentedon Feb 11, 2024
Hey sorry for the delay, as i've been coming back to this in bits and bobs.
No errors in the envoy logs, everything looks "normal" at debug level.
I've tried all sorts of different configurations in case it was some other weird interaction with things we're doing and haven't found anything at all. The only thing that "fixes" it is a window_size of <= 14.
Not sure where else to go with it to help you debug!
Attached some debug logs from our non prod in case you can spot anything, first request worked, second was truncated.
debug.txt
Stono commentedon Feb 11, 2024
Just realised i was logging from the compressor side there, and it's the decompressor side that would likely show any errors. Will have another stab tomorrow.
github-actions commentedon Mar 12, 2024
This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or "no stalebot" or other activity occurs. Thank you for your contributions.
github-actions commentedon Mar 20, 2024
This issue has been automatically closed because it has not had activity in the last 37 days. If this issue is still valid, please ping a maintainer and ask them to label it as "help wanted" or "no stalebot". Thank you for your contributions.
jessebye commentedon Oct 30, 2024
@Stono we just ran into a problem that is similar to this. Most gzipped requests are fine, but then some are truncated. We have our
window_bitsset to 15 as well. Did you manage to grab those decompressor logs?Stono commentedon Oct 30, 2024
Nope, and I haven't been able to recreate it since!
jessebye commentedon Oct 30, 2024
Strange... I'm also confused how this would happen. We applied the compressor globally using an
EnvoyFilterand had no issue for hundreds of our endpoints, but one endpoint gave back truncated responses. Removing the filter solved the issue, but now we're back to uncompressed responses.