-
Notifications
You must be signed in to change notification settings - Fork 374
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
WS_Reset() and overflown workspaces #3194
Labels
Comments
this patch to
result:
|
FTR, I just noticed that we already had a similar issue in #2645 |
nigoroll
added a commit
to nigoroll/varnish-cache
that referenced
this issue
Jan 20, 2020
Since 8baf4a6 we lost information about an overflowed workspace by calling std.log(), which resulted from use of the workspace for construction of a contiguous string from the argument constituents. Since then, we have changed the interface to STRANDS, but this issue remained. We now solve the case for real by pushing the string concatenation down to VSL: New versions of VSL and VSLb (coded by example of VSLv() and VSLbt()) take a strands argument and create a log record without additional copy overhead. These solve varnishcache#3194 for std.log(), make logging more efficient and, in particular, allow use of std.log() in low workspace conditions (because they do not require any). Also improve test coverage for std.log() Ref varnishcache#3194
nigoroll
added
b=bug
c=varnishd
r=6.0
r=6.1
r=trunk
and removed
a=feedback please
labels
Jan 20, 2020
nigoroll
added a commit
to nigoroll/varnish-cache
that referenced
this issue
Jan 20, 2020
We add checks for cases where a WS_Reset() could reset the overflowed workspace marker to ensure that the workspace is not already overflowed before using it. See also varnishcache#3196 for std.log() This concludes the fix for varnishcache#3194 together with the above (except for one h2 case, maybe)
nigoroll
added a commit
to nigoroll/varnish-cache
that referenced
this issue
Jan 20, 2020
Since 8baf4a6 we lost information about an overflowed workspace by calling std.log(), which resulted from use of the workspace for construction of a contiguous string from the argument constituents. Since then, we have changed the interface to STRANDS, but this issue remained. We now solve the case for real by pushing the string concatenation down to VSL: New versions of VSL and VSLb (coded by example of VSLv() and VSLbt()) take a strands argument and create a log record without additional copy overhead. These solve varnishcache#3194 for std.log(), make logging more efficient and, in particular, allow use of std.log() in low workspace conditions (because they do not require any). Also improve test coverage for std.log() Ref varnishcache#3194
nigoroll
added a commit
to nigoroll/varnish-cache
that referenced
this issue
Jan 20, 2020
It now bails out due to a workspace overflow which it acually did trigger all the time, but that remained unnoticed due to varnishcache#3194 The test to multiply the cookie header 128 times (8x in the first regex, 16x in the second) was not realistic even for "modern times", so I do not think we should have workspaces sized by such an example.
nigoroll
added a commit
to nigoroll/varnish-cache
that referenced
this issue
Jan 27, 2020
Since 8baf4a6 we lost information about an overflowed workspace by calling std.log(), which resulted from use of the workspace for construction of a contiguous string from the argument constituents. Since then, we have changed the interface to STRANDS, but this issue remained. We now solve the case for real by pushing the string concatenation down to VSL: New versions of VSL and VSLb (coded by example of VSLv() and VSLbt()) take a strands argument and create a log record without additional copy overhead. These solve varnishcache#3194 for std.log(), make logging more efficient and, in particular, allow use of std.log() in low workspace conditions (because they do not require any). Also improve test coverage for std.log() Ref varnishcache#3194
nigoroll
added a commit
to nigoroll/varnish-cache
that referenced
this issue
Jan 27, 2020
It now bails out due to a workspace overflow which it acually did trigger all the time, but that remained unnoticed due to varnishcache#3194 The test to multiply the cookie header 128 times (8x in the first regex, 16x in the second) was not realistic even for "modern times", so I do not think we should have workspaces sized by such an example.
nigoroll
added a commit
to nigoroll/varnish-cache
that referenced
this issue
Jan 27, 2020
We add checks for cases where a WS_Reset() could reset the overflowed workspace marker to ensure that the workspace is not already overflowed before using it. See also varnishcache#3196 for std.log() This concludes the fix for varnishcache#3194 together with the above (except for one h2 case, maybe)
nigoroll
added a commit
that referenced
this issue
Jan 27, 2020
nigoroll
added a commit
to nigoroll/varnish-cache
that referenced
this issue
Jan 27, 2020
We use workspace overflows to signal to bail out for example after a failing `VRT_SetHdr()`. This is a guarantee that if some serious issue occurred during processing, we rather send an error downstream than an incomplete response or the result of incomplete processing. We use the `WS_Snapshot() ... WS_Reset()` pattern as some kind of second order workspace allocation where the called code itself uses `WS_Reserve()`. With this usage patter, `WS_Reset()` called `ws_ClearOverflow(ws)`, potentially clearing the overflow bit from a previous relevant failure. `WS_Snapshot()` now returns a magic value which gets recognized by `WS_Reset()` to ensure that the overflowed marker gets preserved or even restored. In this case, the workspace does not actually get reset. Fixes varnishcache#3194
nigoroll
added a commit
to nigoroll/varnish-cache
that referenced
this issue
Jan 27, 2020
We use workspace overflows to signal to bail out for example after a failing `VRT_SetHdr()`. This is a guarantee that if some serious issue occurred during processing, we rather send an error downstream than an incomplete response or the result of incomplete processing. We use the `WS_Snapshot() ... WS_Reset()` pattern as some kind of second order workspace allocation where the called code itself uses `WS_Reserve()`. With this usage pattern, `WS_Reset()` called `ws_ClearOverflow(ws)`, potentially clearing the overflow bit from a previous relevant failure. `WS_Snapshot()` now returns a magic value which gets recognized by `WS_Reset()` to ensure that the overflowed marker gets preserved or even restored. In this case, the workspace does not actually get reset. Fixes varnishcache#3194
nigoroll
added a commit
to nigoroll/varnish-cache
that referenced
this issue
Jan 27, 2020
We use workspace overflows to signal to bail out for example after a failing `VRT_SetHdr()`. This is a guarantee that if some serious issue occurred during processing, we rather send an error downstream than an incomplete response or the result of incomplete processing. We use the `WS_Snapshot() ... WS_Reset()` pattern as some kind of second order workspace allocation where the called code itself uses `WS_Reserve()`. With this usage pattern, `WS_Reset()` called `ws_ClearOverflow(ws)`, potentially clearing the overflow bit from a previous relevant failure. `WS_Snapshot()` now returns a magic value which gets recognized by `WS_Reset()` to ensure that the overflowed marker is still present. In this case, the workspace does not actually get reset. Fixes varnishcache#3194
nigoroll
added a commit
to nigoroll/varnish-cache
that referenced
this issue
Jan 27, 2020
We add checks for cases where a WS_Reset() could reset the overflowed workspace marker to ensure that the workspace is not already overflowed before using it. See also varnishcache#3196 for std.log() This concludes the fix for varnishcache#3194 together with the above (except for one h2 case, maybe)
nigoroll
added a commit
to nigoroll/varnish-cache
that referenced
this issue
Jan 30, 2020
We use workspace overflows to signal to bail out for example after a failing `VRT_SetHdr()`. This is a guarantee that if some serious issue occurred during processing, we rather send an error downstream than an incomplete response or the result of incomplete processing. We use the `WS_Snapshot() ... WS_Reset()` pattern as some kind of second order workspace allocation where the called code itself uses `WS_Reserve()`. With this usage pattern, `WS_Reset()` called `ws_ClearOverflow(ws)`, potentially clearing the overflow bit from a previous relevant failure. `WS_Snapshot()` now returns a magic value which gets recognized by `WS_Reset()` to ensure that the overflowed marker is still present. In this case, the workspace does not actually get reset. Fixes varnishcache#3194
nigoroll
added a commit
to nigoroll/varnish-cache
that referenced
this issue
Feb 3, 2020
We add checks for cases where a WS_Reset() could reset the overflowed workspace marker to ensure that the workspace is not already overflowed before using it. See also varnishcache#3196 for std.log() This concludes the fix for varnishcache#3194 together with the above (except for one h2 case, maybe)
nigoroll
added a commit
to nigoroll/varnish-cache
that referenced
this issue
Feb 27, 2020
We use workspace overflows to signal to bail out for example after a failing `VRT_SetHdr()`. This is a guarantee that if some serious issue occurred during processing, we rather send an error downstream than an incomplete response or the result of incomplete processing. We use the `WS_Snapshot() ... WS_Reset()` pattern as some kind of second order workspace allocation where the called code itself uses `WS_Reserve()`. With this usage pattern, `WS_Reset()` called `ws_ClearOverflow(ws)`, potentially clearing the overflow bit from a previous relevant failure. `WS_Snapshot()` now returns a magic value which gets recognized by `WS_Reset()` to ensure that the overflowed marker is still present. In this case, the workspace does not actually get reset. Fixes varnishcache#3194
nigoroll
added a commit
to nigoroll/varnish-cache
that referenced
this issue
Feb 27, 2020
This is an improvement over the second last commit (e.g. ed970de) based on feedback to varnishcache#3202: We now avoid any other unintended clears of the overflow bit by splitting two functions: * WS_Rollback() is now what WS_Reset() used to be: It clears overflows and accepts the zero cookie for a reset-to-start It is only intended for use within varnishd and is thus declared in cache_varnishd.h * WS_Reset() does not touch the overflow bit any longer, ensuring that a once-overflowed workspace stays overflowed With this in place, the magic snap_overflowed cookie would not be needed any more, but it still serves two good purposes: - better debugging and - a safety measure against passing a cookie from an already overflowed workspace to WS_Rollback() Fixes varnishcache#3194
nigoroll
added a commit
to nigoroll/varnish-cache
that referenced
this issue
Mar 2, 2020
We use workspace overflows to signal to bail out for example after a failing `VRT_SetHdr()`. This is a guarantee that if some serious issue occurred during processing, we rather send an error downstream than an incomplete response or the result of incomplete processing. We use the `WS_Snapshot() ... WS_Reset()` pattern as some kind of second order workspace allocation where the called code itself uses `WS_Reserve()`. With this usage pattern, `WS_Reset()` called `ws_ClearOverflow(ws)`, potentially clearing the overflow bit from a previous relevant failure. We now avoid any other unintended clears of the overflow bit by splitting two functions: * WS_Rollback() is now what WS_Reset() used to be: It clears overflows and accepts the zero cookie for a reset-to-start It is only intended for use within varnishd and is thus declared in cache_varnishd.h * WS_Reset() does not touch the overflow bit any longer, ensuring that a once-overflowed workspace stays overflowed `WS_Snapshot()` now returns a magic value which gets recognized by `WS_Reset()` to ensure that the overflowed marker is still present. This serves two purposes: - better debugging and - a safety measure against passing a cookie from an already overflowed workspace to WS_Rollback() Fixes varnishcache#3194
nigoroll
added a commit
to nigoroll/varnish-cache
that referenced
this issue
Mar 2, 2020
We use workspace overflows to signal to bail out for example after a failing `VRT_SetHdr()`. This is a guarantee that if some serious issue occurred during processing, we rather send an error downstream than an incomplete response or the result of incomplete processing. We use the `WS_Snapshot() ... WS_Reset()` pattern as some kind of second order workspace allocation where the called code itself uses `WS_Reserve()`. With this usage pattern, `WS_Reset()` called `ws_ClearOverflow(ws)`, potentially clearing the overflow bit from a previous relevant failure. We now avoid any other unintended clears of the overflow bit by splitting two functions: * WS_Rollback() is now what WS_Reset() used to be: It clears overflows and accepts the zero cookie for a reset-to-start It is only intended for use within varnishd and is thus declared in cache_varnishd.h * WS_Reset() does not touch the overflow bit any longer, ensuring that a once-overflowed workspace stays overflowed `WS_Snapshot()` now returns a magic value which gets recognized by `WS_Reset()` to ensure that the overflowed marker is still present. This serves two purposes: - better debugging and - a safety measure against passing a cookie from an already overflowed workspace to WS_Rollback() Fixes varnishcache#3194
dridi
pushed a commit
that referenced
this issue
Mar 3, 2020
We use workspace overflows to signal to bail out for example after a failing `VRT_SetHdr()`. This is a guarantee that if some serious issue occurred during processing, we rather send an error downstream than an incomplete response or the result of incomplete processing. We use the `WS_Snapshot() ... WS_Reset()` pattern as some kind of second order workspace allocation where the called code itself uses `WS_Reserve()`. With this usage pattern, `WS_Reset()` called `ws_ClearOverflow(ws)`, potentially clearing the overflow bit from a previous relevant failure. We now avoid any other unintended clears of the overflow bit by splitting two functions: * WS_Rollback() is now what WS_Reset() used to be: It clears overflows and accepts the zero cookie for a reset-to-start It is only intended for use within varnishd and is thus declared in cache_varnishd.h * WS_Reset() does not touch the overflow bit any longer, ensuring that a once-overflowed workspace stays overflowed `WS_Snapshot()` now returns a magic value which gets recognized by `WS_Reset()` to ensure that the overflowed marker is still present. This serves two purposes: - better debugging and - a safety measure against passing a cookie from an already overflowed workspace to WS_Rollback() Fixes #3194
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
noticed here: b12af68
We use workspace overflows to signal to bail out for example after a failing
VRT_SetHdr()
. This is a guarantee that if some serious issue occurred during processing, we rather send an error downstream than an incomplete response or the result of incomplete processing.We use the
WS_Snapshot() ... WS_Reset()
pattern as some kind of second order workspace allocation where the called code itself usesWS_Reserve()
.This combination is problematic because
WS_Reset()
callsws_ClearOverflow(ws)
, potentially clearing the overflow bit from a previous relevant failure.I think we should simply forbid
WS_Snapshot()
on overflown workspaces, mandating prior calls ofWS_Overflowed()
for the above mentioned pattern.Feedback welcome
The text was updated successfully, but these errors were encountered: