program exited with exit code -11

[maartenvd]

I'm using this "proxy" http://pastebin.com/c1ssUZ3y and it appears to work fine until suddenly I get an "program exited with exit code -11" for no apparent reason (arch 64 bit)

[s-ludwig]

I probably got it reproduced: an InvalidMemoryOperationError caused by a HTTPClientResponse.~this(). The problem is that the body of the response was not read and something triggers an allocation when doing so its finalizer. That allocation must definitely be removed or replaced by an assertion to indicate a programming error (although I'd much rather see that the GC gets enhanced to be reentrant instead of throwing InvalidMemoryOperationErrors around), but two possibilities to fix this are:

• Add a scope(exit) cres.dropBody(); after the requestHTTP call
• Use the overload of requestHTTP which takes two callbacks - I would recommend this as it also avoids some memory allocations

[s-ludwig]

Actually there is already an assertion that should trigger. So maybe that is the cause for the memory allocation actually. Unfortunately the call stack is useless (at least on Windows/VisualD) so it's hard to tell.

[maartenvd]

Have you managed to get it to reliably crash? because over here it appears
to happen at quite random times.

2013/6/19 Sönke Ludwig notifications@github.com

Actually there is already an assertion that should trigger. So maybe that
is the cause for the memory allocation actually. Unfortunately the call
stack is useless (at least on Windows/VisualD) so it's hard to tell.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/243#issuecomment-19675380
.

[s-ludwig]

It took some requests and was not really deterministic, but it happened pretty consistently.

[maartenvd]

I quickly changed to scope(exit) (will use the second suggestion, only wanted to briefly test).

After surfing for 5 minutes without crashing, it does appear to work quite nicely.
Something I'm getting "operating on closed tcp connection ..." but everything (except a few cases) are getting rendered properly (and those cases can very well be cached failures from previous attempts).

So, is the conclusion that D's gc is failing? Or was I supposed to put a scope(exit)?

Thank you very much for taking your time to decipher my awful bug report (couldn't really simplify that much...)

[s-ludwig]

I'd say due to the GC it is a programming error (I'll need to document that though), but the GC makes graceful handling more or less impossible. However, apart from the crash, there is another logic issue. That is, not reading the body leaves the HTTP connection open indefinitely and will cause a connection timeout at some point. So even with a more powerful GC the response needs to be drained explicitly, because the GC may not run in a timely manner. The second form of requestHTTP takes care of this automatically.

I still noticed issues on some sites (including http://vibed.org, which caused a path with two consecutive slashes). I don't know all the cases off the top of my head, but, even though the code looks simple, I know I had to make a lot of subtle changes in the reverse proxy until it handled "everything" right. A lot can go wrong there if one of the critical headers is off (the protocol version, "Content-Length", "Content-Encoding", "Transfer-Encoding" or "Connection"). For each case I would recommend to use Wireshark or similar (regarded as a "hacker tool" in Germany that can get you for one year in jail o_O) and check those headers in detail.

[mihails-strasuns]

For each case I would recommend to use Wireshark or similar (regarded as a "hacker tool" in Germany > that can get you for one year in jail o_O)

WAT? Have just moved to Germany, something I should know about it? X_x

[s-ludwig]

The slightly good news about it is that I don't know of any case where someone has actually been prosecuted for this, so it is supposedly safe to use such tools for non-criminal activities. It became widely known as the "hacker paragraph" some time ago. The bad thing about it is that it is very vague in the way it is layed out, so that it is always subject to interpretation. What really bothers me is to see how the number of such laws has grown recently and althougth they are applied liberally now, nothing says that this won't change when it is convenient for some reason at some point. But anyway... back to work I guess :)