New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"Reached EOF before reaching end marker" error #744

Closed
SerialVelocity opened this Issue Jul 28, 2014 · 6 comments

Comments

Projects
None yet
4 participants
@SerialVelocity

SerialVelocity commented Jul 28, 2014

Hey,

So I'm trying to read data from server and I end up getting this error (it works fine with curl though):

object.Exception@../../../../.dub/packages/vibe-d-master/source/vibe/stream/operations.d(181): Reached EOF before reaching end marker.
----------------
5   replay-downloader                   0x000000010eb8e93f pure @safe bool std.exception.enforce!(bool).enforce(bool, lazy const(char)[], immutable(char)[], ulong) + 107
6   replay-downloader                   0x000000010ec9d9b0 void vibe.stream.operations.readUntil(vibe.core.stream.InputStream, vibe.core.stream.OutputStream, const(ubyte[]), ulong) + 1648
7   replay-downloader                   0x000000010ec9d2c8 ubyte[] vibe.stream.operations.readUntil(vibe.core.stream.InputStream, const(ubyte[]), ulong, shared(vibe.utils.memory.Allocator)) + 188
8   replay-downloader                   0x000000010ec9d208 ubyte[] vibe.stream.operations.readLine(vibe.core.stream.InputStream, ulong, immutable(char)[], shared(vibe.utils.memory.Allocator)) + 56
9   replay-downloader                   0x000000010ec7eebd vibe.http.client.HTTPClientResponse vibe.http.client.HTTPClientResponse.__ctor(vibe.http.client.HTTPClient, bool, bool, shared(vibe.utils.memory.Allocator)) + 129
10  replay-downloader                   0x000000010ec7da68 vibe.http.client.HTTPClientResponse vibe.http.client.HTTPClient.request(scope void delegate(vibe.http.client.HTTPClientRequest)) + 200
11  replay-downloader                   0x000000010ec7ccf9 vibe.http.client.HTTPClientResponse vibe.http.client.requestHTTP(vibe.inet.url.URL, scope void delegate(scope vibe.http.client.HTTPClientRequest)) + 537
12  replay-downloader                   0x000000010ec7cad4 vibe.http.client.HTTPClientResponse vibe.http.client.requestHTTP(immutable(char)[], scope void delegate(scope vibe.http.client.HTTPClientRequest)) + 100
13  replay-downloader                   0x000000010eb87c0c ubyte[] api.callApi(immutable(char)[]) + 144
14  replay-downloader                   0x000000010eb87ed5 lastchunkinfo.LastChunkInfo api.getLastChunkInfo(immutable(char)[], immutable(char)[], immutable(char)[]) + 113
15  replay-downloader                   0x000000010eb875d5 _Dmain + 1349
16  replay-downloader                   0x000000010ece7571 void rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).runAll().void __lambda1() + 33
17  replay-downloader                   0x000000010ece74bd void rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).tryExec(scope void delegate()) + 45
18  replay-downloader                   0x000000010ece751d void rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).runAll() + 45
19  replay-downloader                   0x000000010ece74bd void rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).tryExec(scope void delegate()) + 45
20  replay-downloader                   0x000000010ece7439 _d_run_main + 449
21  replay-downloader                   0x000000010eb879fa main + 34
22  libdyld.dylib                       0x00007fff93e4a5fd start + 1
23  ???                                 0x0000000000000004 0x0 + 4
Error executing command run: Program exited with code 1

It's kinda hard to reproduce without the full program though. Is there any reason why this would happen?

@SerialVelocity

This comment has been minimized.

Show comment
Hide comment
@SerialVelocity

SerialVelocity Jul 28, 2014

And even then, if you have the full program, you need to parse batch files for input, etc so it's hard to show you the code in action.

SerialVelocity commented Jul 28, 2014

And even then, if you have the full program, you need to parse batch files for input, etc so it's hard to show you the code in action.

@s-ludwig

This comment has been minimized.

Show comment
Hide comment
@s-ludwig

s-ludwig Jul 28, 2014

Member

Does it happen consistently? It looks like the connection gets closed before the server has sent a response line (the first one, like "HTTP/1.1 200 OK\r\n"). The question is just what happened before that. Some possibilities:

  • Something on the client side took too long, so that either the keep-alive time, or the max. request time exceeds and the server shuts down the connection
  • There was some kind of protocol mismatch between client and server, so that the server still thinks that the request is going to be written and eventually times out before writing the response

It would be interesting to see what is in the log for LogLevel.debug_ (with timestamps and thread/fiber id enabled) and the particular failing connection.

Member

s-ludwig commented Jul 28, 2014

Does it happen consistently? It looks like the connection gets closed before the server has sent a response line (the first one, like "HTTP/1.1 200 OK\r\n"). The question is just what happened before that. Some possibilities:

  • Something on the client side took too long, so that either the keep-alive time, or the max. request time exceeds and the server shuts down the connection
  • There was some kind of protocol mismatch between client and server, so that the server still thinks that the request is going to be written and eventually times out before writing the response

It would be interesting to see what is in the log for LogLevel.debug_ (with timestamps and thread/fiber id enabled) and the particular failing connection.

@s-ludwig

This comment has been minimized.

Show comment
Hide comment
Member

s-ludwig commented Jul 28, 2014

@SerialVelocity

This comment has been minimized.

Show comment
Hide comment
@SerialVelocity

SerialVelocity Jul 28, 2014

Well, I only ever have one connection open to the server at any point in time. The server has no such limiting either...
I think it's because the server hasn't fully created its reply in time (it's for a game so the data is still coming in). Is there a parameter I can boost to stop it timing out so quickly?

SerialVelocity commented Jul 28, 2014

Well, I only ever have one connection open to the server at any point in time. The server has no such limiting either...
I think it's because the server hasn't fully created its reply in time (it's for a game so the data is still coming in). Is there a parameter I can boost to stop it timing out so quickly?

@etcimon

This comment has been minimized.

Show comment
Hide comment
@etcimon

etcimon Jul 28, 2014

Contributor

Is there a parameter I can boost to stop it timing out so quickly?

On Git master you'll find a new HTTPClientSettings object you can pass to requestHTTP with a custom keepalive timeout setting.

Contributor

etcimon commented Jul 28, 2014

Is there a parameter I can boost to stop it timing out so quickly?

On Git master you'll find a new HTTPClientSettings object you can pass to requestHTTP with a custom keepalive timeout setting.

etcimon added a commit to etcimon/vibe.d that referenced this issue Jul 30, 2014

etcimon added a commit to etcimon/vibe.d that referenced this issue Jul 30, 2014

Add default keepAliveLimit - fixes #744
Duration must be at the left of SysTime in an addition

Refactored and fixed logical errors

Fixed major issues and refactored some more

etcimon added a commit to etcimon/vibe.d that referenced this issue Jul 30, 2014

Add default keepAliveLimit - fixes #744
Duration must be at the left of SysTime in an addition

Refactored and fixed logical errors

Fixed major issues and refactored some more

Check if connected before checking if keep-alive limit is expired

etcimon added a commit to etcimon/vibe.d that referenced this issue Aug 3, 2014

Add default keepAliveLimit - fixes #744
Duration must be at the left of SysTime in an addition

Refactored and fixed logical errors

Fixed major issues and refactored some more

Check if connected before checking if keep-alive limit is expired

fix bad server_timeout / ignores keep-alive

A keep-alive of 0 means no keep-alive

Add better documentation for HTTPClientSettings, unittests not possible without a stable proxy online

etcimon added a commit to etcimon/vibe.d that referenced this issue Aug 13, 2014

Add default keepAliveLimit - fixes #744
Duration must be at the left of SysTime in an addition

Refactored and fixed logical errors

Fixed major issues and refactored some more

Check if connected before checking if keep-alive limit is expired

fix bad server_timeout / ignores keep-alive

A keep-alive of 0 means no keep-alive

Add better documentation for HTTPClientSettings, unittests not possible without a stable proxy online

Fix summary for settings
@Geod24

This comment has been minimized.

Show comment
Hide comment
@Geod24

Geod24 Aug 30, 2014

Contributor

I also get this one quite frequently. I was not able to reproduce it consistently.

@s-ludwig : This is what happen with LogLevel.debug_:

[3]> list-vms
Socket event on fd 9: 17 (15E44E8 vs 15E44E8)
Connection was closed (fd 9).
object.Exception@../../../../../home/geod24/.dub/packages/vibe-d-0.7.21-alpha.4/source/vibe/stream/operations.d(202): Reached EOF before reaching end marker.
----------------
./auth-boapnet(pure @safe bool std.exception.enforce!(bool).enforce(bool, lazy const(char)[], immutable(char)[], ulong)+0x6b) [0x628db3]
./auth-boapnet(void vibe.stream.operations.readUntil!(vibe.stream.wrapper.StreamOutputRange).readUntil(vibe.core.stream.InputStream, ref vibe.stream.wrapper.StreamOutputRange, const(ubyte[]), ulong)+0x6be) [0x65dd2e]

(Note: It's inside a console, "[3]> list-vms" is a command sending a GET).
Also you might find the output of wireshark interesting: http://pastebin.com/nHgFp3En
The first query (POST /BoapNetAdm-web/manager/keys/, which returns 200) is a login request, and IIRC I've never hit this error on the first query.

Contributor

Geod24 commented Aug 30, 2014

I also get this one quite frequently. I was not able to reproduce it consistently.

@s-ludwig : This is what happen with LogLevel.debug_:

[3]> list-vms
Socket event on fd 9: 17 (15E44E8 vs 15E44E8)
Connection was closed (fd 9).
object.Exception@../../../../../home/geod24/.dub/packages/vibe-d-0.7.21-alpha.4/source/vibe/stream/operations.d(202): Reached EOF before reaching end marker.
----------------
./auth-boapnet(pure @safe bool std.exception.enforce!(bool).enforce(bool, lazy const(char)[], immutable(char)[], ulong)+0x6b) [0x628db3]
./auth-boapnet(void vibe.stream.operations.readUntil!(vibe.stream.wrapper.StreamOutputRange).readUntil(vibe.core.stream.InputStream, ref vibe.stream.wrapper.StreamOutputRange, const(ubyte[]), ulong)+0x6be) [0x65dd2e]

(Note: It's inside a console, "[3]> list-vms" is a command sending a GET).
Also you might find the output of wireshark interesting: http://pastebin.com/nHgFp3En
The first query (POST /BoapNetAdm-web/manager/keys/, which returns 200) is a login request, and IIRC I've never hit this error on the first query.

@s-ludwig s-ludwig closed this in bef592d Sep 3, 2014

marcioapm added a commit to marcioapm/vibe.d that referenced this issue Sep 4, 2014

Add default keepAliveLimit - fixes #744
Duration must be at the left of SysTime in an addition

Refactored and fixed logical errors

Fixed major issues and refactored some more

Check if connected before checking if keep-alive limit is expired

fix bad server_timeout / ignores keep-alive

A keep-alive of 0 means no keep-alive

Add better documentation for HTTPClientSettings, unittests not possible without a stable proxy online

Fix summary for settings

Changed test to unittest

Change variable name for max keep-alive requests
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment