Thread 'main' panicked

[zepouet]

I open this issue with the current docker image. Maybe the problem is corrected outside docker.

Environment :

root@vertx:~/vertx-voting-app# docker version
Client:
 Version:      17.05.0-ce
 API version:  1.29
 Go version:   go1.7.5
 Git commit:   89658be
 Built:        Thu May  4 22:10:54 2017
 OS/Arch:      linux/amd64

Server:
 Version:      17.05.0-ce
 API version:  1.29 (minimum version 1.12)
 Go version:   go1.7.5
 Git commit:   89658be
 Built:        Thu May  4 22:10:54 2017
 OS/Arch:      linux/amd64
 Experimental: false

The Linux is Ubuntu 16.04

root@vertx:~/vertx-voting-app# uname -a
Linux vertx 4.4.0-79-generic #100-Ubuntu SMP Wed May 17 19:58:14 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

The application comes from https://github.com/Treeptik/vertx-voting-app

And the stacktrace:

sozu_1          | 2017-06-19T19:29:39Z	6122965723685	9	DEBUG	HTTP-0	sozu_lib::network::protocol::http	2f5e0dfd-6460-492e-9cb7-a8b6ecec1c89	backend-vote		BACK [36->20036]: wrote 15 bytes of 15
sozu_1          | thread 'main' panicked at 'investigating single buffer usage: the front->back buffer should not be used while parsing and forwarding the response', /sozu-source/lib/src/network/protocol/http.rs:611
sozu_1          | stack backtrace:
sozu_1          |    1:     0x5605bbed94dc - std::sys::imp::backtrace::tracing::imp::write::hf33ae72d0baa11ed
sozu_1          |                         at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:42
sozu_1          |    2:     0x5605bbedf2ce - std::panicking::default_hook::{{closure}}::h59672b733cc6a455
sozu_1          |                         at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/src/libstd/panicking.rs:351
sozu_1          |    3:     0x5605bbedeed4 - std::panicking::default_hook::h1670459d2f3f8843
sozu_1          |                         at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/src/libstd/panicking.rs:367
sozu_1          |    4:     0x5605bbedf76b - std::panicking::rust_panic_with_hook::hcf0ddb069e7beee7
sozu_1          |                         at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/src/libstd/panicking.rs:555
sozu_1          |    5:     0x5605bbca7a13 - std::panicking::begin_panic::hcee73e0b293b8c83
sozu_1          |                         at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/src/libstd/panicking.rs:517
sozu_1          |    6:     0x5605bbda0a0b - <sozu_lib::network::protocol::http::Http<Front>>::back_readable::h4a1246cabf7ecd88
sozu_1          |                         at /sozu-source/lib/src/network/protocol/http.rs:611
sozu_1          |    7:     0x5605bbdbefb3 - <sozu_lib::network::http::Client as sozu_lib::network::proxy::ProxyClient>::back_readable::h876c202785470845
sozu_1          |                         at /sozu-source/lib/src/network/http.rs:272
sozu_1          |    8:     0x5605bbdf8ff5 - <sozu_lib::network::proxy::Server<ServerConfiguration, Client>>::ready::ha28e851867ca024d
sozu_1          |                         at /sozu-source/lib/src/network/proxy.rs:532
sozu_1          |    9:     0x5605bbdf4fc1 - <sozu_lib::network::proxy::Server<ServerConfiguration, Client>>::run::h033dcfd844da99f2
sozu_1          |                         at /sozu-source/lib/src/network/proxy.rs:415
sozu_1          |   10:     0x5605bbdcd748 - sozu_lib::network::http::start::h5253c70194772b51
sozu_1          |                         at /sozu-source/lib/src/network/http.rs:708
sozu_1          |   11:     0x5605bb929174 - sozu::worker::begin_worker_process::hfe3f858642ef9a0f
sozu_1          |                         at /sozu-source/bin/src/worker.rs:117
sozu_1          |   12:     0x5605bb930d4d - sozu::main::h3c24609eeed730d5
sozu_1          |                         at /sozu-source/bin/src/main.rs:74
sozu_1          |   13:     0x5605bbee67da - __rust_maybe_catch_panic
sozu_1          |                         at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/src/libpanic_unwind/lib.rs:98
sozu_1          |   14:     0x5605bbedfed6 - std::rt::lang_start::hd7c880a37a646e81
sozu_1          |                         at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/src/libstd/panicking.rs:436
sozu_1          |                         at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/src/libstd/panic.rs:361
sozu_1          |                         at /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/src/libstd/rt.rs:57
sozu_1          |   15:     0x5605bb946142 - main
sozu_1          |   16:     0x7fefa100182f - __libc_start_main
sozu_1          |   17:     0x5605bb602758 - _start
sozu_1          |   18:                0x0 - <unknown>
sozu_1          | 2017-06-19T19:46:04Z	7107772533523	1	INFO	MASTER	received client order AddInstance(Instance { app_id: "backend-sozu-vertxvotingapp", ip_address: "172.18.0.4", port: 80 }) with tag "HTTP"
sozu_1          | 2017-06-19T19:46:04Z	7107773357481	1	INFO	MASTER	received client order RemoveInstance(Instance { app_id: "backend-sozu-vertxvotingapp", ip_address: "172.19.0.2", port: 80 }) with tag "HTTP"

[Geal]

what kind of features is this app using? Are there websockets, pipelining?

[Geal]

could you show a larger part of the log?

[Geal]

could you also provide a pcap of the network packets? I suspect the app might have sent more data after the end of the response

[zepouet]

Hello @Geal Can you send me a public key ? I will give access to a linux instance.

I use websocket in my app not directly with sock.js and vertx evenbus

[BlackYoup]

I personally get that panic during a file upload. Here is a simple reproduction case:

Server:

const app = require("express")();

app.get("*", (req, res) => {
  console.log(req.headers);
  res.status(200).send("Hello");
});

app.listen(1026, () => { console.log("listening.."); });

Generate file:
dd if=/dev/urandom of=test bs=1M count=1

Send:
curl http://lolcatho.st:8080 -XPOST -d @test

It triggers the problem. I don't know if that's the case for all uploads or even the same source as this panic

[Geal]

ok, this should be fixed with b547f1a, could you try again?

[BlackYoup]

It seems like it fixed that problem (yes, I got a new one now :D, I'll investigate)

[Geal]

file uploads with curl don't work right now because #10 is not implemented yet

[BlackYoup]

They work if you explicitly disable the Expect header: -H "Expect:"

[Geal]

it should work in all cases now with fbde478 :)

[BlackYoup]

It works fine for me now, even with cURL

[Geal]

@zepouet could you try again?