Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

http server dump #781

Open
Howe2015 opened this issue Aug 31, 2020 · 7 comments
Open

http server dump #781

Howe2015 opened this issue Aug 31, 2020 · 7 comments
Labels
http

Comments

@Howe2015
Copy link

Howe2015 commented Aug 31, 2020 •
edited by nyh

i use wrk to test the simple http server, but it dump every time.

wrk -t12 -c100 -d30s --latency http://10.1.18.231:10000/pic/test.jpeg

code

#include <seastar/http/httpd.hh>
#include <seastar/http/handlers.hh>
#include <seastar/http/function_handlers.hh>
#include <seastar/http/file_handler.hh>
#include <seastar/core/seastar.hh>
#include <seastar/core/reactor.hh>
#include <seastar/http/api_docs.hh>
#include <seastar/core/thread.hh>
#include <seastar/core/prometheus.hh>
#include <seastar/core/print.hh>
#include <seastar/net/inet_address.hh>
#include "../../apps/lib/stop_signal.hh"

namespace bpo = boost::program_options;

using namespace seastar;
using namespace httpd;


int main(int ac, char** av) {
    httpd::http_server_control prometheus_server;
    prometheus::config pctx;
    app_template app;

    app.add_options()("port", bpo::value<uint16_t>()->default_value(10000), "HTTP Server port");

    return app.run_deprecated(ac, av, [&] {
        return seastar::async([&] {
            seastar_apps_lib::stop_signal stop_signal;
            auto&& config = app.configuration();

            uint16_t port = config["port"].as<uint16_t>();
            auto server = new http_server_control();
            server->start().get();             
            server->set_routes([](routes& r){
                r.add(seastar::httpd::GET, url("/pic").remainder("path"), new httpd::directory_handler("pic/"));//程序所在目录,记住需要最后的"/"
            }).get();
            server->listen(port).get();

            std::cout << "Seastar HTTP server listening on port " << port << " ...\n";
            engine().at_exit([server] {
                return [] {
                    return make_ready_future<>();
                }().finally([server] {
                    std::cout << "Stoppping HTTP server" << std::endl;
                    return server->stop();
                });
            });

            stop_signal.wait().get();
        });
    });
}

core info

wuhaotao@wuhaotao:~/local_project/seastar_new/seastar/study/http-server$ gdb my_app core 
GNU gdb (Ubuntu 9.1-0ubuntu1) 9.1
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from my_app...
[New LWP 606710]
[New LWP 606714]
[New LWP 606725]
[New LWP 606719]
[New LWP 606716]
[New LWP 606707]
[New LWP 606703]
[New LWP 606715]
[New LWP 606718]
[New LWP 606705]
[New LWP 606702]
[New LWP 606712]
[New LWP 606717]
[New LWP 606706]
[New LWP 606723]
[New LWP 606711]
[New LWP 606722]
[New LWP 606713]
[New LWP 606721]
[New LWP 606708]
[New LWP 606720]
[New LWP 606709]
[New LWP 606724]
[New LWP 606704]

warning: Unexpected size of section `.reg-xstate/606710' in core file.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
--Type <RET> for more, q to quit, c to continue without paging--
Core was generated by `./my_app'.
Program terminated with signal SIGSEGV, Segmentation fault.

warning: Unexpected size of section `.reg-xstate/606710' in core file.
#0  0x0000561d3100431c in seastar::internal::future_base::detach_promise (this=0x6050001a3d28) at ../../include/seastar/core/future.hh:1334
1334	    void schedule(Pr&& pr, Func&& func, Wrapper&& wrapper) noexcept {
[Current thread is 1 (Thread 0x7fac80dff700 (LWP 606710))]
(gdb) bt
#0  0x0000561d3100431c in seastar::internal::future_base::detach_promise (this=0x6050001a3d28) at ../../include/seastar/core/future.hh:1334
#1  seastar::future<>::detach_promise() (this=0x6050001a3d28) at ../../include/seastar/core/future.hh:1331
#2  seastar::future<>::schedule<seastar::internal::promise_base_with_type<>, seastar::httpd::http_server::do_accepts(int)::<lambda(seastar::future<seastar::accept_result>)> mutable::<lambda()>, seastar::future<T>::then_impl_nrvo(Func&&) [with Func = seastar::httpd::http_server::do_accepts(int)::<lambda(seastar::future<seastar::accept_result>)> mutable::<lambda()>; Result = seastar::future<>; T = {}]::<lambda()>::<lambda(pr_type&, seastar::httpd::http_server::do_accepts(int)::<lambda(seastar::future<seastar::accept_result>)> mutable::<lambda()>&, seastar::future_state<>&&)> > (wrapper=..., func=..., pr=..., this=0x6050001a3d28) at ../../include/seastar/core/future.hh:1338
#3  seastar::future<>::<lambda()>::operator() (this=<optimized out>) at ../../include/seastar/core/future.hh:1517
#4  seastar::future<>::then_impl_nrvo<seastar::httpd::http_server::do_accepts(int)::<lambda(seastar::future<seastar::accept_result>)> mutable::<lambda()>, seastar::future<> > (
    func=..., this=0x6050001a3d28) at ../../include/seastar/core/future.hh:1530
#5  seastar::future<>::then_impl<seastar::httpd::http_server::do_accepts(int)::<lambda(seastar::future<seastar::accept_result>)> mutable::<lambda()> > (func=..., 
    this=0x6050001a3d28) at ../../include/seastar/core/future.hh:1545
#6  seastar::internal::call_then_impl<seastar::future<> >::run<seastar::httpd::http_server::do_accepts(int)::<lambda(seastar::future<seastar::accept_result>)> mutable::<lambda()> > (func=..., fut=...) at ../../include/seastar/core/future.hh:1199
#7  seastar::future<>::then<seastar::httpd::http_server::do_accepts(int)::<lambda(seastar::future<seastar::accept_result>)> mutable::<lambda()> > (func=..., this=0x6050001a3d28)
    at ../../include/seastar/core/future.hh:1461
#8  seastar::httpd::http_server::<lambda(seastar::future<seastar::accept_result>)>::operator()(seastar::future<seastar::accept_result>) (f_ar=..., __closure=<optimized out>, 
    __closure=<optimized out>) at ../../src/http/httpd.cc:471
#9  0x0000561d31004ef0 in seastar::futurize<void>::invoke<seastar::httpd::http_server::do_accepts(int)::<lambda(seastar::future<seastar::accept_result>)>, seastar::future<seastar::accept_result> > (func=...) at ../../include/seastar/core/future.hh:498
#10 seastar::future<seastar::accept_result>::then_wrapped_common<true, void, seastar::httpd::http_server::do_accepts(int)::<lambda(seastar::future<seastar::accept_result>)> > (
    func=..., this=0x7fac80dfd100) at ../../include/seastar/core/future.hh:1635
#11 seastar::future<seastar::accept_result>::then_wrapped_maybe_erase<true, void, seastar::httpd::http_server::do_accepts(int)::<lambda(seastar::future<seastar::accept_result>)> >
    (func=..., this=0x7fac80dfd100) at ../../include/seastar/core/future.hh:1584
#12 seastar::future<seastar::accept_result>::then_wrapped<seastar::httpd::http_server::do_accepts(int)::<lambda(seastar::future<seastar::accept_result>)> > (func=..., 
    this=0x7fac80dfd100) at ../../include/seastar/core/future.hh:1575
#13 seastar::httpd::http_server::do_accepts (this=<optimized out>, which=<optimized out>) at ../../src/http/httpd.cc:455
#14 0x0000561d3100481b in seastar::httpd::http_server::<lambda(seastar::future<seastar::accept_result>)>::<lambda()>::operator() (__closure=<optimized out>, 
    __closure=<optimized out>) at ../../src/http/httpd.cc:472
#15 std::__invoke_impl<seastar::future<>, seastar::httpd::http_server::do_accepts(int)::<lambda(seastar::future<seastar::accept_result>)> mutable::<lambda()> > (__f=...)
    at /usr/include/c++/9/bits/invoke.h:60
#16 std::__invoke<seastar::httpd::http_server::do_accepts(int)::<lambda(seastar::future<seastar::accept_result>)> mutable::<lambda()> > (__fn=...)
    at /usr/include/c++/9/bits/invoke.h:96
#17 std::__apply_impl<seastar::httpd::http_server::do_accepts(int)::<lambda(seastar::future<seastar::accept_result>)> mutable::<lambda()>, std::tuple<> > (__t=..., __f=...)
    at /usr/include/c++/9/tuple:1685
#18 std::apply<seastar::httpd::http_server::do_accepts(int)::<lambda(seastar::future<seastar::accept_result>)> mutable::<lambda()>, std::tuple<> > (__t=..., __f=...)
    at /usr/include/c++/9/tuple:1696
#19 seastar::futurize<seastar::future<> >::apply<seastar::httpd::http_server::do_accepts(int)::<lambda(seastar::future<seastar::accept_result>)> mutable::<lambda()> > (args=..., 
    func=...) at ../../include/seastar/core/future.hh:2008
#20 seastar::future<>::then_impl<seastar::httpd::http_server::do_accepts(int)::<lambda(seastar::future<seastar::accept_result>)> mutable::<lambda()> > (func=..., 
    this=0x6050001a3d28) at ../../include/seastar/core/future.hh:1542
#21 seastar::internal::call_then_impl<seastar::future<> >::run<seastar::httpd::http_server::do_accepts(int)::<lambda(seastar::future<seastar::accept_result>)> mutable::<lambda()> >--Type <RET> for more, q to quit, c to continue without paging--
 (func=..., fut=...) at ../../include/seastar/core/future.hh:1199
#22 seastar::future<>::then<seastar::httpd::http_server::do_accepts(int)::<lambda(seastar::future<seastar::accept_result>)> mutable::<lambda()> > (func=..., this=0x6050001a3d28)
    at ../../include/seastar/core/future.hh:1461
#23 seastar::httpd::http_server::<lambda(seastar::future<seastar::accept_result>)>::operator()(seastar::future<seastar::accept_result>) (f_ar=..., __closure=<optimized out>, 
    __closure=<optimized out>) at ../../src/http/httpd.cc:471
#24 0x0000561d31005dbc in seastar::future<seastar::accept_result>::<lambda()>::<lambda(pr_type&, seastar::httpd::http_server::do_accepts(int)::<lambda(seastar::future<seastar::accept_result>)>&, seastar::future_state<seastar::accept_result>&&)>::<lambda()>::operator() (this=<optimized out>, this=<optimized out>) at ../../include/seastar/core/future.hh:498
#25 seastar::futurize<void>::satisfy_with_result_of<seastar::future<T>::then_wrapped_nrvo(Func&&) [with FuncResult = void; Func = seastar::httpd::http_server::do_accepts(int)::<lambda(seastar::future<seastar::accept_result>)>; T = {seastar::accept_result}]::<lambda()>::<lambda(pr_type&, seastar::httpd::http_server::do_accepts(int)::<lambda(seastar::future<seastar::accept_result>)>&, seastar::future_state<seastar::accept_result>&&)> mutable::<lambda()> > (func=..., pr=...) at ../../include/seastar/core/future.hh:1985
#26 seastar::future<seastar::accept_result>::<lambda()>::<lambda(pr_type&, seastar::httpd::http_server::do_accepts(int)::<lambda(seastar::future<seastar::accept_result>)>&, seastar::future_state<seastar::accept_result>&&)>::operator() (this=<optimized out>, state=..., func=..., pr=...) at ../../include/seastar/core/future.hh:1613
#27 seastar::continuation<seastar::internal::promise_base_with_type<>, seastar::httpd::http_server::do_accepts(int)::<lambda(seastar::future<seastar::accept_result>)>, seastar::future<T>::then_wrapped_nrvo(Func&&) [with FuncResult = void; Func = seastar::httpd::http_server::do_accepts(int)::<lambda(seastar::future<seastar::accept_result>)>; T = {seastar::accept_result}; seastar::futurize_t<FuncResult> = seastar::future<>]::<lambda()>::<lambda(pr_type&, seastar::httpd::http_server::do_accepts(int)::<lambda(seastar::future<seastar::accept_result>)>&, seastar::future_state<seastar::accept_result>&&)>, seastar::accept_result>::run_and_dispose(void) (this=0x6050000cbee0) at ../../include/seastar/core/future.hh:647
#28 0x0000561d30f3c7f8 in seastar::reactor::run_tasks (this=0x60500007e000, tq=...) at ../../src/core/reactor.cc:2152
#29 0x0000561d30f3cb0e in seastar::reactor::run_some_tasks (this=this@entry=0x60500007e000) at ../../src/core/reactor.cc:2568
#30 0x0000561d30f6c7e6 in seastar::reactor::run_some_tasks (this=0x60500007e000) at ../../src/core/reactor.cc:2723
#31 seastar::reactor::run (this=0x60500007e000) at ../../src/core/reactor.cc:2723
#32 0x0000561d30f75531 in seastar::smp::<lambda()>::operator()(void) const (__closure=0x600000172900) at ../../include/seastar/core/reactor.hh:738
#33 0x0000561d30f34652 in std::function<void ()>::operator()() const (this=<optimized out>) at /usr/include/c++/9/bits/std_function.h:683
#34 seastar::posix_thread::start_routine (arg=<optimized out>) at ../../src/core/posix.cc:60
#35 0x00007fac843ef609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#36 0x00007fac83f6e103 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
@nyh nyh added the http label Aug 31, 2020
@nyh
Copy link
Contributor

nyh commented Aug 31, 2020

I do not have "wrk" installed, but I tried both "curl" or "ab" and it seems to work without a crash.
Does the crash only happen in this specific wrk workload, or can you crash it in simpler ways?

@psarna
Copy link
Contributor

psarna commented Aug 31, 2020

I tried the wrk workload (with a random 4MiB file acting as index.jpeg) and nothing crashed, so it's not that easy to reproduce the issue... could you post the commit hash o Seastar you're using? Is it current master?

@Howe2015
Copy link
Author

I do not have "wrk" installed, but I tried both "curl" or "ab" and it seems to work without a crash.
Does the crash only happen in this specific wrk workload, or can you crash it in simpler ways?

I use this wrk(https://github.com/wg/wrk), it crash,but when I request a single url on the chrome, it work well.So is it the large workload crash the app?

@Howe2015
Copy link
Author

I tried the wrk workload (with a random 4MiB file acting as index.jpeg) and nothing crashed, so it's not that easy to reproduce the issue... could you post the commit hash o Seastar you're using? Is it current master?

how can I know the version of the seastar I used?😂

@Howe2015
Copy link
Author

Howe2015 commented Sep 4, 2020

I do not have "wrk" installed, but I tried both "curl" or "ab" and it seems to work without a crash.
Does the crash only happen in this specific wrk workload, or can you crash it in simpler ways?

and what's more, I use seastar on ubuntu 20 of vmware ,does it matter?

@psarna
Copy link
Contributor

psarna commented Sep 4, 2020

Seastar is highly optimized for bare metal machines, so running it on vmware will be a performance issue, but it shouldn't lead to faults (unless it ran out of memory, how much RAM does the machine have?). Do you build this app and seastar straight from the git repository, or have you downloaded it from somewhere (e.g. https://github.com/scylladb/seastar/releases)? I'm interested in which version you use, since perhaps it has a bug which was already fixed on current master.

@Howe2015
Copy link
Author

Howe2015 commented Sep 5, 2020

Seastar is highly optimized for bare metal machines, so running it on vmware will be a performance issue, but it shouldn't lead to faults (unless it ran out of memory, how much RAM does the machine have?). Do you build this app and seastar straight from the git repository, or have you downloaded it from somewhere (e.g. https://github.com/scylladb/seastar/releases)? I'm interested in which version you use, since perhaps it has a bug which was already fixed on current master.

My pc has 16GB RAM but I allocate about 4GB for the vmware ubuntu. I build seastar from git repository at about Jul 11,2020.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
http
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@nyh @psarna @Howe2015