HTTP/fetch query can block event loop (tested on Linux)

[lsergiy]

I have http server and http client. Both standard and very minimalist based on examples kindly provided with Nodepp.
I noticed http server may read accepted socked forever blocked. Client side is performing http fetch. Everything is Nodepp based code.

    virtual int __read( char* bf, const ulong& sx ) const noexcept {
        if( process::millis() > get_recv_timeout() || is_closed() )
          { return -1; } if ( sx==0 ) { return 0; }

        SOCKADDR_ST& addr = get_addr(); socklen_t len = sizeof(addr);

        int res = SOCK != SOCK_DGRAM // <<< <<< <<< WE STUCK HERE <<< <<< <<< SOCK is 1 <<< <<< <<<
                ? ::recv    ( obj->fd, bf, sx, 0 )
                : ::recvfrom( obj->fd, bf, sx, 0, (SOCKADDR*) &addr, &len );

Everything runs in separated and dedicated worker thread. Call stack is:

1  __libc_recv                                                                                                                                                                                                                                                                                                                                                                                                                                  recv.c           28  0x7ffff792bc4a 
2  __libc_recv                                                                                                                                                                                                                                                                                                                                                                                                                                  recv.c           23  0x7ffff792bc4a 
3  nodepp::socket_t::__read                                                                                                                                                                                                                                                                                                                                                                                                                     socket.h         629 0x5555556f7f8d 
4  nodepp::socket_t::_read                                                                                                                                                                                                                                                                                                                                                                                                                      socket.h         617 0x5555556f7e75 
5  nodepp::generator::file::read::operator()<nodepp::http_t>                                                                                                                                                                                                                                                                                                                                                                                    generator.h      35  0x55555571c134 
6  nodepp::generator::file::line::operator()<nodepp::http_t>                                                                                                                                                                                                                                                                                                                                                                                    generator.h      117 0x55555570a4e9 
7  nodepp::http_t::read_header                                                                                                                                                                                                                                                                                                                                                                                                                  http.h           160 0x5555556fb284 
8  nodepp::http::fetch(nodepp::fetch_t const&, nodepp::agent_t *, nodepp::function_t<void, nodepp::http_t>)::{lambda(nodepp::function_t<void, nodepp::http_t>, nodepp::function_t<void, nodepp::except_t>)#1}::operator()(nodepp::function_t<void, nodepp::http_t>, nodepp::function_t<void, nodepp::except_t>) const::{lambda(nodepp::http_t)#1}::operator()(nodepp::http_t) const                                                             http.h           244 0x5555556fc81e 
9  nodepp::function_t<void, nodepp::socket_t>::func_impl<nodepp::http::fetch(nodepp::fetch_t const&, nodepp::agent_t *, nodepp::function_t<void, nodepp::http_t>)::{lambda(nodepp::function_t<void, nodepp::http_t>, nodepp::function_t<void, nodepp::except_t>)#1}::operator()(nodepp::function_t<void, nodepp::http_t>, nodepp::function_t<void, nodepp::except_t>) const::{lambda(nodepp::http_t)#1}>::invoke(nodepp::socket_t const&) const function.h       59  0x5555557667cc 
10 nodepp::function_t<void, nodepp::socket_t>::emit                                                                                                                                                                                                                                                                                                                                                                                             function.h       44  0x55555571a5c9 
11 nodepp::function_t<void, nodepp::socket_t>::operator()                                                                                                                                                                                                                                                                                                                                                                                       function.h       40  0x555555709789 
12 nodepp::tcp_t::connect(nodepp::dns_t const&, int, nodepp::function_t<void, nodepp::socket_t>) const::{lambda()#1}::operator()() const                                                                                                                                                                                                                                                                                                        tcp.h            150 0x5555556fa1a1 
13 nodepp::loop_t::add<nodepp::tcp_t::connect(nodepp::dns_t const&, int, nodepp::function_t<void, nodepp::socket_t>) const::{lambda()#1}>(nodepp::tcp_t::connect(nodepp::dns_t const&, int, nodepp::function_t<void, nodepp::socket_t>) const::{lambda()#1}) const::{lambda()#1}::operator()() const                                                                                                                                            loop.h           171 0x55555572bd1e 
14 nodepp::function_t<int>::func_impl<nodepp::loop_t::add<nodepp::tcp_t::connect(nodepp::dns_t const&, int, nodepp::function_t<void, nodepp::socket_t>) const::{lambda()#1}>(nodepp::tcp_t::connect(nodepp::dns_t const&, int, nodepp::function_t<void, nodepp::socket_t>) const::{lambda()#1}) const::{lambda()#1}>::invoke() const                                                                                                            function.h       59  0x555555764b8a 
15 nodepp::function_t<int>::emit                                                                                                                                                                                                                                                                                                                                                                                                                function.h       44  0x555555713b15 
16 nodepp::function_t<int>::operator()                                                                                                                                                                                                                                                                                                                                                                                                          function.h       40  0x5555557036e8 
17 nodepp::loop_t::normal_queue_next() const::{lambda()#1}::operator()() const                                                                                                                                                                                                                                                                                                                                                                  loop.h           82  0x5555556ec78c 
18 nodepp::loop_t::normal_queue_next                                                                                                                                                                                                                                                                                                                                                                                                            loop.h           116 0x5555556ecbaa 
19 nodepp::loop_t::next                                                                                                                                                                                                                                                                                                                                                                                                                         loop.h           156 0x5555556ecd5e 
20 nodepp::kernel_t::next                                                                                                                                                                                                                                                                                                                                                                                                                       kernel.h         277 0x5555556f1c34 
21 nodepp::process::next                                                                                                                                                                                                                                                                                                                                                                                                                        nodepp.h         67  0x5555556f22f7 
22 nodepp::process::wait                                                                                                                                                                                                                                                                                                                                                                                                                        nodepp.h         112 0x5555556f251a 
23 operator()                                                                                                                                                                                                                                                                                                                                                                                                                                   aqt.cpp          79  0x5555557c9746 
24 operator()                                                                                                                                                                                                                                                                                                                                                                                                                                   worker.h         57  0x5555557c9d14 
25 nodepp::function_t<int>::func_impl<nodepp::worker_t::worker_t<bee::net::async_queue_thread::start()::<lambda()>>(bee::net::async_queue_thread::start()::<lambda()>)::<lambda()>>::invoke(void) const                                                                                                                                                                                                                                         function.h       59  0x5555557cbef6 
26 nodepp::function_t<int>::emit                                                                                                                                                                                                                                                                                                                                                                                                                function.h       44  0x555555713b15 
27 nodepp::function_t<int>::operator()                                                                                                                                                                                                                                                                                                                                                                                                          function.h       40  0x5555557036e8 
28 nodepp::worker_t::callback                                                                                                                                                                                                                                                                                                                                                                                                                   worker.h         44  0x5555556f29f9 
29 start_thread                                                                                                                                                                                                                                                                                                                                                                                                                                 pthread_create.c 447 0x7ffff789caa4 
30 clone3                                                                                                                                                                                                                                                                                                                                                                                                                                       clone3.S         78  0x7ffff7929c6c

The key thing to reproduce this issue is same as I described in previous issue: start timer or repeated function call in same thread/event-loop where http server is started.
So, this issue is Linux version of MacOSX issue. I cannot do Windows checks for you, unfortunately. But I think issue should be checked there also because I suspect presence of some logical issue in source code of event loop.

[EDBCREPO]

Hello Sergiy,

Thank you for the detailed report. This led us into a deep investigation regarding the nature of multithreading in asynchronous environments. The issue you are experiencing is not a failure in the network logic, but rather a direct consequence of Thread Local Storage (TLS) and memory isolation.

Below are the findings and the solutions integrated into version nodepp v1.4.1_1:

1. The "Deep Sleep" Problem (Kernel Wakeup)

Nodepp is designed for extreme efficiency; if there are no pending tasks but waiting for network events, the Kernel enters a power-saving mode (sleep) while waiting for an interruption.

• The Issue: If you inject a task (such as a timer or a callback) into a Worker from the main thread, the Worker remains unaware of the pending work until an external network event (like a new connection) triggers a wakeup.
• The Solution: We have integrated wake() automatically into process::add and process::poll. Now, any task injection explicitly notifies the target Kernel to wake up and process the queue immediately.

template< class... T >
ptr_t<task_t> loop_add( const T&... args ) const noexcept {
    ptr_t<task_t> tsk = obj->ev_queue.add( args... ); 
    wake(); // Notifies the kernel to exit sleep-mode
    return tsk; 
}

2. Client/Server Deadlock within the Same Loop

This is a known architectural behavior that I am currently working to "patch," but it requires structural changes. When Nodepp executes a task, it marks it as USED to prevent infinite reentrancy and protect the stack.

When you create a web server and attempt to make HTTP requests to it from the same thread, both sides (read and write) attempt to mark the task as USED simultaneously. This creates a deadlock where neither side yields control.

Recommendation: In production environments, internal requests should be handled via events or invokers. If you strictly require internal HTTP communication, you must separate the Client and the Server into different Workers:

void onMain(){
     worker::add([=](){ server(); process::wait(); return -1; });
     worker::add([=](){ client(); process::wait(); return -1; });
}

3. The Thread-Local Storage (TLS) Phenomenon

This is where C++ behavior becomes counterintuitive. By marking the Kernel as thread_local, the compiler guarantees total memory isolation per thread. Consider the following experiment:

#include <nodepp/nodepp.h>
#include <nodepp/worker.h>
#include <nodepp/timer.h>

using namespace nodepp;

void onMain(){

    event_t<> ev;

    worker::add([=](){
        ev.on([=](){ console::log( "<A>", &process::kernel() ); });
        console::log( "real0:", &process::kernel() );
    return -1; });

    worker::add([=](){
        ev.on([=](){ console::log( "<B>", &process::kernel() ); });
        console::log( "real1:", &process::kernel() );
    return -1; });

    worker::add([=](){
        ev.on([=](){ console::log( "<C>", &process::kernel() ); });
        console::log( "real2:", &process::kernel() );
    return -1; });

    timer::timeout([=](){
        ev.emit();
        console::log( "<D>", &process::kernel() );
    },1000);

}

real0: 0x7d799cfff620 (Actual address in Worker 0)
real1: 0x7d799c7fe620 (Actual address in Worker 1)
<A>    0x7d799d651460 (Address captured inside the Event)
<D>    0x7d799d651460 (Address in the Main Thread)

Notice that realN is the kernel address within its own thread, but <A> (captured in an injected event) points to the address of the thread that emitted the event. C++ detects the variable as thread-local and, upon executing the callback, uses the context of the caller thread instead of the target worker thread. Essentially, you were injecting tasks into the caller thread itself rather than the target worker.

Conclusion: This is not a bug in Nodepp; it is a fundamental C++ isolation feature. To safely inject tasks between threads, you should use channel_t or any other indirection like:

    worker::add([=](){

        auto krn = type::bind( process::kernel() );

        ev.on([=]( coroutine_t callback ){ 
            krn->loop_add( callback );
        });
    
    process::wait(); return -1; });

New API Tools in v1.4.0:

To provide more control over these scenarios, we have added:

• worker::kernel(): Direct access to the thread's real kernel.
• worker::is_sleeping(): To monitor the reactor's idle state.
• worker::wake(): To manually wake the kernel from an external thread.

    expected_t<kernel_t,except_t> kernel() const noexcept {
        if( obj->krn==nullptr ){ return except_t( "kernel not found" ); }
        return * ( type::cast<kernel_t>( obj->krn ) );
    }

    bool is_sleeping() const noexcept {
        if( obj->krn==nullptr ){ return false; }
        return type::cast<kernel_t>( obj->krn )->is_sleeping();
    }

    void wake() const noexcept { 
        if( obj->krn==nullptr ){ return; }
        type::cast<kernel_t>( obj->krn )->wake();
    }

[lsergiy]

Thanks for sharing best-practice details!