Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Application stops after some requests #161

Closed
ddcovery opened this issue Jul 31, 2020 · 23 comments
Closed

Application stops after some requests #161

ddcovery opened this issue Jul 31, 2020 · 23 comments

Comments

@ddcovery
Copy link

ddcovery commented Jul 31, 2020

I'm running hunt-skeleton app
After first request, app stops:

$ ./hunt-skeleton 
2020-Aug-01 00:09:35.3466182 | 14015 | trace | registerAuthUserConfig.__lambda1 | [name: admin, role: ["superuser"], name: guest, role: ["guest"]] | ../../../.dub/packages/hunt-framework-3.2.0/hunt-framework/source/hunt/framework/provider/ConfigServiceProvider.d:113
2020-Aug-01 00:09:35.3466698 | 14015 | trace | registerAuthUserConfig.__lambda1 | [name: superuser, permissions: ["user:add", "user:del"], name: guest, permissions: ["user:query"]] | ../../../.dub/packages/hunt-framework-3.2.0/hunt-framework/source/hunt/framework/provider/ConfigServiceProvider.d:114
2020-Aug-01 00:09:35.3467618 | 14015 | warning | DatabaseServiceProvider.buildEntityManagerFactory | The database is disabled. | ../../../.dub/packages/hunt-framework-3.2.0/hunt-framework/source/hunt/framework/provider/DatabaseServiceProvider.d:53


 ___  ___     ___  ___     ________      _________   
|\  \|\  \   |\  \|\  \   |\   ___  \   |\___   ___\     Hunt Framework 3.2.0
\ \  \\\  \  \ \  \\\  \  \ \  \\ \  \  \|___ \  \_|     
 \ \   __  \  \ \  \\\  \  \ \  \\ \  \      \ \  \      Listening: 0.0.0.0:8080
  \ \  \ \  \  \ \  \\\  \  \ \  \\ \  \      \ \  \     TLS: Disabled
   \ \__\ \__\  \ \_______\  \ \__\\ \__\      \ \__\    
    \|__|\|__|   \|_______|   \|__| \|__|       \|__|    https://www.huntframework.com


Try to browse http://0.0.0.0:8080
$ 

If I try tu debug application (gdb):

(gdb) run
Starting program: /home/antonio/Devel/dlang/myproject2/hunt-skeleton 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7ffff7fca700 (LWP 14083)]
[New Thread 0x7ffff7fc3700 (LWP 14084)]
[New Thread 0x7ffff7fbc700 (LWP 14085)]
[New Thread 0x7ffff7fb5700 (LWP 14086)]
[New Thread 0x7ffff7fae700 (LWP 14087)]
[New Thread 0x7ffff7698700 (LWP 14088)]
[New Thread 0x7ffff7691700 (LWP 14089)]
[New Thread 0x7ffff716a700 (LWP 14090)]

Thread 9 "hunt-skeleton" received signal SIGUSR1, User defined signal 1.
[Switching to Thread 0x7ffff716a700 (LWP 14090)]
0x00007ffff779b361 in __GI___clock_nanosleep (clock_id=clock_id@entry=0, 
    flags=flags@entry=0, req=0x7ffff71688b0, rem=0x7ffff71688c8)
    at ../sysdeps/unix/sysv/linux/clock_nanosleep.c:48
48	../sysdeps/unix/sysv/linux/clock_nanosleep.c: No such file or directory.
(gdb) 

My system:
Ubuntu 20.04.1 LTS
DMD64 D Compiler v2.090.1
DUB version 1.19.0

@ddcovery
Copy link
Author

ddcovery commented Aug 2, 2020

After rebooting my machine it seems to work properly...

@ddcovery ddcovery closed this as completed Aug 2, 2020
@zoujiaqing
Copy link
Member

OK;)

@ddcovery ddcovery reopened this Aug 2, 2020
@ddcovery
Copy link
Author

ddcovery commented Aug 2, 2020

Sorry zoujiaqing :-)
After reboot, it begun to fail

First time I ran the program result was this:

Screenshot from 2020-08-02 19-46-53

Other runs doen't show the warning, but application stop after few requests.

If I run it with dub (dub run -v) then I obtain (after some requests)
Screenshot from 2020-08-02 20-06-00

@Heromyth
Copy link
Contributor

Heromyth commented Aug 3, 2020

It seems so strange. The Hunt Framework 3.3 is comming and will be release soon.
Then you can have an another test.

@ddcovery
Copy link
Author

ddcovery commented Aug 5, 2020

Fails with 3.3.0-rc.2 too. I will wait for 3.3

@ddcovery
Copy link
Author

ddcovery commented Aug 5, 2020

which linux distribution are you using to run/test hunt-framework?

@zoujiaqing
Copy link
Member

Please use 3.3.0 release version.

@ddcovery
Copy link
Author

ddcovery commented Aug 6, 2020

Same problem:

Peek 2020-08-07 00-22

@Heromyth
Copy link
Contributor

Heromyth commented Aug 7, 2020

Can you use gdb to debug it? It's really difficult to reproduce this problem in our environment.

@Heromyth
Copy link
Contributor

Create a file nambed .gdbinit in your Home directory:

set verbose off
handle SIGPIPE nostop noprint
handle SIGUSR1 nostop noprint
handle SIGUSR2 nostop noprint

which can ignore most of the warnings.

You can install the C++ extension for VS code.
20200811091555

And check the debug box in settings to allow breakpoints everywhere:
20200811091904

Then you can debug the application.

More about this, see https://forums.dlangchina.com/thread/10057.

@ddcovery
Copy link
Author

ddcovery commented Aug 19, 2020

Thank you Heromyth.

I was able to configure and place breakpoints... the main problem is the debug errors shown as popups by vscode about missing glibc files.

"Unable to open 'clock_nanosleep.c': Unable to read file '/build/glibc-YYA7BZ/glibc-2.31/sysdeps/unix/sysv/linux/clock_nanosleep.c' (Error: Unable to resolve non-existing file '/build/glibc-YYA7BZ/glibc-2.31/sysdeps/unix/sysv/linux/clock_nanosleep.c')."

@ddcovery
Copy link
Author

ddcovery commented Aug 19, 2020

Finally, I tried with gdbgui (it works nicely without this annoying popups).

The error that stops application is this:

Thread 
11 "hunt-skeleton" received signal SIGPIPE, Broken pipe.
[Switching to Thread 0x7ffff57b6700 (LWP 6678)]
__libc_write (nbytes=1265, buf=0x7ffff70e9000, fd=20) at ../sysdeps/unix/sysv/linux/write.c:26
26  ../sysdeps/unix/sysv/linux/write.c: No such file or directory.

And finally

Program terminated with signal 
SIGPIPE, Broken pipe.
The program no longer exists.

As I understand, doing repeatedly CTRL+F5 on Firefox causes it to abandon previous sockets before server writes the responses. This causes the broken pipe signal on server when it tries to write the response.

I suppose you must manage manually the error if necessary and ignore the SIGPIPE on POSIX systems.

@Heromyth
Copy link
Contributor

Heromyth commented Aug 20, 2020

Do you have another OS? Can you test this problem in it?
Maybe the bad network communication can cause some problem. In my environment, there is no any problem even I press CTRL+F5 quickly.

By the way, you can test the network with the other Hunt libraries like Hunt, Hunt-Http. All of them have an folder named examples where you can run some tests.

@ddcovery
Copy link
Author

Do you refer a non POSIX OS?
Which OS do you primary use to develop and test hunt-framework?

@Heromyth
Copy link
Contributor

Debian, Linux Mint (Ubuntu), Windows 10 etc.
X64 plantform.

@ddcovery ddcovery changed the title Application stops after first request Application stops after some requests Aug 21, 2020
@ddcovery
Copy link
Author

ddcovery commented Aug 30, 2020

I added the code to ignore the SIGPIPE signal (only valid on POSIX systems) on main.d

import hunt.framework;
version(Posix)
{
    import core.sys.posix.signal;
}

void main(string[] args)
{
  version(Posix){
    sigset_t sigset;
    sigemptyset(&sigset);
    sigaction_t siginfo;
    siginfo.sa_mask = sigset;
    siginfo.sa_flags = SA_RESTART;
    siginfo.sa_handler = SIG_IGN;
    sigaction(SIGPIPE, &siginfo, null);
  }
  app().run(args);

}

With this change, the application works without interruption when a pipe brokes (i.e.: Firefox discarding a socket before a response is written by the server).

Once in a while, a warning is shown in console:

2020-Aug-30 02:41:28.8044936 | 9613 | warning | AbstractStream.tryRead | warning on read: fd=20, errno=11, message=Resource temporarily unavailable | ../../../.dub/packages/hunt-1.6.1/hunt/source/hunt/io/channel/posix/AbstractStream.d:123

As I mentioned in a previous comment, it seems that hunt is not managing SIGPIPE properly.

@Heromyth
Copy link
Contributor

It seems that we need to do more searches about SIGPIPE, Broken pipe. and to find a way to fix this.

5 similar comments
@Heromyth
Copy link
Contributor

It seems that we need to do more searches about SIGPIPE, Broken pipe. and to find a way to fix this.

@Heromyth
Copy link
Contributor

It seems that we need to do more searches about SIGPIPE, Broken pipe. and to find a way to fix this.

@Heromyth
Copy link
Contributor

It seems that we need to do more searches about SIGPIPE, Broken pipe. and to find a way to fix this.

@Heromyth
Copy link
Contributor

It seems that we need to do more searches about SIGPIPE, Broken pipe. and to find a way to fix this.

@Heromyth
Copy link
Contributor

It seems that we need to do more searches about SIGPIPE, Broken pipe. and to find a way to fix this.

@Heromyth
Copy link
Contributor

Heromyth commented Sep 12, 2020

It's really difficult to reproduce this problem.
Maybe my GDB and OS are all the latest. When debugging a application, there are no any signals received.
However, I still added your code to ignore the SIGPIPE signal in commit huntlabs/hunt-net@8494ac8.

@Heromyth Heromyth closed this as completed Oct 4, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants