Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Crash (ABORT) on r2171 and 2176 on linux #533

Closed
Nothing4You opened this issue Apr 22, 2018 · 20 comments

Comments

Projects
None yet
2 participants
@Nothing4You
Copy link

commented Apr 22, 2018

Hey,
I've sent you a core dump via email.

This happens within a minute of starting nzbget, although after switching to the debug build I also managed to somehow get it into a state once where it would just be frozen, not producing any logs anymore, timeout on http request. Right now I can't run it for a longer time anymore.

My current queue starts extraction of some successful downloads right away while also downloading more content. this could be caused by either.

Is it possible for me to start nzbget with download and postprocessing paused? That way I could try to narrow it down.

The last few lines in the log are this:

Sun Apr 22 20:10:00 2018    11910   140274958202624 DEBUG   SIMD yEnc decoder can be used (Decoder.cpp:29:Decoder)
Sun Apr 22 20:10:00 2018    11910   140274958202624 DEBUG   SIMD Crc routine can be used (Decoder.cpp:30:Decoder)
Sun Apr 22 20:10:00 2018    11910   140274958202624 DEBUG   Creating ArticleDownloader (ArticleDownloader.cpp:34:ArticleDownloader)
Sun Apr 22 20:10:00 2018    11910   140274958202624 DEBUG   Starting Thread (Thread.cpp:93:Start)
Sun Apr 22 20:10:00 2018    11910   140272474248960 DEBUG   Entering Thread-func (Thread.cpp:164:thread_handler)
Sun Apr 22 20:10:00 2018    11910   140272474248960 DEBUG   Entering ArticleDownloader-loop (ArticleDownloader.cpp:73:Run)
Sun Apr 22 20:10:00 2018    11910   140272474248960 DETAIL  something/somefile.part01.rar [256/273] @ my-nnt-provider (my-nntp-provider)
Sun Apr 22 20:10:00 2018    11910   140272474248960 DEBUG   Opening connection to my-nntp-provider (NntpConnection.cpp:187:Connect)
Sun Apr 22 20:10:00 2018    11910   140274916235008 DEBUG   Checking cache, Allocated: 94537728, FlushEverything: 1 (ArticleWriter.cpp:840:CheckFlush)
Sun Apr 22 20:10:00 2018    11910   140274916235008 DETAIL  Flushing cache for SomethingElse/Else.zip
@hugbug

This comment has been minimized.

Copy link
Member

commented Apr 22, 2018

Thanks for the core dump. I'll look into this.

NZBGet can be started with paused downloading by adding -P to command line:

nzbget -s -P

There is no switch to pause postprocessing though. However nzbget pauses all activities if it detects an error in configuration file. You can either add invalid options into config file or pass them in command line, for example:

nzbget -s -o invalid=yes
@Nothing4You

This comment has been minimized.

Copy link
Author

commented Apr 23, 2018

It seems to happen during download, this is what I just got when I unpaused one of my downloads:

** Error in `/home/user/nzbget/nzbget': double free or corruption (fasttop): 0x00007f43a85e0c10 ***
DETAIL  Writing articles for something.zip
  6          1.51       2.08                        1        26.1                                  Segmentation fault, tracing...

I'll send you another mail with the current core dump later if you need it.

@Nothing4You

This comment has been minimized.

Copy link
Author

commented Apr 23, 2018

Actually, nzbget didn't completely crash from this, I still see the cli interface progress updating, however, it doesn't answer via http anymore and it seems to be ignoring my input in the cli interface.

@hugbug

This comment has been minimized.

Copy link
Member

commented Apr 23, 2018

Yes please. The more core dumps the better 😉

@Nothing4You

This comment has been minimized.

Copy link
Author

commented Apr 23, 2018

I didn't grab a core dump from my attempt earlier, however, I just had somthing ABORT again while the rest of nzbget was still up and running, including the cli interface and the webinterface. I then killed it with kill -6 and grabbed a core dump which I emailed to you, also included a full debug log from that run.

@Nothing4You

This comment has been minimized.

Copy link
Author

commented Apr 26, 2018

@hugbug did you have a chance to look at them yet?

@hugbug

This comment has been minimized.

Copy link
Member

commented Apr 26, 2018

Sorry, not yet. I hope to do that over weekend.

@Nothing4You

This comment has been minimized.

Copy link
Author

commented May 1, 2018

Is there anything I can do to help?

@Nothing4You

This comment has been minimized.

Copy link
Author

commented May 2, 2018

Unfortunately it still crashes the same using r2177. I have added log and core from todays attempt to the shared drive folder. Once again it wasn't a full crash, I had to kill -6 it after it had Aborted already.

@hugbug

This comment has been minimized.

Copy link
Member

commented May 2, 2018

How often does this happen and how long nzbget runs until this occurs?
If that happens often:

  • please send me more dumps+logs.
  • as a test: stop nzbget, cleanup QueueDir (make a backup first), start nzbget. Any difference?
@Nothing4You

This comment has been minimized.

Copy link
Author

commented May 2, 2018

For the last few tests I was able to start nzbget and it would crash within less than 5 min (still has items queued when starting).

It was always the same queue so far, I'll try with a fresh queue later when I have time. I can also create a bunch of dumps + logs later with my current queue.
Since this started I have stopped using nzbget for now since I'd rather get this fixed than clearing my queue and hoping for the best, over this time it was probably 10+ times that it crashed in the same way (+ some more systemd automatic restarts which didn't produce coredumps though)

@hugbug

This comment has been minimized.

Copy link
Member

commented May 3, 2018

I have added log and core from todays attempt to the shared drive folder

Please send me the link to that folder. I only have a link to one core-file.

The idea to test with an empty queue was to check if the issue has something to do with queue state. If it currently crashes in 5 minutes it should be not difficult to test few files for 30 minutes using empty queue. For that test you can even install nzbget into a new directory and test that installation.

I can also create a bunch of dumps + logs later with my current queue.

Yes please.

@Nothing4You

This comment has been minimized.

Copy link
Author

commented May 3, 2018

I've sent you a link to the folder on the 23th, it was my second email about this issue, did you not get that?

Unfortunately I only got home very late yesterday so I wasn't able to do more testing. Today or tomorrow I'll have time to do that though.

@hugbug

This comment has been minimized.

Copy link
Member

commented May 3, 2018

No emails on 23th except from github. Can you please send this link again?

@Nothing4You

This comment has been minimized.

Copy link
Author

commented May 3, 2018

Weird, according to my mailserver logs and a report from google that should have been delivered, could it possibly have ended up in your spam folder?

Anyways, I've just sent it again.

@Nothing4You

This comment has been minimized.

Copy link
Author

commented May 4, 2018

Did you receive my email yesterday? I've just added 7 more core dumps + logs which all happened shortly after starting, all of these caused a full crash resulting in output like this (from 7th run) on my terminal rather than one thread aborting while the rest kept running (even though not downloading).

@Nothing4You

This comment has been minimized.

Copy link
Author

commented May 5, 2018

I've disabled ArticleCache for now and it seems to work fine without it. It was set to 100 before. Disabling it doesn't seem to have a big performance impact - still downloading with 40-60 MB/s.

hugbug added a commit that referenced this issue May 6, 2018

hugbug added a commit that referenced this issue May 6, 2018

@hugbug

This comment has been minimized.

Copy link
Member

commented May 6, 2018

The nzb is damaged. Posting software has produced malformed articles:

=ybegin part=3 total=14911247 line=128 size=1048576 name=example.filename.zip
=ypart begin=2097152 end=1048576

Parameters begin and end point to the begin and end of segment in the file. Therefore end can never be smaller than begin like we have here. Due to a bug in NZBGet such malformed articles might cause crashes. The bug has been fixed.

I've also added printing of warnings for such articles to explain why article downloads fail.

Sending you the fixed version to try.

@Nothing4You

This comment has been minimized.

Copy link
Author

commented May 9, 2018

I'll have the fixed version tested with my backup of the broken queue by tomorrow.

@hugbug

This comment has been minimized.

Copy link
Member

commented May 18, 2018

Closing this as fixed. Feel free to post comments if the issue occurs again.

@hugbug hugbug closed this May 18, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.