Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Crash when flushing article cache after direct rename #529

Closed
hugbug opened this issue Apr 2, 2018 · 5 comments

Comments

Projects
None yet
2 participants
@hugbug
Copy link
Member

commented Apr 2, 2018

Got crash dumps from a user where crashes occurred during flushing of article cache. The program state was "impossible" and it took a fair amount of time (three months) to collect these crash dumps and analyse them. Only after the third crash dump I finally got an idea of what was happening.

The critical place is in direct renamer: after renaming a small amount of data downloaded from par2-files is discarded (one article per par2-file) to avoid holding of info for these files, which are rarely used at all (par2-files are needed only for repair). It turned out the discarding of data isn't properly synchronised. As a result the article cache may attempt to write the data which was just deleted from memory.

A proper synchronisation must be developed.

@hugbug hugbug added the bug label Apr 2, 2018

@hugbug hugbug added this to the v20 milestone Apr 2, 2018

@hugbug hugbug referenced this issue Apr 2, 2018

Closed

NZBGet 20 #522

@JackDandy

This comment has been minimized.

Copy link
Contributor

commented Apr 2, 2018

In this game, at least 50% of the battle is reaching enlightenment - 3 months=nightmare, congrats to you, and kudos to the user.

@hugbug

This comment has been minimized.

Copy link
Member Author

commented Apr 2, 2018

When a bug happens once a month it isn't a big issue for the user but still frustrating for the developer 😉

Will be eliminated soon: 🐛 🔫

@hugbug

This comment has been minimized.

Copy link
Member Author

commented May 1, 2018

There is a bug in the fix which may lead to unsuccessful attempts to flush articles which in turn produces a lot of logging (a user has reported 85GB log file). Other than large log-file the sub-bug seems to be merely harmless.

@hugbug hugbug reopened this May 1, 2018

hugbug added a commit that referenced this issue May 1, 2018

#529: fixed missing file unlocking after direct rename
Also made locking more granular to avoid unnecessary locks for files
whose articles are not going to be discarded.
@JackDandy

This comment has been minimized.

Copy link
Contributor

commented May 5, 2018

Before/After:

Yup, that's a 3.5 Gb log file ;)

@hugbug

This comment has been minimized.

Copy link
Member Author

commented May 5, 2018

Thanks for confirming the fix is working.

These were crazy long logs ;)

@hugbug hugbug closed this May 28, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.