Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Files can be assembled after the job is removed from the queue #1509

Closed
Safihre opened this issue Jun 16, 2020 · 3 comments
Closed

Files can be assembled after the job is removed from the queue #1509

Safihre opened this issue Jun 16, 2020 · 3 comments
Labels

Comments

@Safihre
Copy link
Member

Safihre commented Jun 16, 2020

Using the automated testing I found that in special circumstances it can be that files are still being decoded/assembled while the job has already finished.
It seems to occur because only after the first par2 is received, we postpone the rest of the par2 files we don't need. But in the case of the test and the small files, these were already being downloaded.

Solutions:

  • Discard any articles and files after the job was nzo.deleted. Have to check the impact when extra par2 is added.
  • Post-pone par2 when adding the job.

At first I thought, based on the log output show, that end-of-job was triggered multiple times while parts are still being decoded. This turned out not to be the case, since nzo.remove_article is locked, so the end-of-job is really only triggered once. This took me hours of debugging to find out.

@Safihre Safihre added the Bug label Jun 16, 2020
@sanderjo
Copy link
Contributor

So ... what is the real problem? CPU power wasted? Time wasted until the next download starts? How much?

And does it only happen with small files / downloads?

@Safihre
Copy link
Member Author

Safihre commented Jun 16, 2020

Crash of the assembler (see the URL, a few lines down) resulting in the job disappearing after the restart.

Safihre added a commit that referenced this issue Aug 30, 2020
Relates to #1509
Hopefully it stops the failing tests.
Safihre added a commit that referenced this issue Dec 12, 2020
Removed the par2 files for the unicode job, they caused too much problems. It's a bad "fix" for #1509.
Safihre added a commit that referenced this issue Dec 12, 2020
Removed the par2 files for the unicode job, they caused too much problems. It's a bad "fix" for #1509.
@Safihre
Copy link
Member Author

Safihre commented Feb 14, 2022

Closed by 9ad5bd4 in 3.5.1RC2.

@Safihre Safihre closed this as completed Feb 14, 2022
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 20, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

No branches or pull requests

2 participants