Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Always add to history and post-process #26

Closed
hugbug opened this issue Jun 21, 2015 · 4 comments

Comments

Projects
None yet
1 participant
@hugbug
Copy link
Member

commented Jun 21, 2015

When adding nzbs the nzb-files are sometimes rejected on early stages and are not added to history. They are also not processed by pp-scripts. The cases are for example:

  • send nzb is invalid;
  • url could not be loaded (history entry is created but pp-scripts are not executed);
  • nzb was detected as identical by nzbget and therefore never added at all;
  • nzb was detected as dupe to a history item with success or good status and therefore never added at all;
  • nzb was detected as dupe and added to history;
  • maybe something else.

Every nzb-request must result in a history record and must be post-processed.

@hugbug hugbug added the feature label Jun 21, 2015

@hugbug hugbug added this to the v16.0 milestone Jun 21, 2015

hugbug added a commit that referenced this issue Jul 23, 2015

#26: ignored nzbs are now added to history
When an nzb-file isn’t added to queue for some reason, the file is now
also added to history (in addition to messages printed to log):
- for malformed nzb-files which cannot be parsed the status in history
“DELETE: SCAN”;
- for duplicate files with exactly same content status “DELETE: COPY”;
- for duplicate files having history items with status “GOOD” - status
“DELETE: GOOD”;
- for duplicate files having history items with status “SUCCESS” -
status “DELETE: SUCCESS”;
- history items have log-entries with explanation;
- new values for field “DeleteStatus” of RPC-Method “history”: GOOD,
SUCCESS, COPY, SCAN;
- new values for field “Status” of RPC-Method “history”:
"FAILURE/SCAN”, ”DELETED/COPY”, "DELETED/GOOD”, "DELETED/SUCCESS”;
- one exception: for files added from RSS-feeds no history items are
created, the files are ignored as if they were filtered.
@hugbug

This comment has been minimized.

Copy link
Member Author

commented Jul 23, 2015

Implementation

When an nzb-file isn’t added to queue for some reason, the file is now also added to history (in addition to messages printed to log):

  • for malformed nzb-files which cannot be parsed the status in history “DELETE: SCAN”;
  • for duplicate files with exactly same content status “DELETE: COPY”;
  • for duplicate files having history items with status “GOOD” and for duplicate files having hidden history items with status “SUCCESS” which do not have any visible duplicates - status “DELETE: GOOD”;
  • history items have log-entries with explanation;
  • new values for field “DeleteStatus” of RPC-Method “history”: GOOD, COPY, SCAN;
  • new values for field “Status” of RPC-Method “history”: "FAILURE/SCAN”, ”DELETED/COPY”, "DELETED/GOOD” (they will also be passed to pp-scripts as NZBPP_STATUS);
  • one exception: for files added from RSS-feeds no history items are created for duplicates, the files are ignored as if they were filtered.

So far achieved: for every nzb sent to nzbget a queue or history item is always created, even if nzb were ignored by duplicate check or could not be parsed.

Branch: 26-history.

Next step: execute pp-scripts or queue-scripts.

hugbug added a commit that referenced this issue Jul 24, 2015

#26: RPC-method "append" return ID on parsing failure
… instead of error code “-1” as in previous version. Since now a
history item is created on parsing failure its ID is returned. The
caller can check the status from history.

hugbug added a commit that referenced this issue Jul 24, 2015

#26: removed code "SUCCESS" from DeleteStatus
A more general code “GOOD” is used instead.

hugbug added a commit that referenced this issue Jul 24, 2015

hugbug added a commit that referenced this issue Jul 24, 2015

#26: new queue-script event "NZB_DELETED"
- when an nzb is deleted from queue and moved to history the
queue-scripts are called with event “NZB_DELETED”.
- no scripts are called if items are deleted permanently without
putting to history or if the history is completely disabled. This is
because queue-scripts are executed asynchronously and at the time the
script is executed the item must exist (either in queue or in history)
in order to pass item info to the script.
@hugbug

This comment has been minimized.

Copy link
Member Author

commented Jul 24, 2015

Implementation completed

New queue-script event NZB_DELETED:

  • when an nzb is deleted from queue and moved to history the queue-scripts are called with event “NZB_DELETED”.
  • no scripts are called if items are deleted permanently without putting to history or if the history is completely disabled. This is because queue-scripts are executed asynchronously and at the time the script is executed the item must exist (either in queue or in history) in order to pass item info to the script. This can be improved so that the scripts are called even without history but I'm not sure if a fix is really necessary.

hugbug added a commit that referenced this issue Jul 27, 2015

#26: passing DupeKey, DupeMode and DupeScore to queue- and pp-scripts
- new env. parameters passed to queue-scripts: NZBNA_DUPEKEY,
NZBNA_DUPEMODE, NZBNA_DUPESCORE;
- new env. parameters passed to pp-scripts: NZBPP_DUPEKEY,
NZBPP_DUPEMODE, NZBPP_DUPESCORE.
@hugbug

This comment has been minimized.

Copy link
Member Author

commented Jul 31, 2015

TODO:

  • failures for url-downloads should execute queue-scripts.

hugbug added a commit that referenced this issue Aug 11, 2015

#26: executing queue scripts for URLs
- queue-scripts are now also called for failed URLs;
- new queue event “URL_COMPLETED” with possible values: FAILURE,
SCAN_SKIPPED, SCAN_FAILURE;
- queue scripts are not called when URL was successfully fetched and
added queue; event “NZB_ADDED” is fired in this case;
@hugbug

This comment has been minimized.

Copy link
Member Author

commented Aug 11, 2015

Update

  • queue-scripts are now also called for failed URLs;
  • new queue event “URL_COMPLETED” with possible values: FAILURE,
    SCAN_SKIPPED, SCAN_FAILURE; see description of option QueueScript for a little bit more details;
  • please note that queue scripts are not called for successful URL downloads; in such case the URL-item is replaced with an nzb-item and event “NZB_ADDED” is fired;

I'm considering this feature completed and closing the issue. Should any problems arise during test please reopen the issue.

@hugbug hugbug closed this Aug 11, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.