New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ignore previous import events when grabbing a torrent release that was previously downloaded then deleted #2393

Open
kaysond opened this Issue Feb 2, 2018 · 11 comments

Comments

Projects
None yet
4 participants
@kaysond
Copy link

kaysond commented Feb 2, 2018

As of version 2.0.0.5085, if a release is successfully imported then deleted, a new attempt to grab the same release is not properly monitored or imported by sonarr. The download is tracked internally, but it does not show up in "Activity" and never gets imported. Apparently, this is because the release already has SonarrStage=Imported.

This would effectively prevent any release from ever being re-downloaded, as far as I can tell.

See: https://forums.sonarr.tv/t/unable-to-add-torrent-file-to-deluge/17562/6

@Taloth

This comment has been minimized.

Copy link
Member

Taloth commented Feb 2, 2018

That's by design, i'll detail in the forum thread.

@Taloth Taloth closed this Feb 2, 2018

@Taloth Taloth reopened this Feb 2, 2018

@kaysond

This comment has been minimized.

Copy link

kaysond commented Feb 2, 2018

The main issue here is that releases can't ever be redownloaded, whether by nzb or torrent. If I delete a file through sonarr and don't blacklist the release, I should be able to download that same release again (through sonarr).

I don't think anything should be changed to accommodate manually adding torrents/nzbs to the client directly.

Right now "re-downloads" are tracked just fine, but basically ignored because of the existing import event, despite the file being deleted. It's a little bit of a pain because I actually want to keep the release I downloaded (so I have to add the download through sonarr search, then copy the file over and do a manual import)

@markus101

This comment has been minimized.

Copy link
Member

markus101 commented Feb 2, 2018

The main issue here is that releases can't ever be redownloaded, whether by nzb or torrent.

Usenet is unaffected, every usenet client Sonarr support uses a unique ID per job, whereas torrent clients use the InfoHash for tracking.

I agree that we shouldn't worry about manually added torrents (where we'd be guessing at best).

@markus101 markus101 added the suboptimal label Feb 2, 2018

@kaysond

This comment has been minimized.

Copy link

kaysond commented Feb 2, 2018

Usenet is unaffected, every usenet client Sonarr support uses a unique ID per job, whereas torrent clients use the InfoHash for tracking.

Ah my mistake. What about generating our own unique hash for torrent clients? Something like hash(infohash + date added) would be unique for every job... (This is somewhat separate from changing the monitor/import logic to ignore old imports, but could help make the behavior a little more user friendly)

@markus101

This comment has been minimized.

Copy link
Member

markus101 commented Feb 2, 2018

What about generating our own unique hash for torrent clients? Something like hash(infohash + date added) would be unique for every job

Sonarr would never find that ID hash in the download client though. If Sonarr was able to get the date added for each torrent that'd work, but then changing clients would reset everything and you could end up with hundreds of "new" torrents that need to be imported. That seems like a worse scenario than treating it as already imported if the torrent was added manually after it was already imported.

@kaysond

This comment has been minimized.

Copy link

kaysond commented Feb 2, 2018

I'm not talking about manually adding downloads to torrent clients at all.

Just to make sure I understand the process correctly -

  1. User adds download through sonarr
  2. Sonarr sends the download to client and stores an identifier provided by the client (unique for every download in the case of nzb clients, and unique only for every torrent, but not download, in torrent clients)
  3. Sonarr tracks the downloads by polling clients and monitoring the matching identifier
  4. When the download matching the identifier completes, Sonarr imports the file

Is that correct? Now you're saying for nzb clients, if I download the same nzb file twice, it will have a different identifier the second time, so it will get imported?

But for torrents, it will have the same identifier as the first time, so it won't get imported. If a unique identifier was created for a torrent based on its infohash and the date added, for example, then it would behave exactly the same as an nzb client.

Changing torrent clients would be the same as changing nzb clients, and presumably, if the client has manually added downloads (that weren't added through sonarr), the identifier wouldn't be in the tracking database so sonarr would ignore it.

@markus101

This comment has been minimized.

Copy link
Member

markus101 commented Feb 2, 2018

Is that correct?

Yes

Now you're saying for nzb clients, if I download the same nzb file twice, it will have a different identifier the second time, so it will get imported?

Yes

But for torrents, it will have the same identifier as the first time, so it won't get imported. If a unique identifier was created for a torrent based on its infohash and the date added, for example, then it would behave exactly the same as an nzb client.

If there was a reliable way to create that unique ID with information from the download client when the torrent is added, but clients report back the info hash only in many cases, since that is the internal ID they use. Not something I see torrent clients changing and not something we'd hack in, as there wouldn't be a reliable way to do it.

Changing torrent clients would be the same as changing nzb clients, and presumably, if the client has manually added downloads (that weren't added through sonarr), the identifier wouldn't be in the tracking database so sonarr would ignore it.

When changing usenet clients you don't import the history, you start from scratch. When switching torrent clients, generally you import and seeding torrents to continue seeding them, so the info hash is the same and if Sonarr built an ID based on info hash + added time the added time would now have changed and it'd need to be imported.

Anyways, we're way off topic here, we're not going to create an ID for torrents, we'll continue to use the ID provided use the ID from the torrent client. If torrent clients decide to move to a unique ID then Sonarr can be updated to use that, but otherwise it'll be an InfoHash.

@kaysond

This comment has been minimized.

Copy link

kaysond commented Feb 2, 2018

If there was a reliable way to create that unique ID with information from the download client when the torrent is added, but clients report back the info hash only in many cases, since that is the internal ID they use. Not something I see torrent clients changing and not something we'd hack in, as there wouldn't be a reliable way to do it.

With deluge, after adding the torrent, you send a second request with the hash to the "get_torrents" API method, get the date/time added, and then construct the "sonarr-unique-id". I'm sure all torrent clients have a similar api method, and it would be very reliable.

When switching torrent clients, generally you import and seeding torrents to continue seeding them, so the info hash is the same and if Sonarr built an ID based on info hash + added time the added time would now have changed and it'd need to be imported.

Even if you import and re-seed on a new client, sonarr wouldn't be tracking those torrents at all because their unique id wouldn't match the one that was generated when sonarr added the torrent to the first client. So its not in the download monitor database at all.

I don't really understand the hesitance to do something like this since it seems way more robust, but its ultimately up to you guys. In any case, yes, this is mostly unrelated to the issue of not being able to redownload torrent releases.

It would be great if the import logic could be updated to allow re-downloading torrents

@kaysond kaysond changed the title Ignore previous import events when grabbing a release that was previously downloaded then deleted Ignore previous import events when grabbing a torrent release that was previously downloaded then deleted Feb 2, 2018

@Taloth

This comment has been minimized.

Copy link
Member

Taloth commented Feb 2, 2018

You're making a couple of bad assumptions. Especially regarding the usability of information in deluge and other clients. But also about how Sonarr deals with torrents in general, and which kind of edge cases we have to protect against.

Either way, we already have a possible approach to the issue posted as discussed on the forums.

@arkayenro

This comment has been minimized.

Copy link

arkayenro commented Jan 12, 2019

so this feature is to handle seeding (the job staying in the download client after its been imported), you dont want to continually try to import a seeding torrent so you flag the id as having been imported thus you do it once, flag it, and dont worry about it if you see it again? makes sense.

downside is youre keeping that status forever? so the issue ends up at a later date, and the download is no longer active, and you need to download the file again for sonarr to import

you could clear the imported flag for that id when the user clicks on the download button (manual or auto) in sonarrs web gui so that it will get imported again (i presume youre checking here for the id and not adding an activity to sonarr because it exists?) - that wont fix duplicate items manually added to the download client but it makes sonarr work how it should.

you could purge the import flag for the id after a set amount of time? 24 hours? user configurable per indexer? (probably better so you can match it to each indexers re-seed times). this would cover the majority of scenarios.

i just noticed theres already a re-seed time option for an indexer, could that be utilised for an id import timeout/purge?

https://forums.sonarr.tv/t/assistance-with-script-to-replicate-drone-factory/20860/17?u=rhom

@markus101

This comment has been minimized.

Copy link
Member

markus101 commented Jan 12, 2019

As we've mentioned before (here/in the linked thread) the status is kept forever because it's part of history, items in history are immutable and aren't meant to be changed, unless the series removed then the history for everything related to that series is removed.

i presume youre checking here for the id and not adding an activity to sonarr because it exists?

Unless the item is still in the queue (in which case it'd only work for manual) there is no activity for it.

I don't see any reason to remove/alter/ignore the import history after any period of time unless the item is re-downloaded and that's already been covered.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment