[BIT-HDTV] Configure IMDb ID search to look in both 'URL' and 'Title & Desc.' fields

[RoloSoze]

Is it possible to configure Jackett so that when the IMDb ID is used to search BIT-HDTV, it searches both the URL and Title & Desc.?

That's particularly useful for TV season packs (e.g. tt0068098) and movie packs (e.g. tt0364569) where the uploader may not enter the IMDb ID in the URL field, but mentions it in the torrent description.

[cadatoiva]

It is technically possible, but it's going to add complexity if we want to avoid duplicate results from both searches. I was also looking into adding the IMDB link to the results, but since that's on the details page, and with two search types meaning the link could be in one of two locations, it'll take more effort to get that working bug free.

[RoloSoze]

It is technically possible, but it's going to add complexity if we want to avoid duplicate results from both searches. I was also looking into adding the IMDB link to the results, but since that's on the details page, and with two search types meaning the link could be in one of two locations, it'll take more effort to get that working bug free.

Personally I'm OK with the occasional duplicates. I prefer duplicates over missing relevant results. As for including the IMDb link, I believe the practice so far is to include it only if it appears on the tracker's torrent/browse/search results page. I don't think there is an instance in Jackett where the IMDb link from the details page is included because, as you said, this may take a lot more work. @garfield69 and I briefly had this discussion here:

#4859 (comment)

@garfield69 Can you confirm my understanding of the inclusion of the IMDb link in Jackett?

[garfield69]

I believe the practice so far is to include it only if it appears on the tracker's torrent/browse/search results page
Can you confirm my understanding of the inclusion of the IMDb link in Jackett?

Correct. We do not want to go fetching every torrents details page just to check for the imdbid.
that would mean up to 100 additional http GETs, potentially adding unwelcome traffic load on the servers and possible getting flagged as a DDoS generator.

[cadatoiva]

I know that for YML trackers, they aren't capable of digging into the details page. And for C# trackers, if that's the only thing we're getting off the details page then it'll probably be a lot of traffic for little benefit, which would make sense as to why we wouldn't implement it if that's the only reason.

Following into the details would also let us get the BannerUrl, so maybe? I'll leave it up to the rest of the team to decide on whether it should or shouldn't be implemented.

[RoloSoze]

@cadatoiva I am in favor of searching with the IMDb ID in 2 different locations on BIT-HDTV. It is the only top tracker I tested where searching using the IMDb ID in 2 different fields may return additional / 2 different sets of relevant results. I'm fine with the occasional duplicate results that could be an outcome of doing this. Nevertheless, after testing this for a while, if I find that the we're getting too many duplicates, we can always revert.

However, I'm not in favor of fetching the IMDb ID from the secondary/torrent details page in BIT-HDTV since this would open the door for doing this in most other C# trackers. If this means possibly getting flagged as a DoS attack, or if it's going to slow down Jackett's performance in any way, I don't think it's a good idea.

[garfield69]

up to now, the only time we allow indexers to process the details page to supplement the results from the primary, is when the the sites torrent title in the primary results page leads to multiple download links in the details page.
This mostly applies to anime sites, and some tv sites (mainly Spanish) that have only the series title in the primary, but individual episode torrent download links in the details.

[cadatoiva]

Ok, well I won't implement grabbing ImdbID & BannerUrl for this tracker. Maybe you can talk to the owner about changing the results page to include these somehow @RoloSoze since you're in contact with them.

I'm testing to make sure my changes to search both options works, and then I'll PR

[RoloSoze]

up to now, the only time we allow indexers to process the details page to supplement the results from the primary, is when the sites torrent title in the primary results page leads to multiple download links in the details page.
This mostly applies to anime sites, and some tv sites (mainly Spanish) that have only the series title in the primary, but individual episode torrent download links in the details.

I think it's also being done in PTP to fetch the correct filename. I had this discussion recently with @ngosang about how the proper filenames for some torrents in Morethantv and Awesome-HD #6783 will need to be fetched from the secondary page in order for the names to be accurate and make sense.

[RoloSoze]

Maybe you can talk to the owner about changing the results page to include these somehow @RoloSoze since you're in contact with them.

I'll email you about this.

[ngosang]

@RoloSoze Could you provide an example that works searching imdb in the description?

Btw, It's easy to remove duplicates since the GUID is unique by definition. Just add all results to releases list and then => https://stackoverflow.com/a/9993228

@cadatoiva add me as reviewer when it's ready.

[RoloSoze]

@RoloSoze Could you provide an example that works searching IMDb in the description?

Use the examples I mentioned in my initial post :) As I said, it's particularly useful for TV season packs and movie packs.

BIT-HDTV asks uploaders to enter the TVMaze URL for TV and TV packs uploads so trying to search the URL field for the IMDb ID does not work, but searching the description might work if the uploader chooses to enter the IMDb URL in the description.

The same goes for movie packs. When there's more than one movie in the torrent, the uploader usually enters the IMDb URLs for all the included movies in the description section, not the URL section since they cannot enter more than one URL there.

[cadatoiva]

I decided to skip duplicates by ignoring entries that already link to the same description page this grabbed all 12 unique results from tt0068098 as example listed above.

[garfield69]

Jackett 0.14.49

[RoloSoze]

Is this working for you guys? It's not working on Jackett 0.14.49.0. I tried searching with the IMDb ID. Before this update, it worked well for the URL search, now it's totally broken. I get an error:

@garfield69 @ngosang @cadatoiva

log.txt

[cadatoiva]

The problem was that it was assumed that there would always be a results table on the search page. But when there's no results, that is not the case. It now checks and skips a results page if no results are found.

[garfield69]

Jackett 0.14.79