SDBits Search Broken #10629

ghost · 2020-12-26T06:14:17Z

Back with another issue, this time SDBits.

OS: Linux/Docker

Jackett Version: v0.17.159

Are you using a proxy or VPN? [yes/no] no

Description

The search on SDBits is broken and has been since I added it (more than 4 months at the very least). The search returns a lot of torrents none of which include the search terms. And the torrent that was searched for doesn't appear in the list despite searching the exact title that appears on the actual tracker.

How to Reproduce the error

Search any title you can think of and see the results obtained. Always 17 torrents returned and it's the same exact torrents every time.

Screenshots

https://thesungod.xyz/4d25FB_Q.png
https://thesungod.xyz/ZdMVBx5u.png

ilike2burnthing · 2020-12-26T06:34:31Z

No maintainers currently have an account. Do you have an invite available, or can you post an html dump of the search page? https://github.com/Jackett/Jackett/wiki/Troubleshooting#how-to-provide-html-source

ghost · 2020-12-26T07:21:06Z

Unfortunately, I've never had any invites over there but I can provide the html dump. Should I send to Garfield's email which I have used before or can I get your email?

ilike2burnthing · 2020-12-26T17:46:29Z

Post the dump here using this format:

<details>
<summary>HTML dump</summary>

'''
<html>
HTML dump goes here
</html>
'''
</details>

But replace ' with `

It will give you:

HTML dump

<html>
HTML dump goes here
</html>

Otherwise use something like pastebin.

ilike2burnthing · 2021-01-12T23:03:11Z

Working again?

ghost · 2021-01-12T23:15:31Z

Nah here is the link to both jackett logs and the html
https://mega.nz/folder/5AFQRJZC#xsVlIK7a9cFYVl_mxTSDHA

ilike2burnthing · 2021-01-13T00:13:44Z

Should be fixed in the next release, will publish it in <1hr.

Looks like IMDB search could be added, as well as sorting results, but I'm not going to try that without an account.

ilike2burnthing · 2021-01-13T00:25:51Z

v0.17.261

ghost · 2021-01-13T02:45:33Z

Working perfectly now, thanks for fixing it. I'll send you an invite if I ever get one, tho I doubt that'll happen since I'm not that active on the tracker.

ghost · 2021-01-13T03:41:06Z

@ilike2burnthing I think I jumped the gun a lil bit here. So, for the most part, the search is working. Now the problem not sure where it lies between Jackett, Sonarr and Radarr. So everything on the tracker is SD Quality but the tracker has 5 subcategories Capture, DVD, Encode, Remux, and WEB-DL. I've done a couple of searches from Sonarr and Radarr and these are errors so far;
a) Can't Parse release name & quality even tho the movie is named as MovieName NTSC DVD
b) Failed to find movie title in release name
c) Some shows don't show up completely in sonarr, possibly due to being multi-season pack and the naming not including season numbers i.e ShowName Complete NTSC Remux

Maybe that IMDB search is needed after all to correctly map the movie and show names idk. Some movies map well, some dont.

ilike2burnthing · 2021-01-14T01:06:41Z

a & b) if you search for these in Jackett, are they found and the titles displayed correctly? If not, then provide enhanced logs of the issues. If they are, then yeah we might need to play about with the IMDB search

c) yea, I'm not sure there's a solution for those case unfortunately, but it might be worth asking over on https://forums.sonarr.tv/ if whole series packs (with no season numbers) can be/are supported

ghost · 2021-01-14T23:08:29Z

I think the fix is fine for the most part. You can go ahead and close the issue. I think the fix is sufficient but if someone else is having an issue they should send the invite to make it easier for you to diagnose the issue. Otherwise, if you still need the logs just to countercheck I can send them tomorrow.

ilike2burnthing · 2021-01-15T01:21:34Z

I think I've got a grip on the IMDB search for the site (there are three 'imdb' options, but the first two look like ratings).

The only thing I need to know is can you just search by imdbid, or does the search require a title and the imdbid? In other words, can you just use tt7286456 in the imdb search, or do you also have to search for Joker?

Either way it should be a pretty simple addition.

ghost · 2021-01-15T01:30:57Z

I did try yesterday searching just the imdb search but it did not work. Maybe it requires you to search it in a certain format like imdb:tt0230323. Not sure, but just tt0xxxxxxx doesn't work, I also think the imdb is just for ratings but I'll try again later on tonight.

ghost · 2021-01-15T20:30:27Z

@ilike2burnthing IMDB search on the tracker site works without the preceding tt if you could implement that I could test it out.
The problem I encountered now is I don't think everything is imdb linked. Searched a show that had 3 torrents (season 1,2 and 5) but the imdb search only had 2 results (season 1&2, 5 was left out). For movies I didn't encounter any errors so far maybe you can make it search imdb main and fallback to title search idk.

ilike2burnthing · 2021-01-15T23:32:39Z

See #10646 (comment) for instructions of how to test out an indexer when using Jackett in Docker. You'll need to create a new file sdbits-test.yml (or download the one in the link above and just rename it) and then paste this into it using a text editor:

sdbits-test.yml

---
id: sdbits-test
name: SDBits Test
description: "SDBits Test is a small tracker that focuses on SD movies and tv."
language: en-us
type: private
encoding: UTF-8
links:
  - https://sdbits.org/
legacylinks:
  - http://sdbits.org/

caps:
  categorymappings:
    - {id: 6, cat: Audio, desc: "Audio"}
    - {id: 3, cat: TV/Documentary, desc: "Documentary"}
    - {id: 1, cat: Movies, desc: "Movies"}
    - {id: 4, cat: Audio, desc: "Music"}
    - {id: 5, cat: TV/Sport, desc: "Sports"}
    - {id: 7, cat: TV/Other, desc: "Stand-up Comedy"}
    - {id: 2, cat: TV, desc: "TV"}

  modes:
    search: [q]
    tv-search: [q, season, ep, imdbid]
    movie-search: [q, imdbid]
    music-search: [q]

login:
  path: takeloginn3.php
  method: post
  inputs:
    uname: "{{ .Config.username }}"
    password: "{{ .Config.password }}"
    returnto: "/"
  error:
    - selector: td.embedded:has(h2:contains("failed")+table)
  test:
    path: browse.php
    selector: span.smallfont:has(a[href="logout.php"])

search:
  paths:
    - path: browse.php
  inputs:
    $raw: "{{ range .Categories }}c{{.}}=1&{{end}}"
    search: "{{ if .Query.IMDBID }}{{ else }}{{ .Keywords }}{{ end }}"
    incldead: 1
    descriptions: 0
    imdbgt: 0
    imdblt : 10
    imdb: "{{ .Query.IMDBIDShort }}"

  rows:
    selector: table#torrent-list > tbody > tr[id]

  fields:
    title:
      selector: td:nth-child(3) > b > a
    category:
      selector: a[href^="?cat="]
      attribute: href
      filters:
        - name: querystring
          args: cat
    details:
      selector: td:nth-child(3) > b > a
      attribute: href
    download:
      selector: a[href^="download.php?id="]
      attribute: href
    size:
      selector: td:nth-child(6)
    grabs:
      selector: td:nth-child(7)
      filters:
        - name: regexp
          args: (\d+)
    seeders:
      selector: td:nth-child(8)
    leechers:
      selector: td:nth-child(9)
    date:
      selector: td:nth-child(5)
      filters:
        - name: append
          args: " ago"
    imdb:
      selector: a[href*="imdb.com/title/tt"]
      attribute: href
    downloadvolumefactor:
      case:
        "a[style=\"color:#000099\"][href^=\"details.php?\"]": 0
        "*": 1
    uploadvolumefactor:
      text: 1
# engine tbd

Save the file, restart Jackett, add SDBits Test, and try it out with Jackett and Sonarr/Radarr (you'll need to add the indexer there as well).

If you're up for testing sorting at the same time, use this one instead:

sdbits-test.yml + sorting

---
id: sdbits-test
name: SDBits Test
description: "SDBits Test is a small tracker that focuses on SD movies and tv."
language: en-us
type: private
encoding: UTF-8
links:
  - https://sdbits.org/
legacylinks:
  - http://sdbits.org/

caps:
  categorymappings:
    - {id: 6, cat: Audio, desc: "Audio"}
    - {id: 3, cat: TV/Documentary, desc: "Documentary"}
    - {id: 1, cat: Movies, desc: "Movies"}
    - {id: 4, cat: Audio, desc: "Music"}
    - {id: 5, cat: TV/Sport, desc: "Sports"}
    - {id: 7, cat: TV/Other, desc: "Stand-up Comedy"}
    - {id: 2, cat: TV, desc: "TV"}

  modes:
    search: [q]
    tv-search: [q, season, ep, imdbid]
    movie-search: [q, imdbid]
    music-search: [q]

settings:
  - name: username
    type: text
    label: Username
  - name: password
    type: password
    label: Password
  - name: sort
    type: select
    label: Sort requested from site
    default: added&h=7
    options:
      added&h=7: created
      seeders&h=10: seeders
      size&h=8: size
      name&h=2: title
  - name: type
    type: select
    label: Order requested from site
    default: DESC
    options:
      DESC: desc
      ASC: asc

login:
  path: takeloginn3.php
  method: post
  inputs:
    uname: "{{ .Config.username }}"
    password: "{{ .Config.password }}"
    returnto: "/"
  error:
    - selector: td.embedded:has(h2:contains("failed")+table)
  test:
    path: browse.php
    selector: span.smallfont:has(a[href="logout.php"])

search:
  paths:
    - path: browse.php
  inputs:
    $raw: "{{ range .Categories }}c{{.}}=1&{{end}}"
    search: "{{ if .Query.IMDBID }}{{ else }}{{ .Keywords }}{{ end }}"
    incldead: 1
    descriptions: 0
    imdbgt: 0
    imdblt : 10
    imdb: "{{ .Query.IMDBIDShort }}"
    sort: "{{ .Config.sort }}"
    d: "{{ .Config.type }}"

  rows:
    selector: table#torrent-list > tbody > tr[id]

  fields:
    title:
      selector: td:nth-child(3) > b > a
    category:
      selector: a[href^="?cat="]
      attribute: href
      filters:
        - name: querystring
          args: cat
    details:
      selector: td:nth-child(3) > b > a
      attribute: href
    download:
      selector: a[href^="download.php?id="]
      attribute: href
    size:
      selector: td:nth-child(6)
    grabs:
      selector: td:nth-child(7)
      filters:
        - name: regexp
          args: (\d+)
    seeders:
      selector: td:nth-child(8)
    leechers:
      selector: td:nth-child(9)
    date:
      selector: td:nth-child(5)
      filters:
        - name: append
          args: " ago"
    imdb:
      selector: a[href*="imdb.com/title/tt"]
      attribute: href
    downloadvolumefactor:
      case:
        "a[style=\"color:#000099\"][href^=\"details.php?\"]": 0
        "*": 1
    uploadvolumefactor:
      text: 1
# engine tbd

Use a keywordless/empty search, or something with lots of results like 2020, and make sure the results are the same between the site and Jackett when using different types of sorting.

Hopefully all goes well. Thanks in advance!

ghost · 2021-01-16T00:40:21Z

So I'm using LinuxServer/Jackett on Synology and I can't find the definitions folder anywhere on my NAS.
Jackett says its

Loading Cardigann indexers from: /config/cardigann/definitions/, /etc/xdg/cardigan/definitions/, /app/Jackett/Definitions

I don't have any of those directories on the nas.

I was looking into this issue and apparently, /definitions/ folder is held within the container itself, or am I mistaken? I tried the solution in that issue but couldn't get it to work maybe I'm doing something wrong. Not so good with this docker stuff.

linuxserver/docker-jackett#72

ilike2burnthing · 2021-01-16T01:00:19Z

You should have the volume /config mapped for Jackett's container (it has the sub-folder Jackett with the logs, config file, indexers, etc.). If not do so, e.g. /volume1/Docker/Jackett/config - not sure about Synology's folder structure, so that might need changing.

Navigate to that folder (for me, that's just via a mapped network drive on my computer), create a sub-folder cardigann, and a sub-folder within that one definitions.

Pretty sure by default config is already mapped (basically mirroring the folder inside the container to somewhere more accessible), so you would just need to navigate to (for example) /Docker/Jackett/config in Synology's File Station, create those new subfolders, and upload the file there.

If that doesn't help, I can push the fix, wait for the new Docker image to be posted, and you can test it out. I'd just prefer not to publish something untested.

ghost · 2021-01-16T01:25:21Z

oh ok, I think I'm making progress. Jackett creates a folder inside it called jackett with 2 other folders inside it and I was creating the cardigann folder inside it. I moved it and now I have /volume1/docker/apps/jackett/cardigann/definitions/ and I threw the SDBitsTest.yml in it but it doesn't appear in the list when I try to add the new tracker + now I'm getting this error in logs:

Error while creating Cardigann instance from definition ID=sdbits-test: System.Exception: Not supported movie-search param: imdb
at Jackett.Common.Models.TorznabCapabilities.ParseMovieSearchParams(IEnumerable1 paramsList) in /home/vsts/work/1/s/src/Jackett.Common/Models/TorznabCapabilities.cs:line 141 at Jackett.Common.Models.TorznabCapabilities.ParseCardigannSearchModes(Dictionary2 modes) in /home/vsts/work/1/s/src/Jackett.Common/Models/TorznabCapabilities.cs:line 88
at Jackett.Common.Indexers.CardigannIndexer..ctor(IIndexerConfigurationService configService, WebClient wc, Logger l, IProtectionService ps, ICacheService cs, IndexerDefinition Definition) in /home/vsts/work/1/s/src/Jackett.Common/Indexers/CardigannIndexer.cs:line 117
at Jackett.Common.Services.IndexerManagerService.<>c__DisplayClass15_0.b__5(IndexerDefinition definition) in /home/vsts/work/1/s/src/Jackett.Common/Services/IndexerManagerService.cs:line 181

ilike2burnthing · 2021-01-16T01:29:33Z

Whoops, that's my bad.
movie-search: [q, imdb] > movie-search: [q, imdbid]
Fixed the typos above if you want to try again

ghost · 2021-01-16T01:48:06Z

Ok finally got it to work. Now the issue is searching directly from Jackett returns no results whatsoever and when trying to add the tracker to radarr I got Query successful, but no results were returned from your indexer. This may be an issue with the indexer or your indexer category settings.

ilike2burnthing · 2021-01-16T02:02:54Z

Was that for both versions?

ghost · 2021-01-16T02:04:22Z

No, the 2nd one with sorting let me try the 1st.

ghost · 2021-01-16T17:38:12Z

Ok so,
Doing manual searches for both IMDB and Title works perfectly inside of jackett.
Problems;

Radarr
Always returns 15 results none of which is the required torrent. Same problem that you fixed initially. But initially Jackett was also returning wrong results.

2021-01-16 10:13:57 | Info | Torznab search in SDBits Test => Found 15 releases
-- | -- | --
2021-01-16 10:13:50 | Info | Torznab search in SDBits Test => Found 15 releases (from cache)

Sonarr
Sonarr quickly goes to no results found. So it actually doesn't even execute the search based on the jackett logs.

I think for TV shows it would be better to search just the IMDB number or Title and get rid of the season numbers completely since alot of the shows posted are the complete multi-season packs which sonarr doesn't support anyways. Tracker doesn't have individual episodes either.

ilike2burnthing · 2021-01-17T03:14:59Z

Enable enhanced logs in Jackett and run searches in Jackett (keyword and imdbid), Sonarr, and Radarr. Let me know the search URLs being used for each (https://sdbits.org/browse.php...).

ilike2burnthing · 2021-02-09T20:52:33Z

@Paul5656 ping

ghost · 2021-02-10T01:24:13Z

My bad I just went back to using the original one that you fixed. I think that will suffice. Release names are all over the place, especially the tv cat, hence Sonarr/Jackett not finding releases that are actually there. The ones that follow scene naming standards do get found and mapped correctly. It works fine for the most part and I think you can close the issue. The fine-tuning I think would just require you to login to the site, but without an invite....¯_(ツ)_/¯

ghost · 2021-02-10T01:29:00Z

What I would suggest personally for TV is to get rid of the season numbers in the search and just search the name only so that everything matching can be returned, I manual search most of the time in sonarr anyways, so that would help me.

ilike2burnthing · 2021-02-10T13:05:56Z

I'll leave this open for someone with an account/when they open registration.

mekarawr · 2021-04-22T16:02:17Z

I have an SDbits account is there some way I can assist with troubleshooting? Or we could set up a zoom session for example for you to troubleshoot using my account

ilike2burnthing · 2021-04-23T00:39:16Z

If you have an invite free or you're happy to email your account details I can take a look at it, otherwise it will likely be a long repeat of the above - trial and error based on enhanced logs.

mekarawr · 2021-04-24T12:30:58Z

if you contact me on
redacted
and i will provide you login details, cheers

ilike2burnthing · 2021-04-24T16:13:52Z

Sent.

ilike2burnthing · 2021-04-24T22:53:30Z

Thanks @mekarawr, you can change your password now.

Added the ability to sort results and search by IMDB ID, just slight changes made to what I had done before from the work with @Paul5656 (thanks to them for all that effort).

There's also a new note about changing the torrents per page in your account settings to 100, so you may both want to do that.

There are very few results which include episode numbers, but they do exist, so I kept that ability.

The naming scheme not being consistent isn't really something we can do anything about without completely removing the ability to search by season. There are other issues with the search which affect Sonarr and Radarr (e.g. not being able to find results which have an apostrophe in the name if the search term doesn't include it), so I'm leaving it at is for now. If that ends up being more of an issue, leave a comment here and it can be removed.

If you want to test it yourself, just change:

  modes:
    search: [q]
    tv-search: [q, season, ep, imdbid]
    movie-search: [q, imdbid]
    music-search: [q]

to:

  modes:
    search: [q]
    tv-search: [q, imdbid]
    movie-search: [q, imdbid]
    music-search: [q]

save the file and restart Jackett.

New release should be out in ~6hrs.

garfield69 · 2021-04-25T07:02:26Z

v0.17.946

ghost · 2021-04-26T03:24:31Z

Thanks for the fix. What I never realized when we were going back and forth above, is that Radarr requires a minimum of 4 things to match in order to correctly match and map a release. Movie Name, Year, Resolution, and the Source Medium. If one of those is missing the release will always get rejected/not map correctly. Thus a lot of releases from SDBits get rejected bcoz they miss one of those details. But as you mentioned not much can be done for the tracker naming inconsistencies, nonetheless I appreciate you guys working together to improve it 🙏.

ilike2burnthing added More information needed Needs Investigations labels Dec 26, 2020

jackettbot bot removed the More information needed label Dec 26, 2020

ghost closed this as completed Jan 12, 2021

ilike2burnthing removed the Needs Investigations label Jan 13, 2021

ilike2burnthing self-assigned this Jan 13, 2021

ilike2burnthing added this to In progress in Jackett Indexers Jan 13, 2021

ilike2burnthing reopened this Jan 13, 2021

ilike2burnthing closed this as completed in f5c23e7 Jan 13, 2021

ilike2burnthing reopened this Jan 14, 2021

ilike2burnthing added Full logs needed More information needed labels Jan 14, 2021

jackettbot bot removed Full logs needed More information needed labels Jan 14, 2021

ilike2burnthing added the On hold label Feb 9, 2021

ilike2burnthing removed their assignment Feb 10, 2021

ilike2burnthing added the Invite needed label Feb 10, 2021

ilike2burnthing moved this from In progress to Blocked in Jackett Indexers Feb 10, 2021

ilike2burnthing closed this as completed in b2c0cb6 Apr 24, 2021

garfield69 moved this from Blocked to Done in Jackett Indexers Apr 25, 2021

garfield69 removed the On hold label Apr 25, 2021

SDBits Search Broken #10629

SDBits Search Broken #10629

Comments

ghost commented Dec 26, 2020

Description

How to Reproduce the error

Screenshots

ilike2burnthing commented Dec 26, 2020

ghost commented Dec 26, 2020

ilike2burnthing commented Dec 26, 2020 • edited

ilike2burnthing commented Jan 12, 2021

ghost commented Jan 12, 2021 • edited by ghost

ilike2burnthing commented Jan 13, 2021

ilike2burnthing commented Jan 13, 2021

ghost commented Jan 13, 2021

ghost commented Jan 13, 2021 • edited by ghost

ilike2burnthing commented Jan 14, 2021

ghost commented Jan 14, 2021

ilike2burnthing commented Jan 15, 2021

ghost commented Jan 15, 2021 • edited by ghost

ghost commented Jan 15, 2021

ilike2burnthing commented Jan 15, 2021 • edited

ghost commented Jan 16, 2021 • edited by ghost

ilike2burnthing commented Jan 16, 2021 • edited

ghost commented Jan 16, 2021

ilike2burnthing commented Jan 16, 2021

ghost commented Jan 16, 2021

ilike2burnthing commented Jan 16, 2021

ghost commented Jan 16, 2021

ghost commented Jan 16, 2021

ilike2burnthing commented Jan 17, 2021

ilike2burnthing commented Feb 9, 2021

ghost commented Feb 10, 2021 • edited by ghost

ghost commented Feb 10, 2021 • edited by ghost

ilike2burnthing commented Feb 10, 2021

mekarawr commented Apr 22, 2021

ilike2burnthing commented Apr 23, 2021

mekarawr commented Apr 24, 2021 • edited

ilike2burnthing commented Apr 24, 2021

ilike2burnthing commented Apr 24, 2021 • edited

garfield69 commented Apr 25, 2021

ghost commented Apr 26, 2021

ilike2burnthing commented Dec 26, 2020 •

edited

ghost commented Jan 12, 2021 •

edited by ghost

ghost commented Jan 13, 2021 •

edited by ghost

ghost commented Jan 15, 2021 •

edited by ghost

ilike2burnthing commented Jan 15, 2021 •

edited

ghost commented Jan 16, 2021 •

edited by ghost

ilike2burnthing commented Jan 16, 2021 •

edited

ghost commented Feb 10, 2021 •

edited by ghost

ghost commented Feb 10, 2021 •

edited by ghost

mekarawr commented Apr 24, 2021 •

edited

ilike2burnthing commented Apr 24, 2021 •

edited