Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[New Tracker] Erai-Raws #11484

Merged
merged 9 commits into from
Apr 18, 2021
Merged

Conversation

6cUbi57z
Copy link
Contributor

@6cUbi57z 6cUbi57z commented Apr 5, 2021

Resolves #4116

As on the issue, there is a problem with this in that Erai-Raws isn't really an indexer but is a release site. As such, it has no search, no stats such as files, size, seeders, leechers and the implementation has to work based on the RSS feed (to work reliably) which has limited results. Also the releases don't follow the usual naming conventions on indexers so are not easily detected by Sonarr.

I have managed to put this together which seems to work fairly well but this is rather different to most other indexers (exc. ShowRSS). Like ShowRSS, it is grabbing all items from the RSS feed and then performing a basic filter within Jackett. I also added an option which will try to extract the season and episode numbers from the title and move them to the end in the format S1E1 so that they are detected by Sonarr. It may need another option to exclude the quality tag from the search as searching for episode "10" is also matching "1080p".

...but I'm unsure if it's a good idea to put this in given it's so different. Open to other suggestions if there are better ways to implement it.

@jackettbot
Copy link

jackettbot bot commented Apr 5, 2021

Hi @6cUbi57z,

Thanks for your contribution to Jackett!

If you are adding a new indexer, please ensure that you've added it to the readme as well

A human will be along soon to review

@ilike2burnthing
Copy link
Contributor

Apologies for taking so long to look at this.

Noticed an issue in the title detail parsing, where it seems to be taking any two numbers and making them into SxExx, e.g. Yuukoku no Moriarty Part 2 – 02 (Multi) becomes Yuukoku no Moriarty Part – (Multi) S2E02. That might be a bit of an edge case, but maybe put an exception in for part x?

It could be useful to also move the quality at the end, so instead of [1080p] blahblah - 01 it becomes something like blahblah - E01 - 1080p.

The size, seeds, leechers, DL factor, and UL factor should all be given generic values, so 512MB, 1, 1, 0, and 1 respectively (you can find these defaults here even though it's for YAML indexers). An example of them being used in a C# indexer:

Size = 0,
Seeders = 1,
Peers = 2,
DownloadVolumeFactor = 0,
UploadVolumeFactor = 1

(just realising ShowRSS' size also gives 0B, that needs changed)

If you want to give yourself more work you could make the generic files sizes different depending on the quality, e.g. 512MB for 720p releases and 1GB for 1080p releases, etc., or whatever is more suitable (increments of 256MB is probably as accurate as you need to be).

6cUbi57z added 2 commits April 17, 2021 23:42
- Add support for seasons in "Part X" format
- Move quality tags to end to improve sorting
- Remove quality tags from search to avoid "Episode 10" matching everything with "1080p"
- Add default values for Grabs, Seeds, Leechers, DL and UL Factors
- Add size estimating based on quality
- Add tests for title parsing
- Fix excess whitespace in parsed titles



Move quality to end of title and remove from search text


Fix wrong size unit


Adjust title parsing to insert identifiers after odd hyphen


Add size estimates


Fix title parsing tests
@6cUbi57z
Copy link
Contributor Author

Apologies for taking so long to look at this.

No problem. Great feedback.

Noticed an issue in the title detail parsing, where it seems to be taking any two numbers and making them into SxExx, e.g. Yuukoku no Moriarty Part 2 – 02 (Multi) becomes Yuukoku no Moriarty Part – (Multi) S2E02. That might be a bit of an edge case, but maybe put an exception in for part x?

Seems it was matching on the last season search regex where it checks for " - ". I've added a check before this for " Part - " so it should detect this now as season numbers. Not ideal as I expect there are some anime where "Part 2" is not actually season 2 but I expect there will be a few edge cases like this which is why it can be disabled.

It could be useful to also move the quality at the end, so instead of [1080p] blahblah - 01 it becomes something like blahblah - E01 - 1080p.

Good point. Sorting by name won't work correctly with it at the front. I've moved it to the end. Seemed sensible and fairly safe to do this even with title parsing off. While I was changing this, I also adjusted it so that the search string doesn't apply to the quality. This means that searching for episode "10" of something won't return everything which is 1080p.

The size, seeds, leechers, DL factor, and UL factor should all be given generic values, so 512MB, 1, 1, 0, and 1 respectively (you can find these defaults here even though it's for YAML indexers). An example of them being used in a C# indexer:

Size = 0,
Seeders = 1,
Peers = 2,
DownloadVolumeFactor = 0,
UploadVolumeFactor = 1

(just realising ShowRSS' size also gives 0B, that needs changed)

Fixed in both Erai-Raws and ShowRSS.

If you want to give yourself more work you could make the generic files sizes different depending on the quality, e.g. 512MB for 720p releases and 1GB for 1080p releases, etc., or whatever is more suitable (increments of 256MB is probably as accurate as you need to be).

Done. Most seem to have at most 540p, 720p and 1080p so I just checked 3 of the releases and put those sizes as the estimates. It will be wildly inaccurate for batch releases but should be fine for single releases.

Sorry if you had already looked through the code, I ended up refactoring quite a lot for the quality and title parsing stuff.

@ilike2burnthing
Copy link
Contributor

Thanks for the contribution, looks good and is working here.

I changed peers to 2, as it seems Jackett subtracts 1 from whatever is returned. Not entirely sure why (maybe removing your own potential footprint from the count, which doesn't really make sense to me as you won't be leeching many if any yet), but ShowRSS has the same and it works, so 🤷

@ilike2burnthing ilike2burnthing merged commit 42b6aae into Jackett:master Apr 18, 2021
@garfield69
Copy link
Contributor

what is erairawstests.cs ?

@6cUbi57z
Copy link
Contributor Author

what is erairawstests.cs ?

Unit tests currently only for the title parsing. Debugging the title parsing required titles with certain naming (e.g. "Part 2" above) to appear in the titles which were returned. Adding some tests made this easier to debug and develop.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[REQ] New Public Indexer erai-raws
3 participants