Request: Add support for custom search providers #959

Closed
Erik10206 opened this Issue Dec 12, 2012 · 23 comments

Projects

None yet

6 participants

@Erik10206

With the recent events of nzbmatrix bowing out, I am requesting that addition of support for custom providers. I have at least 5 other indexers that I use but they are not in the interface.

@rembo10
Owner
rembo10 commented Dec 13, 2012

Yeah I'm gonna add a field where you can enter the url, api/password and
category ids - it's just parsing the data that might be a little tricky

On Thu, Dec 13, 2012 at 3:28 AM, Erik10206 notifications@github.com wrote:

With the recent events of nzbmatrix bowing out, I am requesting that
addition of support for custom providers. I have at least 5 other indexers
that I use but they are not in the interface.


Reply to this email directly or view it on GitHubhttps://github.com/rembo10/headphones/issues/959.

@Erik10206

Sounds good, let me know if you need help testing.

@samvdb
samvdb commented Dec 13, 2012

Yeah now that nzbmatrix is down we could really use this feature.
Hope you find the time to add it soon!
Great project man

@rembo10
Owner
rembo10 commented Dec 14, 2012

Working on it now, will hopefully have something in the develop branch in
the next few hours. Testers definitely appreciated - I'll update this
thread when i have something up

Thanks

On Thu, Dec 13, 2012 at 4:32 PM, Sam Van der Borght <
notifications@github.com> wrote:

Yeah now that nzbmatrix is down we could really use this feature.
Hope you find the time to add it soon!
Great project man


Reply to this email directly or view it on GitHubhttps://github.com/rembo10/headphones/issues/959#issuecomment-11354359.

@rembo10
Owner
rembo10 commented Dec 17, 2012

Sorry - it's taking a bit longer than I thought to get this working right. It might not be so user friendly and the info parsing might be pretty hit or miss - but it's worth a shot :-)

will have something up for testing today

@Erik10206

No worries. I appreciate you taking a look at it so quickly.

@rembo10
Owner
rembo10 commented Dec 17, 2012

It would absolutely make my life a lot easier if we can set it up so the user just enters in the full api url, except for the query..... so something like:
"http://whatever.us/api?id=harry&apikey=12312abc&maxage=1024&cat=3040&q="

then i can just add the query in to the end. The only problem I guess is that, for example, if you have "cat" as an mp3 category, and decide to search for a flac, with this provider it'll still use the mp3 cat

But to do it the other way, I'd have to separately find out all the variable names (e.g. "age" or "maxage", "q" or "query" or "searchterm" or whatever, all that stuff), plus the variables, plus each category for lossless, mp3 & audiobooks.... I dunno.....

thoughts?

@rembo10
Owner
rembo10 commented Dec 17, 2012

I just don't want it to be too complicated for the user

@Erik10206

I have contacts at one of the index sites. Let me hit them up and see if i
can get a run down of the syntax and all options. If we can figure this one
out , then we can apply it to most of the other indexers since they all use
the same index site engine/template.
On Dec 16, 2012 10:57 PM, "rembo10" notifications@github.com wrote:

I just don't want it to be too complicated for the user


Reply to this email directly or view it on GitHubhttps://github.com/rembo10/headphones/issues/959#issuecomment-11430530.

@rembo10
Owner
rembo10 commented Dec 17, 2012

Is it a newznab site?

@Erik10206

Yes
On Dec 16, 2012 11:03 PM, "rembo10" notifications@github.com wrote:

Is it a newznab site?


Reply to this email directly or view it on GitHubhttps://github.com/rembo10/headphones/issues/959#issuecomment-11430609.

@rembo10
Owner
rembo10 commented Dec 17, 2012

You can already add custom newznab providers.... I was working on a way to
add any type of provider, by setting up the right variables, and then
Headphones can parse the data by trying to search for the appropriate
fields.

Does the custom newznab provider section not work for you?

On Mon, Dec 17, 2012 at 12:11 AM, Erik10206 notifications@github.comwrote:

Yes
On Dec 16, 2012 11:03 PM, "rembo10" notifications@github.com wrote:

Is it a newznab site?


Reply to this email directly or view it on GitHub<
https://github.com/rembo10/headphones/issues/959#issuecomment-11430609>.


Reply to this email directly or view it on GitHubhttps://github.com/rembo10/headphones/issues/959#issuecomment-11430688.

@Erik10206

I wasn't aware that. I just thought i had to use what was in the GUI. If
you could instruct me, that would be great. I have a few others that are
no. Newznab. I can try and reach the admins there for syntax and options.
Let me know.
On Dec 16, 2012 11:48 PM, "rembo10" notifications@github.com wrote:

You can already add custom newznab providers.... I was working on a way to
add any type of provider, by setting up the right variables, and then
Headphones can parse the data by trying to search for the appropriate
fields.

Does the custom newznab provider section not work for you?

On Mon, Dec 17, 2012 at 12:11 AM, Erik10206 notifications@github.comwrote:

Yes
On Dec 16, 2012 11:03 PM, "rembo10" notifications@github.com wrote:

Is it a newznab site?


Reply to this email directly or view it on GitHub<
https://github.com/rembo10/headphones/issues/959#issuecomment-11430609>.


Reply to this email directly or view it on GitHub<
https://github.com/rembo10/headphones/issues/959#issuecomment-11430688>.


Reply to this email directly or view it on GitHubhttps://github.com/rembo10/headphones/issues/959#issuecomment-11431172.

@brunnels

I think the OP may be asking for multiple custom newznab providers like sickbeard does. I was wanting this too because I have memberships to a few newznab sites that only allow a certain number of api requests and nzb downloads each day. What should happen is that if one site fails or denies a request it moves on to the next in the list. Sickbeard also allows ordering through the gui which is also very useful.

I may take a stab at pulling the search provider setup from sickbeard into headphones.

@rembo10
Owner
rembo10 commented Dec 18, 2012

Dude if you can get sickbeards provider setup into headphones, that would be sick. I've had a hard time doing it, getting the ordering in, etc. But the ability to add as many custom newznab providers as you want is in there (just no falling over to the next one if the first one fails, no ordering, etc - it just gathers results from all providers and tries to pick the best release)

But anyways, I might just go ahead and merge the backend code for building your own search url by providing categories, variables, api url, etc. into the branch so whenever this is set up we can easily add the option to add your own non-newznab providers

@brunnels

I'll make a sickproviders branch and work on that today then.

On Tue, Dec 18, 2012 at 4:29 AM, rembo10 notifications@github.com wrote:

Dude if you can get sickbeards provider setup into headphones, that would
be sick. I've had a hard time doing it, getting the ordering in, etc. But
the ability to add as many custom newznab providers as you want is in there
(just no falling over to the next one if the first one fails, no ordering,
etc - it just gathers results from all providers and tries to pick the best
release)

But anyways, I might just go ahead and merge the backend code for building
your own search url by providing categories, variables, api url, etc. into
the branch so whenever this is set up we can easily add the option to add
your own non-newznab providers


Reply to this email directly or view it on GitHubhttps://github.com/rembo10/headphones/issues/959#issuecomment-11480945.

@Erik10206

That would be awesome! I hope you have some luck!! Let me know if there is
anything I can help with.

Erik

On Mon, Dec 17, 2012 at 3:37 PM, Benjamin Runnels
notifications@github.comwrote:

I think the OP may be asking for multiple custom newznab providers like
sickbeard does. I was wanting this too because I have memberships to a few
newznab sites that only allow a certain number of api requests and nzb
downloads each day. What should happen is that if one site fails or denies
a request it moves on to the next in the list. Sickbeard also allows
ordering through the gui which is also very useful.

I may take a stab at pulling the search provider setup from sickbeard into
headphones.


Reply to this email directly or view it on GitHubhttps://github.com/rembo10/headphones/issues/959#issuecomment-11462684.

@brunnels

I've been working a little on it today to wedge it in there but there is a
lot that is different between the headphones and sickbeard as far as the
base goes. It looks like they diverged from a common source long ago
though. Sickbeard is now much more object oriented than headphones so it's
not as simple as pulling a couple of methods from one and plopping into the
other. Getting a class from sickbeard often involves getting many classes
because of inheritance. Many of the inherited classes have methods and
includes that are not needed in headphones so there is a lot of rework to
do.

Has there been any thought put into forking current sickbeard or
couchpotato and re-factoring into a new headphones? Many of the base
functions are common to all three. Would primarily be an exercise of
culling out all thats not common and then moving the backend logic for
searching and post processing over and then creating the interface. Maybe
get it all going at the api level and then code the web interface to talk
to the API?

On Tue, Dec 18, 2012 at 8:37 AM, Erik10206 notifications@github.com wrote:

That would be awesome! I hope you have some luck!! Let me know if there is
anything I can help with.

Erik

On Mon, Dec 17, 2012 at 3:37 PM, Benjamin Runnels
notifications@github.comwrote:

I think the OP may be asking for multiple custom newznab providers like
sickbeard does. I was wanting this too because I have memberships to a
few
newznab sites that only allow a certain number of api requests and nzb
downloads each day. What should happen is that if one site fails or
denies
a request it moves on to the next in the list. Sickbeard also allows
ordering through the gui which is also very useful.

I may take a stab at pulling the search provider setup from sickbeard
into
headphones.


Reply to this email directly or view it on GitHub<
https://github.com/rembo10/headphones/issues/959#issuecomment-11462684>.


Reply to this email directly or view it on GitHubhttps://github.com/rembo10/headphones/issues/959#issuecomment-11488309.

@rembo10
Owner
rembo10 commented Dec 19, 2012

Yeah it's a bit of a mess. Only really the init stuff is based on
sickbeard/sabnzbd. The rest is just crazy, messy intertwining functions. It
needs a refactoring, just not sure if now's the time to do it. Even though
i've been putting it off forever, it's more important to me to get the
provider stuff as flexible as possible first, before this sort of major
undertaking :-)

Honestly, taking sickbeard and trying to rework it back into headphones
would be a lot more work than making the current codebase more object
oriented.... Although it's not a bad idea and would be nice to have a solid
foundation to work from. Lemme think about it
On Dec 19, 2012 4:48 AM, "Benjamin Runnels" notifications@github.com
wrote:

I've been working a little on it today to wedge it in there but there is a
lot that is different between the headphones and sickbeard as far as the
base goes. It looks like they diverged from a common source long ago
though. Sickbeard is now much more object oriented than headphones so it's
not as simple as pulling a couple of methods from one and plopping into
the
other. Getting a class from sickbeard often involves getting many classes
because of inheritance. Many of the inherited classes have methods and
includes that are not needed in headphones so there is a lot of rework to
do.

Has there been any thought put into forking current sickbeard or
couchpotato and re-factoring into a new headphones? Many of the base
functions are common to all three. Would primarily be an exercise of
culling out all thats not common and then moving the backend logic for
searching and post processing over and then creating the interface. Maybe
get it all going at the api level and then code the web interface to talk
to the API?

On Tue, Dec 18, 2012 at 8:37 AM, Erik10206 notifications@github.com
wrote:

That would be awesome! I hope you have some luck!! Let me know if there
is
anything I can help with.

Erik

On Mon, Dec 17, 2012 at 3:37 PM, Benjamin Runnels
notifications@github.comwrote:

I think the OP may be asking for multiple custom newznab providers
like
sickbeard does. I was wanting this too because I have memberships to a
few
newznab sites that only allow a certain number of api requests and nzb
downloads each day. What should happen is that if one site fails or
denies
a request it moves on to the next in the list. Sickbeard also allows
ordering through the gui which is also very useful.

I may take a stab at pulling the search provider setup from sickbeard
into
headphones.


Reply to this email directly or view it on GitHub<
https://github.com/rembo10/headphones/issues/959#issuecomment-11462684>.


Reply to this email directly or view it on GitHub<
https://github.com/rembo10/headphones/issues/959#issuecomment-11488309>.


Reply to this email directly or view it on GitHubhttps://github.com/rembo10/headphones/issues/959#issuecomment-11506763.

@Erik10206

I just got the new updates and have plugged in my newznab
providers. Thanks for turning this around so quickly!!

Erik

On Wed, Dec 19, 2012 at 2:46 AM, rembo10 notifications@github.com wrote:

Yeah it's a bit of a mess. Only really the init stuff is based on
sickbeard/sabnzbd. The rest is just crazy, messy intertwining functions.
It
needs a refactoring, just not sure if now's the time to do it. Even though
i've been putting it off forever, it's more important to me to get the
provider stuff as flexible as possible first, before this sort of major
undertaking :-)

Honestly, taking sickbeard and trying to rework it back into headphones
would be a lot more work than making the current codebase more object
oriented.... Although it's not a bad idea and would be nice to have a
solid
foundation to work from. Lemme think about it
On Dec 19, 2012 4:48 AM, "Benjamin Runnels" notifications@github.com
wrote:

I've been working a little on it today to wedge it in there but there is
a
lot that is different between the headphones and sickbeard as far as the
base goes. It looks like they diverged from a common source long ago
though. Sickbeard is now much more object oriented than headphones so
it's
not as simple as pulling a couple of methods from one and plopping into
the
other. Getting a class from sickbeard often involves getting many
classes
because of inheritance. Many of the inherited classes have methods and
includes that are not needed in headphones so there is a lot of rework
to
do.

Has there been any thought put into forking current sickbeard or
couchpotato and re-factoring into a new headphones? Many of the base
functions are common to all three. Would primarily be an exercise of
culling out all thats not common and then moving the backend logic for
searching and post processing over and then creating the interface.
Maybe
get it all going at the api level and then code the web interface to
talk
to the API?

On Tue, Dec 18, 2012 at 8:37 AM, Erik10206 notifications@github.com
wrote:

That would be awesome! I hope you have some luck!! Let me know if
there
is
anything I can help with.

Erik

On Mon, Dec 17, 2012 at 3:37 PM, Benjamin Runnels
notifications@github.comwrote:

I think the OP may be asking for multiple custom newznab providers
like
sickbeard does. I was wanting this too because I have memberships to
a
few
newznab sites that only allow a certain number of api requests and
nzb
downloads each day. What should happen is that if one site fails or
denies
a request it moves on to the next in the list. Sickbeard also allows
ordering through the gui which is also very useful.

I may take a stab at pulling the search provider setup from
sickbeard
into
headphones.


Reply to this email directly or view it on GitHub<
https://github.com/rembo10/headphones/issues/959#issuecomment-11462684>.


Reply to this email directly or view it on GitHub<
https://github.com/rembo10/headphones/issues/959#issuecomment-11488309>.


Reply to this email directly or view it on GitHub<
https://github.com/rembo10/headphones/issues/959#issuecomment-11506763>.


Reply to this email directly or view it on GitHubhttps://github.com/rembo10/headphones/issues/959#issuecomment-11521905.

@Diovarus

@Erik10206 is there a list online witch i can use best for headphones?

@Erik10206

There is not really one list that has all of the indexers. It takes some
searching but I use nzb.su and dognzb.cr

On Sat, Dec 22, 2012 at 6:08 AM, Diovarus notifications@github.com wrote:

@Erik10206 https://github.com/Erik10206 is there a list online witch i
can use best for headphones?


Reply to this email directly or view it on GitHubhttps://github.com/rembo10/headphones/issues/959#issuecomment-11636597.

@dbenyakar

The ability to enter custom search providers is almost a must these days.
Can this be added?
Or at least a provider that is available in the providers list.

@rembo10 rembo10 closed this Apr 18, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment