[sabnzbd develop branch] spotweb gives error when pushing to sabnzbd

[Hydranet]

I am running the develop branch of sabnzbd, I just noticed today that when I login to spotweb and push an nzb to sabnzbd the green download error gets a red cross through it as if the pushing has failed. However sabnzbd does receive the nzb and it does start downloading, I tried it with other titles and I get the same result there.

I see this in the apache logs:
[Thu Jun 11 19:56:33.655419 2020] [php7:notice] [pid 6121] [client 0.0.0.0.29:33658] sabnzbd push : http returncode: 200 / / {'status': True, 'nzo_ids': ['SABnzbd_nzo_j4cmwk47']}\n, referer: https://spotweb.feedmebits.nl/?page=index&search[tree]=cat0_z0_a9&search[value][]=

In my sabnzbd logs I see sabnzbd receiving the file from spotweb and starting to download:
removed

Here is what I see in spotweb when I try it:

The really strange thing is when I add the download to sonarr/radarr and then request my searcher to look for it on my spotweb indexer I don't get any errors in de apache logs. When I then refresh spotweb it shows it correctly download the a green check mark.

I am using:

• python36 (3.6.8)
• CentOS8
• apache 2.4 (Running sabnzbd behind apache via a reverse proxy)

spotweb hasn't been updated in a few months:

commit bf7d95d18576707cfb4c1dc77181fca3fba7cc97
Author: Sweepr <Sweepr@users.noreply.github.com>
Date:   Sat Apr 4 22:10:48 2020 +0200

    Update README.md

So thinking it may have been one of the recent updates to the develop branch.

I don't log into spotweb very often since I use radarr and sonarr so I wouldn't be able to tell you since when this started.

[sanderjo]

You have 79 Tracebacks in your log! Shouldn't you solve that first?

Traceback (most recent call last):
  File "/opt/sabnzbd/sabnzbd/filesystem.py", line 529, in create_all_dirs
    os.makedirs(path, mode=mask, exist_ok=True)
  File "/usr/lib64/python3.6/os.py", line 210, in makedirs
    makedirs(head, mode, exist_ok)
  File "/usr/lib64/python3.6/os.py", line 220, in makedirs
    mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/data/downloads/tv'

Disk / rights problem?

Traceback (most recent call last):
  File "/usr/local/lib64/python3.6/site-packages/cherrypy/_cprequest.py", line 638, in respond
    self._do_respond(path_info)
  File "/usr/local/lib64/python3.6/site-packages/cherrypy/_cprequest.py", line 697, in _do_respond
    response.body = self.handler()
  File "/usr/local/lib64/python3.6/site-packages/cherrypy/lib/encoding.py", line 219, in __call__
    self.body = self.oldhandler(*args, **kwargs)
  File "/usr/local/lib64/python3.6/site-packages/cherrypy/_cpdispatch.py", line 54, in __call__
    return self.callable(*self.args, **self.kwargs)
  File "/opt/sabnzbd/sabnzbd/interface.py", line 403, in api
    return api_handler(kwargs)
  File "/opt/sabnzbd/sabnzbd/api.py", line 116, in api_handler
    response = _api_table.get(mode, (_api_undefined, 2))[0](name, output, kwargs)
  File "/opt/sabnzbd/sabnzbd/api.py", line 777, in _api_browse
    paths = folders_at_path(name, True, show_hidden)
  File "/opt/sabnzbd/sabnzbd/utils/pathbrowser.py", line 103, in folders_at_path
    for filename in os.listdir(path):
PermissionError: [Errno 13] Permission denied: '/data/downloads/sabnzb'
2020-06-09 15:55:42,082::INFO::[_cplogging:216] [09/Jun/2020:15:55:42] HTTP 
Request Headers:
  Remote-Addr: 127.0.0.1
  HOST: localhost:8080
  USER-AGENT: Mozilla/5.0 (X11; Linux x86_64; rv:77.0) Gecko/20100101 Firefox/77.0
  ACCEPT: application/json, text/javascript, */*; q=0.01
  ACCEPT-LANGUAGE: en-US,en;q=0.5
  ACCEPT-ENCODING: gzip, deflate, br
  REFERER: https://downloads.feedmebits.nl/sabnzbd/config/folders/
  X-REQUESTED-WITH: XMLHttpRequest
  COOKIE: login_cookie=<HASH>ab5c44c9775c2ef; login_salt=532
  DNT: 1
  X-FORWARDED-FOR: 10.31.251.11
  X-FORWARDED-HOST: downloads.feedmebits.nl
  X-FORWARDED-SERVER: downloads.feedmebits.nl
  CONNECTION: Keep-Alive

more disk problems ...

[Safihre]

I did make a change to the API, the output has changed slightly.
Maybe Spotnet can't handle that. But I also haven't made a release yet since that change, so nobody could pick up on it.
Can you see the request spotnet made for this response?

[Thu Jun 11 19:56:33.655419 2020] [php7:notice] [pid 6121] [client 0.0.0.0.29:33658] sabnzbd push : http returncode: 200 / / {'status': True, 'nzo_ids': ['SABnzbd_nzo_j4cmwk47']}\n, referer: https://spotweb.feedmebits.nl/?page=index&search[tree]=cat0_z0_a9&search[value][]=

[Hydranet]

@sanderjo Those were from yesterday, I solved those yesterday. The entries with today's date don't have any tracebacks, I did check those. I checked the downloads I did today via spotweb and there I didn't see any tracebacks, neither when I scrolled through all today's logs.

[Hydranet]

I did make a change to the API, the output has changed slightly.
Maybe Spotnet can't handle that. But I also haven't made a release yet since that change, so nobody could pick up on it.
Can you see the request spotnet made for this response?

[Thu Jun 11 19:56:33.655419 2020] [php7:notice] [pid 6121] [client 0.0.0.0.29:33658] sabnzbd push : http returncode: 200 / / {'status': True, 'nzo_ids': ['SABnzbd_nzo_j4cmwk47']}\n, referer: https://spotweb.feedmebits.nl/?page=index&search[tree]=cat0_z0_a9&search[value][]=

@Safihre
I tried that, when I open that url it just opens the category the movie is from which I tried downloading via spotweb.

The screenshots here is from after I tried download the same download via radarr, hence you not seeing a download arrow with a red cross in it.

[Safihre]

The change I made is this:
Long, long time ago, SABnzbd didn't support JSON output and all the API-calls would return plain text responses. When adding an NZB it would just respond ok.
Around 2012 we added JSON output in which we respond with the ID's of the jobs that were added (since it can be multiple). For backwards-compatibility a special mode was added that could still output ok, this is what Spotnet uses.
After 8 more years, I decided to remove this backwards compatibility.

[Hydranet]

In which commit id did you remove that? I could try undoing that commit and then seeing how spotweb reacts before that commit in the develop branch.

[Safihre]

308c0ec

[Hydranet]

When I switch to the previous commit it works as expected

commit 2f45a87f125aa1bf6c2b595b767f3eb0c4b247e5
Author: Safihre <safihre@sabnzbd.org>
Date:   Mon Jun 8 16:57:18 2020 +0200

    API-method addurl only supports adding 1 URL at a time

[Safihre]

While annoying, I do want to use this major release to including breaking changes. Otherwise it might be another 8 years 🤔
Spotnet will have to adjust..

[thezoggy]

looks like spotnab sends output=jsonp which is invalid. should be json. so all this time sab just silently handled that typo and now we dont

https://github.com/spotweb/spotweb/blob/develop/lib/services/NzbHandler/Services_NzbHandler_Clientsabnzbd.php#L14

[thezoggy]

and about the issue adding, spotnab uses output=text and then checks if 'ok' was returned
https://github.com/spotweb/spotweb/blob/develop/lib/services/NzbHandler/Services_NzbHandler_Pushsabnzbd.php#L56

they should just update to json and handle that.. as that will fix 3.x+ support and still work fine with 2.x and older as well

[Safihre]

Thanks for looking into that!

[Hydranet]

@thezoggy

Thanks! I created an issue for spotweb referring back to this one.

[sanderjo]

@thezoggy

Thanks! I created an issue for spotweb referring back to this one.

With the two advices of @thezoggy , maybe you can patching it yourself and if it works send a PR to spotweb?

[mesa57]

@Safihre :
Could you also amend you're api documentation ?

Request types

Supported output types are json, xml and text and can be specified in the request:

http://host:port/sabnzbd/api?output=json

[Safihre]

Text is still supported, but it doesn't just output ok anymore but more details. That's all that changed 🙂

[thezoggy]

as a side note, this change also breaks the original sickbeard (as it also checked for 'ok'). which is one of the reasons why shy left this the way it was for so long
https://github.com/midgetspy/Sick-Beard/blob/development/sickbeard/sab.py#L101

original sb has been abandoned forever.. but there is a lot of people that still use it. those people really should move to one of the various sb forks that does get updated (checked those and they all use json output).

[sanderjo]

Thanks to analysis of @thezoggy I've reported a bug report at Spotweb.

[Sweepr]

This fix has now been pushed into master branch and released as version 1.4.6 as of yesterday.

See: https://github.com/spotweb/spotweb/tree/1.4.6