Failure to Post Process and Tag Comic

[raph521]

Describe the bug
Hi, I've recently updated my docker container from linuxserver/mylar to lsiodev/mylar3, using the existing app config and it mostly works great, except that post processing / tagging fails. It also fails when I manually try to tag the comic from the comic details page.

CarePackage
carepackage.zip

Environment (please complete the following information):

• OS: https://hub.docker.com/r/lsiodev/mylar3 / Raspbian
• Branch / commit: latest

Additional information
The specific error message looks to be the following:

27-Aug-2020 10:28:49 - WARNING :: mylar.run.257 : CP Server Thread-8 : [META-TAGGER][COMIC-TAGGER][CBR-TO-CBZ] Failed to convert cbr to cbz - check permissions on folder : /config/mylar/cache and/or the location where Mylar is trying to tag the files from.

I don't think there are any issues with permissions on that directory as I can very briefly see the mylar_ directory get created inside of cache with the cbr file inside when I repeatedly type "ls" after initiating a manual tag.

I took a stab at trying to debug by having the value of "out" in cmtagmylar.py get printed - it's visible in the carepackage, I think but here's a snippet of it:

Archive failed to export!
Unexpected exception in readArchiveFile(): [Cannot execute unrar (rarfile.UNRAR_TOOL='')] for /config/mylar/cache/mylar_h1rmtgvo/...

[raph521]

After comparing the Dockerfile of lsiodev/mylar3 to hotio/mylar3, I noticed that lsio's container uses the requirements provided within mylar3's requirements.txt, while hotio's container installs a hardcoded list of requirements.

I tried just installing all of the requirements in hotio's Dockerfile directly in my lsio container and it seems to have fixed it:

apk add --no-cache nodejs python3 py3-openssl py3-setuptools py3-six py3-openssl py3-requests py3-urllib3 libjpeg-turbo unrar && \
    apk add --no-cache --virtual=build-dependencies py3-pip gcc zlib-dev libjpeg-turbo-dev python3-dev musl-dev make g++ && \
    pip3 install --no-cache-dir --upgrade \
        APScheduler>=3.6.3 \
        beautifulsoup4>=4.8.2 \
        cfscrape>=2.0.8 \
        cheroot==8.2.1 \
        CherryPy>=18.5.0 \
        configparser>=4.0.2 \
        feedparser>=5.2.1 \
        Mako>=1.1.0 \
        natsort>=3.5.2 \
        Pillow>=4.2.1,~=6.2.2 \
        portend>=2.6 \
        pytz>=2019.3 \
        simplejson>=3.17.0 \
        tzlocal>=2.0.0 \
        unrar>=0.3 \
        unrar-cffi==0.1.0a5 && \
    apk del --purge build-dependencies

Post processing now works and I can manually tag.

I noticed that unrar and unrar-cffi were removed from requirements.txt at some point. Do they need to be re-added?

[evilhero]

Actually I think that all we need to do is add back in the unrar dependency. The unrar-cffi is an external dependency that was created by the CT owner in order to try and speed up unraring. It works, but it also doesn't have a wheel file for anything above python3.8.x for Windows users, and some of the linux distributions have had problems compiling due to the deep dependencies that it requires.

The traceback error is saying it can't find the unrar binary in your path, which makes sense if you're running Docker and it was never installed for usage. I'll have to run thru some tests and see since I removed the unrar-cffi from the CT within Mylar, I might have just missed something, and / or it could be a simple as just adding back in that dependency.

EDIT. Just tested - uninstalled unrar-cffi and unrar (cause I had both). Unset my mylar3/.ComicTagger/settings file that had rar_exe_path set. Tried to tag- got exact same error as you. Then realized what I said above was moot. From pip unrar and unrar-cffi aren't needed for actual unraring as they're really just doing the job of a wrapper to the actual unrar binary.

So it actually needs the unrar binary installed on your system somewhere where it can access it. Once you have it installed, and if it's not installed in your path and/or you're still getting the error you posted initially, you'd need to set the rar_exe_path in the .ComicTagger/settings to the path location of the unrar binary. Now the .ComicTagger directory will be in one of 2 places depending on your installation - either in the user's home directory, hidden cause of the ., or and this probably applies to you as you run docker, it would be in root of the mylar directory.

#428 should fix the problem with the unrar binary not being detected if it's in the path, but the rar_exe_path is not initially set (like on first-run, new installations).

[raph521]

Awesome, thanks for the quick response and thorough explanation!

[barbequesauce]

Raph521 did the commit fix your issue? Are we OK to close this?

[raph521]

Sorry for the delay, I was waiting for the commit to show up in the master build.

I've re-built the lsiodev/mylar3 image with the python3-dev branch and the issue is MOSTLY resolved.

The only gotcha is that this image relies on the requirements stated in requirements.txt, and requirements.txt is still missing unrar. Once I installed unrar manually within my container (via apk add --no-cache unrar), the issue was completely resolved.

Could requirements.txt be updated to add unrar?

[evilhero]

This is still related to #428, unrar via pip is not required for unrar'ing - it's a wrapper for the actual unrar library that should be located on your system somewhere. #428 hasn't been pushed to master as of yet, so you'll still experience this error if the unrar binary isn't in your path.

The apk add will install the unrar binary on your docker install - which is then obviously in your PATH environment which would then be found by Mylar. The requirements.txt is for pip installation (ie. python module installs, not binaries) - so adding unrar back into requirements.txt will just add in the unrar wrapper, not the actual binary.

The Unrar requirement has been added to the README.md indicating that the binary must be on your system somewhere so that it's either accessible via the PATH environment, or if not in the path - so that it can be added to the unrar_exe_path field in the settings file for ComicTagger manually.

[raph521]

Ah thanks for clearing that up!

I took a look at linuxserver/mylar3's Dockerfile and I do now see that they're only using requirements.txt for pip installation - my mistake.

I'll reply back on linuxserver/docker-mylar3#1 that unrar should be added to their image and I think that should close both of these issues.