[Package]: Flaresolverr #631

Vojtak42 · 2023-10-07T21:08:10Z

Name

Flaresolverr

Home page URL

No response

Source code URL

https://github.com/FlareSolverr/FlareSolverr

Packaging policy acknowledgement

I understand Tur Package Policy.

Additional information

Officially arm64 is only supported on Docker (also due to chromium dependency) but there is (untested) guide from author for installing it on Arm64: FlareSolverr/FlareSolverr#88.
Thanks in advance
I would really appreciate it.

licy183 · 2023-10-11T18:32:43Z

It is hard to package this. Maybe it will be able to package using venv as pyinstaller doesn't work on Termux.

romanovj · 2023-10-12T16:57:35Z

you can proot into docker container, here is my output (aarch64, termux from f-droid, no root)

2023-10-12 16:37:17 INFO     FlareSolverr 3.3.6
2023-10-12 16:37:17 INFO     Testing web browser installation...
2023-10-12 16:37:17 INFO     Platform: Linux-4.19.136-perf+-aarch64-with-glibc2.31
2023-10-12 16:37:17 INFO     Chrome / Chromium path: /usr/bin/chromium
2023-10-12 16:37:17 INFO     Chrome / Chromium major version: 116
2023-10-12 16:37:17 INFO     Launching web browser...
2023-10-12 16:37:20 INFO     FlareSolverr User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36
2023-10-12 16:37:20 INFO     Test successful!
2023-10-12 16:37:20 INFO     Serving on http://0.0.0.0:8191

2023-10-12 16:48:45 INFO     Incoming request => POST /v1 body: {'cmd': 'request.get', 'url': 'https://iptorrents.com/', 'maxTimeout': 60000}
2023-10-12 16:48:49 INFO     Challenge detected. Title found: Just a moment...
2023-10-12 16:48:59 INFO     Challenge solved!
2023-10-12 16:48:59 INFO     Response in 14.253 s
2023-10-12 16:48:59 INFO     127.0.0.1 POST http://localhost:8191/v1 200 OK

use proot-distro to proot into debian/ubuntu, update, install skopeo(for downloading) and umoci (for unpacking)

cd /data/data/com.termux/files/home
skopeo copy docker://ghcr.io/flaresolverr/flaresolverr:v3.3.6 oci:flaresolverr:v3.3.6
umoci unpack --image flaresolverr:v3.3.6 rootfs

proot into this rootfs from termux and execute:

su flaresolverr
python -u /app/flaresolverr.py

Vojtak42 · 2023-10-12T18:38:45Z

you can proot into docker container, here is my output (aarch64, termux from f-droid, no root)

2023-10-12 16:37:17 INFO     FlareSolverr 3.3.6
2023-10-12 16:37:17 INFO     Testing web browser installation...
2023-10-12 16:37:17 INFO     Platform: Linux-4.19.136-perf+-aarch64-with-glibc2.31
2023-10-12 16:37:17 INFO     Chrome / Chromium path: /usr/bin/chromium
2023-10-12 16:37:17 INFO     Chrome / Chromium major version: 116
2023-10-12 16:37:17 INFO     Launching web browser...
2023-10-12 16:37:20 INFO     FlareSolverr User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36
2023-10-12 16:37:20 INFO     Test successful!
2023-10-12 16:37:20 INFO     Serving on http://0.0.0.0:8191

2023-10-12 16:48:45 INFO     Incoming request => POST /v1 body: {'cmd': 'request.get', 'url': 'https://iptorrents.com/', 'maxTimeout': 60000}
2023-10-12 16:48:49 INFO     Challenge detected. Title found: Just a moment...
2023-10-12 16:48:59 INFO     Challenge solved!
2023-10-12 16:48:59 INFO     Response in 14.253 s
2023-10-12 16:48:59 INFO     127.0.0.1 POST http://localhost:8191/v1 200 OK

use proot-distro to proot into debian/ubuntu, update, install skopeo(for downloading) and umoci (for unpacking)

cd /data/data/com.termux/files/home
skopeo copy docker://ghcr.io/flaresolverr/flaresolverr:v3.3.6 oci:flaresolverr:v3.3.6
umoci unpack --image flaresolverr:v3.3.6 rootfs

proot into this rootfs from termux and execute:

su flaresolverr
python -u /app/flaresolverr.py

Thanks, but su flaresolverr doesn't work for me (root@localhost:/data/data/com.termux/files/home/rootfs/rootfs/app# su flaresolverr su: user flaresolverr does not exist or the user entry does not contain all the required fields). And when i cd to the app folder and run python3 -u flaresolverr.py, it can't import certifi and when i try to install it, i get this:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pip/_vendor/platformdirs/android.py", line 85, in _android_folder
    from jnius import autoclass
ModuleNotFoundError: No module named 'jnius'
                                                                  During handling of the above exception, another exception occurred:

Traceback (most recent call last):                                  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/usr/lib/python3/dist-packages/pip/__main__.py", line 29, in <module>
    from pip._internal.cli.main import main as _main
  File "/usr/lib/python3/dist-packages/pip/_internal/cli/main.py", line 9, in <module>
    from pip._internal.cli.autocompletion import autocomplete
  File "/usr/lib/python3/dist-packages/pip/_internal/cli/autocompletion.py", line 10, in <module>
    from pip._internal.cli.main_parser import create_main_parser
  File "/usr/lib/python3/dist-packages/pip/_internal/cli/main_parser.py", line 8, in <module>
    from pip._internal.cli import cmdoptions
  File "/usr/lib/python3/dist-packages/pip/_internal/cli/cmdoptions.py", line 23, in <module>
    from pip._internal.cli.parser import ConfigOptionParser         File "/usr/lib/python3/dist-packages/pip/_internal/cli/parser.py", line 12, in <module>                                               from pip._internal.configuration import Configuration, ConfigurationError                                                         File "/usr/lib/python3/dist-packages/pip/_internal/configuration.py", line 26, in <module>                                            from pip._internal.utils.logging import getLogger
  File "/usr/lib/python3/dist-packages/pip/_internal/utils/logging.py", line 27, in <module>
    from pip._internal.utils.misc import ensure_dir
  File "/usr/lib/python3/dist-packages/pip/_internal/utils/misc.py", line 39, in <module>                                               from pip._internal.locations import get_major_minor_version
  File "/usr/lib/python3/dist-packages/pip/_internal/locations/__init__.py", line 14, in <module>
    from . import _distutils, _sysconfig
  File "/usr/lib/python3/dist-packages/pip/_internal/locations/_distutils.py", line 19, in <module>
    from .base import get_major_minor_version
  File "/usr/lib/python3/dist-packages/pip/_internal/locations/base.py", line 12, in <module>
    USER_CACHE_DIR = appdirs.user_cache_dir("pip")
  File "/usr/lib/python3/dist-packages/pip/_internal/utils/appdirs.py", line 17, in user_cache_dir                                      return _appdirs.user_cache_dir(appname, appauthor=False)
  File "/usr/lib/python3/dist-packages/pip/_vendor/platformdirs/__init__.py", line 114, in user_cache_dir
    return PlatformDirs(appname=appname, appauthor=appauthor, version=version, opinion=opinion).user_cache_dir
  File "/usr/lib/python3/dist-packages/pip/_vendor/platformdirs/android.py", line 43, in user_cache_dir
    return self._append_app_name_and_version(_android_folder(), "cache")
  File "/usr/lib/python3/dist-packages/pip/_vendor/platformdirs/android.py", line 97, in _android_folder                                raise OSError("Cannot find path to android app folder")
OSError: Cannot find path to android app folder

EDIT: I successfully installe certifi from source

romanovj · 2023-10-12T19:09:54Z

@Vojtak42 after downloading and unpacking container you should exit from debian/ubuntu and proot from native termux into new rootfs by using script

#!/data/data/com.termux/files/usr/bin/bash
cd $(dirname $0)
## unset LD_PRELOAD in case termux-exec is installed
unset LD_PRELOAD
command="proot"
## uncomment following line if you are having FATAL: kernel too old message.
#command+=" -k 4.14.81"
command+=" --link2symlink"
command+=" -0"
command+=" -r rootfs"
command+=" -b /dev"
command+=" -b /proc"
command+=" -b /sys"
command+=" -b rootfs/tmp:/dev/shm"
#command+=" -b /data/data/com.termux"
#command+=" -b /:/host-rootfs"
#command+=" -b /sdcard"
#command+=" -b /storage"
command+=" -b /mnt"
command+=" -w /root"
command+=" /usr/bin/env -i"
command+=" HOME=/root"
command+=" PATH=/usr/local/sbin:/usr/local/bin:/bin:/usr/bin:/sbin:/usr/sbin:/usr/games:/usr/local/games"
command+=" TERM=$TERM"
command+=" LANG=C.UTF-8"
command+=" /bin/bash --login"
com="$@"
if [ -z "$1" ];then
    exec $command
else
    $command -c "$com"
fi

place this script in ~/rootfs folder inside which you have another rootfs folder

romanovj · 2023-10-12T19:59:51Z

you shouldn't install anything

post output

/app/chromedriver

Vojtak42 · 2023-10-12T20:03:25Z

/app/chromedriver: error while loading shared libraries: libatomic.so.1: cannot open shared object file: No such file or directory

Vojtak42 · 2023-10-12T20:04:28Z

The python -u flaresolverr.py outputs this:

2023-10-12 19:39:28 INFO     FlareSolverr 3.3.6
2023-10-12 19:39:28 INFO     Testing web browser installation...
2023-10-12 19:39:28 INFO     Platform: Linux-4.14.180-perf-g4b73fd3-aarch64-with-glibc2.31
2023-10-12 19:39:28 INFO     Chrome / Chromium path: /usr/bin/chromium
2023-10-12 19:39:28 INFO     Chrome / Chromium major version: 117
2023-10-12 19:39:28 INFO     Launching web browser...
Traceback (most recent call last):
  File "/app/utils.py", line 296, in get_user_agent
    driver = get_webdriver()
             ^^^^^^^^^^^^^^^
  File "/app/utils.py", line 177, in get_webdriver
    driver = uc.Chrome(options=options, browser_executable_path=browser_executable_path,
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/undetected_chromedriver/__init__.py", line 474, in __init__
    super(Chrome, self).__init__(
  File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/chrome/webdriver.py", line 45, in __init__
    super().__init__(
  File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/chromium/webdriver.py", line 53, in __init__
    self.service.start()
  File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/common/service.py", line 116, in start
    raise WebDriverException(f"Can not connect to the Service {self._path}")
selenium.common.exceptions.WebDriverException: Message: Can not connect to the Service /app/chromedriver


During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/app/flaresolverr.py", line 105, in <module>
    flaresolverr_service.test_browser_installation()
  File "/app/flaresolverr_service.py", line 72, in test_browser_installation
    user_agent = utils.get_user_agent()
                 ^^^^^^^^^^^^^^^^^^^^^^
  File "/app/utils.py", line 300, in get_user_agent
    raise Exception("Error getting browser User-Agent. " + str(e))
Exception: Error getting browser User-Agent. Message: Can not connect to the Service /app/chromedriver

romanovj · 2023-10-12T20:06:15Z

wtf you did with rootfs?

ldd /app/chromedriver
        linux-vdso.so.1 (0x000000798b3ac000)
        libatomic.so.1 => /usr/lib/aarch64-linux-gnu/libatomic.so.1
.....

Vojtak42 · 2023-10-12T20:07:25Z

Before i installed chromium using apt, the only different thing was that there was: Chrome / Chromium major version: 116

Vojtak42 · 2023-10-12T20:22:09Z

I will probably tomorrow start from beginning and i will see.

romanovj · 2023-10-12T20:24:14Z

from root user
apt reinstall libatomic1:arm64

or just delete and unpack again, do not update/upgrade anything

Vojtak42 · 2023-10-12T20:45:25Z

I reinstalled everything and now i get

/app/chromedriver
Starting ChromeDriver 116.0.5845.180 (fa9889b94260f950a6ab6bfc140732263651f355-refs/branch-heads/5845@{#1746}) on port 9515
Only local connections are allowed.
Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
ChromeDriver was started successfully.

But same error with flaresolverr.

romanovj · 2023-10-12T20:59:16Z

disable vpn if you have

start with
LOG_LEVEL=debug HOST=localhost python -u /app/flaresolverr.py

romanovj · 2023-10-12T21:40:33Z

upd. do not resolve localhost
echo 127.0.0.1 localhost > /etc/hosts

also you can try to change dns (default 1.1.1.1)

echo nameserver 8.8.8.8 > /etc/resolv.conf
error Exception: Error getting browser User-Agent. Message: Can not connect to the Service /app/chromedriver

echo nameserver 9.9.9.9 > /etc/resolv.conf
ok, but isn't working with random vpn

echo nameserver 192.168.1.1 > /etc/resolv.conf
ok, ok with vpn (it's my router ip)

PS. I don't know why it's try to resolve localhost and why CF and google dns aren't returning 127.0.0.1 for localhost
PPS HOST=127.0.0.1 doesn't change anything

#google
nslookup localhost 8.8.8.8                                    Server:         8.8.8.8
Address:        8.8.8.8#53

** server can't find localhost: NXDOMAIN

#CF
flaresolverr@localhost:~$ nslookup localhost 1.1.1.1
Server:         1.1.1.1
Address:        1.1.1.1#53

** server can't find localhost: NXDOMAIN

#quad dns
flaresolverr@localhost:~$ nslookup localhost 9.9.9.9
Server:         9.9.9.9
Address:        9.9.9.9#53

Non-authoritative answer:
Name:   localhost
Address: 127.0.0.1
Name:   localhost
Address: ::1

#adguard 
nslookup localhost 94.140.14.14
Server:         94.140.14.14
Address:        94.140.14.14#53

Name:   localhost
Address: 127.0.0.1
Name:   localhost
Address: ::1

romanovj · 2023-10-12T21:49:09Z

also add 'dumb-init' in order to kill zombie processes

LOG_LEVEL=debug HOST=localhost dumb-init python -u /app/flaresolverr.py

Vojtak42 · 2023-10-13T05:46:57Z

Thank you! It works well with this: echo 127.0.0.1 localhost > /etc/hosts

Vojtak42 added the Package request label Oct 7, 2023

Vojtak42 closed this as completed Oct 23, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Package]: Flaresolverr #631

[Package]: Flaresolverr #631

Vojtak42 commented Oct 7, 2023

licy183 commented Oct 11, 2023

romanovj commented Oct 12, 2023 •

edited

Loading

Vojtak42 commented Oct 12, 2023 •

edited

Loading

romanovj commented Oct 12, 2023 •

edited

Loading

romanovj commented Oct 12, 2023 •

edited

Loading

Vojtak42 commented Oct 12, 2023

Vojtak42 commented Oct 12, 2023

romanovj commented Oct 12, 2023

Vojtak42 commented Oct 12, 2023 •

edited

Loading

Vojtak42 commented Oct 12, 2023 •

edited

Loading

romanovj commented Oct 12, 2023

Vojtak42 commented Oct 12, 2023

romanovj commented Oct 12, 2023 •

edited

Loading

romanovj commented Oct 12, 2023 •

edited

Loading

romanovj commented Oct 12, 2023

Vojtak42 commented Oct 13, 2023

[Package]: Flaresolverr #631

[Package]: Flaresolverr #631

Comments

Vojtak42 commented Oct 7, 2023

Name

Home page URL

Source code URL

Packaging policy acknowledgement

Additional information

licy183 commented Oct 11, 2023

romanovj commented Oct 12, 2023 • edited Loading

Vojtak42 commented Oct 12, 2023 • edited Loading

romanovj commented Oct 12, 2023 • edited Loading

romanovj commented Oct 12, 2023 • edited Loading

Vojtak42 commented Oct 12, 2023

Vojtak42 commented Oct 12, 2023

romanovj commented Oct 12, 2023

Vojtak42 commented Oct 12, 2023 • edited Loading

Vojtak42 commented Oct 12, 2023 • edited Loading

romanovj commented Oct 12, 2023

Vojtak42 commented Oct 12, 2023

romanovj commented Oct 12, 2023 • edited Loading

romanovj commented Oct 12, 2023 • edited Loading

romanovj commented Oct 12, 2023

Vojtak42 commented Oct 13, 2023

romanovj commented Oct 12, 2023 •

edited

Loading

Vojtak42 commented Oct 12, 2023 •

edited

Loading

romanovj commented Oct 12, 2023 •

edited

Loading

romanovj commented Oct 12, 2023 •

edited

Loading

Vojtak42 commented Oct 12, 2023 •

edited

Loading

Vojtak42 commented Oct 12, 2023 •

edited

Loading

romanovj commented Oct 12, 2023 •

edited

Loading

romanovj commented Oct 12, 2023 •

edited

Loading