Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run on Raspberry Pi -> requires 64-bit OS #3

Closed
vogler opened this issue Jan 10, 2022 · 15 comments
Closed

Run on Raspberry Pi -> requires 64-bit OS #3

vogler opened this issue Jan 10, 2022 · 15 comments
Assignees
Labels
feature New feature or request help wanted Extra attention is needed

Comments

@vogler
Copy link
Owner

vogler commented Jan 10, 2022

Chromium runs on Raspberry Pi, but Playwright seems to not support it out-of-the-box.
Only tried running headless via ssh, maybe it works otherwise.
Reported architectures (uname -m) are
rpi3: armv7l
rpi4: aarch64 (64-bit kernel via arm_64bit=1)

$ npx playwright open google.de
browserType.launch: Browser closed.
==================== Browser output: ====================
<launching> /home/pi/.cache/ms-playwright/chromium-939194/chrome-linux/chrome --disable-background-networking --enable-features=NetworkService,NetworkServiceInProcess --disable-background-timer-throttling --disable-backgrounding-occluded-windows --disable-breakpad --disable-client-side-phishing-detection --disable-component-extensions-with-background-pages --disable-default-apps --disable-dev-shm-usage --disable-extensions --disable-features=ImprovedCookieControls,LazyFrameLoading,GlobalMediaControls,DestroyProfileOnBrowserClose,MediaRouter,AcceptCHFrame --allow-pre-commit-input --disable-hang-monitor --disable-ipc-flooding-protection --disable-popup-blocking --disable-prompt-on-repost --disable-renderer-backgrounding --disable-sync --force-color-profile=srgb --metrics-recording-only --no-first-run --enable-automation --password-store=basic --use-mock-keychain --no-service-autorun --no-sandbox --user-data-dir=/tmp/playwright_chromiumdev_profile-FjzIbQ --remote-debugging-pipe --no-startup-window
<launched> pid=4731
[pid=4731][err] /home/pi/.cache/ms-playwright/chromium-939194/chrome-linux/chrome: 1: Syntax error: Unterminated quoted string
[pid=4731] <process did exit: exitCode=2, signal=null>
[pid=4731] starting temporary directories cleanup
=========================== logs ===========================
<launching> /home/pi/.cache/ms-playwright/chromium-939194/chrome-linux/chrome --disable-background-networking --enable-features=NetworkService,NetworkServiceInProcess --disable-background-timer-throttling --disable-backgrounding-occluded-windows --disable-breakpad --disable-client-side-phishing-detection --disable-component-extensions-with-background-pages --disable-default-apps --disable-dev-shm-usage --disable-extensions --disable-features=ImprovedCookieControls,LazyFrameLoading,GlobalMediaControls,DestroyProfileOnBrowserClose,MediaRouter,AcceptCHFrame --allow-pre-commit-input --disable-hang-monitor --disable-ipc-flooding-protection --disable-popup-blocking --disable-prompt-on-repost --disable-renderer-backgrounding --disable-sync --force-color-profile=srgb --metrics-recording-only --no-first-run --enable-automation --password-store=basic --use-mock-keychain --no-service-autorun --no-sandbox --user-data-dir=/tmp/playwright_chromiumdev_profile-FjzIbQ --remote-debugging-pipe --no-startup-window
<launched> pid=4731
[pid=4731][err] /home/pi/.cache/ms-playwright/chromium-939194/chrome-linux/chrome: 1: Syntax error: Unterminated quoted string
[pid=4731] <process did exit: exitCode=2, signal=null>
[pid=4731] starting temporary directories cleanup
============================================================
    at launchContext (/home/pi/epicgames-claimer/node_modules/playwright-core/lib/cli/cli.js:298:37)
    at Command.listener [as _actionHandler] (/home/pi/epicgames-claimer/node_modules/commander/lib/command.js:488:17)
    at /home/pi/epicgames-claimer/node_modules/commander/lib/command.js:1227:65
    at Command._chainOrCall (/home/pi/epicgames-claimer/node_modules/commander/lib/command.js:1144:12)
    at Command._parseCommand (/home/pi/epicgames-claimer/node_modules/commander/lib/command.js:1227:27)
    at Command._dispatchSubcommand (/home/pi/epicgames-claimer/node_modules/commander/lib/command.js:1050:25)
    at Command._parseCommand (/home/pi/epicgames-claimer/node_modules/commander/lib/command.js:1193:19)
    at Command.parse (/home/pi/epicgames-claimer/node_modules/commander/lib/command.js:897:10) {
  name: 'Error'
}

Adding executablePath: '/usr/bin/chromium-browser' to https://github.com/vogler/epicgames-claimer/blob/fd3f947223733b187c5e1903be1209f8fd3b618c/main.stealth.js#L26 gives

$ npm start

> start
> node main.stealth

node:internal/process/promises:246
          triggerUncaughtException(err, true /* fromPromise */);
          ^

browserType.launchPersistentContext: Browser closed.
==================== Browser output: ====================
<launching> /usr/bin/chromium-browser --disable-background-networking --enable-features=NetworkService,NetworkServiceInProcess --disable-background-timer-throttling --disable-backgrounding-occluded-windows --disable-breakpad --disable-client-side-phishing-detection --disable-component-extensions-with-background-pages --disable-default-apps --disable-dev-shm-usage --disable-extensions --disable-features=ImprovedCookieControls,LazyFrameLoading,GlobalMediaControls,DestroyProfileOnBrowserClose,MediaRouter,AcceptCHFrame --allow-pre-commit-input --disable-hang-monitor --disable-ipc-flooding-protection --disable-popup-blocking --disable-prompt-on-repost --disable-renderer-backgrounding --disable-sync --force-color-profile=srgb --metrics-recording-only --no-first-run --enable-automation --password-store=basic --use-mock-keychain --no-service-autorun --no-sandbox --user-data-dir=userDataDir --remote-debugging-pipe about:blank
<launched> pid=25655
[pid=25655][err] [0110/114804.067560:ERROR:elf_dynamic_array_reader.h(61)] tag not found
=========================== logs ===========================
<launching> /usr/bin/chromium-browser --disable-background-networking --enable-features=NetworkService,NetworkServiceInProcess --disable-background-timer-throttling --disable-backgrounding-occluded-windows --disable-breakpad --disable-client-side-phishing-detection --disable-component-extensions-with-background-pages --disable-default-apps --disable-dev-shm-usage --disable-extensions --disable-features=ImprovedCookieControls,LazyFrameLoading,GlobalMediaControls,DestroyProfileOnBrowserClose,MediaRouter,AcceptCHFrame --allow-pre-commit-input --disable-hang-monitor --disable-ipc-flooding-protection --disable-popup-blocking --disable-prompt-on-repost --disable-renderer-backgrounding --disable-sync --force-color-profile=srgb --metrics-recording-only --no-first-run --enable-automation --password-store=basic --use-mock-keychain --no-service-autorun --no-sandbox --user-data-dir=userDataDir --remote-debugging-pipe about:blank
<launched> pid=25655
[pid=25655][err] [0110/114804.067560:ERROR:elf_dynamic_array_reader.h(61)] tag not found
============================================================
    at /home/pi/epicgames-claimer/main.stealth.js:26:34
    at Object.<anonymous> (/home/pi/epicgames-claimer/main.stealth.js:137:3) {
  name: 'Error'
}

but that's likely due to my rpi4 running headless since it shows the same error without Playwright:

$ chromium-browser
[0110/122436.415559:ERROR:elf_dynamic_array_reader.h(61)] tag not found
[1]    16438 trace trap  chromium-browser
@vogler vogler self-assigned this Jan 10, 2022
@vogler vogler added feature New feature or request help wanted Extra attention is needed labels Jan 20, 2022
@vogler
Copy link
Owner Author

vogler commented Feb 15, 2022

Apparently Ubuntu is supported (only 32 bit?):

rpi3:

$ npx playwright install chrome
Switching to root user to install dependencies...
++ arch
+ [[ armv7l == \a\a\r\c\h\6\4 ]]
+ [[ ! -f /etc/os-release ]]
++ bash -c 'source /etc/os-release && echo $ID'
+ ID=raspbian
+ [[ raspbian != \u\b\u\n\t\u ]]
+ echo 'ERROR: cannot install on raspbian distribution - only Ubuntu is supported'
ERROR: cannot install on raspbian distribution - only Ubuntu is supported
+ exit 1
Failed to install browsers
Error: Failed to install chrome

rpi4:

$ npx playwright install chrome
Switching to root user to install dependencies...
++ arch
+ [[ aarch64 == \a\a\r\c\h\6\4 ]]
+ echo 'ERROR: not supported on Linux Arm64'
ERROR: not supported on Linux Arm64
+ exit 1
Failed to install browsers
Error: Failed to install chrome
```

@vogler
Copy link
Owner Author

vogler commented Apr 9, 2022

Just tried #11 on my RPi4 (SSD): docker build took 6m53s.
It's a 64bit kernel, 32bit userland.

Launching the browser failed:

$ uname -m
aarch64

$ dpkg --print-architecture
armhf

$ docker run --rm -it \
    -p 5900:5900 -p 6080:6080 \
    -v "$(pwd)/userDataDir:/fgc/userDataDir" \
    -v "$(pwd)/screenshots:/fgc/screenshots" \
    -v "$(pwd)/epic-games.js:/fgc/epic-games.js" \
    --name free-games-claimer \
    free-games-claimer

node:internal/process/esm_loader:94
    internalBinding('errors').triggerUncaughtException(
                              ^

browserType.launchPersistentContext: Browser closed.
==================== Browser output: ====================
<launching> /root/.cache/ms-playwright/chromium-978106/chrome-linux/chrome --disable-background-networking --enable-features=NetworkService,NetworkServiceInProcess --disable-background-timer-throttling --disable-backgrounding-occluded-windows --disable-breakpad --disable-client-side-phishing-detection --disable-component-extensions-with-background-pages --disable-default-apps --disable-dev-shm-usage --disable-extensions --disable-features=ImprovedCookieControls,LazyFrameLoading,GlobalMediaControls,DestroyProfileOnBrowserClose,MediaRouter,AcceptCHFrame,AutoExpandDetailsElement,CertificateTransparencyComponentUpdater --allow-pre-commit-input --disable-hang-monitor --disable-ipc-flooding-protection --disable-popup-blocking --disable-prompt-on-repost --disable-renderer-backgrounding --disable-sync --force-color-profile=srgb --metrics-recording-only --no-first-run --enable-automation --password-store=basic --use-mock-keychain --no-service-autorun --export-tagged-pdf --no-sandbox --disable-session-crashed-bubble --restore-last-session --user-data-dir=/fgc/userDataDir --remote-debugging-pipe about:blank
<launched> pid=68
[pid=68][err] /root/.cache/ms-playwright/chromium-978106/chrome-linux/chrome: 1: y: not found
[pid=68][err] /root/.cache/ms-playwright/chromium-978106/chrome-linux/chrome: 1: ?: not found
[pid=68][err] /root/.cache/ms-playwright/chromium-978106/chrome-linux/chrome: 1: cannot open DLS]dq}����������������
[pid=68][err] $-5DTY_hr: No such file
[pid=68][err] /root/.cache/ms-playwright/chromium-978106/chrome-linux/chrome: 1: ������������: not found
[pid=68][err] /root/.cache/ms-playwright/chromium-978106/chrome-linux/chrome: 1: @GKRahqx�������������
                                                                                                      9@MZp: not found
[pid=68][err] /root/.cache/ms-playwright/chromium-978106/chrome-linux/chrome: 1: ������������: not found
[pid=68][err] /root/.cache/ms-playwright/chromium-978106/chrome-linux/chrome: 1: cannot create �W@xO@8
                                                                                                      @$@@@�����
[pid=68][err] W�
[pid=68][err] W�
[pid=68][err] W�W�Wлл��7
                        ��7
                           ��7
                              y�)�
                                  �Y�
                                     �Y�
                                        ��ypfyP�td�F��F��F��7�7Q�td���XX/lib64/ld-linux-x86-64.so.2GNU INFOCrashpad4�
                                                                                                                    GNU&+�U9���Ba p � �������9: Directory nonexistent
[pid=68][err] /root/.cache/ms-playwright/chromium-978106/chrome-linux/chrome: 1: ELF: not found
[pid=68][err] /root/.cache/ms-playwright/chromium-978106/chrome-linux/chrome: 17: Syntax error: word unexpected (expecting ")")
[pid=68] <process did exit: exitCode=2, signal=null>
[pid=68] starting temporary directories cleanup
=========================== logs ===========================
... same as above
============================================================
    at async file:///fgc/epic-games.js:13:17 {
  name: 'Error'
}

Originally posted by @vogler in #11 (comment)

@vogler
Copy link
Owner Author

vogler commented Dec 20, 2022

Tried with firefox on rpi4:

$ npx playwright install --with-deps firefox
BEWARE: your OS is not officially supported by Playwright; installing dependencies for Ubuntu as a fallback.
Installing dependencies...
BEWARE: your OS is not officially supported by Playwright; downloading fallback build.

$ node epic-games.js
node:internal/process/esm_loader:97
    internalBinding('errors').triggerUncaughtException(
                              ^

browserType.launchPersistentContext: Browser.enable): Browser closed.
==================== Browser output: ====================
<launching> /home/pi/.cache/ms-playwright/firefox-1369/firefox/firefox -no-remote -wait-for-browser -foreground -profile /home/pi/free-games-claimer/data/browser -juggler-pipe --hide-crash-restore-bubble about:blank
<launched> pid=26206
[pid=26206][err] /home/pi/.cache/ms-playwright/firefox-1369/firefox/firefox: 1: ELF: not found
[pid=26206][err] /home/pi/.cache/ms-playwright/firefox-1369/firefox/firefox: 2: p: not found
[pid=26206][err] /home/pi/.cache/ms-playwright/firefox-1369/firefox/firefox: 3: p: not found
[pid=26206][err] /home/pi/.cache/ms-playwright/firefox-1369/firefox/firefox: 4: ����p
                                                                                     p/
                                                                                       p/
                                                                                         P�5@-
                                                                                              @=
                                                                                                @=
                                                                                                  @���DDp
                                                                                                         p/
                                                                                                           p/
                                                                                                             : not found
[pid=26206][err] /home/pi/.cache/ms-playwright/firefox-1369/firefox/firefox: 5: ��: not found
[pid=26206][err] /home/pi/.cache/ms-playwright/firefox-1369/firefox/firefox: 6: ��: not found
[pid=26206][err] /home/pi/.cache/ms-playwright/firefox-1369/firefox/firefox: 7: Syntax error: ")" unexpected
[pid=26206] <process did exit: exitCode=2, signal=null>
[pid=26206] starting temporary directories cleanup
=========================== logs ===========================
<launching> /home/pi/.cache/ms-playwright/firefox-1369/firefox/firefox -no-remote -wait-for-browser -foreground -profile /home/pi/free-games-claimer/data/browser -juggler-pipe --hide-crash-restore-bubble about:blank
<launched> pid=26206
[pid=26206][err] /home/pi/.cache/ms-playwright/firefox-1369/firefox/firefox: 1: ELF: not found
[pid=26206][err] /home/pi/.cache/ms-playwright/firefox-1369/firefox/firefox: 2: p: not found
[pid=26206][err] /home/pi/.cache/ms-playwright/firefox-1369/firefox/firefox: 3: p: not found
[pid=26206][err] /home/pi/.cache/ms-playwright/firefox-1369/firefox/firefox: 4: ����p
                                                                                     p/
                                                                                       p/
                                                                                         P�5@-
                                                                                              @=
                                                                                                @=
                                                                                                  @���DDp
                                                                                                         p/
                                                                                                           p/
                                                                                                             : not found
[pid=26206][err] /home/pi/.cache/ms-playwright/firefox-1369/firefox/firefox: 5: ��: not found
[pid=26206][err] /home/pi/.cache/ms-playwright/firefox-1369/firefox/firefox: 6: ��: not found
[pid=26206][err] /home/pi/.cache/ms-playwright/firefox-1369/firefox/firefox: 7: Syntax error: ")" unexpected
[pid=26206] <process did exit: exitCode=2, signal=null>
[pid=26206] starting temporary directories cleanup
============================================================
    at async file:///home/pi/free-games-claimer/epic-games.js:36:17 {
  name: 'Error'
}

Node.js v18.10.0

Same happens for docker:

$ docker build --tag free-games-claimer .
...
$ docker run --rm -it -p 6080:6080 -v fgc:/fgc/data free-games-claimer
Xvfb display server created screen with resolution 1280x1280.
VNC is running on port 5900 (no password!).
noVNC is running on http://localhost:6080

node:internal/process/esm_loader:97
    internalBinding('errors').triggerUncaughtException(
                              ^

browserType.launchPersistentContext: Browser.enable): Browser closed.
==================== Browser output: ====================
<launching> /root/.cache/ms-playwright/firefox-1369/firefox/firefox -no-remote -wait-for-browser -foreground -profile /fgc/data/browser -juggler-pipe --hide-crash-restore-bubble about:blank
<launched> pid=124
[pid=124][err] /root/.cache/ms-playwright/firefox-1369/firefox/firefox: 1: ELF: not found
...

@RobDaNub
Copy link

Works for me on Raspberry Pi 4 with Ubuntu Server 22.04.1 LTS with docker

@vogler
Copy link
Owner Author

vogler commented Jan 12, 2023

docker: no matching manifest for linux/arm/v7 in the manifest list entries.

Did you build the image yourself?

Would be good to know why it doesn't work on mine.

$ lsb_release -d
Description:    Raspbian GNU/Linux 11 (bullseye)
$ uname -a
Linux rpi4 5.15.76-v8+ #1597 SMP PREEMPT Fri Nov 4 12:16:41 GMT 2022 aarch64 GNU/Linux

@RobDaNub
Copy link

Did you build the image yourself?

I tried the prebuild one and I also build it myself.

So i ran

$ docker run --rm -it -p 6080:6080 -v fgc:/fgc/data ghcr.io/vogler/free-games-claimer

and

$ docker build --tag free-games-claimer .
$ docker run --rm -it -p 6080:6080 -v fgc:/fgc/data free-games-claimer

Both work.

I'm using a Pi 4 model B with 4 GB Ram.

$ uname -a
Linux ubuntu 5.15.0-1022-raspi #24-Ubuntu SMP PREEMPT Mon Dec 5 09:58:33 UTC 2022 aarch64 aarch64 aarch64 GNU/Linux

$ lsb_release -d
Description:    Ubuntu 22.04.1 LTS

Here I get arm64 instead of armhf in your post...

$ dpkg --print-architecture
arm64

@vogler
Copy link
Owner Author

vogler commented Jan 12, 2023

Ok, right, the problem is that my Raspbian only has a 64bit kernel and apt packages are still 32bit.
By now they offer 64bit images for everything, but migrating is not supported: https://www.raspberrypi.com/news/raspberry-pi-os-64-bit/

Thanks, I'll mention it in the readme!

@vogler vogler changed the title Run on Raspberry Pi Run on Raspberry Pi -> requires 64-bit OS Jan 12, 2023
@WiLuX-Source
Copy link

Hello I just want to clarify things since nothing is mentioned at the readme at the moment.

You guys want to use playwright but it gives error on raspbian OS. apt packages are still 32bit is the reason why it don't work with 64-bit OS.

Current given solution is use docker and docker uses another OS which is ubuntu and then installs nodejs runs the needed scripts etc etc.

Did you guys try with one with desktop environment?

@WiLuX-Source
Copy link

Raspberry Pi (3, 4, Zero 2): Raspbian won't work since it's 32-bit, but Raspberry Pi OS (64-bit) or Ubuntu will.

Oh my bad the commit isn't mentioned in this issue. Is there any specific reason you guys want to use this with Raspbian?

@vogler
Copy link
Owner Author

vogler commented Feb 8, 2023

apt packages are still 32bit is the reason why it don't work with 64-bit OS.

Playwright also downloads patched browser binaries which are not available for armhf, but for arm64.

Current given solution is use docker and docker uses another OS which is ubuntu and then installs nodejs runs the needed scripts etc etc.

Docker doesn't change anything about the issue, see above. The docker image is for ease of use and originally was introduced as a workaround since epic-games showed captchas when Playwright was run headless. But with Firefox instead of Chrome this seems to no longer be the case.
Playwright only supports Ubuntu and not Alpine which would be preferred for smaller image size. See #13.

Did you guys try with one with desktop environment?

Don't understand what you're asking. You can VNC into the container.

Is there any specific reason you guys want to use this with Raspbian?

It has been the default for a long time, so I guess many people still use it since migration to Raspberry OS is not supported.

@WiLuX-Source
Copy link

Playwright also downloads patched browser binaries which are not available for armhf, but for arm64.

Thanks for clarifying, I understand better now.

Docker doesn't change anything about the issue, see above. The docker image is for ease of use and originally was introduced as a workaround since epic-games showed captchas when Playwright was run headless. But with Firefox instead of Chrome this seems to no longer be the case.

I see. Docker is made for ease of use not for solving raspbian problem.

Don't understand what you're asking. You can VNC into the container.

In the above "Only tried running headless via ssh, maybe it works otherwise." I understood that as I run my raspberry pi OS without desktop environment. Does it mean something else?

It has been the default for a long time, so I guess many people still use it since migration to Raspberry OS is not supported.

I see thanks for clarifying.

@vogler
Copy link
Owner Author

vogler commented Feb 8, 2023

I assume you mean that you use Raspberry Pi OS without desktop environment. What does dpkg --print-architecture say?
If it's arm64 you have the 64-bit version and should be able to either

  1. run the scripts with node (default is headless), or
  2. run via docker (default is non-headless inside the container, you can control the browser with noVNC)

@WiLuX-Source
Copy link

I run my OS with desktop environment. I am not having issues, just wanted to clarify things because I saw something related to rpi.

@vogler
Copy link
Owner Author

vogler commented Feb 11, 2023

Tried qemu emulation of the amd64 image on my armhf-userland, aarch64 rpi4, which seems to work, but is way too slow:

$ docker run --rm --privileged aptman/qus -s -- -p x86_64
cat ./qemu-binfmt-conf.sh | sh -s -- --path=/qus/bin -p x86_64 --suffix -static
Setting /qus/bin/qemu-x86_64-static as binfmt interpreter for x86_64
$ docker run --rm amd64/alpine uname -a
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm/v8) and no specific platform was requested
Linux 55015c61e4c5 5.15.84-v8+ #1613 SMP PREEMPT Thu Jan 5 12:03:08 GMT 2023 x86_64 Linux
$  docker run --rm -it -p 6080:6080 -v fgc:/fgc/data --platform linux/amd64 ghcr.io/vogler/free-games-claimer
Xvfb display server created screen with resolution 1280x1280
VNC is running on port 5900 (no password!)
noVNC (VNC via browser) is running on http://localhost:6080

2023-02-11 16:45:01.982 started checking epic-games
node:internal/process/esm_loader:100
    internalBinding('errors').triggerUncaughtException(
                              ^

browserType.launchPersistentContext: Timeout 180000ms exceeded.

Running the arm64 image somehow didn't work:

$  docker run --rm -it -p 6080:6080 -v fgc:/fgc/data --platform linux/arm64 ghcr.io/vogler/free-games-claimer
Xvfb display server created screen with resolution 1280x1280

@vogler
Copy link
Owner Author

vogler commented Mar 30, 2023

Closing this since it's solved (64-bit OS work, Playwright prob. won't support 32-bit OS) and linked in the Readme.

@vogler vogler closed this as completed Mar 30, 2023
free-games-claimer automation moved this from To do to Done Mar 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature or request help wanted Extra attention is needed
Projects
Development

No branches or pull requests

3 participants