Chromium randomly hangs before exit sometimes causing Singlefile, DOM, PDF, and Screenshot methods to timeout

[gerroon]

Just checking to see if it is just me, or a network issue or a new version glitch.

Try this maybe which is in my list

https://arstechnica.com/?p=1995005

ArchiveBox v0.7.2+editable COMMIT_HASH=825b882 BUILD_TIME=2024-01-10 05:23:33 1704864213
IN_DOCKER=True IN_QEMU=False ARCH=x86_64 OS=Linux PLATFORM=Linux-5.14.0-4-amd64-x86_64-with-glibc2.36 PYTHON=Cpython
FS_ATOMIC=True FS_REMOTE=True FS_USER=1000:1000 FS_PERMS=644
DEBUG=False IS_TTY=True TZ=UTC SEARCH_BACKEND=sonic LDAP=False

[i] Dependency versions:
 √  PYTHON_BINARY         v3.11.7         valid     /usr/local/bin/python3.11
 √  SQLITE_BINARY         v2.6.0          valid     /usr/local/lib/python3.11/sqlite3/dbapi2.py
 √  DJANGO_BINARY         v3.1.14         valid     /usr/local/lib/python3.11/site-packages/django/__init__.py
 √  ARCHIVEBOX_BINARY     v0.7.2          valid     /usr/local/bin/archivebox

 √  CURL_BINARY           v8.5.0          valid     /usr/bin/curl
 √  WGET_BINARY           v1.21.3         valid     /usr/bin/wget
 √  NODE_BINARY           v20.10.0        valid     /usr/bin/node
 √  SINGLEFILE_BINARY     v1.1.46         valid     /app/node_modules/single-file-cli/single-file
 √  READABILITY_BINARY    v0.0.11         valid     /app/node_modules/readability-extractor/readability-extractor
 √  MERCURY_BINARY        v1.0.0          valid     /app/node_modules/@postlight/parser/cli.js
 √  GIT_BINARY            v2.39.2         valid     /usr/bin/git
 √  YOUTUBEDL_BINARY      v2023.12.30     valid     /usr/local/bin/yt-dlp
 √  CHROME_BINARY         v120.0.6099.28  valid     /usr/bin/chromium-browser
 √  RIPGREP_BINARY        v13.0.0         valid     /usr/bin/rg

[i] Source-code locations:
 √  PACKAGE_DIR           23 files        valid     /app/archivebox
 √  TEMPLATES_DIR         3 files         valid     /app/archivebox/templates
 -  CUSTOM_TEMPLATES_DIR  -               disabled  None

[i] Secrets locations:
 -  CHROME_USER_DATA_DIR  -               disabled  None
 -  COOKIES_FILE          -               disabled  None

[i] Data locations:
 √  OUTPUT_DIR            13 files @      valid     /data
 √  SOURCES_DIR           200 files       valid     ./sources
 √  LOGS_DIR              1 files         valid     ./logs
 √  ARCHIVE_DIR           123 files       valid     ./archive
 √  CONFIG_FILE           554.0 Bytes     valid     ./ArchiveBox.conf
 √  SQL_INDEX             1.6 MB          valid     ./index.sqlite3

[pirate]

Looks like Chromium 120.0.6099.28 is randomly hanging before exit (again) which causes Screenshot, DOM, PDF, and Singlefile to fail.

Ugh, this issue pops up every few Chromium releases, it's not the first time it's happened. cypress-io/cypress#27264

It seems to run successfully and then just hang when its supposed to exit.
A bunch of the failing methods actually produce correct output, but ArchiveBox logs them as failed because they time out.

At the moment there's not a ton I can do as it's not something ArchiveBox can control, but I'll keep troubleshooting.
For now you can downgrade chrome or wait a few hours and try again, it often just fixes itself.

You can verify that it's Chrome failing by running a very simple chrome headless command and seeing if it hangs indefinitely:

"/Applications/Google Chrome.app/Contents/MacOS/Google Chrome" --headless=new --no-sandbox --no-zygote --disable-dev-shm-usage --disable-sync --dump-dom "https://arstechnica.com/?p=1995005"
# or in docker
docker run -it -v $PWD/data:/data archivebox:test /usr/bin/chromium-browser --headless=new --no-sandbox --no-zygote --disable-dev-shm-usage --disable-sync --dump-dom "https://arstechnica.com/?p=1995005"

[pirate]

Seems to be working now https://demo.archivebox.io

I'm rebuilding the docker image on Docker Hub just in case, you can try pulling :0.7.2 (or :0.7.3 pre-release) and trying again if it doesn't start working on its own.

Maybe it's caused by docker root volume getting full? Or maybe autoupdating checks? No idea tbh I can never reproduce it long enough to figure out why it happens.

[gerroon]

It feels much better with the latest update, I have been getting less issues with the single file.