Question: Docker on Windows archiving to an SMB path that doesn't support FSYNC

[aluhrs13]

I've been fighting with this long enough to hope that someone with more ArchiveBox and/or Docker knowledge can spot something I'm doing wrong. I'm attempting to run ArchiveBox on Windows following the docker-compose directions, with my Volume on an SMB path (my NAS, not my server).

First I make the volume:
docker volume create --driver local --opt type=cifs --opt device=//<ipaddress>/ArchiveBox --opt o=username=<username>,password=<pass> ArchiveBox

Then I run the first command in the directions:
docker-compose run archivebox init --setup

And hit errors:

Creating network "archivebox3_default" with the default driver
Creating archivebox3_archivebox_run ... done
[i] [2021-04-24 19:07:10] ArchiveBox v0.6.2: archivebox init --setup
    > /data

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/logging/config.py", line 564, in configure
    handler = self.configure_handler(handlers[name])
  File "/usr/local/lib/python3.9/logging/config.py", line 745, in configure_handler
    result = factory(**kwargs)
  File "/usr/local/lib/python3.9/logging/handlers.py", line 153, in __init__
    BaseRotatingHandler.__init__(self, filename, mode, encoding=encoding,
  File "/usr/local/lib/python3.9/logging/handlers.py", line 58, in __init__
    logging.FileHandler.__init__(self, filename, mode=mode,
  File "/usr/local/lib/python3.9/logging/__init__.py", line 1146, in __init__
    StreamHandler.__init__(self, self._open())
  File "/usr/local/lib/python3.9/logging/__init__.py", line 1175, in _open
    return open(self.baseFilename, self.mode, encoding=self.encoding,
PermissionError: [Errno 13] Permission denied: '/data/logs/errors.log'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/bin/archivebox", line 33, in <module>
    sys.exit(load_entry_point('archivebox', 'console_scripts', 'archivebox')())
  File "/app/archivebox/cli/__init__.py", line 140, in main
    run_subcommand(
  File "/app/archivebox/cli/__init__.py", line 74, in run_subcommand
    setup_django(in_memory_db=subcommand in fake_db, check_db=cmd_requires_db and not init_pending)
  File "/app/archivebox/config.py", line 1136, in setup_django
    django.setup()
  File "/usr/local/lib/python3.9/site-packages/django/__init__.py", line 19, in setup
    configure_logging(settings.LOGGING_CONFIG, settings.LOGGING)
  File "/usr/local/lib/python3.9/site-packages/django/utils/log.py", line 75, in configure_logging
    logging_config_func(logging_settings)
  File "/usr/local/lib/python3.9/logging/config.py", line 809, in dictConfig
    dictConfigClass(config).configure()
  File "/usr/local/lib/python3.9/logging/config.py", line 571, in configure
    raise ValueError('Unable to configure handler '
ValueError: Unable to configure handler 'logfile'
ERROR: 1

A folder named "logs" is created, but nothing else. Obviously seems to be a permissions issue of some sort, but I don't know enough to figure out what it is. Any suggestions or alternative approaches?

[pirate]

Try adding a :z flag to your volume:

services:
    archivebox:
        volumes:
            - ArchiveBox:/data:z

Network drives throw red-herring permissions errors sometimes because we use atomic_write to FSYNC all writes, and some filesystems don't support it. The :z flag seems to often fix that quirk by having docker middleman the writes (which seems to hide the lack of FSYNC support and fix the error as a side effect).

[aluhrs13]

No dice 😢, here's the contents of my docker-compose.yml in case it's something there.

version: '2.4'

volumes:
  vol:
    external: true
    name: ArchiveBox

services:
    archivebox:
        image: ${DOCKER_IMAGE:-archivebox/archivebox:latest}     
        command: server --quick-init 0.0.0.0:8000
        ports:
            - 3081:8000
        environment:
            - ALLOWED_HOSTS=*
            - MEDIA_MAX_SIZE=750m
            - SUBMIT_ARCHIVE_DOT_ORG=False
        volumes:
            - vol:/data:z

[pirate]

docker-compose run --rm archivebox /bin/ls -lah /data
docker-compose run --rm archivebox /bin/ls -lah /data/logs

Can you post the output of these ^

[aluhrs13]

Here you go, thanks for the help!

C:\Users\aluhr\Docker\ArchiveBox3>docker-compose run --rm archivebox /bin/ls -lah /data
Creating archivebox3_archivebox_run ... done
total 4.0K
drwxr-xr-x 2 root root    0 Apr 14 08:57 .
drwxr-xr-x 1 root root 4.0K Apr 25 01:05 ..
drwxr-xr-x 2 root root    0 Apr 14 08:57 logs

C:\Users\aluhr\Docker\ArchiveBox3>docker-compose run --rm archivebox /bin/ls -lah /data/logs
Creating archivebox3_archivebox_run ... done
total 0
drwxr-xr-x 2 root root 0 Apr 14 08:57 .
drwxr-xr-x 2 root root 0 Apr 14 08:57 ..

[pirate]

Can you try giving all permissions on the dir? chmod -R 777 data (or whatever the windows equivalent of chmod is).

You may also have to change the settings on the cifs cluster to allow read/write for all users, as often permissions are enforced on the server side of the fileshare.

[aluhrs13]

The permissions definitely changed according to SSH and Synology's dashboard, but re-running the the last commands still show the same permissions from Docker's perspective and I get the same error.

I wonder if I should re-think my approach entirely. Is there any way to change what folder archived data is saved to after initial setup? I didn't see an environment variable to configure that. Or maybe I finally go nuclear and swap this server to Linux like I should've done from the start 😅.

[aluhrs13]

Alright I've bit a partial bullet and opted to just run ArchiveBox on my Synology instead of trying to fight this more. Feel free to close this out, unless there's something here you want to investigate further, I'm happy to continue trying troubleshooting steps.

[pirate]

Note I've added a new DB/filesystem troubleshooting area to the wiki that may help people arriving here from Google: https://github.com/ArchiveBox/ArchiveBox/wiki/Upgrading-or-Merging-Archives#database-troubleshooting

Contributions/suggestions welcome there.