Docker: Set explicit HOME env for Darktable & RawTherapee

[michalskalski]

Hi,

I see in logs for all my raw files:

time="2021-09-13T08:27:31Z" level=warning msg="index: failed converting 20141221-poz066.DNG to jpeg (path lookup '/.config/darktable' fails with: 'No such file or directory'\n)"

I run photoprism using below docker compose:

  photoprism:
    image: photoprism/photoprism:preview
    container_name: photoprism
    security_opt:
      - seccomp:unconfined
      - apparmor:unconfined
    networks:
      multimedia:
        ipv4_address: 192.168.143.16
    environment:
      PHOTOPRISM_ADMIN_PASSWORD: "removed"          # PLEASE CHANGE: Your initial admin password (min 4 characters)
      PHOTOPRISM_ORIGINALS_LIMIT: 5000               # File size limit for originals in MB (increase for high-res video)
      PHOTOPRISM_HTTP_COMPRESSION: "gzip"            # Improves transfer speed and bandwidth utilization (none or gzip)
      PHOTOPRISM_DEBUG: "false"                      # Run in debug mode (shows additional log messages)
      PHOTOPRISM_PUBLIC: "false"                     # No authentication required (disables password protection)
      PHOTOPRISM_READONLY: "true"                   # Don't modify originals directory (reduced functionality)
      PHOTOPRISM_EXPERIMENTAL: "false"               # Enables experimental features
      PHOTOPRISM_DISABLE_WEBDAV: "false"             # Disables built-in WebDAV server
      PHOTOPRISM_DISABLE_SETTINGS: "false"           # Disables Settings in Web UI
      PHOTOPRISM_DISABLE_TENSORFLOW: "false"         # Disables using TensorFlow for image classification
      PHOTOPRISM_DARKTABLE_PRESETS: "false"          # Enables Darktable presets and disables concurrent RAW conversion
      PHOTOPRISM_DETECT_NSFW: "false"                # Flag photos as private that MAY be offensive (requires TensorFlow)
      PHOTOPRISM_UPLOAD_NSFW: "true"                 # Allow uploads that MAY be offensive
      # PHOTOPRISM_DATABASE_DRIVER: "sqlite"         # SQLite is an embedded database that doesn't require a server
      PHOTOPRISM_DATABASE_DRIVER: "mysql"            # Use MariaDB (or MySQL) instead of SQLite for improved performance
      PHOTOPRISM_DATABASE_SERVER: "mariadb:3306"     # MariaDB database server (hostname:port)
      PHOTOPRISM_DATABASE_NAME: "photoprism"         # MariaDB database schema name
      PHOTOPRISM_DATABASE_USER: "photoprism"         # MariaDB database user name
      PHOTOPRISM_DATABASE_PASSWORD: "removed"       # MariaDB database user password
      PHOTOPRISM_SITE_URL: "removed"  # Public PhotoPrism URL
      PHOTOPRISM_SITE_TITLE: "PhotoPrism"
      PHOTOPRISM_SITE_CAPTION: "Browse Your Life"
      PHOTOPRISM_SITE_DESCRIPTION: ""
      PHOTOPRISM_SITE_AUTHOR: ""
      PHOTOPRISM_JPEG_QUALITY: "95"
    user: "489:499"
    volumes:
      - "/data/photos:/photoprism/originals"
      - "/photoprism/storage:/photoprism/storage"

Both dartable-cli and rawtherapee-cli return errors in interactive mode:

$ docker-compose exec photoprism bash
groups: cannot find name for group ID 499
I have no name!@3f570ab729bc:/photoprism$ darktable-cli
path lookup '/.config/darktable' fails with: 'No such file or directory'

I have no name!@3f570ab729bc:/photoprism$ rawtherapee-cli

FATAL ERROR:
Creation of the user's processing profile directory "/.config/RawTherapee/profiles" failed!

I checked that HOME dir was set to '/' this is where above commands tries to create files, but '/' belongs to root so it can't create it there:

I have no name!@3f570ab729bc:/photoprism$ env | grep HOME
HOME=/

I changed HOME variable to point to '/photoprism' dir which allow me to use both commands:

I have no name!@3f570ab729bc:/photoprism$ export HOME=/photoprism
I have no name!@3f570ab729bc:~$ rawtherapee-cli
RawTherapee, version 5.8, command line.
Terminating without anything to do.
I have no name!@3f570ab729bc:~$ rawtherapee-cli ^C
I have no name!@3f570ab729bc:~$ darktable-c

I have no name!@3f570ab729bc:~$ darktable-cli
usage: darktable-cli [<input file or dir>] [<xmp file>] <output destination> [options] [--core <darktable options>]

options:
   --width <max width> default: 0 = full resolution
   --height <max height> default: 0 = full resolution
   --bpp <bpp>, unsupported
   --hq <0|1|false|true> default: true
   --upscale <0|1|false|true>, default: false
   --export_masks <0|1|false|true>, default: false
   --style <style name>
   --style-overwrite
   --apply-custom-presets <0|1|false|true>, default: true
                          disable for multiple instances
   --out-ext <extension>, default from output destination or '.jpg'
                          if specified, takes preference over output
   --import <file or dir> specify input file or dir, can be used'
                          multiple times instead of input file
   --icc-type <type> specify icc type, default to NONE
                     use --help icc-type for list of supported types
   --icc-file <file> specify icc filename, default to NONE
   --icc-intent <intent> specify icc intent, default to LAST
                     use --help icc-intent for list of supported intents
   --verbose
   --help,-h [option]
   --version

On the end I extended my docker compose file with HOME: "/photoprism" in env section and conversion is working now.
Maybe it is worth to extend sample docker compose file with above information?

[lastzero]

Yes, please. I've recently added an explicit workdir value for this, not sure if setting HOME is better? Can you test if the new docker-compose.yml files in our develop branch work for you?

[lastzero]

Note we're on vacation until later this week.

[michalskalski]

Ok I finally finished indexing my photos, looks great! So after that I checked with

photoprism/docker/examples/docker-compose.yml

Line 78 in 6d30419

working_dir: "/photoprism"

but issue still persist.

It seems it is pretty common problem, when try to use custom user inside container. In my case in docker-compose I use:

 user: "489:499"

so that will be the user/group in which context rawtherapee-cli and darktable-cli will operate. However this user(id/gid) is not known inside container:

I have no name!@c1f40404d596:/photoprism$ cat /etc/passwd
root:x:0:0:root:/root:/bin/bash
daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin
bin:x:2:2:bin:/bin:/usr/sbin/nologin
sys:x:3:3:sys:/dev:/usr/sbin/nologin
sync:x:4:65534:sync:/bin:/bin/sync
games:x:5:60:games:/usr/games:/usr/sbin/nologin
man:x:6:12:man:/var/cache/man:/usr/sbin/nologin
lp:x:7:7:lp:/var/spool/lpd:/usr/sbin/nologin
mail:x:8:8:mail:/var/mail:/usr/sbin/nologin
news:x:9:9:news:/var/spool/news:/usr/sbin/nologin
uucp:x:10:10:uucp:/var/spool/uucp:/usr/sbin/nologin
proxy:x:13:13:proxy:/bin:/usr/sbin/nologin
www-data:x:33:33:www-data:/var/www:/usr/sbin/nologin
backup:x:34:34:backup:/var/backups:/usr/sbin/nologin
list:x:38:38:Mailing List Manager:/var/list:/usr/sbin/nologin
irc:x:39:39:ircd:/run/ircd:/usr/sbin/nologin
gnats:x:41:41:Gnats Bug-Reporting System (admin):/var/lib/gnats:/usr/sbin/nologin
nobody:x:65534:65534:nobody:/nonexistent:/usr/sbin/nologin
_apt:x:100:65534::/nonexistent:/usr/sbin/nologin
systemd-timesync:x:101:101:systemd Time Synchronization,,,:/run/systemd:/usr/sbin/nologin
systemd-network:x:102:103:systemd Network Management,,,:/run/systemd:/usr/sbin/nologin
systemd-resolve:x:103:104:systemd Resolver,,,:/run/systemd:/usr/sbin/nologin
messagebus:x:104:105::/nonexistent:/usr/sbin/nologin
davfs2:x:105:106::/var/cache/davfs2:/usr/sbin/nologin
photoprism:x:1000:1000::/photoprism:/bin/sh

so also there is no HOME dir for this user, and when rawtherapee-cli or darktable-cli will try create some configuration files, they will try to do it in "/" dir where only root can do this. In my case I populate HOME env variable set to ' /photoprism' which have permissions allowing to create files for everyone:

I have no name!@c1f40404d596:/photoprism$ ls -ld /photoprism
drwxrwxrwx 7 photoprism photoprism 10 Sep  6 14:27 /photoprism

including the user which I pass as in docker-compose:

I have no name!@ba3369f79f50:~$ ls -ld /photoprism/.config/
drwxr-xr-x 3 489 499 3 Sep 14 10:28 /photoprism/.config/

[lastzero]

Seems setting an explicit HOME environment variable directly in a Dockerfile breaks multi-stage builds, at least in combination with multi-arch images. Removed it and the preview build stopped failing... for now. Will keep it in our docker-compose.yml examples, unless that also turns out to cause issues.