Trying to run the demo

[krth8]

Hi,

The demo is stuck on "Connecting server"

Both server and image would run of the same machine.

[12:03:27 PM] Auth error at 'http://192.168.0.20:8080/api/health': 502 - Bad Gateway

`clapshot-1 | --- Browse http://192.168.0.20:8080/ for Clapshot
clapshot-1 | --- or http://192.168.0.20:8080/htadmin/ for user management
clapshot-1 | ---
clapshot-1 | --- Default users:
clapshot-1 | --- - admin:admin (can edit other people's videos)
clapshot-1 | --- - demo:demo
clapshot-1 | --- - alice:alice123
clapshot-1 | ---
clapshot-1 | --- User management admin:
clapshot-1 | --- - htadmin:admin (only for /htadmin)
clapshot-1 | ==============================================================
clapshot-1 |
clapshot-1 | # Dig up start command from systemd script and run it as docker user instead of www-data
clapshot-1 | CMD=$(grep 'Exec' /lib/systemd/system/clapshot-server.service | sed 's/^.=//')
clapshot-1 | sudo -u docker $CMD &
clapshot-1 |
clapshot-1 | # Follow server log
clapshot-1 | tail -f /var/log/clapshot.log
clapshot-1 | 2024-05-21T09:59:33+00 INFO ORG: [stdout] [bf.after_migration] Running post-migration checks...
clapshot-1 | 2024-05-21T09:59:33+00 INFO srv->org handshake done (org->srv not connected yet).
clapshot-1 | 2024-05-21T09:59:33+00 INFO Starting video processing pipeline.
clapshot-1 | 2024-05-21T09:59:33+00 INFO MD: Starting. n_workers=4
clapshot-1 | 2024-05-21T09:59:33+00 INFO INCOMING: Starting. dir="/mnt/clapshot-data/data" poll_interval=3.0 resubmit_delay=15.0
clapshot-1 | 2024-05-21T09:59:33+00 INFO COMPR: Starting. n_workers=8
clapshot-1 | 2024-05-21T09:59:33+00 INFO API: org->srv connected, bidirectional gRPC established. org_name="clapshot.organizer.basic_folders" description="Basic folders for the UI" version="0.6.1"
clapshot-1 | 2024-05-21T09:59:33+00 INFO API: Starting frontend API server. port=8095
clapshot-1 | 2024-05-21T09:59:33+00 INFO API: Allowed CORS origins: [""]
clapshot-1 | 2024-05-21T09:59:33+00 WARN API: !! SECURITY RISK !! – Using CORS origin '' allows any website to access your video annotation system. This exposes your users' videos to potential API attacks. Do NOT use '' in production! Instead, specify the allowed origin, such as 'https://clapshot.example.com'.
clapshot-1 | Traceback (most recent call last):

clapshot-1 | File "", line 3, in

clapshot-1 | File "/usr/lib/python3.11/configparser.py", line 713, in read

clapshot-1 | self._read(fp, filename)

clapshot-1 | File "/usr/lib/python3.11/configparser.py", line 1112, in _read

clapshot-1 | raise DuplicateOptionError(sectname, optname,

clapshot-1 | configparser.DuplicateOptionError: While reading from '/etc/clapshot-server.conf' [line 59]: option 'cors' in section 'general' already exists

clapshot-1 | �[1m�[31merror:�[0m the following required arguments were not provided:

clapshot-1 | �[32m--data-dir
�[0m

clapshot-1 | �[32m--url-base �[0m

clapshot-1 |

clapshot-1 | �[1m�[4mUsage:�[0m �[1mclapshot-server�[0m �[1m--data-dir�[0m
�[1m--url-base�[0m

clapshot-1 |

clapshot-1 | For more information, try '�[1m--help�[0m'.
`
Would love for this to work, Also, a docker image suitable for production would be real neat.

[elonen]

Do you have a modified docker-entry_htadmin.sh and/or Docker image? Wondering how did URL_BASE become http://192.168.0.20:8080 . Another odd thing about the log is CORS origins [""] (should be ["*"] in the demo image). The python errors seem to suggest that clapshot-server.conf in the image already has a cors=... entry, and the entry script then adds another one, leading to the arg parser error.

That aside, do you have a clear idea on how exactly should a Docker-based production deployment to work? E.g. HTTPS cert management, config file, logging, storage. I almost never use Docker for production myself, so I'm not really familiar with the best practices.

[krth8]

To be honest, i am quite the novice at this stuff. However, i love this tool and would love to get it to work and preferably as a docker image as i run all of my other services this way. Using cloudflare tunnels for access.

Running your docker cli command with htadmin gives these results:
[8:46:53 PM] Network error fetching 'http://127.0.0.1:8080/api/health'

clapshot-1 | --- Browse http://127.0.0.1:8080/ for Clapshot
clapshot-1 | --- or http://127.0.0.1:8080/htadmin/ for user management
clapshot-1 | ---
clapshot-1 | --- Default users:
clapshot-1 | --- - admin:admin (can edit other people's videos)
clapshot-1 | --- - demo:demo
clapshot-1 | --- - alice:alice123
clapshot-1 | ---
clapshot-1 | --- User management admin:
clapshot-1 | --- - htadmin:admin (only for /htadmin)
clapshot-1 | ==============================================================
clapshot-1 |
clapshot-1 | # Dig up start command from systemd script and run it as docker user instead of www-data
clapshot-1 | CMD=$(grep 'Exec' /lib/systemd/system/clapshot-server.service | sed 's/^.=//')
clapshot-1 | sudo -u docker $CMD &
clapshot-1 |
clapshot-1 | # Follow server log
clapshot-1 | tail -f /var/log/clapshot.log
clapshot-1 | 2024-05-24T18:02:04+00 INFO ORG: Spawning (shell cmd): "sh" "-c" "/usr/bin/clapshot-organizer-basic-folders /mnt/clapshot-data/data/grpc-srv-to-org.sock"
clapshot-1 | 2024-05-24T18:02:04+00 INFO Backing up database before migration. file=/mnt/clapshot-data/data/clapshot.sqlite backup=/mnt/clapshot-data/data/clapshot.backup-2024-05-24T18_02_04.tar.gz
clapshot-1 | 2024-05-24T18:02:04+00 INFO Applying (Clapshot server) DB migration: 2022-10-04-155935_setup_tables
clapshot-1 | 2024-05-24T18:02:04+00 INFO Applying (Clapshot server) DB migration: 2023-02-08-153341_add_title
clapshot-1 | 2024-05-24T18:02:04+00 INFO Applying (Clapshot server) DB migration: 2023-02-09-211348_add_thumb_sheet_dims
clapshot-1 | 2024-05-24T18:02:04+00 INFO Applying (Clapshot server) DB migration: 2023-04-17-100000_cleanup_foreign_keys
clapshot-1 | 2024-05-24T18:02:04+00 INFO Applying (Clapshot server) DB migration: 2023-04-18-190209_change_video_primkey
clapshot-1 | 2024-05-24T18:02:04+00 INFO Applying (Clapshot server) DB migration: 2023-04-18-190300_add_cascade_rules
clapshot-1 | 2024-05-24T18:02:04+00 INFO Applying (Clapshot server) DB migration: 2024-05-13-093800_add_users_table
clapshot-1 | 2024-05-24T18:02:04+00 INFO All server migrations applied. file=/mnt/clapshot-data/data/clapshot.sqlite
clapshot-1 | 2024-05-24T18:02:04+00 INFO API: Starting gRPC server for org->srv.
clapshot-1 | 2024-05-24T18:02:04+00 INFO gRPC server for org->srv: Binding to 'unix:///mnt/clapshot-data/data/grpc-org-to-srv.sock'
clapshot-1 | 2024-05-24T18:02:04+00 INFO Connecting gRPC srv->org...
clapshot-1 | 2024-05-24T18:02:04+00 INFO ORG: [stdout] [bf] Organizer listening on '/mnt/clapshot-data/data/grpc-srv-to-org.sock'
clapshot-1 | 2024-05-24T18:02:05+00 INFO Connected to organizer (on attempt 1). Doing handshake.
clapshot-1 | 2024-05-24T18:02:05+00 INFO ORG: [stdout] [bf] Got handshake from server.
clapshot-1 | 2024-05-24T18:02:05+00 INFO ORG: [stdout] [bf] Connecting back to Clapshot server...
clapshot-1 | 2024-05-24T18:02:05+00 INFO ORG: [stdout] [bf] Clapshot server connected.
clapshot-1 | 2024-05-24T18:02:05+00 INFO ORG: [stdout] [bf] Opening SQLite database at '/mnt/clapshot-data/data/clapshot.sqlite'
clapshot-1 | 2024-05-24T18:02:05+00 INFO Calling check_migrations on organizer.
clapshot-1 | 2024-05-24T18:02:05+00 INFO ORG: [stdout] [bf] check_migrations(): current schema version = '', max version = '0001_initial_schema', 1 pending migration alternatives
clapshot-1 | 2024-05-24T18:02:05+00 WARN MIGRATION DEPENDENCY RESOLVER NOT YET PROPERLY IMPLEMENTED! Blindly applying organizer migration: Migration { uuid: "basic_folders_2024-05-01_1610", version: "0001_initial_schema", dependencies: [Dependency { name: "clapshot.server", min_ver: Some("2024-05-13-093800_add_users_table"), max_ver: None }, Dependency { name: "clapshot.organizer.basic_folders", min_ver: None, max_ver: Some("") }], description: "Initial schema. One video per folder, folders are user-specific and can be nested." }
clapshot-1 | 2024-05-24T18:02:05+00 INFO ORG: [stdout] [bf] apply_migration('basic_folders_2024-05-01_1610')
clapshot-1 | 2024-05-24T18:02:05+00 INFO ORG: [stdout] [bf.after_migration] Running post-migration checks...
clapshot-1 | 2024-05-24T18:02:05+00 INFO srv->org handshake done (org->srv not connected yet).
clapshot-1 | 2024-05-24T18:02:05+00 INFO Starting video processing pipeline.
clapshot-1 | 2024-05-24T18:02:05+00 INFO MD: Starting. n_workers=4
clapshot-1 | 2024-05-24T18:02:05+00 INFO INCOMING: Starting. dir="/mnt/clapshot-data/data" poll_interval=3.0 resubmit_delay=15.0
clapshot-1 | 2024-05-24T18:02:05+00 INFO COMPR: Starting. n_workers=8
clapshot-1 | 2024-05-24T18:02:05+00 INFO API: org->srv connected, bidirectional gRPC established. org_name="clapshot.organizer.basic_folders" description="Basic folders for the UI" version="0.6.1"
clapshot-1 | 2024-05-24T18:02:05+00 INFO API: Starting frontend API server. port=8095
clapshot-1 | 2024-05-24T18:02:05+00 INFO API: Allowed CORS origins: [""]
clapshot-1 | 2024-05-24T18:02:05+00 WARN API: !! SECURITY RISK !! – Using CORS origin '' allows any website to access your video annotation system. This exposes your users' videos to potential API attacks. Do NOT use '' in production! Instead, specify the allowed origin, such as 'https://clapshot.example.com'.

[krth8]

Got it to work with this compose script. Such a nice tool!

However, database crashed when uploading a prores file containing a alpha channel. This is not a functionality that is important, but it could be nice in some cases where the alpha channel gets removed on transcode but download original will contain it.

version: '3.8'

services:
clapshot:
image: elonen/clapshot:latest-demo-htadmin
container_name: clapshot_demo
environment:
- CLAPSHOT_URL_BASE=192.168.0.20:8080/
ports:
- "8080:80"
volumes:
- clapshot-demo:/mnt/clapshot-data/data
restart: unless-stopped

volumes:
clapshot-demo:

[krth8]

after rebooting the server will not start it seems. Is this because of the cors settings? Also cant seem to find the clapshot-server.conf file in the data folder.

Just while i try to figure this stuff out i am installing with this command.
docker run --rm -it -p 192.168.0.20:8080:80 -v clapshot-demo:/mnt/clapshot-data/data elonen/clapshot:latest-demo

Boots up nicely after install, but restarting the container cripples it.

clapshot-1 | ==============================================
clapshot-1 | _____ _ _ _
clapshot-1 | / | | | | | |
clapshot-1 | | | | | __ _ _ __ | | ___ | |
clapshot-1 | | | | |/ ` | ' / _| ' \ / _ | __|
clapshot-1 | | || | (| | |) __ | | | | () | |
clapshot-1 | _||_,| ./||| ||___/ _|
clapshot-1 | | |
clapshot-1 | ||
clapshot-1 | --- Browse http://192.168.0.20:8080/ ---
clapshot-1 | ==============================================
clapshot-1 |
clapshot-1 | # Dig up start command from systemd script and run it as docker user instead of www-data
clapshot-1 | CMD=$(grep 'Exec' /lib/systemd/system/clapshot-server.service | sed 's/^.=//')
clapshot-1 | sudo -u docker $CMD &
clapshot-1 |
clapshot-1 | # Follow server log
clapshot-1 | tail -f /var/log/clapshot.log
clapshot-1 | 2024-05-26T11:32:50+00 INFO Starting video processing pipeline.
clapshot-1 | 2024-05-26T11:32:50+00 INFO MD: Starting. n_workers=4
clapshot-1 | 2024-05-26T11:32:50+00 INFO COMPR: Starting. n_workers=8
clapshot-1 | 2024-05-26T11:32:50+00 INFO INCOMING: Starting. dir="/mnt/clapshot-data/data" poll_interval=3.0 resubmit_delay=15.0
clapshot-1 | 2024-05-26T11:32:50+00 INFO API: org->srv connected, bidirectional gRPC established. org_name="clapshot.organizer.basic_folders" description="Basic folders for the UI" version="0.6.1"
clapshot-1 | 2024-05-26T11:32:50+00 INFO API: Starting frontend API server. port=8095
clapshot-1 | 2024-05-26T11:32:50+00 INFO API: Allowed CORS origins: [""]
clapshot-1 | 2024-05-26T11:32:50+00 WARN API: !! SECURITY RISK !! – Using CORS origin '' allows any website to access your video annotation system. This exposes your users' videos to potential API attacks. Do NOT use '' in production! Instead, specify the allowed origin, such as 'https://clapshot.example.com'.
clapshot-1 | 2024-05-26T11:32:53+00 INFO ORG: [stdout] [bf] No root folder for user 'docker', creating one now.
clapshot-1 | 2024-05-26T11:33:02+00 INFO ws_session{sid=2 user=docker}: Got websocket close message.
clapshot-1 | Traceback (most recent call last):

clapshot-1 | File "", line 3, in

clapshot-1 | File "/usr/lib/python3.11/configparser.py", line 713, in read

clapshot-1 | self._read(fp, filename)

clapshot-1 | File "/usr/lib/python3.11/configparser.py", line 1112, in _read

clapshot-1 | raise DuplicateOptionError(sectname, optname,

clapshot-1 | configparser.DuplicateOptionError: While reading from '/etc/clapshot-server.conf' [line 59]: option 'cors' in section 'general' already exists

clapshot-1 | �[1m�[31merror:�[0m the following required arguments were not provided:

clapshot-1 | �[32m--data-dir
�[0m

clapshot-1 | �[32m--url-base �[0m

clapshot-1 |

clapshot-1 | �[1m�[4mUsage:�[0m �[1mclapshot-server�[0m �[1m--data-dir�[0m
�[1m--url-base�[0m

clapshot-1 |

clapshot-1 | For more information, try '�[1m--help�[0m'.

[krth8]

After tinkering a bit it seems that its the nginx config that is messing up? Is it possible to bypass this completely?

if i use CLAPSHOT_URL_BASE=192.168.0.20:8095/ it works great on initial setup everytime, but then bugs off on a container restart.

The duplicated cors settings i have no idea why happens, i thought it could be some old files from a previous install, but after cleaning up it seems that was not the case. Will deleting that duplicated cors line in the .conf solve all of this?

[elonen]

Can you share a small test video with alpha channel that crashed the transcoding? I've added a DNxHR file with alpha on the test suite on the new 0.7.0 release, but couldn't reproduce the problem.

RE: the Docker stuff, what kind of deployment / use case are you aiming at?
LAN only, one to a few users who completely trust each other (to the degree that leaking passwords is ok), no access to any local DNS?

Exposing the service on a numeric IP address (instead of DNS name), port 8080, without HTTPS to protect you passwords, and CORS=* allowing any malicious website to tap into your open API sessions truly isn't something I'd recommend for any kind of production use.

Docker Compose would in principle be a good way to setup a container-based production deployment, but at the very least it would need to fix CORS=* to something else to protect you from cross-site-scripting attacks. HTTPS (perhaps through Let's Encrypt) would be my strong next recommendation, and likely best implemented as a separate Nginx reverse proxy container in the same Composer script.

[krth8]

DNxHR 12bit with alpha works perfectly, Prores seems to crash the server, however it seems the log does not say that.

Regarding docker and web security is where i am at deep waters. Preferably i would use docker and expose it to the web. For my other services i use Cloudflare tunnels and would want to use it here as well. To my understanding cloudflare tunnels are safer than Reverse proxy, it has HTTPS, certs and doesn't expose your IP.

I use Frame.io professionally, but Clapshot is so nice and fast. Also I appreciate open source projects. I just need to get it working properly on my server so i can use it more thoroughly.

Besides prores with alpha crashing the server everything is fine until a restart, after which the seems to be missing the gRPC server section of the log. This is why i am stuck at connecting to server i guess.

clapshot_demo` | --- Browse http://192.168.0.20:8095/ ---
clapshot_demo | ==============================================
clapshot_demo |
clapshot_demo | # Dig up start command from systemd script and run it as docker user instead of www-data
clapshot_demo | CMD=$(grep 'Exec' /lib/systemd/system/clapshot-server.service | sed 's/^.*=//')
clapshot_demo | sudo -u docker $CMD &
clapshot_demo |
clapshot_demo | # Follow server log
clapshot_demo | tail -f /var/log/clapshot.log
clapshot_demo | 2024-05-29T09:05:13+00 INFO ffmpeg_transcode{thread=ThreadId(53)}: ffmpeg finished
clapshot_demo | 2024-05-29T09:06:20+00 INFO File uploaded. dst="/mnt/clapshot-data/data/upload/15042cbf-1205-4212-bb4b-8c6644798eb0/SmallWoodHit-004.mov"
clapshot_demo | 2024-05-29T09:06:20+00 INFO MD: Scanning file. file=SmallWoodHit-004.mov user="docker"
clapshot_demo | 2024-05-29T09:06:20+00 INFO MD: Calling mediainfo
clapshot_demo | 2024-05-29T09:06:20+00 INFO PIPELINE:INGEST_MEDIA{media_id=bca83309 user="docker" filename=SmallWoodHit-004.mov}: Ingesting file.
clapshot_demo | 2024-05-29T09:06:20+00 INFO COMPR: Media file transcode request. id=bca83309 type=Video user=docker file=SmallWoodHit-004.mov
clapshot_demo | 2024-05-29T09:06:20+00 INFO COMPR: Media file thumbnail request. id=bca83309 type=Video user=docker file=SmallWoodHit-004.mov
clapshot_demo | 2024-05-29T09:06:20+00 INFO run_ffmpeg_transcode{media_file=bca83309 user=docker thread=ThreadId(30)}: Transcoder called. bitrate=54810510 media_type=Video
clapshot_demo | 2024-05-29T09:06:20+00 INFO run_ffmpeg_thumbnailer{media_file=bca83309 media_type=Video user=docker thread=ThreadId(31)}: Thumbnailer called. needs_poster=true needs_sheet=true
clapshot_demo | 2024-05-29T09:06:28+00 INFO ffmpeg_transcode{thread=ThreadId(59)}: ffmpeg finished
clapshot_demo | Traceback (most recent call last):
clapshot_demo | File "", line 3, in
clapshot_demo | File "/usr/lib/python3.11/configparser.py", line 713, in read
clapshot_demo | self._read(fp, filename)
clapshot_demo | File "/usr/lib/python3.11/configparser.py", line 1112, in _read
clapshot_demo | raise DuplicateOptionError(sectname, optname,
clapshot_demo | configparser.DuplicateOptionError: While reading from '/etc/clapshot-server.conf' [line 59]: option 'cors' in section 'general' already exists
clapshot_demo | error: the following required arguments were not provided:
clapshot_demo | --data-dir
clapshot_demo | --url-base
clapshot_demo |
clapshot_demo | Usage: clapshot-server --data-dir
--url-base
clapshot_demo |
clapshot_demo | For more information, try '--help'.

[elonen]

Cloudfare Tunnel + Docker

Oh I see, so Cloudfare Tunnel is a sort of HTTPS Reverse Proxy + DNS as a Service?
Do you get public name and HTTPS certificate, so that https://clapshot.myaccount.cloudfare.com/ (or something) is transparently forwarded to your LAN http://192.168.0.20:8095/? If that's the case, this gets a lot simpler. You'll basically:

1. set CLAPSHOT_URL_BASE="https://clapshot.myaccount.cloudfare.com/"
2. leave cors empty (it will default to URL_BASE)
3. Change the htaccess passwords

...and you should be good to go.
Cloudfare probably also offers some URI-based access rules, so blocking /htadmin from anyone else besides yourself would also be a good idea.

The only issue is that the demo Docker image is not quite designed for that and would need to be slightly modified. If you have some kind of a virtualization platform I'd still recommend LXC (or even Qemu) VM with a full Debian Bookworm instead, due to saner log and service management, but if you definitely want to use Docker I can help turning the current Dockerfile.demo into a production-usable one.

Prores

You might want to try the new v0.7.0 release from yesterday. It pulls a bit newer FFMPEG from Debian Multimedia than what's in the Bookworm main repo. Don't know if it contains (better) support for Prores but it's worth a try.

For debugging, if you are able to reliably reproduce the crash, could you convert this file...
https://github.com/elonen/clapshot/blob/master/server/src/tests/assets/alpha-test_dnxhr-444-12bit-dnxhr.mov
...into a Prores one and if it crashes, send it to me as a test case?

[elonen]

BTW, for debugging transcode issues, you can go the the videos/ dir and check the stderr.txt/stdout.txt for FFMPEG logs:

videos
├── febd207b
│   ├── orig
│   │   └── Apollo11-108-ksc-69p-3253.webp
│   ├── stderr.txt
│   ├── stdout.txt
│   ├── thumbs
│   │   ├── stderr.txt
│   │   ├── stdout.txt
│   │   └── thumb.webp
│   ├── transcoded_br2500000_0e6219e8-013b-45f3-a92b-550b738100d0.mp4
│   ├── transcoded_br2500000_0e6219e8-013b-45f3-a92b-550b738100d0.pipe
│   └── video.mp4 -> transcoded_br2500000_0e6219e8-013b-45f3-a92b-550b738100d0.mp4

[krth8]

Exactly, i have my different services running at projects.mysite.com, share.mysite.com and so on.

Running 0.7 now and i still get the bug. as soon as i get this up and working i will for sure do some tests and can use your .mov as base.

Setting my access as https at cloudflare and in environment variable makes the log look like this. i guess http:// is added at the back?

--- Browse http://https://review.bestsiteintheworld.no/ ---

Entering through my cloudflare url set to http with clapshot the same i get access to clapshots "connecting server" interface

[elonen]

I've update the Docker image to work better for your use case @krth8.

There a new shell script https://github.com/elonen/clapshot/blob/master/test/run-cloudflare.sh to demonstrate it with Cloudflare. This seems like a reasonable way to run Clapshot in Docker to me (provided you have a static domain - unlike the example script that dynamically creates a random one in trycloudflare.com), thanks for the tip!

[krth8]

Absolute legend... Where is your patreon or buymeacoffee account?

Entering easily now through my domain.. Also no problems after restarting the container.

How do i also log on locally? Can i have two CLAPSHOT_URL_BASE?

One for my local ip and one for my domain?

[elonen]

Great to hear! https://ko-fi.com/elonen if you want to drop a ☕️.

Due to how CORS rules and video playback links work, it's quite difficult to support more than one URL base at a time.

That said, nothing prevents you from running a localhost or LAN address container most of the time, and only firing up the Cloudflare variation when you actually need to share to the Internet. Just bind mount the same directory on different containers. In principle you could even run them simultaneously on the same data dir (sqlite is fine with that), but you'd need to start the second Clapshot container with different log and organizer .sock paths, to avoid them mixing up with the first one.

BTW: Clapshot supports external authenticators through HTTP headers, so with some configuration, one could also use Cloudflare to proxy Google/Github/Facebook OAuth SSO logins, for example, instead of dealing with passwords in Htadmin like it's the 90s. Unfortunately Cloudflare gave me some obscure "Unable to find your Access organization" Zero Trust errors when trying to configure auth on it so I couldn't actually test it, but once set up correctly, it would provide both better security and nicer user experience for the users.

[krth8]

Dropped you a ko-fi, Thanks!

No worries about the URL. Not an issue just using the domain address.

So one could actually just login in with their google or facebook account? That would be real neat!

[elonen]

Thank you!
Yes, the idea is to pass a user ID in header X-Remote-User-Id to Clapshot API from the reverse proxy if (and only if) a user is authorized. If you have such an external auth mechanism (e.g. carefully configured Cloudflare), you can drop the PHP Htadmin middle-layer entirely. Advanced Authentication discusses the headers in more detail.

[krth8]

Will dive some into this! Still need to figure out the more advanced cloudflare details.

In the meantime, or if someone will not want to or able to use these external auth accounts, is it possible to have a basic UI for the login?

Typically something like this:

[elonen]

The login UI is external also, as it depends on the exact login method and the auth provider.
Clapshot doesn't have any authentication code, so I can't actually add login screen there.
The user/password prompt in the PHP Htadmin deployment actually comes from this block in Nginx config:

# Use HTTP basic auth, this can be edited by
# htadmin (see below)
auth_basic "Clapshot login";
auth_basic_user_file /var/www/.htpasswd;
proxy_set_header X-Remote-User-Id $remote_user;
proxy_set_header X-Remote-User-Name $remote_user;

What /htadmin does, is it edits /var/www/.htpasswd.
You could switch this scheme to any other auth/user manager if you find one that you like, and Clapshot would be none the wiser.

In general, authentication UI might be a username/password prompt like in your screenshot, but it could also have 2FA, perhaps Fido2 Webauthn by a hardware token or something else - like in Okta for example, or Active Directory.

The way this kind of thing ususually works is that when a user tries to open a login-protected page, your reverse proxy checks for some auth token (often a JWT in a cookie), and if it's not valid, redirects the browser to the authentication service's login page (e.g. Okta.com, Google.com etc) and passes them a return address in the URL.
The user then logs in at that external auth service. The auth server set a new valid JWT token in a cookie and redirects browser back to the original page (e.g. https://clapshot.example.com/). Your reverse proxy then checks the token again, and now that it's valid, proxies user to the protected page/app (Nginx serving Clapshot Client and Server API in the backend in this case).

This way, the authentication process is completely detached from the backend service, and will have a login page of their own. Cloudflare seems to support this kind of thing also, and probably makes it simpler than it sounds since they are the reverse proxy and can do most of this behind the scenes.