-
-
Notifications
You must be signed in to change notification settings - Fork 165
YouTube Proof‐of‐Origin Tokens
To generate tokens TubeSync is using a plugin for yt-dlp from:
Support for using the plugin was added in: https://github.com/meeb/tubesync/pull/1005
This plugin communicates with a web service to retrieve tokens, for which it uses a default URL of: http://127.0.0.1:4416
Because for TubeSync nginx is what most users will connect to with their web browser, the configuration to listen on port 4416 has been added.
If you are using another web server instead, you should configure similar proxying of the requests, or configure a different URL for the plugin to use.
Tip
You can set the base URL that the plugin tries to access with a custom local_settings.py file.
YOUTUBE_DEFAULTS = {
'extractor_args': {
'youtubepot-bgutilhttp': {
'base_url': ['http://127.0.0.1:4416'],
},
},
# ... all the other yt_dlp settings
}Docker image: brainicism/bgutil-ytdlp-pot-provider:1.3.1
$ docker run -d \
--name bgutil-ytdlp-pot-service \
-p 4416:4416 \
--restart unless-stopped \
brainicism/bgutil-ytdlp-pot-provider:1.3.1Because most users do not already have a custom local_settings.py file, and are not using a custom web server, an easier way to configure the plugin was developed.
Note
You will need to add a new container so that your TubeSync container can access it.
After that is running, you can set the new environment variables to use the web services provided by that container:
$ TUBESYNC_POT_PORT=4416
$ export TUBESYNC_POT_PORT
$ TUBESYNC_POT_IPADDR=[Whatever IP you are using]
$ export TUBESYNC_POT_IPADDRImportant
Don't forget to add -e TUBESYNC_POT_IPADDR -e TUBESYNC_POT_PORT to the docker run command that you are using with TubeSync as well.
Tip
Setting TUBESYNC_POT_HTTPS to True will connect using https://¹ instead of http://.
¹: Setting up valid certificates for the web service is outside the scope of this guide.
When you have both containers on the same docker host, it can be very convenient to use this method.
Environment variables are automatically created by docker, and support for using those was included.
For a web service container named bgutil-ytdlp-pot-service you would add this --link option to your docker run command for TubeSync:
$ docker run --link 'bgutil-ytdlp-pot-service:POTServer' # everything elseThe web service provides a /ping endpoint.
$ no_proxy='127.0.0.1' curl 'http://127.0.0.1:4416/ping' ; echo
{"token_ttl_hours":6,"server_uptime":1633.034876421,"version":"0.8.4"}If the curl command above works from inside the TubeSync container, then the web server configuration is proxying the requests as expected.
When it is not configured, the same /ping request receives a HTTP 502 error.
Server:
Added environment variables:
- TUBESYNC_POT_IPADDR
- TUBESYNC_POT_PORT
- TUBESYNC_POT_HTTPS (use https:// when set)
To generate tokens TubeSync is using a plugin for yt-dlp from:
Support for using the plugin was added in: https://github.com/meeb/tubesync/pull/1005
This plugin communicates with a web service to retrieve tokens, for which it uses a default URL of: http://127.0.0.1:4416
Because for TubeSync nginx is what most users will connect to with their web browser, the configuration to listen on port 4416 has been added.
If you are using another web server instead, you should configure similar proxying of the requests, or configure a different URL for the plugin to use.
This method runs the token provider inside its own independent Docker container.
Deploy the provider container on your host. It listens on default port 4416.
When deploying containers inside a unified docker compose file, you can leverage native DNS name resolution. Because the TUBESYNC_POT_IPADDR environment variable does not accept a hostname, you can choose to bypass it completely by overriding the plugin's base URL via a custom local_settings.py file.
services:
bgutil-ytdlp-pot-service:
image: brainicism/bgutil-ytdlp-pot-provider:1.3.1
ports:
- "4416:4416"
restart: unless-stopped
tubesync:
image: ghcr.io/meeb/tubesync:latest
volumes:
- /path/to/your/local_settings.py:/config/tubesync/local_settings.py:roTo complete this layout, create your custom configuration file on the host and configure the base URL to target your service name:
YOUTUBE_DEFAULTS = {
# The rest of the YOUTUBE_DEFAULTS copied from settings.py
'extractor_args': {
# The other extractor_args keys copied from settings.py
'youtubepot-bgutilhttp': {
'base_url': ['http://bgutil-ytdlp-pot-service:4416'],
},
},
}Alternatively, if you prefer utilizing the native TUBESYNC_POT_IPADDR environment variable inside a docker compose file without modifying python configuration files, you can define a custom network bridge and assign a fixed static internal IP address to the provider service.
networks:
tubesync-net:
driver: bridge
ipam:
config:
- subnet: 172.20.0.0/16
services:
bgutil-ytdlp-pot-service:
image: brainicism/bgutil-ytdlp-pot-provider:1.3.1
ports:
- "4416:4416"
networks:
tubesync-net:
ipv4_address: 172.20.0.100
restart: unless-stopped
tubesync:
image: ghcr.io/meeb/tubesync:latest
environment:
- TUBESYNC_POT_IPADDR=172.20.0.100
- TUBESYNC_POT_PORT=4416
networks:
- tubesync-netImportant
Compose Isolation Warning: If your tubesync and bgutil-ytdlp-pot-service blocks are split across distinct docker-compose.yml files, they cannot resolve each other via internal DNS out of the box. If they are in separate files, you must map a shared external network or use a static host IP address routing configuration.
docker run -d \
--name bgutil-ytdlp-pot-service \
-p 4416:4416 \
--restart unless-stopped \
brainicism/bgutil-ytdlp-pot-provider:1.3.1If you deployed standalone containers using docker run instead of a unified docker compose file, select one of the two strategies below to connect the containers.
If both independent containers run on the same Docker host, you can use the --link flag to connect them.
Important
To use this built-in routing mechanism, the link alias must be exactly POTServer.
docker run --link 'bgutil-ytdlp-pot-service:POTServer' [YOUR_OTHER_TUBESYNC_ARGS]You can directly tell Nginx where to route traffic by defining the environment variables explicitly.
Warning
TUBESYNC_POT_IPADDR strictly accepts a raw IP address. Passing a hostname or container name here will cause resolution to fail.
# Assign your host's LAN IP or a fixed bridge network IP where the provider container can be reached
docker run -e TUBESYNC_POT_IPADDR=[YOUR_DOCKER_HOST_OR_BRIDGE_IP] -e TUBESYNC_POT_PORT=4416 [YOUR_OTHER_TUBESYNC_ARGS]This method activates a built-in provider inside the existing TubeSync container using deno. It saves network overhead but adds ~240 MB of storage to the container runtime.
When this service is enabled, the background provider runs locally within the container and binds to port 4406. Nginx was already listening on port 4416 and will start using port 4406 as a fallback as soon as it becomes available without additional configuration.
Inject an empty file into the s6-overlay user bundle to flag the service for activation.
# Create an empty anchor file on your host system
touch /path/to/your/empty_fileMount the empty host file into the container's specific s6-overlay destination path as read-only.
services:
tubesync:
image: ghcr.io/meeb/tubesync:latest
volumes:
- "/path/to/your/empty_file:/etc/s6-overlay/s6-rc.d/user/contents.d/bgutil-ytdlp-pot-provider:ro"docker run -v "/path/to/your/empty_file:/etc/s6-overlay/s6-rc.d/user/contents.d/bgutil-ytdlp-pot-provider:ro" [YOUR_OTHER_TUBESYNC_ARGS]If the container is already running, start the embedded background service from inside your TubeSync container using:
/command/s6-rc start bgutil-ytdlp-pot-providerThe bundled configuration listens on port 4416 and acts as a dynamic reverse proxy. The configuration evaluates targets in this order of precedence:
-
Fallback / Failure Recovery: If a previous connection attempt failed, the balancer limits retry loops by forcing a configuration fallback straight to the default internal loopback (
127.0.0.1:4406). -
Legacy Linking (
POTServer): If present via a container link, it routes traffic to that linked container's address and port. -
Explicit Environment Keys: Evaluates
TUBESYNC_POT_IPADDRandTUBESYNC_POT_PORT. If provided, traffic is proxied to this explicit external address. It will also switch tohttpsifTUBESYNC_POT_HTTPSevaluates to true. -
Absolute Default: If no environment overrides or container link aliases exist, the configuration falls back to targeting
127.0.0.1:4406(the default port for the Embedded Container Service).
Execute a connection test directly from inside your TubeSync container using the /ping endpoint.
# Execute connection request test against the target port
no_proxy='127.0.0.1' curl 'http://127.0.0.1:4416/ping' ; printf -- '\n' ;{"token_ttl_hours":6,"server_uptime":1633.034876421,"version":"0.8.4"}-
Error HTTP 502: The Nginx proxy layer cannot reach the targeted background token server. Check that your container IP addresses match your network topology, or verify that your link alias is defined exactly as
POTServer.
| Variable | Description |
|---|---|
TUBESYNC_POT_IPADDR |
Target IP address of the provider service. Does not accept hostnames. |
TUBESYNC_POT_PORT |
Target port of the provider service. |
TUBESYNC_POT_HTTPS |
Uses https:// if set to True. Requires additional configuration for certificates. |
- Provider Dockerfile Source
- Provider Application Entrypoint
- Provider Plugin Source
- TubeSync Nginx Configuration File
[ TubeSync Container ] --(Port 4416)--> [ Nginx Proxy ] ----> [ bgutil-ytdlp-pot-service ]
Tip
Start the server inside an already running container with:
/command/s6-rc start bgutil-ytdlp-pot-providerUsing deno for node compatibility, we can run the POT web service inside the container.
This adds hundreds of megabytes (~240 MB) in extra storage, so I don't want to bundle it for everyone.
The end goal is, that anyone who wants this configuration, would only need to add one file to the user bundle.
$ touch "${HOME}/empty"Add the following to your docker run arguments:
-v "${HOME}/empty:/etc/s6-overlay/s6-rc.d/user/contents.d/bgutil-ytdlp-pot-provider:ro"
This is likely a win in the common case where you only have one TubeSync container.
It also plays nicely for anyone who wants to modify the Dockerfile so that the extra files can live in the image once instead of the overlay of each container.
On the other hand, if you have a pool of bgutil-ytdlp-pot-provider containers that all your TubeSync containers access over HTTP, then it is wasted space in the image with no benefit.