Skip to content

Commit

Permalink
* Several updates 2022_03_20. See full commit log.
Browse files Browse the repository at this point in the history 
* Configured Prowlarr, which is amazing.
* With prowlarr working, there is no need for Jackett and NZBHydra. Obsoleted both.
* Added flaresolverr to circumvent cloudflare challenge for some of the torrent trackers.
* Minor changes to bash_aliases.
* Added YT-DLP bash_aliases.
* Updated example systemd scripts for Rclone (rclone-drive.service.example and rclone-crypt.service.example) and MergerFS (merge-media.service.example).
* Added a check to see if rclone media upload is already running prior to starting a new one (upload-media.sh.example).
* Updated media-services.txt to add airsonic-advanced. This should have container names and not service names.
* Updated README.
* Obsoleted Navidrom and added Airsonic-Advanced with MariaDB backend. Couldn't live without the features of Airsonic.
* Added dupeguru temporarily - I am organizing my music collection and there a ton of duplicate files.
* Replaced StatPing with Uptime-Kuma, which is awesome.
* Tried Youtube-DL-Material and MeTube - Couldn't get these to work - Could not resolve host name error. YT-DLP works great.
  • Loading branch information
@anandslab
anandslab committed Mar 20, 2022
1 parent 447f265 commit 6c917d6
Show file tree
Hide file tree
Showing 10 changed files with 280 additions and 138 deletions.
5 changes: 3 additions & 2 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
@@ -1,16 +1,17 @@
###################### THIS CHANGELOG IS NO LONGER MAINTAINED. SEE DETAILED COMMIT LOGS. ##########################

# Changelog

- Only showing high-level changes. Smaller changes are too many to list. See commits.

## Planned (notes for future):

- Add projectsend, embystat, nextcloud, nut-upsd, HealthChecks, FileRun, fail2ban, ofelia, scrutiny to NUC, Wireguard, traktarr, listrr, uptimerobot, subliminal, netdata, FlareSolverr
- Add projectsend, embystat, nextcloud, nut-upsd, HealthChecks, FileRun, fail2ban, ofelia, scrutiny to NUC, Wireguard, traktarr, listrr, Subliminal, netdata
- Check Cloudbox/cloudbox - plex autoscan, cloudplow, plexdupefinder, plextraktsync
- implement secrets and remove variables from .env
- add prometheus, glances to influxdb, speedtest to influxdb
- Web GUI for rclone
- Switch to Hotio.dev's cloudflareddns,
- Remove lazylibrarian (added Readarr)
- Check traefik buffering - to avoid http error 431 - header request size too large
- Implement thelounge, privatebin

Expand Down
46 changes: 23 additions & 23 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ The apps I use are scattered around in several different docker-compose files. S

- Traefik - Reverse Proxy
- Docker Socket Proxy - Secure Proxy for Docker API
- Traefik Custom Error Pages (OBSOLETE)
- Traefik Custom Error Pages
- OAuth - Google OAuth 2 Forward Authentication
- Authelia - Private Forward Authentication
- Portainer - Container Management
Expand All @@ -99,30 +99,30 @@ The apps I use are scattered around in several different docker-compose files. S
### SMART HOME

- Home Assistant Core - Home Automation
- HA-Dockermon - Manage Docker containers in Home Assistant (OBSOLETE)
- HA-Dockermon - Manage Docker containers in Home Assistant
- Mosquitto - MQTT Broker
- MotionEye - Video Surveillance
- ZoneMinder - Video Surveillance (OBSOLETE)
- MiFlora - MiFlora MQTT Daemon (MiFlora Plant Sensors) (OBSOLETE)
- ZoneMinder - Video Surveillance
- MiFlora - MiFlora MQTT Daemon (MiFlora Plant Sensors)

### DATABASE

- MariaDB - MySQL Database
- phpMyAdmin - Database management
- InfluxDB - Database for sensor data
- Postgres - Database (OBSOLETE)
- Postgres - Database
- Grafana - Graphical data visualization for InfluxDB data
- Varken - Monitor Plex, Sonarr, Radarr, and Other Data (OBSOLETE)
- Varken - Monitor Plex, Sonarr, Radarr, and Other Data
- Redis - Key value store
- Redis Commander - Redis management

### DOWNLOADERS

- jDownloader - Download management
- TransmissionBT with VPN - Torrent Downloader with [IPVanish](https://www.smarthomebeginner.com/go/ipvanish) VPN.
- SABnzbd - Binary newsgrabber, NZB downloader (OBSOLETE)
- SABnzbd - Binary newsgrabber, NZB downloader
- Nzbget - Binary newsgrabber, NZB downloader
- qBittorrent with VPN - Torrent downloader (OBSOLETE)
- qBittorrent with VPN - Torrent downloader

### INDEXERS

Expand All @@ -140,20 +140,20 @@ The apps I use are scattered around in several different docker-compose files. S

### MEDIA SERVER

- AirSonic - Music Server (OBSOLETE)
- AirSonic - Music Server
- NaviDrome - Music Server
- FunkWhale - Music Server
- Calibre - Ebook/Audiobook Server (OBSOLETE)
- Calibre-Web - Ebook/Audiobook Reader (OBSOLETE)
- Calibre - Ebook/Audiobook Server
- Calibre-Web - Ebook/Audiobook Reader
- Plex - Media Server
- Emby - Media Server (OBSOLETE)
- Emby - Media Server
- Jellyfin - Media Server
- Ombi - Media Requests (OBSOLETE)
- Ombi - Media Requests
- Tautulli - Previously PlexPy. Plex statistics and monitoring
- Plex-Sync - For Syncing watched status between plex servers
- PhotoShow - Personal Photo Gallery and viewer (OBSOLETE)
- TellyTv- IPTV proxy for Plex (OBSOLETE)
- xTeve- IPTV proxy for Plex (OBSOLETE)
- PhotoShow - Personal Photo Gallery and viewer
- TellyTv- IPTV proxy for Plex
- xTeve- IPTV proxy for Plex

### MEDIA FILE MANAGEMENT

Expand All @@ -175,16 +175,16 @@ The apps I use are scattered around in several different docker-compose files. S
- Dozzle - Docker logs viewer
- qDirStat - Directory Statistics
- StatPing - Status Page & Monitoring Server
- SmokePing - Network Latency Monitoring (OBSOLETE)
- SmokePing - Network Latency Monitoring
- VS Code Server - Code Editor
- Logarr - Log Management (OBSOLETE)
- Monitorr - Webfront to display the status of any webapp or service (OBSOLETE)
- Cloud Commander - Web File Manager (OBSOLETE)
- Cloud9 - Cloud IDE (OBSOLETE)
- SMTP To Telegram - Sends all incoming Email messages to Telegram (OBSOLETE)
- Logarr - Log Management
- Monitorr - Webfront to display the status of any webapp or service
- Cloud Commander - Web File Manager
- Cloud9 - Cloud IDE
- SMTP To Telegram - Sends all incoming Email messages to Telegram
- UniFi Controller - Controller for Ubiquiti UniFi Network Gear
- Rclone - Mount Cloud/Google Drive
- MergerFS - Merge local and remote file systems (OBSOLETE)
- MergerFS - Merge local and remote file systems

### WEB

Expand Down
139 changes: 138 additions & 1 deletion docker-compose-t2-obsolete.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1684,4 +1684,141 @@ services:
- "traefik.http.routers.navidrome-rtr.middlewares=chain-no-auth@file"
## HTTP Services
- "traefik.http.routers.navidrome-rtr.service=navidrome-svc"
- "traefik.http.services.navidrome-svc.loadbalancer.server.port=4533"
- "traefik.http.services.navidrome-svc.loadbalancer.server.port=4533"

# Jackett - Torrent proxy
# Set url_base in Jackett settings if using PathPrefix
jackett:
<<: *common-keys-apps # See EXTENSION FIELDS at the top
image: lscr.io/linuxserver/jackett:latest
container_name: jackett
# network_mode: container:transmission-vpn
networks:
t2_proxy:
ipv4_address: 192.168.90.161
# ports:
# - "$JACKETT_PORT:9117"
volumes:
- $DOCKERDIR/appdata/jackett:/config
- $DATADIR/downloads:/data/downloads
- "/etc/localtime:/etc/localtime:ro"
environment:
<<: *default-tz-puid-pgid
labels:
- "traefik.enable=true"
## HTTP Routers
- "traefik.http.routers.jackett-rtr.entrypoints=https"
- "traefik.http.routers.jackett-rtr.rule=Host(`jackett.$DOMAINNAME0`)"
## Middlewares
- "traefik.http.routers.jackett-rtr.middlewares=chain-oauth@file"
## HTTP Services
- "traefik.http.routers.jackett-rtr.service=jackett-svc"
- "traefik.http.services.jackett-svc.loadbalancer.server.port=9117"

# NZBHydra2 - NZB meta search
hydra:
<<: *common-keys-apps # See EXTENSION FIELDS at the top
image: linuxserver/nzbhydra2:latest
container_name: hydra
ports:
- "$NZBHYDRA_PORT:5076"
volumes:
- $DOCKERDIR/appdata/hydra2:/config
- $DATADIR/downloads:/data/downloads
environment:
<<: *default-tz-puid-pgid
labels:
- "traefik.enable=true"
## HTTP Routers
- "traefik.http.routers.hydra-rtr.entrypoints=https"
- "traefik.http.routers.hydra-rtr.rule=Host(`hydra.$DOMAINNAME0`)"
## Middlewares
- "traefik.http.routers.hydra-rtr.middlewares=chain-oauth@file"
## HTTP Services
- "traefik.http.routers.hydra-rtr.service=hydra-svc"
- "traefik.http.services.hydra-svc.loadbalancer.server.port=5076"

# MeTube - Youtube downloader
# Could not get this to work - "Could not resolve name" error
metube:
<<: *common-keys-apps # See EXTENSION FIELDS at the top
image: alexta69/metube
container_name: metube
user: $PUID:$PGID
volumes:
- $DATADIR/downloads/youtube:/downloads
- $DOCKERDIR/appdata/metube/config:/config
- $DOCKERDIR/appdata/metube/cookies:/cookies
environment:
- STATE_DIR=/config
- YTDL_OPTIONS={"cookiefile":"/cookies/cookies.txt"}
labels:
- "traefik.enable=true"
## HTTP Routers
- "traefik.http.routers.metube-rtr.entrypoints=https"
- "traefik.http.routers.metube-rtr.rule=Host(`metube.$DOMAINNAME0`)"
## Middlewares
- "traefik.http.routers.metube-rtr.middlewares=chain-oauth@file"
## HTTP Services
- "traefik.http.routers.metube-rtr.service=metube-svc"
- "traefik.http.services.metube-svc.loadbalancer.server.port=8081"

# Youtube DL Material - Youtube downloader
# Could not get this to work - "Could not resolve name" error
youtube-dl:
<<: *common-keys-apps # See EXTENSION FIELDS at the top
image: tzahi12345/youtubedl-material:nightly
container_name: youtube-dl
volumes:
- $DOCKERDIR/appdata/youtube-dl:/app/appdata
- $DATADIR/downloads/youtube:/app/downloads
environment:
ALLOW_CONFIG_MUTATIONS: "true"
UID: $PUID
GID: $PGID
labels:
- "traefik.enable=true"
## HTTP Routers
- "traefik.http.routers.youtube-dl-rtr.entrypoints=https"
- "traefik.http.routers.youtube-dl-rtr.rule=Host(`ytdl.$DOMAINNAME0`)"
## Middlewares
- "traefik.http.routers.youtube-dl-rtr.middlewares=chain-oauth@file"
## HTTP Services
- "traefik.http.routers.youtube-dl-rtr.service=youtube-dl-svc"
- "traefik.http.services.youtube-dl-svc.loadbalancer.server.port=17442"

# StatPing - Status Page & Monitoring Server
# Not adding $TZ to linuxserver/mariadb resulted in a blank page
# Had to change default username admin in MariaDB
# dexec into mariadb and run mysql_tzinfo_to_sql /usr/share/zoneinfo | mysql -u root mysql -p
statping:
<<: *common-keys-core # See EXTENSION FIELDS at the top
image: statping/statping:latest
container_name: statping
networks:
- t2_proxy
- default
volumes:
- $DOCKERDIR/appdata/statping:/app
environment:
<<: *default-tz-puid-pgid
DB_CONN: mysql
DB_HOST: $MARIADB_HOST
DB_PORT: $MARIADB_PORT
DB_USER: $STATPING_DB_USER
DB_PASS: $STATPING_DB_PASS
DB_DATABASE: $STATPING_DB
IS_DOCKER: "true"
DISABLE_LOGS: "false"
NAME: StatPing
DESCRIPTION: Monitor web services
labels:
- "traefik.enable=true"
## HTTP Routers
- "traefik.http.routers.statping-rtr.entrypoints=https"
- "traefik.http.routers.statping-rtr.rule=Host(`sp.$DOMAINNAME0`)"
## Middlewares
- "traefik.http.routers.statping-rtr.middlewares=chain-oauth@file"
## HTTP Services
- "traefik.http.routers.statping-rtr.service=statping-svc"
- "traefik.http.services.statping-svc.loadbalancer.server.port=8080"

10 comments on commit 6c917d6

@DennisGaida
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seeing these changes / commits is like an addiction. Also deprecated statping (the -ng version) in favor of uptime-kuma.

@Nickous
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello
Yes! I'm a fan! And waiting any news 🥇
I wonder what you put in your basic-services.txt file?
Say like that, it sounds very inquisitive but it's for the Science... ;))

@anandslab
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you guys! It motivates me to share more. @DennisGaida Uptime-Kuma is great. There was this product that i was really interested in. It was low on stock and it went back and forth being in stock. I setup a HTTPS with keyword monitor on Uptime-kuma to get a telegram of when it came in stock. Sure enough it worked, I was able to grab the product within few hours. Of course, its capabilities go well beyond this.

@Nickous I am not sure why i did not share basic-services.txt. I will include it in the next commit. Basically its all services with this key: common-keys-core.

For me, the biggest issue i see is i could not get cloudflare-companion to work with the latest docker compose version. The maximum i could go was 2.1.1. If anyone has success i would love to hear.

@DennisGaida
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Latest docker-compose I don't know, I am on 3.8 and cloudflare-companion:latest works.

@robflate
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@DennisGaida I assume @htpcBeginner means the version you get with, docker-compose --version rather than the file format version you specify in the .yml file.

https://docs.docker.com/compose/compose-file/

I'm on compose 1.29.2 using the following for cf-companion which is working;

###############################################################################
# Cloudflare-Companion - Automatic CNAME DNS Creation
# https://hub.docker.com/r/tiredofit/traefik-cloudflare-companion
#
# - See https://www.smarthomebeginner.com/cloudflare-settings-for-traefik-docker/
#
# notes: Works
# - No secrets.
###############################################################################
  cf-companion:
    container_name: cf-companion
    image: tiredofit/traefik-cloudflare-companion:latest
    security_opt:
      - no-new-privileges:true
    restart: unless-stopped
    networks:
      - socket_proxy
    environment:
      TIMEZONE: $TZ
      TRAEFIK_VERSION: 2
      CF_TOKEN: $CLOUDFLARE_API_TOKEN
      TARGET_DOMAIN: $DOMAINNAME
      DOMAIN1: $DOMAINNAME
      DOMAIN1_ZONE_ID: $CLOUDFLARE_ZONEID # Copy from Cloudflare Overview page
      DOMAIN1_PROXIED: "TRUE"
      DOCKER_HOST: tcp://socket-proxy:2375
    labels:
      # Add hosts specified in rules here to force cf-companion to create the CNAMEs
      - "traefik.http.routers.cf-companion-rtr.rule=Host(`pihole.$DOMAINNAME`)"
      - "traefik.http.routers.cf-companion-rtr.rule=Host(`skyrouter.$DOMAINNAME`)"

Also, my working code for Youtube-DL-Material that @htpcBeginner referenced as not working in this commit;

###############################################################################
# YouTubeDL Material - Material Design frontend for youtube-dl
# https://hub.docker.com/r/tzahi12345/youtubedl-material
#
# notes: Works
###############################################################################
  youtubedl-material:
    image: tzahi12345/youtubedl-material:latest
    container_name: youtubedl-material
    restart: unless-stopped
    networks:
      - t2_proxy
    security_opt:
      - no-new-privileges:true
    # ports:
    #   - "$YOUTUBEDLMATERIAL_PORT:17442"
    volumes:
      - $CONFIG/youtubedl-material/appdata:/app/appdata
      - $DATA/direct/youtubedl-material/audio:/app/audio
      - $DATA/direct/youtubedl-material/video:/app/video
      - $DATA/direct/youtubedl-material/subscriptions:/app/subscriptions
      - $DATA/direct/youtubedl-material/users:/app/users
    environment:
      UID: $PUID
      GID: $PGID
    labels:
      - "traefik.enable=true"
      ## HTTP Routers
      - "traefik.http.routers.youtubedl-material-rtr.entrypoints=https"
      - "traefik.http.routers.youtubedl-material-rtr.rule=Host(`youtubedl-material.$DOMAINNAME`)"
      ## Middlewares
      - "traefik.http.routers.youtubedl-material-rtr.middlewares=chain-oauth@file"
      ## HTTP Services
      - "traefik.http.routers.youtubedl-material-rtr.service=youtubedl-material-svc"
      - "traefik.http.services.youtubedl-material-svc.loadbalancer.server.port=17442"

@anandslab
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello Yes! I'm a fan! And waiting any news 🥇 I wonder what you put in your basic-services.txt file? Say like that, it sounds very inquisitive but it's for the Science... ;))

6c917d6

@anandslab
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@robflate 1.29.2 for docker-compose is quite old (but it still works). Newer versions has several udpates (including some cool asthetics). I am trying to upgrade to the latest. But the farthest i could go without enountering issues was 2.1.1.

@anandslab
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@robflate to clarify, Youtube-DL-Material worked during my testing, in that, the web ui opened up. But I could not download anytihng.....I got the "unable to resolve host address" error or something similar.

@anandslab
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@robflate I still get the following error with your Youtube-DL compose
ERROR: unable to download video data: <urlopen error [Errno -2] Name does not resolve>

@robflate
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, brain not in gear, I was checking the wrong user. My compose version for this stack is v2.2.3 and cloudflare companion works. I just did a test and it successfully added a subdomain to Cloudflare.

I just tested Youtube-DL-Material and succesfully downloaded a video. Youtube-DL-Material reports as v4.2 in the Settings.

Newer versions has several udpates (including some cool asthetics).

Maybe you've managed to customise the new compose but I really dislike its Terminal output. The previous compose would stack all the changed containers at the bottom of the output and colour them differently. With this Compose, everything is the same colour on completion so it's really hard to see which containers have updated. Any tips on improving readability appreciated.

Please sign in to comment.