feat: support for postgresql

[zackhow]

Near the same as my PR in overseerr. Opening this here for others to test out. I haven't run into any issues myself so a second set of eyes would be good.

I also tested migrating an overseerr sqlite db but would also like more testing for that if possible.

[Fallenbagel]

I will be reviewing this and creating a preview tag this week c:

[zackhow]

For anyone wanting to test migrating you can use pgloader to test. Swap out the username, password, ip and database. You have to run jellyseerr once to create the schema before migrating.

docker run --rm -v /tmp/db.sqlite3:/db.sqlite3:ro --network=host ghcr.io/roxedus/pgloader --with "quote identifiers" --with "data only" /db.sqlite3 'postgresql://$username:$password@ip/$database'

[teambvd]

I've been testing this for the last ~wk (8 days) with no issues. Had to add the following for the migration to work though:

--with "prefetch rows = 100" --with "batch size = 1MB"

( From here )

I added 9 multi-season shows in rapid succession, something that normally would've resulted in shows getting added to sonarr, but either not searched, set to unmonitored, or equally odd behaviors - this time, it went through flawlessly.

GREAT work!! 👍

[zackhow]

I've been testing this for the last ~wk (8 days) with no issues. Had to add the following for the migration to work though:

--with "prefetch rows = 100" --with "batch size = 1MB"

( From here )

I added 9 multi-season shows in rapid succession, something that normally would've resulted in shows getting added to sonarr, but either not searched, set to unmonitored, or equally odd behaviors - this time, it went through flawlessly.

GREAT work!! 👍

Thanks! Doing dev work I always expect users to somehow put their head underwater if I asked them to QA a toilet, which can be helpful to find those edge case bugs.

[ralgar]

Great work so far, @zackhow! I managed to get this working after rebasing on v1.7.0, and migrating the initial SQLite DB from Jellyseerr (which was a huge chore, see below for details). I also built a container based on this PR.

I notice this hasn't had any progress in a while, are you still able/willing to work on it, @zackhow? I'm willing to lend a hand, or take it to the finish line myself if you're unable.

I've noticed these finishing touches would be required, in order for this to be production-ready:

• TLS settings - there should at least be a way to provide a self-signed CA cert, and a flag to toggle strict CA verification. I just hard-coded this for my test case, but it's easy to add more env vars here as well.
• Needs numerous changes to the Jellyseerr migrations - a couple of the idioms used (datetime for example) are SQLite-specific, and the current handling of FK constraints during migration (by disabling their checks) doesn't appear to be at all Postgres-friendly either. I've never dealt with migrations before, so I'm not entirely sure how this is typically handled. Would be good to get input from @Fallenbagel here.

Docker Container

I built a Docker container based on this PR, and also disabled (hardcoded) strict CA verification for anyone using self-signed certs on their Postgres instance. My requests stack is now backed entirely by Postgres (Jellyseerr, Prowlarr, Radarr, and Sonarr), yay!

ghcr.io/ralgar/jellyseerr-pgsql:v1.7.0-preview

My fun with migrating

I was unable to use the WITH data only option suggested above. I also ran into this issue.

I ended up having to build a custom container from this PR, and also using this pgloader script:

LOAD DATABASE
    FROM {{DATA_SOURCE}}
    INTO postgresql://{{POSTGRES_USER}}:{{POSTGRES_PASSWORD}}@{{POSTGRES_HOST}}:{{POSTGRES_PORT}}/{{POSTGRES_DATABASE}}?sslmode=allow

    WITH
        quote identifiers

    CAST type datetime to timestamptz drop default
    AFTER LOAD DO
        $$ alter table season_request alter "createdAt" set default now() $$,
        $$ alter table season_request alter "updatedAt" set default now() $$

; /* Finalize script */

You can download the seed DB I used (there's no data in it, just the schema) from here. It's from Jellyseerr v1.7.0, and it matches with the container I linked above.

Hopefully this info will help anyone wanting to test and/or develop this PR.

[KtaHD]

Hello,
Where do i put the username, password, db_host and port? settings.json?
I want to use postgress with my jellyseerr.

[ralgar]

@KtaHD use environment variables.

DB_TYPE=postgres
DB_HOST
DB_PORT
DB_USER
DB_PASS
DB_NAME

[KtaHD]

Thank you @ralgar for the response:

This is my setup:

version: '3'
services:
jellyseerr:
image: ghcr.io/ralgar/jellyseerr-pgsql:v1.7.0-preview
container_name: jellyseerr-psql
environment:
- LOG_LEVEL=debug
- TZ=Europe/Bucharest
- DB_TYPE=postgres
- DB_HOST=
- DB_PORT=5432
- DB_USER=jellyseerr
- DB_PASS=
- DB_NAME=jellyseerr
ports:
- 5056:5055
volumes:
- /data/docker/jellyseerr-psql/config:/app/config
restart: unless-stopped

I keep receiving this error, i don't know how should i setup sslmode=disable or something:

2023-12-16T04:35:51.934Z [info]: Commit Tag: abd80c1
2023-12-16T04:35:52.284Z [info]: Starting Overseerr version 1.7.0
warn - You have enabled experimental features (scrollRestoration, largePageDataBytes) in next.config.js.
warn - Experimental features are not covered by semver, and may cause unexpected or broken application behavior. Use at your own risk.
2023-12-16T04:35:52.972Z [error]: Error: The server does not support SSL connections
at Socket. (/app/node_modules/pg/lib/connection.js:77:37)
at Object.onceWrapper (node:events:628:26)
at Socket.emit (node:events:513:28)
at addChunk (node:internal/streams/readable:315:12)
at readableAddChunk (node:internal/streams/readable:289:9)
at Socket.Readable.push (node:internal/streams/readable:228:10)
at TCP.onStreamRead (node:internal/stream_base_commons:190:23)
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
yarn run v1.22.19

[ralgar]

@KtaHD Thanks for catching this. This is partially an issue with how TypeORM has handled TLS connection parameters. The only way to set sslmode explicitly is using a connection string, however that opens up another can of worms because self-signed TLS certs would then also need to be specified as part of the string. Reading the TypeORM docs, I can see no other options here.

I've come up with a simple workaround though. You won't be able to choose a specific SSL mode, but you can either enable or disable SSL entirely with DB_USE_SSL={true,false}, with disabled being the default. If SSL is enabled, then we simply skip CA verification for now.

I've built a new container image, under the same tag as before. Please let me know if it works for you.

[dr-carrot]

If anyone else is having trouble with the discover page not working - I had to change the column types for type and order in the discover_slider table from bigint to int. Javascript doesn't support numbers that large and was changing them to strings.

[KtaHD]

@KtaHD Thanks for catching this. This is partially an issue with how TypeORM has handled TLS connection parameters. The only way to set sslmode explicitly is using a connection string, however that opens up another can of worms because self-signed TLS certs would then also need to be specified as part of the string. Reading the TypeORM docs, I can see no other options here.

I've come up with a simple workaround though. You won't be able to choose a specific SSL mode, but you can either enable or disable SSL entirely with DB_USE_SSL={true,false}, with disabled being the default. If SSL is enabled, then we simply skip CA verification for now.

I've built a new container image, under the same tag as before. Please let me know if it works for you.

@ralgar Thanks for the fix, i configured SSL on my Postgres instance anyway just in case and it works.
@dr-carrot It works perfectly after i changed the type of "type" and "order" and i attached a screenshot from where you can find the settings in case anyone needs it (using pgadmin), just right click on the column and proprieties.

The issues that i tough i can fix is not yet resolved by using postgres, the app is still loading painfully slow (using ip, not domain).

[Fallenbagel]

@KtaHD that could be dns. Try adding

dns:
     - 1.1.1.1

If a container

[KtaHD]

Always the DNS... Thank you @Fallenbagel, you're a legend. I still have issues making Docker Network accepting Pi-Hole as his Lord and Savior.
It's works good now, i will try the cache function too.

[KtaHD]

If anyone else is having trouble with the discover page not working - I had to change the column types for type and order in the discover_slider table from bigint to int. Javascript doesn't support numbers that large and was changing them to strings.

I don't know if we can fix the Requests menu using this, if i delete something from there it reappears. Also there must be something with the notifications, because i left the container opened today and it sent like 10 emails to confirm that a movie is available now

[Fallenbagel]

superseded by #628