How to "translate" database connection parameters to their equivalent Rocket-listened-to-at-runtime environment variables...

[x10an14]

Hi!

Apologies for any undertones/rantiness in this question - I hope I can be forgiven the frustration of banging my head against this wall for ~6 hours the last 18.

What I want

A rocket-based rust application that on startup reads environment variables containing <database name>, <database port>, <database host address>, etc to dynamically generate the connection url string (or whatever internally equivalent is sent to SQLx from Rocket) at startup.

NB: If I have to manually merge these together with a pre-flight script, into a Rocket.toml, I can live with that - but sounds very unnecessary to me =/.

What I have tried

1. Following the SQLx example, ensuring that the DB names are matching in the Rocket.tomls debug.databases.DB (as described in the documentation here) as well as in the struct in the below code:

   Rocket.toml

   [debug.database.DB]
   url = "postgres://postgres@localhost:5432/exampledb"
   # user = "postgres" # Doesn't help

   Rust code:

   #[derive(Database)]
   #[database("DB")]
   struct Db(sqlx::PgPool);
   
   async fn run_migrations(rocket: Rocket<Build>) -> fairing::Result {
   	match Db::fetch(&rocket) {
   		Some(db) => match sqlx::migrate!("./database-migrations").run(&**db).await {
   			Ok(_) => Ok(rocket),
   			Err(e) => {
   				error!("Failed to initialize SQLx database: {}", e);
   				Err(rocket)
   			},
   		},
   		None => Err(rocket),
   	}
   }
   
   pub fn stage() -> AdHoc {
   	AdHoc::on_ignite("SQLx Stage", |rocket| async {
   		rocket
   			.attach(Db::init())
   			.attach(AdHoc::try_on_ignite("SQLx Migrations", run_migrations))
   	})
   }
   
   #[rocket::main]
   async fn main() -> Result<()> {
   	color_eyre::install()?;
   
   	debug!("Starting up rocket...");
   	let rocket_result = rocket::build()
   		.attach(db::stage()) // Ensure db migrations are run
   		.mount("/", routes![root_index, greet, offend, offend2])
   		.launch()
   		.await?;
   	info!("Rocket has finished with result: {rocket_result:#?}");
   	Ok(())
   }

   That only gave me this feedback:

   2023-05-27T12:56:29.344318Z ERROR rocket_db_pools::database: failed to initialize database: bad configuration: missing field `url`
   2023-05-27T12:56:29.344366Z ERROR rocket_db_pools::database: Attempted to fetch unattached database `my_app::db::Db`.
   2023-05-27T12:56:29.344381Z  INFO rocket_db_pools::database::_: `my_app::db::Db::init()` fairing must be attached prior to using this database.
   

2. Following the failure previously mentioned example, I tried to jump to my ideal solution of just specifying the correct env-vars.
   However, I cannot for the life of me figure out after reading the documentation how a database's connection url/host/username is set by leveraging env-vars.

   Specifically, I struggle figuring out the map between an arbitrary Rocket.toml DB connection config setting <-> full name of env-var.

   Here are the other resources I claim to have read, which refence this possibility, but not in any way I managed to understand sufficiently to understand the previously mentioned mapping function:

   1. this other discussion thread's comment,
   2. rocket_db_pools Config documentation,
   3. rocket_db_pools Configuration documentation,
   4. and Rocket's own default-provider/figment documenation

[SergioBenitez]

Oh no. The key is databases, not database. Hopefully adding the s fixes things?

It would be great if we could get a "hey, we found this configuration which is close to what we expected but isn't actually; did you mean that?"

[x10an14]

Oh no. The key is databases, not database. Hopefully adding the s fixes things?

Thanks for the find! =) I would have been more than happy for my idiocy/blindness to have been the culprit here, but alas the problem still persists.

Rust code is still as specified above, Rocket.toml has been modified (it now reads: debug.databases.DB), and still I have the below output:

   [2023-05-27 18:36:21] 0 x10an14@home-desktop:~/Documents/my-app
   -> $ sudo netstat -atulpn | grep LISTEN | grep 5432
   tcp        0      0 0.0.0.0:5432            0.0.0.0:*               LISTEN      3002/rootlesskit
   tcp6       0      0 :::5432                 :::*                    LISTEN      3002/rootlesskit
   [2023-05-27 18:36:24] 0 x10an14@home-desktop:~/Documents/my-app
   -> $ docker ps --filter status=running --format "{{.ID}}({{.Image}}): {{.Status}} -> {{.Ports}}"
   1c5b133f7fa7(postgres:15): Up 2 minutes (healthy) -> 0.0.0.0:5432->5432/tcp, :::5432->5432/tcp
   [2023-05-27 18:36:31] 0 x10an14@home-desktop:~/Documents/my-app
   -> $ cargo run
   warning: dependency (rocket_db_pools) specified without providing a local path, Git repository, or version to use. This will be considered an error in future versions
       Finished dev [unoptimized + debuginfo] target(s) in 0.10s
        Running `target/debug/my-app`
   2023-05-27T16:36:40.094656Z ERROR rocket_db_pools::database: failed to initialize database: failed to initialize database: error communicating with database: Connection reset by peer (os error 104)
   2023-05-27T16:36:40.094749Z ERROR rocket_db_pools::database: Attempted to fetch unattached database `my_app::db::Db`.
   2023-05-27T16:36:40.094768Z  INFO rocket_db_pools::database::_: `my_app::db::Db::init()` fairing must be attached prior to using this database.
   Error:
      0: launch fairing(s) failed

You can see from above bash snippet that I've got postgres running, listening on port, yet the below Rocket.toml won't match =/

[debug.databases.DB]
# See https://rocket.rs/v0.5-rc/guide/configuration/#custom-providers
# Can be overriden with env ROCKET_DATABASE_EXAMPLEDB_URL?? Doesn't seem like it
url = "postgres://postgres@localhost:5432/exampledb"
user = "postgres"

---

It would be great if we could get a "hey, we found this configuration which is close to what we expected but isn't actually; did you mean that?"

I am not 100% certain what you meant by this, but if you meant it as a response to my wording and frustration, then you're completely correct.
Mea culpa, I should've found a way to calm down after getting so frustrated with the lack of understanding/progress.

I do appreciate the fast reply/suggestion!

[x10an14]

Seems to be that the rootless docker daemon had to be restarted, something finicky going on there.
I can now agree that it works as expected/documented with the Rocket.toml file!

Now to figure out how to selectively replace database values for the following:

• hostname address
• port
• user
• db

[SergioBenitez]

The syntax for environment variable values is that of structured values as in TOML. For the specific case, the configuration docs have an example.

[x10an14]

Jupp, thanks! Same as I documented in my answer here =) #2550 (comment)

[x10an14]

Your answer and suggestion is much appreciated @SergioBenitez, but it's not an answer to what I thought I'd worded the discussion for =/
Yes, it's on the path of solving the "immediate" probem (connecting my rocket instance to my postgresql server).

But no, it doesn't help me with the request:

Specifically, I struggle figuring out the map between an arbitrary Rocket.toml DB connection config setting <-> full name of env-var.

[x10an14]

Solution to my question:

As of version 0.5.0-rc.3, you must make a script that parses/reads/incorporates values from other files/env-vars into a Rocket.toml file or ROCKET_DATABASES env-var.
There is no other choice. (read: No mapping as inquired exists).

Aka (for a given rust code shown below):

# contents of some shellscript to start the compiled rocket binary/program
dbName="${DB_NAME}"
dbAddr="${DB_HOST}"
dbPort="${DB_PORT}"
dbUser="${DB_USER}"
dbPassword="${DB_PASSWORD}"

ROCKET_DATABASES="APP_DATABASE={url='postgres://${dbUser}:${dbPassword}@${dbAddr}/${dbName}'}" target/(debug|release)/my-app

And Rust code:

#[derive(Database)]
#[database("APP_DATABASE")]
struct Db(sqlx::PgPool);

---

https://api.rocket.rs/v0.5-rc/rocket_db_pools/index.html#configuration (mentioned in my opening post) does document this. I just could not get it to work at first due to my docker daemon issue mentioned here.