Define Datasource Options with Environment Variables

[jscruggs]

We are encountering an issue where we can't compile our project because we are using environment variables to set the database password, username, and jdbc-url in def datasource-options. Turning the database-options and database defs into functions cause the following error when we run our test suite:

Exception in thread "main" com.zaxxer.hikari.pool.HikariPool$PoolInitializationException: Failed to initialize pool: Listener refused the connection with the following error:

ORA-12516, TNS:listener could not find available handler with matching protocol stack
 , compiling:(/private/var/folders/hg/6z5kczsn5jg8z9r_n68ng7nm0000gn/T/form-init3653725058901894960.clj:1:142)
	at clojure.lang.Compiler.load(Compiler.java:7391)
	at clojure.lang.Compiler.loadFile(Compiler.java:7317)
	at clojure.main$load_script.invokeStatic(main.clj:275)
	at clojure.main$init_opt.invokeStatic(main.clj:277)
	at clojure.main$init_opt.invoke(main.clj:277)
	at clojure.main$initialize.invokeStatic(main.clj:308)
	at clojure.main$null_opt.invokeStatic(main.clj:342)
	at clojure.main$null_opt.invoke(main.clj:339)
	at clojure.main$main.invokeStatic(main.clj:421)
	at clojure.main$main.doInvoke(main.clj:384)
	at clojure.lang.RestFn.invoke(RestFn.java:421)
	at clojure.lang.Var.invoke(Var.java:383)
	at clojure.lang.AFn.applyToHelper(AFn.java:156)
	at clojure.lang.Var.applyTo(Var.java:700)
	at clojure.main.main(main.java:37)

Oddly enough, some of the tests that hit the database pass while later tests in the run return the above error.

Here is our code (calls to 'env' are using 'environ' to get environment variables):

(defn datasource-options []
    {:jdbc-url    (str "jdbc:oracle:" (env :db-subname))
     :user        (env :db-user)
     :password    (env :db-password)})
     
(defn datasource []
  (hikari/make-datasource (datasource-options)))

Do the datasource-options and datasource defs need to be defs? If so, how do we avoid hardcoding sensitive information (passwords) into our applications and for use in different environments?

[tomekw]

Hello @jscruggs thank you for the report.

I will try to take a look today. Or maybe @jstepien will help :)

[jscruggs]

We seemed to have solved the problem by using 'memoize' like so:

(defn datasource-options []
    {:jdbc-url    (str "jdbc:oracle:" (env :db-subname))
     :user        (env :db-user)
     :password    (env :db-password)})
     
(defn datasource []
  (hikari/make-datasource (datasource-options)))
  
(def datasource-memo 
  (memoize datasource))

Then we can use 'datasource-memo' like so:

(jdbc/with-db-connection [conn {:datasource (datasource-memo)}]
        (jdbc/query conn
...

Everything appears to work fine: Let me know if this is a non-approved way of using hikari-cp.

[tomekw]

Everything appears to work fine: Let me know if this is a non-approved way of using hikari-cp.

@jscruggs if it works for you... it's OK :) Maybe a good idea would be to update the README to describe the usage. Would you be so kind and prepare a PR? :)

[jstepien]

Thanks for taking the time to file the issue @jscruggs.

Do the datasource-options and datasource defs need to be defs?

The options map you pass to hikari/make-datasource can be a value coming from an arbitrary source. It can be defined using def, it can be a value returned by a function, a let binding, or anything else. The Environ-based approach you describe above is correct; it should work without any problems. The call to memoize should not be necessary.

Oddly enough, some of the tests that hit the database pass while later tests in the run return the above error.

Let's take a closer look at this problem. Are you sure that the datasource-options function invoked in your tests always returns the same map? Are you sure that you're closing Hikari connection pools after using them in tests? Maybe you're leaking open connections and the database used in tests refuses to open more connections?

Let us know. A minimal test case allowing us to reproduce the issue against an open source database would be wonderful. Cheers!

[tomekw]

Closing due to inactivity.