Read environment variables in config/config.exs

[tute]

config/config.exs loads before lib/APP_NAME.ex.

How can we have envy read .env vars before the config.exs file is read? I'm specifically configuring https://github.com/stavro/arc#s3-configuration.

Thank you, Blake! :)

[tute]

If I could use the syntax {:system, "AWS_S3_BUCKET"} this would work.

[BlakeWilliams]

If you run the following in config/config.exs below use Mix.Config, does it work?

unless Mix.env == "prod" do
  Envy.auto_load
end

[tute]

Dependencies are not loaded when config runs, so it doesn't.

[BlakeWilliams]

Can you call Application.load(:envy) or does that also not work in the mix config?

[tute]

Adding the following:

unless Mix.env == :prod do
  Envy.auto_load
end

Fails with:

** (Mix.Config.LoadError) could not load config config/config.exs
    ** (UndefinedFunctionError) function Envy.auto_load/0 is undefined (module Envy is not available)
    Envy.auto_load()

With Application.load(:envy):

Application.load(:envy)
unless Mix.env == :prod do
  Envy.auto_load
end

It doesn't change. In a pry session:

pry(2)> Envy.auto_load
** (UndefinedFunctionError) function Envy.auto_load/0 is undefined (module Envy is not available)
    Envy.auto_load()
pry(2)> Application.load :envy
{:error, {'no such file or directory', 'envy.app'}}

[BlakeWilliams]

Ah! Turns out Mix.Config doesn't allow dependencies to be used, or easily compiled and used in the config file.

There is a workaround, which I think may be a good addition to Envy if it works for you.

You can run the following to re-eval the config:

Mix.Config.read!("config/config.exs") |> Mix.Config.persist

I think you'd want to put it directly after Envy.auto_load in your mix config. I think there may be potential side effects, but I'm not totally sure.

[tute]

Yep! Following patch works well:

From adebf8561a99b5cd2054e2d97c16e28aff078147 Mon Sep 17 00:00:00 2001
From: Tute Costa <tutecosta@gmail.com>
Date: Thu, 30 Jun 2016 12:18:52 -0400
Subject: [PATCH] Add Envy for environment variables

---
 .sample.env     | 3 +++
 lib/app_name.ex | 5 +++++
 mix.exs         | 1 +
 mix.lock        | 1 +
 4 files changed, 10 insertions(+)
 create mode 100644 .sample.env

diff --git a/.sample.env b/.sample.env
new file mode 100644
index 0000000..707fbf9
--- /dev/null
+++ b/.sample.env
@@ -0,0 +1,3 @@
+AWS_ACCESS_KEY_ID=abcd
+AWS_SECRET_ACCESS_KEY=abcd
+AWS_S3_BUCKET=abcd
diff --git a/lib/app_name.ex b/lib/app_name.ex
index f2b6334..5899679 100644
--- a/lib/app_name.ex
+++ b/lib/app_name.ex
@@ -6,6 +6,11 @@ defmodule AppName do
   def start(_type, _args) do
     import Supervisor.Spec, warn: false

+    unless Mix.env == :prod do
+      Envy.auto_load
+      Mix.Config.read!("config/config.exs") |> Mix.Config.persist
+    end
+
     children = [
       # Start the endpoint when the application starts
       supervisor(AppName.Endpoint, []),
diff --git a/mix.exs b/mix.exs
index 8f2feb2..98f8472 100644
--- a/mix.exs
+++ b/mix.exs
@@ -46,6 +46,7 @@ defmodule AppName.Mixfile do
       {:arc, "~> 0.5.2"},
       {:comeonin, "~> 2.4"},
       {:cowboy, "~> 1.0"},
+      {:envy, "~> 0.0.2"},
       {:ex_aws, "~> 0.4.10"},
       {:ex_machina, "~> 1.0.0-beta.1", github: "thoughtbot/ex_machina"},
       {:floki, "~> 0.8.1"},
diff --git a/mix.lock b/mix.lock
index bb52130..ef01aad 100644
--- a/mix.lock
+++ b/mix.lock
@@ -7,6 +7,7 @@
   "db_connection": {:hex, :db_connection, "0.2.5", "3e5e28019e0ec744345568d22a2f5206109bff0e2571f4d7819e0d14cf955f3e", [:mix], [{:sbroker, "~> 0.7", [hex: :sbroker, optional: true]}, {:poolboy, "~> 1.5", [hex: :poolboy, optional: true]}, {:connection, "~> 1.0.2", [hex: :connection, optional: false]}]},
   "decimal": {:hex, :decimal, "1.1.2", "79a769d4657b2d537b51ef3c02d29ab7141d2b486b516c109642d453ee08e00c", [:mix], []},
   "ecto": {:hex, :ecto, "2.0.0-rc.5", "a95184a260edb89669be7746b3b270725ce6940e378b05d19df82eecf52544cb", [:mix], [{:sbroker, "~> 0.7", [hex: :sbroker, optional: true]}, {:postgrex, "~> 0.11.1", [hex: :postgrex, optional: true]}, {:poolboy, "~> 1.5", [hex: :poolboy, optional: false]}, {:poison, "~> 1.5 or ~> 2.0", [hex: :poison, optional: true]}, {:mariaex, "~> 0.7.1", [hex: :mariaex, optional: true]}, {:decimal, "~> 1.0", [hex: :decimal, optional: false]}]},
+  "envy": {:hex, :envy, "0.0.2", "278045cf7a7c7d46bd607e9cd6632b7d3fc71ddaa9c26bf364ec11af961b313b", [:mix], []},
   "ex_aws": {:hex, :ex_aws, "0.4.19", "84284a2ce936b1aed3d6db4dac4ccf28105f2b617201194b2804d737cd0ef818", [:mix], [{:jsx, "~> 2.5", [hex: :jsx, optional: true]}, {:httpotion, "~> 2.0", [hex: :httpotion, optional: true]}, {:poison, "~> 1.2 or ~> 2.0", [hex: :poison, optional: true]}, {:httpoison, "~> 0.8", [hex: :httpoison, optional: true]}, {:sweet_xml, "~> 0.5", [hex: :sweet_xml, optional: true]}]},
   "ex_machina": {:git, "https://github.com/thoughtbot/ex_machina.git", "a24c02618b460094bb81ea90f41e052355383e5a", []},
   "floki": {:hex, :floki, "0.8.1", "06aa75bf2d1e01cda7d2ad54f68614be653a11e76b474d8fcb1d838f8c1e0ad1", [:mix], [{:mochiweb_html, "~> 2.15", [hex: :mochiweb_html, optional: false]}]},
--
2.8.1

Thank you Blake!

[BlakeWilliams]

No problem!

I think this would be nice as Envy.reload_config or some better name. Feel free to open a PR, if not I'll try to work on it today.

[BlakeWilliams]

Then I think it'd be worth bumping up to 1.0 since Envy has seen good use and is pretty stable. 😃

[jamescmartinez]

👍

[BlakeWilliams]

This was fixed in 1.0. Closing.

Thanks for the report!

[jakecraige]

I'm running into this same issue, but when configuring another app, pushex specifically, it seems to have a presence validation when it starts up its app and that throws an error since it isn't set at boot time.

Error looks like so:

[info] Application pushex exited: exited in: Pushex.App.start(:normal, [])
    ** (EXIT) an exception was raised:
        ** (Pushex.ValidationError) error on :auth_key with :presence validator: must be present. error on :auth_key with :type validator: must be of type :string
            (pushex) lib/pushex/gcm/app.ex:28: Pushex.GCM.App.create!/1
            (elixir) lib/enum.ex:1184: Enum."-map/2-lists^map/1-0-"/2
            (pushex) lib/pushex/config.ex:104: Pushex.Config.load_apps/3
            (pushex) lib/pushex/config.ex:63: Pushex.Config.make_common_config/1
            (pushex) lib/pushex/config.ex:42: Pushex.Config.make_defaults/1
            (pushex) lib/pushex/app.ex:12: Pushex.App.start/2
            (kernel) application_master.erl:273: :application_master.start_it_old/4
[info] Application apns exited: :stopped
[info] Application vex exited: :stopped
[info] Application postgrex exited: :stopped
[info] Application db_connection exited: :stopped
[info] Application connection exited: :stopped
[info] Application phoenix_pubsub exited: :stopped
[info] Application phoenix_html exited: :stopped
[info] Application phoenix_ecto exited: :stopped
[info] Application ecto exited: :stopped
[info] Application poolboy exited: :stopped
[info] Application decimal exited: :stopped
[info] Application httpoison exited: :stopped
[info] Application hackney exited: :stopped
[info] Application metrics exited: :stopped
[info] Application ssl_verify_fun exited: :stopped
[info] Application certifi exited: :stopped
[info] Application mimerl exited: :stopped
[info] Application idna exited: :stopped
[info] Application ex_aws exited: :stopped
[info] Application gen_stage exited: :stopped
[info] Application cowboy exited: :stopped
[info] Application cowlib exited: :stopped
[info] Application ranch exited: :stopped
[info] Application arc_ecto exited: :stopped

=INFO REPORT==== 18-Dec-2016::13:58:35 ===
    application: logger
    exited: stopped
    type: temporary
** (Mix) Could not start application pushex: exited in: Pushex.App.start(:normal, [])
    ** (EXIT) an exception was raised:
        ** (Pushex.ValidationError) error on :auth_key with :presence validator: must be present. error on :auth_key with :type validator: must be of type :string
            (pushex) lib/pushex/gcm/app.ex:28: Pushex.GCM.App.create!/1
            (elixir) lib/enum.ex:1184: Enum."-map/2-lists^map/1-0-"/2
            (pushex) lib/pushex/config.ex:104: Pushex.Config.load_apps/3
            (pushex) lib/pushex/config.ex:63: Pushex.Config.make_common_config/1
            (pushex) lib/pushex/config.ex:42: Pushex.Config.make_defaults/1
            (pushex) lib/pushex/app.ex:12: Pushex.App.start/2
            (kernel) application_master.erl:273: :application_master.start_it_old/4

Any idea how to work around this?

[BlakeWilliams]

Not sure if it would work, but if we made Envy an application and put it at the start that might work.

I'll try to write something up real quick, push it up to a branch, and see if that works. Feel free to ping me in our #elixir channel too btw.

[BlakeWilliams]

@jakecraige I just pushed the branch bmw-application, try using that and putting :envy at the beginning of your applications list in mix.exs and see if that works around it.

[retgoat]

Hi @BlakeWilliams
it seems your quick hack doesn't work.

I'm trying to use Envy to manage KafkaEx config.

config :kafka_ex,
  brokers: [
    {System.get_env("KAFKA_BROKER_HOST"), 9092}
  ],
  consumer_group: System.get_env("KAFKA_CONSUMER_GROUP"),
  auto_commit: false,
  use_ssl: false

But I'm getting an error:

(Mix) Could not start application kafka_ex: KafkaEx.start(:normal, []) returned an error: an exception was raised:
    ** (FunctionClauseError) no function clause matching in Regex.scan/3
        (elixir) lib/regex.ex:396: Regex.scan(~r/^(\d{1,3})\.(\d{1,3})\.(\d{1,3})\.(\d{1,3})$/, nil, [])
        (kafka_ex) lib/kafka_ex/network_client.ex:56: KafkaEx.NetworkClient.format_host/1
        (kafka_ex) lib/kafka_ex/network_client.ex:9: KafkaEx.NetworkClient.create_socket/4
        (kafka_ex) lib/kafka_ex/server_0_p_9_p_0.ex:56: anonymous fn/3 in KafkaEx.Server0P9P0.kafka_server_init/1
        (elixir) lib/enum.ex:1229: Enum."-map/2-lists^map/1-0-"/2
        (kafka_ex) lib/kafka_ex/server_0_p_9_p_0.ex:56: KafkaEx.Server0P9P0.kafka_server_init/1
        (stdlib) gen_server.erl:328: :gen_server.init_it/6
        (stdlib) proc_lib.erl:247: :proc_lib.init_p_do_apply/3

I'm reading values from config/dev.exs and trying to use bmw-application branch.

# mix.lock
"envy": {:git, "https://github.com/BlakeWilliams/envy.git", "446ec3f609e5da1ece1870ed45e6edcf295cc3e9", [branch: "bmw-application"]},

# mix.exs
  def application do
    # Specify extra applications you'll use from Erlang/Elixir
    [extra_applications: (Mix.env == :dev && [:exsync] || []) ++ [:envy, :logger, :maru, :timex, :ecto, :postgrex,
                                                                  :timex_ecto, :kafka_ex],
     mod: {Geronimo, []}]

  end

Any ideas?

Thanks in advance!