Cannot get conform to apply settings from app.conf at app start

[localshred]

I'm just going to say ahead of time this is my first Elixir/Erlang deploy (with or without exrm/conform) so be kind. I'm also assuming this is my own error and not a bug. I'm just not sure of a better way to get help/feedback. I appreciate your time.

I am using exrm 1.0.0-rc7 and conform 1.0.0-rc8.

When I build on the same machine I want to deploy on, the conf file that has the correct values for our production env are not being applied correctly. Specifically, I have a list of kafka brokers defined as a [list: :ip] data type in the schema. The conf file has this list of broker ips defined, but app start fails.

From the crash dump:

$ head erl_crash.dump
=erl_crash_dump:0.3
Wed Jan 13 22:41:57 2016
Slogan: Kernel pid terminated (application_controller) ({application_start_failure,kafka_ex,{{function_clause,[{prim_inet,setopts,[nil,[binary,{packet,4},{active,false}]],[]},{'Elixir.KafkaEx.NetworkClient'
System version: Erlang/OTP 18 [erts-7.2] [source] [64-bit] [smp:2:2] [async-threads:10] [kernel-poll:false]

When I enter the console I get a KafkaNetwork exception as it's trying to connect to a local Kafka instance (the default value in the schema).

$ gretel console
Using /app/gretel/current/releases/0.0.1/gretel.sh
Exec: /app/gretel/current/erts-7.2/bin/erlexec -boot /app/gretel/current/releases/0.0.1/gretel -boot_var ERTS_LIB_DIR /app/gretel/current/erts-7.2/../lib -env ERL_LIBS /app/gretel/current/lib -config /app/gretel/current/running-config/sys.config -args_file /app/gretel/current/running-config/vm.args -user Elixir.IEx.CLI -extra --no-halt +iex -- console
Root: /app/gretel/current
/app/gretel/current
Erlang/OTP 18 [erts-7.2] [source] [64-bit] [smp:2:2] [async-threads:10] [kernel-poll:false]


22:50:26.996 [info]  Child Logger.ErrorHandler of Supervisor Logger.Supervisor started
Pid: #PID<0.68.0>
Start Call: Logger.Watcher.watcher(:error_logger, Logger.ErrorHandler, {true, true, 500}, :link)
Restart: :permanent
Shutdown: 5000
Type: :worker

22:50:27.012 [info]  Application logger started at :"gretel@127.0.0.1"

22:50:27.014 [info]  Application poison started at :"gretel@127.0.0.1"

22:50:27.014 [error] Could not connect to broker "127.0.0.1" on port 9092
{"Kernel pid terminated",application_controller,"{application_start_failure,kafka_ex,{{function_clause,[{prim_inet,setopts,[nil,[binary,{packet,4},{active,false}]],[]},{'Elixir.KafkaEx.NetworkClient',send_sync_request,3,[{file,\"lib/kafka_ex/network_client.ex\"},{line,30}]},{'Elixir.Enum',do_find_value,3,[{file,\"lib/enum.ex\"},{line,2490}]},{'Elixir.KafkaEx.Server',metadata,5,[{file,\"lib/kafka_ex/server.ex\"},{line,249}]},{'Elixir.KafkaEx.Server',init,1,[{file,\"lib/kafka_ex/server.ex\"},{line,26}]},{gen_server,init_it,6,[{file,\"gen_server.erl\"},{line,328}]},{proc_lib,init_p_do_apply,3,[{file,\"proc_lib.erl\"},{line,240}]}]},{'Elixir.KafkaEx',start,[normal,[]]}}}"}

Crash dump is being written to: erl_crash.dump...done
Kernel pid terminated (application_controller) ({application_start_failure,kafka_ex,{{function_clause,[{prim_inet,setopts,[nil,[binary,{packet,4},{active,false}]],[]},{'Elixir.KafkaEx.NetworkClient'

# schema file
# ...
    "kafka_ex.brokers": [
      doc: "Provide documentation for kafka_ex.brokers here.",
      to: "kafka_ex.brokers",
      datatype: [ list: :ip ],
      default: [{"127.0.0.1", 9092}]
    ],
# ...

# gretel.conf
kafka_ex.brokers = some.ip.range:9092, some.ip.range:9092, some.ip.range:9092

I have placed my gretel.conf in the release version directory /app/gretel/current/releases/0.0.1/gretel.conf. With previous versions of the hex package, the gretel.conf was copied to /app/gretel/current/running-config/gretel.conf, but this current version of conform doesn't do that.

At this point I'm unsure where to go. As indicated I've tried rolling back to the last stable releases of both exrm and conform, but that ran me into a separate set of issues. Specifically, running escript with the built conform file was failing with a message saying that conform_escript function was undefined or something.

I'm sorry if that's too much info, but trying to do my homework here and not submit a bogus issue. It appears that I'm missing a simple step, but I'll be damned if I can find it. Any help would be greatly appreciated. Thanks!

[bitwalker]

Could you describe your deployment workflow for me, just so we're on the same page?

Conform itself isn't responsible for copying the config, that's actually up to the boot script in exrm. However the config is copied from the release directory to running-config during upgrade, downgrade, console, foreground, and start, so if you are copying the config manually after one of those commands are executed, then that would explain why the config used is the old one.

Once I know more about your workflow, I'll try and reproduce locally.

[localshred]

After reading some more code I realized that I was constraining conform and exrm and conform_exrm to dev/test only but that doesn't work when trying to build a prod release, so I removed those restrictions. The conf file is now being copied over correctly and the conform script is now being built into my release. I am now back to the issue I encountered with the old release, namely when the app.sh script tries to generate the config it is failing on this line:

# ...
result="$("$BINDIR/escript" "$__conform" --conf "$__conform_file" --schema "$__schema_file" --config "$SYS_CONFIG" --output-dir "$GENERATED_CONFIG_DIR")"
# ...

Which, expanded out is:

$ /app/gretel/current/erts-7.2/bin/escript /app/gretel/current/releases/0.0.1/conform --conf /app/gretel/current/running-config/gretel.conf --schema /app/gretel/current/releases/0.0.1/gretel.schema.exs --config /app/gretel/current/running-config/sys.config --output-dir /app/gretel/current/running-config

When I run that command I get the following error that I was getting before:

$ /app/gretel/current/erts-7.2/bin/escript /app/gretel/current/releases/0.0.1/conform --conf /app/gretel/current/running-config/gretel.conf --schema /app/gretel/current/releases/0.0.1/gretel.schema.exs --config /app/gretel/current/running-config/sys.config --output-dir /app/gretel/current/running-config
escript: exception error: undefined function conform_escript:main/1
  in function  escript:run/2 (escript.erl, line 757)
  in call from escript:start/1 (escript.erl, line 277)
  in call from init:start_it/1
  in call from init:start_em/1

Hoping that provides more insight.

[bitwalker]

That's a new one, does the conform binary actually exist at /app/gretel/current/releases/0.0.1/conform?

[localshred]

Sure, my current deploy process is to build and run on the same machine (for now). The config/gretel.conf file is already on this machine and in the config directory. /app/gretel/current/bin is on the $PATH as well.

1. Pull latest commits on build server
2. MIX_ENV=prod mix do compile, release
3. mkdir /app/gretel/releases/{version} && cd /app/gretel/releases/{version}
4. tar -xvf {path to gretel code}/rel/gretel/releases/{version}/gretel.tar.gz
5. rm -f /app/gretel/current && ln -sF /app/gretel/releases/{version}/ /app/gretel/current
6. gretel console

[localshred]

The conform binary is only present when all three deps are available for the env I'm building for.

[bitwalker]

That's expected, let me review your process and see if I can reproduce, should just be a few minutes.

[localshred]

Ok thx!

[localshred]

My app deps, if it matters:

  defp deps do
    [
      { :conform, "~> 1.0.0-rc8", override: true },
      { :conform_exrm, "~> 0.2.0" },
      { :dialyxir, "~> 0.2", only: ~w(dev test)a },
      { :dogma, "~> 0.0", only: ~w(dev test)a },
      { :exrm, "~> 1.0.0-rc7", override: true },
      { :insert_ordered_set, "~> 0.0.1" },
      { :kafka_ex, "0.3.0" },
      { :mongodb, "0.1.1" },
      { :poison, "~> 1.5" },
      { :poolboy, "~> 1.5.1" },
      { :timex, "~> 1.0.0-rc4" },
    ]
  end

[bitwalker]

Very strange, your process works for me locally (I have a little test app which I dumped your config in):

⟩ /tmp/test/current/bin/test console
Using /tmp/test/current/releases/0.0.3/test.sh
copying /tmp/test/current/releases/0.0.3/test.conf to /tmp/test/current/running-config/test.conf ...
using /tmp/test/current/running-config/test.conf to populate "/tmp/test/current/running-config".
Exec: /tmp/test/current/erts-7.2.1/bin/erlexec -boot /tmp/test/current/releases/0.0.3/test -boot_var ERTS_LIB_DIR /tmp/test/current/erts-7.2.1/../lib -env ERL_LIBS /tmp/test/current/lib -config /tmp/test/current/running-config/sys.config -args_file /tmp/test/current/running-config/vm.args -conform_schema /tmp/test/current/releases/0.0.3/test.schema.exs -conform_config /tmp/test/current/releases/0.0.3/test.conf -running_conf /tmp/test/current/running-config/test.conf -user Elixir.IEx.CLI -extra --no-halt +iex -- console
Root: /tmp/test/current
/tmp/test/current
Erlang/OTP 18 [erts-7.2.1] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]

Interactive Elixir (1.2.0) - press Ctrl+C to exit (type h() ENTER for help)
iex(test@127.0.0.1)1> Application.get_all_env(:test)
[env: :prod, another_val: "none",
 brokers: [{"192.168.99.101", "9092"}, {"192.168.99.104", "9092"},
  {"192.168.99.107", "9092"}], debug_level: :info, included_applications: []]
iex(test@127.0.0.1)2>
BREAK: (a)bort (c)ontinue (p)roc info (i)nfo (l)oaded
       (v)ersion (k)ill (D)b-tables (d)istribution
a

Only difference being that I changed kafka_ex to test in the schema and conf. What happens when you run the following?

$ /app/gretel/current/erts-7.2/bin/escript /app/gretel/current/releases/0.0.1/conform

You should see the output below if it's working properly:

Conform - Translate the provided .conf file to a .config file using the given schema
-------
usage: conform --conf foo.conf --schema foo.schema.exs [options]

Options:
  --filename <name>:    Names the output file <name>.config
  --output-dir <path>:  Outputs the .config file to <path>/<sys|name>.config
  --config <config>:    Merges the translated configuration over the top of
                        <config> before output
  -h | --help:          Prints this help

What OS, Erlang and Elixir versions are you running on this machine?

[localshred]

Same result, undefined function:

$ /app/gretel/current/erts-7.2/bin/escript /app/gretel/current/releases/0.0.1/conform
escript: exception error: undefined function conform_escript:main/1
  in function  escript:run/2 (escript.erl, line 757)
  in call from escript:start/1 (escript.erl, line 277)
  in call from init:start_it/1
  in call from init:start_em/1

$ erl
Erlang/OTP 18 [erts-7.2] [source] [64-bit] [smp:2:2] [async-threads:10] [kernel-poll:false]

$ elixir -v
Erlang/OTP 18 [erts-7.2] [source] [64-bit] [smp:2:2] [async-threads:10] [kernel-poll:false]

Elixir 1.2.0

$ uname -a
Linux gretel-stage-nuvis01 3.13.0-48-generic #80-Ubuntu SMP Thu Mar 12 11:16:15 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux