Slim release support

[shino]

PR for #7 .

By adding {excl_lib, otp_root} to reltool.config,
generate a slim release which does not include erts and otp libraries.
Also the runner script uses appropriate options for slim release.

Example:

$ mkdir slim_sample
$ cd slim_sample
$ cp /path/to/rebar/rebar .
$ ./rebar create-app appid=slim_sample
$ ./rebar compile

$ rm -rf rel
$ mkdir rel
$ cd rel
$ ../rebar create-node nodeid=slim_sample
$ cp reltool.config reltool.config.original

$ vi reltool.config  # edit
$ diff -u reltool.config.original reltool.config
--- reltool.config.original     2013-01-13 22:23:31.597614577 +0900
+++ reltool.config      2013-01-13 22:23:31.601613550 +0900
@@ -1,5 +1,6 @@
 {sys, [
        {lib_dirs, []},
+       {excl_lib, otp_root},
        {erts, [{mod_cond, derived}, {app_file, strip}]},
        {app_file, strip},
        {rel, "slim_sample", "1",
@@ -22,14 +23,14 @@
        {excl_sys_filters, ["^bin/.*", "^erts.*/bin/(dialyzer|typer)",
                            "^erts.*/(doc|info|include|lib|man|src)"]},
        {excl_app_filters, ["\.gitignore"]},
-       {app, slim_sample, [{mod_cond, app}, {incl_cond, include}]}
+       {app, hipe, [{incl_cond, exclude}]},
+       {app, slim_sample, [{mod_cond, app}, {incl_cond, include}, {lib_dir, ".."}]}
       ]}.

 {target_dir, "slim_sample"}.

 {overlay, [
            {mkdir, "log/sasl"},
-           {copy, "files/erl", "\{\{erts_vsn\}\}/bin/erl"},
            {copy, "files/nodetool", "releases/\{\{rel_vsn\}\}/nodetool"},
            {copy, "files/slim_sample", "bin/slim_sample"},
            {copy, "files/slim_sample.cmd", "bin/slim_sample.cmd"},

$ ../rebar generate

• The erts_vsn directory is not created in slim release as it should be.
  Therefore nodetool script is moved to under releases/$APP_VSN/ (as template)
  and erl script should be removed from overlay (by editing).
• In slim release, rebar generate creates a marker file
  $RUNNER_BASE_DIR/releases/$APP_VSN/runner_script.data.
  runner scripts know whether slim release or not(=normal) by its content.
• Exclusion of hipe application is not related to slim release support,
  just because my Erlang/OTP is built without hipe.

[ghost]

@joewilliams?

[joewilliams]

Have hot upgrade been tested against a slim release?

[shino]

Not yet. I will try soon.

[shino]

Oops, upgrade does not work against a slim release :-/ ,
I got an error as below.

Eshell V5.9.3.1  (abort with ^G)
(dummy@127.0.0.1)1> release_handler:unpack_release("dummy_2").
{error,{enoent,"/home/shino/tmp/dynamic_upgrade_slim/dummy/rel/dummy_1/releases/dummy_2.rel"}}

[ghost]

@shino, ping?

[choptastic]

Hey guys,

Has any progress been made in the slim releases?

[shino]

Rebased on master to avoid merge conflict.

Error in upgrade is caused by release_handler. It seems to be not yet slim release compatible.

• release_handler:extract_tar extracts releases/*.rel under root directory (code:get_root())
• release_handler:check_rel try to open extracted file under releases directory under slim release.

So this slim release support does not work with upgrade at present.
Can be merged to master? Or is some effort to Erlang/OTP needed before merge?

P.S.

Trace of release_handler calls by redbug (for curious people):

/opt/erlang/R15B03_1_zdss/ is Erlang root and /tmp/slim/dummy/rel/dummy_1/ is
reltool generated directory.

redbug:start("release_handler:'_' -> return", [{print_depth, 10}, {msgs, 1000}]).
ok
(dummy@127.0.0.1)18> release_handler:unpack_release("dummy_2").

(dummy@127.0.0.1)1> redbug:start("release_handler:'_' -> return", [{print_depth, 10}, {msgs, 1000}]).
ok
(dummy@127.0.0.1)2> release_handler:unpack_release("dummy_2").

22:52:53 <{erlang,apply,2}> {release_handler,unpack_release,["dummy_2"]}

22:52:53 <{erlang,apply,2}> {release_handler,call,
                                [{unpack_release,"dummy_2"}]}

22:52:53 <release_handler> {release_handler,handle_call,
                               [{unpack_release,"dummy_2"},
                                {<0.70.0>,#Ref<0.0.0.246>},
                                {state,[],[...],...}]}

22:52:53 <release_handler> {release_handler,do_unpack_release,
                               ["/opt/erlang/R15B03_1_zdss/bin/../lib/erlang",
                                "/tmp/slim/dummy/rel/dummy_1/releases",
                                "dummy_2",
                                [{release,...}]]}

22:52:53 <release_handler> {release_handler,do_check_file,
                               ["/tmp/slim/dummy/rel/dummy_1/releases/dummy_2.tar.gz",
                                regular]}

22:52:53 <release_handler> {release_handler,do_check_file,2} -> ok

22:52:53 <release_handler> {release_handler,extract_rel_file,
                               ["releases/dummy_2.rel",
                                "/tmp/slim/dummy/rel/dummy_1/releases/dummy_2.tar.gz",
                                "/opt/erlang/R15B03_1_zdss/bin/../lib/erlang"]}

22:52:53 <release_handler> {release_handler,extract_rel_file,3} -> ok
{error,{enoent,"/tmp/slim/dummy/rel/dummy_1/releases/dummy_2.rel"}}

22:52:53 <release_handler> {release_handler,check_rel,
                               ["/opt/erlang/R15B03_1_zdss/bin/../lib/erlang",
                                "/tmp/slim/dummy/rel/dummy_1/releases/dummy_2.rel",
                                false]}

22:52:53 <release_handler> {release_handler,check_rel,
                               ["/opt/erlang/R15B03_1_zdss/bin/../lib/erlang",
                                "/tmp/slim/dummy/rel/dummy_1/releases/dummy_2.rel",
                                [],false]}

22:52:53 <release_handler> {release_handler,consult,
                               ["/tmp/slim/dummy/rel/dummy_1/releases/dummy_2.rel",
                                false]}

22:52:53 <release_handler> {release_handler,consult,2} -> {error,enoent}

22:52:53 <release_handler> {release_handler,check_rel,4} -> {throw,
                                                             {error,
                                                              {enoent,
                                                               "/tmp/slim/dummy/rel/dummy_1/releases/dummy_2.rel"}}}

22:52:53 <release_handler> {release_handler,check_rel,3} -> {throw,
                                                             {error,
                                                              {enoent,
                                                               "/tmp/slim/dummy/rel/dummy_1/releases/dummy_2.rel"}}}

22:52:53 <release_handler> {release_handler,do_unpack_release,4} -> {throw,
                                                                     {error,
                                                                      {enoent,
                                                                       "/tmp/slim/dummy/rel/dummy_1/releases/dummy_2.rel"}}}

22:52:53 <release_handler> {release_handler,handle_call,3} -> {reply,
                                                               {error,
                                                                {enoent,
                                                                 "/tmp/slim/dummy/rel/dummy_1/releases/dummy_2.rel"}},
                                                               {state,[],
                                                                "/opt/erlang/R15B03_1_zdss/bin/../lib/erlang",
                                                                "/tmp/slim/dummy/rel/dummy_1/releases",
                                                                [{...}],
                                                                undefined,...}}

22:52:53 <{erlang,apply,2}> {release_handler,call,1} -> {error,
                                                         {enoent,
                                                          "/tmp/slim/dummy/rel/dummy_1/releases/dummy_2.rel"}}

22:52:53 <{erlang,apply,2}> {release_handler,unpack_release,1} -> {error,
                                                                   {enoent,
                                                                    "/tmp/slim/dummy/rel/dummy_1/releases/dummy_2.rel"}}

[shino]

Reported: http://erlang.org/pipermail/erlang-bugs/2013-November/003859.html

[shino]

Siri told me the workaround

• unpack relup archive manually, and
• use release_hander:set_unpacked/2

So now (very simple) upgrading can be done in slim release.
Steps are illustrated at https://github.com/shino/slim-upgrade-sample .

[shino]

Workaround to upgrade is found and it may be implemented by upgrade script.
That work is not done yet but filed as the issue above.
Also I added TODO comment to install_upgrade script at the commit 61982e3.

I hope it is now ready for review again :)

[ghost]

@jaredmorrow ping?

[jaredmorrow]

This looks fine to us. Can you rebase this so it merges cleanly and then I'll merge.

[shino]

Rebased on current master fa1a2b5 .

[runcom]

ping! can this be merged? (It would be good to add some lines in the wiki as well @shino )