relx relup crashes, deletes previous release libraries

[blt]

I have a an application that I'm trying to introduce relx into. Here's the pertinent portion of my relx.config (I've extracted the overlays):

{release,
 {fire_engine, "2013.12"},
 [
  fe_db
  , fe_tftp
  , fire_engine
  , fe_api
 ]
}.

{extended_start_script, true}.
{generate_start_script, false}.

Each sub-application has a .app defined in apps/APP_NAME/ebin/ but is otherwise typical for a rebar-driven project. I can generate a release well enough:

> relx
Starting relx build process ...
Resolving OTP Applications from directories:
    /Users/blt/projects/com/rackspace/fire_engine/apps
    /Users/blt/projects/com/rackspace/fire_engine/deps
    /Users/blt/.kerl/installs/R15B03/lib

Resolving available releases from directories:
    /Users/blt/projects/com/rackspace/fire_engine/apps
    /Users/blt/projects/com/rackspace/fire_engine/deps
    /Users/blt/.kerl/installs/R15B03/lib

Resolved fire_engine-2013.12
release successfully created!

I find that _rel/lib contains all the libraries it should:

> ls _rel/lib/                  
asn1-1.8           fire_engine-2013.1 kernel-2.15.3      ssh-2.1.2
bson-0             folsom-0.7.3       lager-1.2.0        ssl-5.1.1
compiler-4.8.2     gen_batch-1.0.2    mochiweb-1.5.1p3   stdlib-1.18.3
crypto-2.2         gen_script-1.0     mongodb-0.2.1      syntax_tools-1.6.9
eldap-1.0          ibrowse-3.0.4      public_key-0.17    webmachine-1.9.3
fe_api-2013.1      inets-5.9.2        rax_snmp-1.1.16
fe_db-2013.1       iron-0.0.1         sasl-2.2.1
fe_tftp-2013.1     jiffy-0.5.2        snmp-4.22.1

and that _rel/releases looks sensible:

> ls _rel/releases 
fire_engine-2013.12

After bumping the version in the fire_engine sub-application's app file to 2013.2, modifying relx.config to have a release target of 2013.13 and creating the following appup:

> cat apps/fire_engine/ebin/fire_engine.appup 
{"2013.2",
 [{"2013.1", [{load_module, fire_engine}]}],
 [{"2013.1", [{load_module, fire_engine}]}]
}.

I find that creating a relup is not so successful:

> relx relup
Starting relx build process ...
Resolving OTP Applications from directories:
    /Users/blt/projects/com/rackspace/fire_engine/apps
    /Users/blt/projects/com/rackspace/fire_engine/deps
    /Users/blt/.kerl/installs/R15B03/lib
    /Users/blt/projects/com/rackspace/fire_engine/_rel

Resolving available releases from directories:
    /Users/blt/projects/com/rackspace/fire_engine/apps
    /Users/blt/projects/com/rackspace/fire_engine/deps
    /Users/blt/.kerl/installs/R15B03/lib
    /Users/blt/projects/com/rackspace/fire_engine/_rel

Resolved fire_engine-2013.13
Errors generating release 
    fire_engine: File not found: "fire_engine.app"


Usage: relx [-n <relname>] [-v <relvsn>] [-g <goal>] [-u <upfrom>]
            [-o <output_dir>] [-l <lib_dir>]
            [--disable-default-libs [<disable_default_libs>]]
            [-V [<log_level>]] [-a <override_app>] [-c [<config>]]
            [-r <root_dir>] [*release-specification-file*]

  -n, --relname           Specify the name for the release that will be 
                          generated
  -v, --relvsn            Specify the version for the release
  -g, --goal              Specify a target constraint on the system. These 
                          are usually the OTP
  -u, --upfrom            Only valid with relup target, specify the 
                          release to upgrade from
  -o, --output-dir        The output directory for the release. This is 
                          `./` by default.
  -l, --lib-dir           Additional dirs that should be searched for OTP 
                          Apps
  --disable-default-libs  Disable the default system added lib dirs (means 
                          you must add them all manually [default: false]
  -V, --verbose           Verbosity level, maybe between 0 and 2 [default: 
                          1]
  -a, --override_app      Provide an app name and a directory to override 
                          in the form <appname>:<app directory>
  -c, --config            The path to a config file [default: ]
  -r, --root              The project root directory

The relup was not generated, of course. Of particular concern is that _rel/lib/fire_engine-2013.1 has been deleted.

> ls _rel/lib  
asn1-1.8           fe_tftp-2013.1     jiffy-0.5.2        sasl-2.2.1
bson-0             folsom-0.7.3       kernel-2.15.3      snmp-4.22.1
compiler-4.8.2     gen_batch-1.0.2    lager-1.2.0        ssh-2.1.2
crypto-2.2         gen_script-1.0     mochiweb-1.5.1p3   ssl-5.1.1
eldap-1.0          ibrowse-3.0.4      mongodb-0.2.1      stdlib-1.18.3
fe_api-2013.1      inets-5.9.2        public_key-0.17    syntax_tools-1.6.9
fe_db-2013.1       iron-0.0.1         rax_snmp-1.1.16    webmachine-1.9.3

It seems that only the fire_engine sub-application was lost.

Problem summary:

• creating a relup failed unexpectedly and
• relup creation destroyed library required by a previously generated release.

I'd be happy to share any more information that might be useful. The project is proprietary but, if desired, I could try to make a minimal reproducing project and open-source that.

[blt]

I've replicated the issue in a simple, open project: beat. Reproduction details are in the README.

[tsloughter]

@blt can I close this now?

[blt]

Absolutely. Thank you again for your help.