Hex App not included in `escript.build`

[maennchen]

Elixir and Erlang/OTP versions

Erlang/OTP 28 [erts-16.1.1] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [jit:ns]

Elixir 1.19.2 (compiled with Erlang/OTP 28)

Operating system

NixOS 25.05 (Warbler)

Current behavior

When adding :hex to mix.exs / application / included_applications, it is not added to any escripts built with mix escript.build.

Expected behavior

Hex is included.

Context

This is important for Build Tooling like SBoM generation since those require Hex to be present.

Detailed reproduction: https://github.com/maennchen/escript_issue_reproduction

[maennchen]

I planned to send a PR with the following changes:

diff --git a/lib/mix/lib/mix/tasks/escript.build.ex b/lib/mix/lib/mix/tasks/escript.build.ex
index 991f496ba..60a5593bb 100644
--- a/lib/mix/lib/mix/tasks/escript.build.ex
+++ b/lib/mix/lib/mix/tasks/escript.build.ex
@@ -287,7 +287,7 @@ defp extra_apps_in_app_tree(app) do
         applications =
           Keyword.get(spec, :applications, []) ++ Keyword.get(spec, :included_applications, [])
 
-        Enum.flat_map(applications, &extra_apps_in_app_tree/1)
+        applications ++ Enum.flat_map(applications, &extra_apps_in_app_tree/1)
     end
   end
 
diff --git a/lib/mix/test/fixtures/escript_test/lib/escript_test.ex b/lib/mix/test/fixtures/escript_test/lib/escript_test.ex
index c5b6b4efe..6207ba06e 100644
--- a/lib/mix/test/fixtures/escript_test/lib/escript_test.ex
+++ b/lib/mix/test/fixtures/escript_test/lib/escript_test.ex
@@ -22,6 +22,12 @@ def main(["--app-paths"]) do
     })
   end
 
+  def main(["--loaded-apps"]) do
+    Application.loaded_applications()
+    |> Enum.map(fn {app, _, _} -> app end)
+    |> IO.inspect()
+  end
+
   def main(["--list-priv", app_name]) do
     # Note: :erl_prim_loader usage with Escript is currently deprecated,
     # but we use it only in tests for convenience
diff --git a/lib/mix/test/mix/tasks/escript_test.exs b/lib/mix/test/mix/tasks/escript_test.exs
index ce363baf9..7d0e74355 100644
--- a/lib/mix/test/mix/tasks/escript_test.exs
+++ b/lib/mix/test/mix/tasks/escript_test.exs
@@ -15,6 +15,10 @@ def project do
         escript: [main_module: EscriptTest, name: "escript_test", embed_elixir: true]
       ]
     end
+
+    def application do
+      [included_applications: [:xmerl]]
+    end
   end
 
   defmodule EscriptErlangWithDeps do
@@ -51,6 +55,10 @@ test "generate escript" do
       # Does not include priv by default
       assert System.cmd("escript", ["escript_test", "--list-priv", "escript_test"]) ==
                {":error\n", 0}
+
+      # Adds Included Application to loadpath
+      assert {out, 0} = System.cmd("escript", ["escript_test", "--loaded-apps"])
+      assert out =~ "xmerl"
     end)
   end

But that seems to break the priv tests:

  2) test generate escript with priv (Mix.Tasks.EscriptTest)
     lib/mix/test/mix/tasks/escript_test.exs:251
     Assertion with == failed
     code:  assert System.cmd("escript", ["escript_test", "--list-priv", "ok"]) == {~s/{:ok, [~c"sample"]}\n/, 0}
     left:  {"{:ok, [~c\"sample\", ~c\"sample\"]}\n", 0}
     right: {"{:ok, [~c\"sample\"]}\n", 0}
     stacktrace:
       (elixir 1.20.0-dev) lib/file.ex:2013: File.cd!/2
       lib/mix/test/test_helper.exs:174: MixTest.Case.in_fixture/3
       lib/mix/test/mix/tasks/escript_test.exs:252: (test)

[josevalim]

Btw, is there a reason why you are using included_applications in particular?

[josevalim]

I don't think the issue is about included_applications. The issue is that hex comes from an archive and then its source is not being included. This is something very particular to Hex. I think simply adding hex to the list of special cases should be enough to address it.

[josevalim]

Please give #14953 a try!

[maennchen]

Btw, is there a reason why you are using included_applications in particular?

Yes, otherwise hex might choke if mix is not already running. For example this happens with Burrito if I move hex to extra_applications:

11:08:44.071 [notice] Application hex exited: exited in: Hex.Application.start(:normal, [])
    ** (EXIT) an exception was raised:
        ** (ArgumentError) errors were found at the given arguments:

  * 1st argument: the table identifier does not refer to an existing ETS table

            (stdlib 7.1) :ets.lookup_element(Mix.State, :scm, 2)
            (mix 1.19.2) lib/mix/state.ex:42: Mix.State.update/2
            (hex 2.3.1) lib/hex/application.ex:9: Hex.Application.start/2
            (kernel 10.4.1) application_master.erl:299: :application_master.start_it_old/4
Kernel pid terminated (application_controller) ("{application_start_failure,hex,{bad_return,{{'Elixir.Hex.Application',start,[normal,[]]},{'EXIT',{badarg,[{ets,lookup_element,['Elixir.Mix.State',scm,2],[{error_info,#{cause => id,module => erl_stdlib_errors}}]},{'Elixir.Mix.State',update,2,[{file,\"lib/mix/state.ex\"},{line,42}]},{'Elixir.Hex.Application',start,2,[{file,\"lib/hex/application.ex\"},{line,9}]},{application_master,start_it_old,4,[{file,\"application_master.erl\"},{line,299}]}]}}}}}")

So instead, i only include it and call Mix.Hex.start() in my application module.

Please give #14953 a try!

That works, however I'm wondering if that is the right strategy. Theoretically this could also be an issue with other mix archives. Should we not include any archive if its name is listed in the mix project / application/0?
Especially if a project also specifies project/0 / archives.

[josevalim]

That works, however I'm wondering if that is the right strategy.

I don't think we should have special provisions for archives because all of them, except for hex, can be a dependency, and we should prefer explicit dependencies.

I can add a more explicit error message if this is an issue in the future but right now, except for Hex, they are not included.