mix deps.get crashes on 26.2.3 - M2 Mac

[Hermanverschooten]

Describe the bug
Trying to do a mix deps.get on a Mac M2 with 26.2.3 causes a crash dump.

To Reproduce

• Install Erlang 26.2.3
• Install Elixir 1.16.1-otp-26
• mix deps.get

Expected behavior
No error and deps downloaded

Affected versions
26.2.3

[Hermanverschooten]

https://elixirforum.com/t/crash-dump-when-installing-phoenix-on-mac-m2-eheap-alloc-cannot-allocate-x-bytes-of-memory-of-type-heap-frag/62154/8

Resolving Hex dependencies...
eheap_alloc: Cannot allocate 976733209832459912 bytes of memory (of type "heap_frag").

Crash dump is being written to: erl_crash.dump...beam/erl_term.h:1492:tag_val_def() Assertion failed: tag_val_def error
[1]    13119 abort      mix deps.get

If you want I can supply the erl_crash.dump

[garazdawi]

Does the fault happen in 26.2.2?

[mkuratczyk]

No. I also just run into this - 26.2.2 is fine, 26.2.3 is not

[Hermanverschooten]

No 26.2.2 works without any issues Verstuurd vanaf mijn iPhoneOp 8 mrt 2024 om 14:19 heeft Michal Kuratczyk ***@***.***> het volgende geschreven: No. I also just run into this - 26.2.2 is fine, 26.2.3 is not —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: ***@***.***>

[josevalim]

I will try to reproduce it locally and, if so, provide a minimal example.

[josevalim]

Here is how far I could isolate it so this far:

$ mix local.hex
$ elixir -e "Mix.Local.append_archives(); Mix.start; Hex.start; Hex.RemoteConverger.converge([], %{})"

Note Hex is installed from an archive. I cannot reproduce this if I compile Hex from scratch.

So my theory is either:

1. a file compiled with an earlier OTP 26 version does not work on OTP 26.2.3
2. there is something wrong with the precompiled Hex file (maybe some chunks missing, we did this mistake in the past)

[josevalim]

Here you go. Compile this Erlang file with Erlang/OTP 24.3:

-module(foo).
-compile(export_all).

example(X) ->
  Total = X / 1000,
  io:format("~p~n", [<<"Example:", (erlang:float_to_binary(Total))/binary>>]),
  ok.

And now run it with Erlang/OTP 26.2.3:

$ erl -eval "foo:example(100)." -s erlang halt
Erlang/OTP 26 [erts-14.2.3] [source] [64-bit] [smp:10:10] [ds:10:10:10] [async-threads:1] [jit]

eheap_alloc: Cannot allocate 434041037028460176 bytes of memory (of type "heap_frag").

Crash dump is being written to: erl_crash.dump...Segmentation fault: 11

I believe 24.3 should still work on 26.2.3. So that's indeed a bug!

[nickkaltner]

Same problem on a raspberry pi 5

[josevalim]

Btw, a temporary fix is to run mix archive.install github hexpm/hex branch latest before you run mix deps.get. This will force latest Hex to be compiled with your current OTP version.

[jimsynz]

I am seeing the same issue on my M1 Mac and Linux on arm64, but also more:

Running mix works:

# mix
** (Mix) "mix" with no arguments must be executed in a directory with a mix.exs file

Usage: mix [task]

Examples:

    mix             - Invokes the default task (mix run) in a project
    mix new PATH    - Creates a new Elixir project at the given path
    mix help        - Lists all available tasks
    mix help TASK   - Prints documentation for a given task

The --help and --version options can be given instead of a task for usage and versioning information.

Weirdly, mix deps.get on an existing Elixir project just seems to deadlock and never seems to exit.

But running pretty much anything else fails with a segfault:

# elixir --version
Erlang/OTP 26 [erts-14.2.3] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [jit]

zsh: segmentation fault (core dumped)  elixir --version

Additionally, @josevalim's suggested fix of recompiling hex also results in a segfault:

# mix archive.install github hexpm/hex branch latest
zsh: segmentation fault (core dumped)  mix archive.install github hexpm/hex branch latest

(The above commands were all executed inside a clean Ubuntu container with a fresh install of ASDF, Erlang 26.2.3 and Elixir 1.16.2).

[conradwt]

@jimsynz After running @josevalim temporary fix, I'm unable to reproduce any of the issues that you're experiencing with my M1 MacBook Pro. For example,

➜ asdf version                    
v0.14.0-ccdd47d

➜ cat .tool-versions
elixir 1.16.2-otp-26
erlang 26.2.3

➜ mix archive.install github hexpm/hex branch latest
* Getting new package (https://github.com/hexpm/hex.git - origin/latest)
remote: Enumerating objects: 13790, done.        
remote: Counting objects: 100% (1245/1245), done.        
remote: Compressing objects: 100% (605/605), done.        
remote: Total 13790 (delta 755), reused 1003 (delta 583), pack-reused 12545        
All dependencies are up to date
warning: in order to compile .xrl files, you must add "compilers: [:leex] ++ Mix.compilers()" to the "def project" section of hex's mix.exs
  (mix 1.16.2) lib/mix/tasks/compile.leex.ex:69: Mix.Tasks.Compile.Leex.preload/1
  (mix 1.16.2) lib/mix/compilers/erlang.ex:66: Mix.Compilers.Erlang.compile/6
  (mix 1.16.2) lib/mix/task.ex:478: anonymous fn/3 in Mix.Task.run_task/5
  (mix 1.16.2) lib/mix/tasks/compile.all.ex:124: Mix.Tasks.Compile.All.run_compiler/2
  (mix 1.16.2) lib/mix/tasks/compile.all.ex:104: Mix.Tasks.Compile.All.compile/4
  (mix 1.16.2) lib/mix/tasks/compile.all.ex:93: Mix.Tasks.Compile.All.with_logger_app/2

Compiling 1 file (.xrl)
Compiling 14 files (.erl)
Compiling 81 files (.ex)
Generated hex app
Generated archive "hex-2.0.6.ez" with MIX_ENV=prod
Found existing entry: /Users/conradwt/.mix/archives/hex-2.0.6
Are you sure you want to replace it with "hex-2.0.6.ez"? [Yn] y
* creating /Users/<username>/.mix/archives/hex-2.0.6

➜ elixir --version
Erlang/OTP 26 [erts-14.2.3] [source] [64-bit] [smp:10:10] [ds:10:10:10] [async-threads:1] [jit]

Elixir 1.16.2 (compiled with Erlang/OTP 26)

➜ mix deps.get                                      
Resolving Hex dependencies...
Resolution completed in 0.037s
Unchanged:
  bandit 1.0.0
  bunt 1.0.0
  castore 1.0.5
  credo 1.7.5
  expo 0.4.1
  file_system 1.0.0
  gettext 0.22.3
  hpax 0.1.2
  jason 1.4.1
  mime 2.0.5
  phoenix 1.7.11
  phoenix_html 4.1.1
  phoenix_live_dashboard 0.8.3
  phoenix_live_view 0.20.12
  phoenix_pubsub 2.1.3
  phoenix_template 1.0.4
  plug 1.15.3
  plug_crypto 2.0.0
  telemetry 1.2.1
  telemetry_metrics 0.6.2
  telemetry_poller 1.0.0
  thousand_island 1.3.5
  websock 0.5.3
  websock_adapter 0.5.5
All dependencies are up to date

The only difference that I can see from your examples is that you're using an Ubuntu container versus macOS here. However, I was able to rebuild an app's Docker container image with the temporary Hex fix and execute without any issues. For example,

➜ docker compose ps                                          
NAME                     IMAGE                  COMMAND             SERVICE   CREATED          STATUS          PORTS
hasher-elixir-hasher-1   hasher-elixir-hasher   "/app/bin/server"   hasher    26 minutes ago   Up 26 minutes   0.0.0.0:8002->80/tcp

➜ curl http://localhost:8002        
HASHER running on fec5cc18a710

➜ curl -X POST http://localhost:8002
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

[jimsynz]

@conradwt so weird. Here's the output on my M1 Max MacBook Pro (macOS Sonoma 14.3.1 build 23D60).

> asdf version
v0.14.0-ccdd47d

> cat .tool-versions
erlang 26.2.3
elixir 1.16.2

>  mix archive.install github hexpm/hex branch latest
# hangs if in an existing mix project directory, otherwise:
zsh: segmentation fault  mix archive.install github hexpm/hex branch latest

> mix deps.get
# hangs if in an existing mix project directory, otherwise:
zsh: segmentation fault  mix deps.get

Supplementary: I noticed that you're using elixir-1.16.2 otp-26 whereas I'm using elixir 1.16.2 - here's the same thing repeated with that change made:

> cat .tool-versions
erlang 26.2.3
elixir 1.16.2-otp-26

>  mix archive.install github hexpm/hex branch latest
* Getting new package (https://github.com/hexpm/hex.git - origin/latest)
remote: Enumerating objects: 13790, done.
remote: Counting objects: 100% (1245/1245), done.
remote: Compressing objects: 100% (602/602), done.
remote: Total 13790 (delta 755), reused 1008 (delta 586), pack-reused 12545
All dependencies are up to date
warning: in order to compile .xrl files, you must add "compilers: [:leex] ++ Mix.compilers()" to the "def project" section of hex's mix.exs
  (mix 1.16.2) lib/mix/tasks/compile.leex.ex:69: Mix.Tasks.Compile.Leex.preload/1
  (mix 1.16.2) lib/mix/compilers/erlang.ex:66: Mix.Compilers.Erlang.compile/6
  (mix 1.16.2) lib/mix/task.ex:478: anonymous fn/3 in Mix.Task.run_task/5
  (mix 1.16.2) lib/mix/tasks/compile.all.ex:124: Mix.Tasks.Compile.All.run_compiler/2
  (mix 1.16.2) lib/mix/tasks/compile.all.ex:104: Mix.Tasks.Compile.All.compile/4
  (mix 1.16.2) lib/mix/tasks/compile.all.ex:93: Mix.Tasks.Compile.All.with_logger_app/2

Compiling 1 file (.xrl)
Compiling 14 files (.erl)
Compiling 81 files (.ex)
Generated hex app
Generated archive "hex-2.0.6.ez" with MIX_ENV=prod
Found existing entry: /Users/jmshrtn/.asdf/installs/elixir/1.16.2-otp-26/.mix/archives/hex-2.0.6
Are you sure you want to replace it with "hex-2.0.6.ez"? [Yn]
* creating /Users/jmshrtn/.asdf/installs/elixir/1.16.2-otp-26/.mix/archives/hex-2.0.6

> mix deps.get
Resolving Hex dependencies...
Resolution completed in 0.022s
Unchanged:
  bunt 1.0.0
  credo 1.7.4
  decimal 2.1.1
  dialyxir 1.4.3
  doctor 0.21.0
  earmark 1.4.46
  earmark_parser 1.4.39
  erlex 0.2.6
  ex_doc 0.31.1
  file_system 1.0.0
  git_cli 0.3.0
  git_ops 2.6.0
  jason 1.4.1
  makeup 1.1.1
  makeup_elixir 0.16.1
  makeup_erlang 0.1.4
  nimble_parsec 1.4.0
All dependencies are up to date

So it looks like @josevalim is correct (as expected TBH) that it's an issue with compatibility with artefacts from older versions of OTP.

[conradwt]

@jimsynz Yes, I typically use an Elixir release that's compatible with the underlying OTP version.

[Cellesti]

I have also noticed this issue while running mix local.hex --force on Alpine Linux's aarch64 CI.

Alpine's edge development branch has Erlang 26.2.3 and Elixir 1.16.2 in repository for 8 architectures, and it seems aarch64 is the only one having problems with the precompiled Hex.

[bjorng]

The linked pull request fixes the bug.

[jdmarshall]

I am also seeing the same thing on an M2 with

elixir 1.16.1-otp-26
erlang 26.2.3

What's the workaround? downgrade to 26.2.2?

[josevalim]

@jdmarshall #8238 (comment)