Compiling problems with Elixir 1.8, OTP21 and OSX 10.14.2 #17

andrelip · 2019-01-28T19:23:33Z

Can't compile the dependency

System:

Erlang/OTP 21 [erts-10.2.1] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [hipe] [dtrace]

Elixir 1.8.0 (compiled with Erlang/OTP 21)

Output:

===> Compiling prometheus_process_collector
cc prometheus_process_collector_nif.o prometheus_process_info_macos.o -L/usr/local/opt/openssl/lib -shared -L /usr/local/Cellar/erlang/21.2.2/lib/erlang/lib/erl_interface-3.10.4/lib -lerl_interface -lei -lstdc++ -o /Users/andresouza/arvore/deps/prometheus_process_collector/c_src/../priv/prometheus_process_collector.so
Undefined symbols for architecture x86_64:
  "_enif_make_atom", referenced from:
      on_load(enif_environment_t*, void**, unsigned long) in prometheus_process_collector_nif.o
  "_enif_make_double", referenced from:
      Prometheus::get_process_info(enif_environment_t*, int, unsigned long const*) in prometheus_process_collector_nif.o
  "_enif_make_int", referenced from:
      Prometheus::get_process_info(enif_environment_t*, int, unsigned long const*) in prometheus_process_collector_nif.o
  "_enif_make_list_from_array", referenced from:
      Prometheus::get_process_info(enif_environment_t*, int, unsigned long const*) in prometheus_process_collector_nif.o
  "_enif_make_long", referenced from:
      Prometheus::get_process_info(enif_environment_t*, int, unsigned long const*) in prometheus_process_collector_nif.o
  "_enif_make_tuple", referenced from:
      Prometheus::get_process_info(enif_environment_t*, int, unsigned long const*) in prometheus_process_collector_nif.o
  "_enif_make_ulong", referenced from:
      Prometheus::get_process_info(enif_environment_t*, int, unsigned long const*) in prometheus_process_collector_nif.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [/Users/andresouza/arvore/deps/prometheus_process_collector/c_src/../priv/prometheus_process_collector.so] Error 1
===> Hook for compile failed!

The text was updated successfully, but these errors were encountered:

mberrueta · 2019-05-04T10:18:04Z

same thing here.

#define ATOM(Id, Value)                         \
    {                                                              \
        Id = enif_make_atom(env, Value);        \
    }

there is any way to include it?
the code is https://github.com/erlang/otp/blob/master/erts/emulator/beam/erl_nif.c but no idea how to include / reference

mberrueta · 2019-05-04T16:16:06Z

I tried to help here, but my no knowledge in c make it impossible

here there is more details if helps in any way

matiasberrueta@mac:~/c/p/w|update_libs⚡*?
➤ env DEBUG=1  mix deps.compile prometheus_process_collector
===> Expanded command sequence to be run: []
===> Provider: {default,do}
===> Expanded command sequence to be run: [{default,app_discovery},
                                           {bare,compile}]
===> Provider: {default,app_discovery}
===> Provider: {bare,compile}
===> Compiling prometheus_process_collector
===> sh info:
	cwd: "/Users/matiasberrueta/code/p/w/deps/prometheus_process_collector"
	cmd: make -C c_src

===> 	opts: [use_stdout,
               {cd,"/Users/matiasberrueta/code/p/w/deps/prometheus_process_collector"},
               {env,[{"REBAR_DEPS_DIR",
                      "/Users/matiasberrueta/code/p/w/deps/prometheus_process_collector/_build/default/lib"},
                     {"REBAR_BUILD_DIR",
                      "/Users/matiasberrueta/code/p/w/deps/prometheus_process_collector/_build/default"},
                     {"REBAR_ROOT_DIR",
                      "/Users/matiasberrueta/code/p/w/deps/prometheus_process_collector/."},
                     {"REBAR_CHECKOUTS_DIR",
                      "/Users/matiasberrueta/code/p/w/deps/prometheus_process_collector/_checkouts"},
                     {"REBAR_PLUGINS_DIR",
                      "/Users/matiasberrueta/code/p/w/deps/prometheus_process_collector/_build/default/plugins"},
                     {"REBAR_GLOBAL_CONFIG_DIR",
                      "/Users/matiasberrueta/.config/rebar3"},
                     {"REBAR_GLOBAL_CACHE_DIR",
                      "/Users/matiasberrueta/.cache/rebar3"},
                     {"REBAR_TEMPLATE_DIR",
                      "/Users/matiasberrueta/.config/rebar3/templates"},
                     {"REBAR_APP_DIRS",
                      "/Users/matiasberrueta/code/p/w/deps/prometheus_process_collector/_build/default/apps/*:/Users/matiasberrueta/code/p/w/deps/prometheus_process_collector/_build/default/lib/*:/Users/matiasberrueta/code/p/w/deps/prometheus_process_collector/_build/default/."},
                     {"REBAR_SRC_DIRS",[]},
                     {"ERLANG_ERTS_VER","10.3.4"},
                     {"ERLANG_ROOT_DIR",
                      "/usr/local/Cellar/erlang/21.3.7/lib/erlang"},
                     {"ERL",
                      "/usr/local/Cellar/erlang/21.3.7/lib/erlang/bin/erl"},
                     {"ERLC",
                      "/usr/local/Cellar/erlang/21.3.7/lib/erlang/bin/erlc"},
                     {"ERLANG_ARCH","64"},
                     {"ERLANG_TARGET","21.3.7-x86_64-apple-darwin18.5.0-64"},
                     {"ERLANG_LIB_DIR_erl_interface",
                      "/usr/local/Cellar/erlang/21.3.7/lib/erlang/lib/erl_interface-3.11.2"},
                     {"ERLANG_LIB_VER_erl_interface","3.11.2"}]},
               {abort_on_error,"Hook for compile failed!\n"}]

===> Port Cmd: make -C c_src
Port Opts: [{cd,"/Users/matiasberrueta/code/p/w/deps/prometheus_process_collector"},
            {env,[{"REBAR_DEPS_DIR",
                   "/Users/matiasberrueta/code/p/w/deps/prometheus_process_collector/_build/default/lib"},
                  {"REBAR_BUILD_DIR",
                   "/Users/matiasberrueta/code/p/w/deps/prometheus_process_collector/_build/default"},
                  {"REBAR_ROOT_DIR",
                   "/Users/matiasberrueta/code/p/w/deps/prometheus_process_collector/."},
                  {"REBAR_CHECKOUTS_DIR",
                   "/Users/matiasberrueta/code/p/w/deps/prometheus_process_collector/_checkouts"},
                  {"REBAR_PLUGINS_DIR",
                   "/Users/matiasberrueta/code/p/w/deps/prometheus_process_collector/_build/default/plugins"},
                  {"REBAR_GLOBAL_CONFIG_DIR",
                   "/Users/matiasberrueta/.config/rebar3"},
                  {"REBAR_GLOBAL_CACHE_DIR",
                   "/Users/matiasberrueta/.cache/rebar3"},
                  {"REBAR_TEMPLATE_DIR",
                   "/Users/matiasberrueta/.config/rebar3/templates"},
                  {"REBAR_APP_DIRS",
                   "/Users/matiasberrueta/code/p/w/deps/prometheus_process_collector/_build/default/apps/*:/Users/matiasberrueta/code/p/w/deps/prometheus_process_collector/_build/default/lib/*:/Users/matiasberrueta/code/p/w/deps/prometheus_process_collector/_build/default/."},
                  {"REBAR_SRC_DIRS",[]},
                  {"ERLANG_ERTS_VER","10.3.4"},
                  {"ERLANG_ROOT_DIR",
                   "/usr/local/Cellar/erlang/21.3.7/lib/erlang"},
                  {"ERL","/usr/local/Cellar/erlang/21.3.7/lib/erlang/bin/erl"},
                  {"ERLC",
                   "/usr/local/Cellar/erlang/21.3.7/lib/erlang/bin/erlc"},
                  {"ERLANG_ARCH","64"},
                  {"ERLANG_TARGET","21.3.7-x86_64-apple-darwin18.5.0-64"},
                  {"ERLANG_LIB_DIR_erl_interface",
                   "/usr/local/Cellar/erlang/21.3.7/lib/erlang/lib/erl_interface-3.11.2"},
                  {"ERLANG_LIB_VER_erl_interface","3.11.2"}]},
            exit_status,
            {line,16384},
            use_stdio,stderr_to_stdout,hide,eof]

cc prometheus_process_collector_nif.o prometheus_process_info_macos.o -L/usr/local/opt/openssl/lib -shared -L /usr/local/Cellar/erlang/21.3.7/lib/erlang/lib/erl_interface-3.11.2/lib -lerl_interface -lei -lstdc++ -o /Users/matiasberrueta/code/p/w/deps/prometheus_process_collector/c_src/../priv/prometheus_process_collector.so
Undefined symbols for architecture x86_64:
  "_enif_make_atom", referenced from:
      on_load(enif_environment_t*, void**, unsigned long) in prometheus_process_collector_nif.o
  "_enif_make_double", referenced from:
      Prometheus::get_process_info(enif_environment_t*, int, unsigned long const*) in prometheus_process_collector_nif.o
  "_enif_make_int", referenced from:
      Prometheus::get_process_info(enif_environment_t*, int, unsigned long const*) in prometheus_process_collector_nif.o
  "_enif_make_list_from_array", referenced from:
      Prometheus::get_process_info(enif_environment_t*, int, unsigned long const*) in prometheus_process_collector_nif.o
  "_enif_make_long", referenced from:
      Prometheus::get_process_info(enif_environment_t*, int, unsigned long const*) in prometheus_process_collector_nif.o
  "_enif_make_tuple", referenced from:
      Prometheus::get_process_info(enif_environment_t*, int, unsigned long const*) in prometheus_process_collector_nif.o
  "_enif_make_ulong", referenced from:
      Prometheus::get_process_info(enif_environment_t*, int, unsigned long const*) in prometheus_process_collector_nif.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [/Users/matiasberrueta/code/p/w/deps/prometheus_process_collector/c_src/../priv/prometheus_process_collector.so] Error 1
===> Hook for compile failed!

** (Mix) Could not compile dependency :prometheus_process_collector, "/Users/matiasberrueta/.mix/rebar3 bare compile --paths "/Users/matiasberrueta/code/p/w/_build/dev/lib/*/ebin"" command failed. You can recompile this dependency with "mix deps.compile prometheus_process_collector", update it with "mix deps.update prometheus_process_collector" or clean it with "mix deps.clean prometheus_process_collector"
matiasberrueta@mac:~/c/p/w|update_libs⚡*?

I tried compiling myself https://github.com/erlang/otp that has the code but I don't know what else is missing

also now is not working with the previous version neither, so not sure what else was changed :(

mberrueta · 2019-05-06T12:03:44Z

https://bugs.erlang.org/browse/ERL-935

with
Erlang/OTP 21 [erts-10.3.4] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [hipe] [dtrace] Eshell V10.3.4 (abort with ^G) fails

but with

Erlang/OTP 21 [erts-10.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1]

Eshell V10.2  (abort with ^G)

works

deadtrickster · 2019-05-06T17:25:02Z

Just compiled here:

Erlang/OTP 21 [erts-10.3] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe]

Eshell V10.3  (abort with ^G)

no idea what's up.

josevalim · 2019-05-17T17:41:28Z

@andrelip @mberrueta what is your mix-installed rebar3 version? You can run this:

~/.mix/rebar3 version

Please try running mix local.rebar and trying again.

krisalyssa · 2019-05-17T19:30:24Z

@josevalim:

$ ~/.asdf/installs/elixir/1.8.1/.mix/rebar3 version
rebar 3.6.1 on Erlang/OTP 21 Erts 10.3.4

mberrueta · 2019-05-17T19:43:22Z

now it's working for me, but I don't know what solve it. I update and restart everything

josevalim · 2019-05-17T19:46:16Z

@mberrueta what is your rebar3 version? I think this is actually a build tool issue, so perhaps it is worth closing the bug report in bugs.erlang.org until we can trim it down?

mberrueta · 2019-05-17T20:21:19Z

~/.mix/rebar3 --version
rebar 3.6.1 on Erlang/OTP 21 Erts 10.3.5

I opened a rebar issue before, but they said that wasn't related
erlang/rebar3#2064

krisalyssa · 2019-05-20T14:31:28Z

@andrelip @mberrueta I think your problem may be this:

-L/usr/local/opt/openssl/lib

I had CPPFLAGS and LDFLAGS set in my environment for compiling in support for MySQL, and their being set was preventing the prometheus_process_collector Makefile from setting them correctly itself. I unset both env vars and now this compiles fine.

krisalyssa · 2019-05-20T14:33:22Z

My Makefile skills are rusty, but I think the line at https://github.com/deadtrickster/prometheus_process_collector/blob/master/c_src/Makefile#L25 says "set LDFLAGS if it's not already set". It probably should be appending to LDFLAGS instead.

deadtrickster · 2019-05-20T14:34:38Z

that's true!

krisalyssa · 2019-05-20T15:11:02Z

@deadtrickster I'm not trying to duck the responsibility of creating a PR for my suggestion -- I'm just elbows deep in something else at the moment. 😄

deadtrickster · 2019-05-20T15:15:17Z

oh, no expectations at all. just agreeing.

clifton-mcintosh · 2019-05-24T21:23:55Z

This problem is occurring for me as well. While troubleshooting, one of my colleagues and I tried running make on the c_src and it fails. My colleague, whose experience with C is more extensive than mine, says it seems to him like the problem is with ld. I have a newer version of ld. Here is the output from running make on the c_src.

https://gist.github.com/clifton-mcintosh/9e5c9d546f8f45bd1d12377c1d709ed9

This is my version of ld, which fails when running make on the c_src:

ld -v
@(#)PROGRAM:ld  PROJECT:ld64-450.3
BUILD 18:45:16 Apr  4 2019
configured to support archs: armv6 armv7 armv7s arm64 arm64e arm64_32 i386 x86_64 x86_64h armv6m armv7k armv7m armv7em
LTO support using: LLVM version 10.0.1, (clang-1001.0.46.4) (static support for 22, runtime is 22)
TAPI support using: Apple TAPI version 10.0.1 (tapi-1001.0.4.1)

My colleague's earlier version of ld, which works when running make on the c_src.

ld -v
@(#)PROGRAM:ld  PROJECT:ld64-409.12
BUILD 02:04:28 Aug 14 2018
configured to support archs: armv6 armv7 armv7s arm64 i386 x86_64 x86_64h armv6m armv7k armv7m armv7em
LTO support using: LLVM version 10.0.0, (clang-1000.10.44.2) (static support for 21, runtime is 21)
TAPI support using: Apple TAPI version 10.0.0 (tapi-1000.10.8)

acrogenesis · 2019-07-08T22:45:24Z

I'm also having trouble compiling.

===> Compiling prometheus_process_collector
cc prometheus_process_collector_nif.o prometheus_process_info_macos.o -arch x86_64 -flat_namespace -undefined suppress -shared -L /usr/local/Cellar/erlang/22.0.5/lib/erlang/lib/erl_interface-3.12/lib -lerl_interface -lei -lstdc++ -o /Users/acrogenesis/Library/CloudStorage/iCloud Drive/Documents/Development/valiot/valiot-jobs/deps/prometheus_process_collector/c_src/../priv/iCloud iCloud prometheus_process_collector.so
clang: error: no such file or directory: 'Drive/Documents/Development/valiot/valiot-jobs/deps/prometheus_process_collector/c_src/../priv/iCloud'
clang: error: no such file or directory: 'iCloud'
clang: error: no such file or directory: 'prometheus_process_collector.so'
make: *** [/Users/acrogenesis/Library/CloudStorage/iCloud] Error 1
===> Hook for compile failed!

~/.mix/rebar3 --version

rebar 3.6.1 on Erlang/OTP 22 Erts 10.4.4

ld -v

BUILD 10:27:00 May 27 2019
configured to support archs: armv6 armv7 armv7s arm64 arm64e arm64_32 i386 x86_64 x86_64h armv6m armv7k armv7m armv7em
LTO support using: LLVM version 11.0.0, (clang-1100.0.20.17) (static support for 23, runtime is 23)
TAPI support using: Apple TAPI version 11.0.0 (tapi-1100.0.10.2)

acrogenesis · 2019-07-08T22:47:39Z

It seems it only fails if I'm compiling inside an iCloud Drive folder.

ghost · 2019-09-08T08:14:10Z

Also an issue on Erlang 22.0.6

ghost · 2019-09-08T08:16:01Z

Also an issue on Erlang 22.0.6

unset LDFLAGS
unset LD_LIBRARY_PATH
unset CFLAGS

and it compiled fine.

ghost · 2019-09-08T08:29:55Z

@deadtrickster Created PR to fix this issue - tested on OSX / Docker (alpine) - would be sweet if it made it into a hex release.

ghost · 2019-09-08T10:45:27Z

Also fixed the issue (Elixir) where compilation fails with error message:

                                                                                                                                                                                  
/code/common/horde_test/ DEBUG=1 mix compile                                                                                                                                      
==> prometheus_process_collector                                                                                                                                                  
make: *** No targets specified and no makefile found.  Stop.                                                                                                                      
could not compile dependency :prometheus_process_collector, "mix compile" failed. You can recompile this dependency with "mix deps.compile prometheus_process_collector", update it with "mix deps.update prometheus_process_collector" or clean it with "mix deps.clean prometheus_process_collector"                                                              
==> horde_test                                                                                                                                                                    
** (Mix) Could not compile with "make" (exit status: 2).                                                                                                                          
You need to have gcc and make installed. Try running the                                                                                                                          
commands "gcc --version" and / or "make --version". If these programs                                                                                                             
are not installed, you will be prompted to install them.

bieniusa · 2019-09-11T07:42:44Z

It seems that with @bryanhuntesl PR in v1.4.4 elixir_make became a dependency.
This is a bit unfortunate for Erlang-only projects as it apparently requires some configuration of mix and a working Elixir installation. Was this intended?

===> Compiling elixir_make
===> Error building application elixir_make:
     No project builder is configured for type mix

deadtrickster · 2019-09-11T07:45:25Z

Woops, nope. hmmm

ghost · 2019-09-11T07:58:09Z

It seems that with @bryanhuntesl PR in v1.4.4 elixir_make became a dependency.
This is a bit unfortunate for Erlang-only projects as it apparently requires some configuration of mix and a working Elixir installation. Was this intended?
===> Compiling elixir_make
===> Error building application elixir_make:
     No project builder is configured for type mix

@bieniusa - Is this analysis correct ?

There are two build systems for this project - the mix.exs used by Elixir and the rebar.config used by hex. Although I only added the dependency to the mix.exs, a dependency has been introduced for 'elixir_make' when using as a rebar3 dependency ?

@deadtrickster - how is this package published - how did the dependency creep in to rebar3 ?

ghost · 2019-09-11T08:00:03Z

@bieniusa - where/how are you using this as a dependency ?

bieniusa · 2019-09-11T08:00:28Z

It does not show up in rebar.config, but hex lists it on the webpage.

bieniusa · 2019-09-11T08:01:12Z

I just upgraded my AntidoteDB dependencies (for some other reason) and there it showed up.
Guess it was only noticeable for me because I don't have an Elixir install on my machine.

ghost · 2019-09-11T08:05:21Z

Awww. This is going to be a nightmare to fix. If we want to have erlang and elixir support it would probably be best to dump the mix stuff completely and use rebar3 / port compiler / rebar3_publish and nothing else. The reason I came to this stuff was that build failed on Elixir projects because of the broken mix config.

ghost · 2019-09-11T08:07:09Z

What do you think @deadtrickster ?

ghost · 2019-09-11T08:10:55Z

Or make the 'elixir_make' dependency ':optional' - but that will create headaches for the Elixir people

josevalim · 2019-09-11T10:13:25Z

This is going to be a nightmare to fix. If we want to have erlang and elixir support it would probably be best to dump the mix stuff completely and use rebar3 / port compiler / rebar3_publish and nothing else.

Yup, I would say this is the way to go. elixir_make is meant to be an alternative to the conveniences provided by rebar for handling native code anyway. So both should take you to the same place.

deadtrickster · 2019-09-12T06:34:26Z

published 1.4.5 without mix

ghost · 2019-09-12T21:01:33Z

Successfully builds on OSX (clang/homebrew) + docker (elixir:1.8.1 (some debian)) - thanks @deadtrickster

ghost · 2019-09-16T08:50:29Z

@andrelip have you tried / tested - can this be closed ?

dimitridewit · 2019-11-07T12:05:01Z

I had the same issue today. Elixir 1.9.1 with Erlang/OTP 22.
I spotted the openssl part L/usr/local/opt/openssl/lib in @andrelip output. Commented them out in my .zshrc and it would compile. But, I can't seem to reproduce it ... ? 💁‍♂️

ghost · 2021-05-13T11:52:37Z

Good to close this ticket now ?

krisalyssa · 2021-05-13T13:21:49Z

I'm going to vote "yes", with the qualification that I haven't used Elixir 1.8, OTP 21, or macOS 10.14 in some time. 😄

bieniusa · 2021-05-13T14:21:37Z

Yes!

ghost mentioned this issue Sep 8, 2019

use and append existing compile flags #19

Merged

Compiling problems with Elixir 1.8, OTP21 and OSX 10.14.2 #17

Compiling problems with Elixir 1.8, OTP21 and OSX 10.14.2 #17

Comments

andrelip commented Jan 28, 2019 • edited Loading

mberrueta commented May 4, 2019

mberrueta commented May 4, 2019 • edited Loading

mberrueta commented May 6, 2019

deadtrickster commented May 6, 2019

josevalim commented May 17, 2019

krisalyssa commented May 17, 2019

mberrueta commented May 17, 2019

josevalim commented May 17, 2019

mberrueta commented May 17, 2019

krisalyssa commented May 20, 2019

krisalyssa commented May 20, 2019

deadtrickster commented May 20, 2019

krisalyssa commented May 20, 2019

deadtrickster commented May 20, 2019

clifton-mcintosh commented May 24, 2019

acrogenesis commented Jul 8, 2019

acrogenesis commented Jul 8, 2019

ghost commented Sep 8, 2019

ghost commented Sep 8, 2019

ghost commented Sep 8, 2019

ghost commented Sep 8, 2019

bieniusa commented Sep 11, 2019

deadtrickster commented Sep 11, 2019

ghost commented Sep 11, 2019

ghost commented Sep 11, 2019

bieniusa commented Sep 11, 2019

bieniusa commented Sep 11, 2019 • edited Loading

ghost commented Sep 11, 2019

ghost commented Sep 11, 2019

ghost commented Sep 11, 2019

josevalim commented Sep 11, 2019

deadtrickster commented Sep 12, 2019

ghost commented Sep 12, 2019

ghost commented Sep 16, 2019

dimitridewit commented Nov 7, 2019

ghost commented May 13, 2021

krisalyssa commented May 13, 2021

bieniusa commented May 13, 2021

andrelip commented Jan 28, 2019 •

edited

Loading

mberrueta commented May 4, 2019 •

edited

Loading

bieniusa commented Sep 11, 2019 •

edited

Loading