'fatal error: 'openssl/hmac.h' file not found' on OSX 10.11 when building openssl-sys-extras

[Byron]

When trying to build openssl-sys-extras, the build seems to fail with fatal error: 'openssl/hmac.h' file not found on OSX 10.11, even if OPENSSL_INCLUDE_DIR is set to /usr/local/opt/openssl/include.

The Fix

Looking at the build script of openssl-sys-extras, you will see that it uses DEP_OPENSSL_INCLUDE to determine additional include directories. Thus running cargo with something like this ...

OPENSSL_INCLUDE_DIR=/usr/local/opt/openssl/include DEP_OPENSSL_INCLUDE=/usr/local/opt/openssl/include cargo build

... will do the trick.

Proposition

To prevent others from running into the issue as well, one could use OPENSSL_INCLUDE_DIR as secondary option in the build script.

[sfackler]

The value of OPENSSL_INCLUDE_DIR should end up getting passed through DEP_OPENSSL_INCLUDE if it was set when openssl-sys was compiled. It seems not super uncommon for people to build a thing that depends on it, get to the point that openssl-sys-extras compiles and then remember to add the environment variable though...

[Byron]

Unfortunately, the pass-through doesn't seem to work under the conditions I am using it. All I do is to use openssl as a dependency, without any extra configuration.

However, to reproduce the issue, it seems to be enough to do the following on OSX 10.11:

➜  /tmp  git clone https://github.com/sfackler/rust-openssl
Cloning into 'rust-openssl'...
cremote: Counting objects: 19706, done.
remote: Total 19706 (delta 0), reused 0 (delta 0), pack-reused 19706
Receiving objects: 100% (19706/19706), 6.85 MiB | 2.23 MiB/s, done.
Resolving deltas: 100% (16053/16053), done.
Checking connectivity... done.
➜  /tmp  cd rust-openssl
➜  rust-openssl git:(master) cargo build
Could not find `Cargo.toml` in `/private/tmp/rust-openssl` or any parent directory
➜  rust-openssl git:(master) ll
total 32
-rw-r--r--  1 byron  wheel   2.7K Dec  7 10:03 README.md
-rw-r--r--  1 byron  wheel   6.9K Dec  7 10:03 THIRD_PARTY
-rw-r--r--  1 byron  wheel   828B Dec  7 10:03 appveyor.yml
drwxr-xr-x  6 byron  wheel   204B Dec  7 10:03 openssl
drwxr-xr-x  6 byron  wheel   204B Dec  7 10:03 openssl-sys
drwxr-xr-x  5 byron  wheel   170B Dec  7 10:03 openssl-sys-extras
➜  rust-openssl git:(master) cd openssl
➜  openssl git:(master) cargo build
    Updating registry `https://github.com/rust-lang/crates.io-index`
 Downloading net2 v0.2.19
 Downloading ws2_32-sys v0.2.1
 Downloading cfg-if v0.1.0
   Compiling libc v0.2.2
   Compiling pkg-config v0.3.6
   Compiling bitflags v0.3.3
   Compiling lazy_static v0.1.15
   Compiling gcc v0.3.20
   Compiling openssl-sys v0.7.1 (file:///private/tmp/rust-openssl/openssl)
   Compiling openssl-sys-extras v0.7.1 (file:///private/tmp/rust-openssl/openssl)
Build failed, waiting for other jobs to finish...
failed to run custom build command for `openssl-sys-extras v0.7.1 (file:///private/tmp/rust-openssl/openssl)`
Process didn't exit successfully: `/private/tmp/rust-openssl/openssl/target/debug/build/openssl-sys-extras-0ac187301cabb077/build-script-build` (exit code: 101)
--- stdout
TARGET = Some("i686-apple-darwin")
OPT_LEVEL = Some("0")
PROFILE = Some("debug")
TARGET = Some("i686-apple-darwin")
debug=true opt-level=0
HOST = Some("i686-apple-darwin")
TARGET = Some("i686-apple-darwin")
TARGET = Some("i686-apple-darwin")
HOST = Some("i686-apple-darwin")
CC_i686-apple-darwin = None
CC_i686_apple_darwin = None
HOST_CC = None
CC = None
HOST = Some("i686-apple-darwin")
TARGET = Some("i686-apple-darwin")
HOST = Some("i686-apple-darwin")
CFLAGS_i686-apple-darwin = None
CFLAGS_i686_apple_darwin = None
HOST_CFLAGS = None
CFLAGS = None
running: "cc" "-O0" "-ffunction-sections" "-fdata-sections" "-g" "-m32" "-o" "/private/tmp/rust-openssl/openssl/target/debug/build/openssl-sys-extras-0ac187301cabb077/out/src/openssl_shim.o" "-c" "src/openssl_shim.c"
ExitStatus(Code(1))


command did not execute successfully, got: exit code: 1



--- stderr
src/openssl_shim.c:1:10: fatal error: 'openssl/hmac.h' file not found
#include <openssl/hmac.h>
         ^
1 error generated.
thread '<main>' panicked at 'explicit panic', /Users/byron/.multirust/cargo/registry/src/github.com-121aea75f9ef2ce2/gcc-0.3.20/src/lib.rs:772

With only OPENSSL_INCLUDE_DIR set, it will show similar issues when compiling openssl-sys-extras:

➜  openssl git:(master) OPENSSL_INCLUDE_DIR=/usr/local/opt/openssl/include cargo build
   Compiling openssl-sys-extras v0.7.1 (file:///private/tmp/rust-openssl/openssl)
failed to run custom build command for `openssl-sys-extras v0.7.1 (file:///private/tmp/rust-openssl/openssl)`
Process didn't exit successfully: `/private/tmp/rust-openssl/openssl/target/debug/build/openssl-sys-extras-0ac187301cabb077/build-script-build` (exit code: 101)
--- stdout
TARGET = Some("i686-apple-darwin")
OPT_LEVEL = Some("0")
PROFILE = Some("debug")
TARGET = Some("i686-apple-darwin")
debug=true opt-level=0
HOST = Some("i686-apple-darwin")
TARGET = Some("i686-apple-darwin")
TARGET = Some("i686-apple-darwin")
HOST = Some("i686-apple-darwin")
CC_i686-apple-darwin = None
CC_i686_apple_darwin = None
HOST_CC = None
CC = None
HOST = Some("i686-apple-darwin")
TARGET = Some("i686-apple-darwin")
HOST = Some("i686-apple-darwin")
CFLAGS_i686-apple-darwin = None
CFLAGS_i686_apple_darwin = None
HOST_CFLAGS = None
CFLAGS = None
running: "cc" "-O0" "-ffunction-sections" "-fdata-sections" "-g" "-m32" "-o" "/private/tmp/rust-openssl/openssl/target/debug/build/openssl-sys-extras-0ac187301cabb077/out/src/openssl_shim.o" "-c" "src/openssl_shim.c"
ExitStatus(Code(1))


command did not execute successfully, got: exit code: 1



--- stderr
src/openssl_shim.c:1:10: fatal error: 'openssl/hmac.h' file not found
#include <openssl/hmac.h>
         ^
1 error generated.
thread '<main>' panicked at 'explicit panic', /Users/byron/.multirust/cargo/registry/src/github.com-121aea75f9ef2ce2/gcc-0.3.20/src/lib.rs:772

With both, OPENSSL_INCLUDE_DIR and DEP_OPENSSL_INCLUDE set, it works. The codified fix is provided via #317.

➜  openssl git:(master) OPENSSL_INCLUDE_DIR=/usr/local/opt/openssl/include DEP_OPENSSL_INCLUDE=/usr/local/opt/openssl/include cargo build
   Compiling openssl-sys-extras v0.7.1 (file:///private/tmp/rust-openssl/openssl)
   Compiling openssl v0.7.1 (file:///private/tmp/rust-openssl/openssl)
➜  openssl git:(master)

Maybe there are other ways to fix it, after all, it was compiling just fine on OSX 10.10 - until now the only solution was to set environment variables though. Having to set two distinct ones with seemingly similar effect appears to be something worth looking at from a usability standpoint.

Am I missing something obvious ?

[andrewtj]

I think you're demonstrating the behaviour @sfackler described. If you do a clean build with just the non dep prefixed environment variables set it ought to build fine.

[Byron]

Thanks for chiming in. However, the comment raises more questions:

• How can a build be cleaner than starting with a fresh checkout ?
• What is the non dep prefixed environment variable ? or Is there OPENSSL_INCLUDE as well ? or Is there a difference between OPENSSL_INCLUDE and OPENSSL_INCLUDE_DIR ?
• Is it intended to now require two environment variables pointing to the same directory where one would only need a single one in the past ?

[andrewtj]

When you ran cargo build the first time without any environment set it tarnished the build. Do cargo clean and then run cargo build with OPENSSL_LIB_DIR and OPENSSL_INCLUDE_DIR set and it should build okay.

[Byron]

Amazing, thanks for the clarification and the immense amount of patience ! The build did work fine when providing the correct environment variable, OPENSSL_INCLUDE_DIR right from the start.

It's interesting that I never needed OPENSSL_LIB_DIR - apparently it manages to find some library to link against, which is compatible enough.

Considering this, #317 shouldn't be needed anymore, even though it might not hurt either.

[andrewtj]

You'll want to set OPENSSL_LIB_DIR as well. The version of OpenSSL bundled with OS X is an odd-duck: OpenSSL doesn't have a stable ABI so for compatibility purposes Apple have maintained a fork that's compatible with one of the earlier ABIs. They deprecated OpenSSL in 10.7 and finally dropped the headers in 10.11 to push OS X app developers toward bundling their own OpenSSL or using their frameworks. The dylibs have been left around so apps that haven't been updated don't break. You can still link against them but you're opening yourself up to odd compatibility issues by doing so (unless you grab the headers from an earlier OS X release).

[Byron]

Now I see ! Thank you !

Considering the importance of using the correct version, do you think it would be worth adjusting the build routines to help assure people don't accidentally link against the wrong version ?
I am not sure how that would be accomplished best, or if this even is the place for discussing it.

[sfackler]

Yeah, I'm not sure of an easy way to check if you're linking against the right version unfortunately. I'll add a message to openssl-sys-extras's build script to tell people what to do if compilation fails.

[sfackler]

Ah ha! https://www.openssl.org/docs/manmaster/crypto/OPENSSL_VERSION_NUMBER.html

I'll put a thing into openssl-sys-extras's build script to check this.

[sfackler]

This is blocked on rust-lang/cc-rs#63 for now.