Unable to run cargo test on macOS; fatal error: 'NEON_2_SSE.h' file not found

[tallamjr]

Firstly I would like say thank you for this work! This looks like a fantastic project and I am excited to get involved if I can.

I wanted to run a fresh build and run cargo test but I encountered several errors that mean I am a bit stuck on what to do next.

The first error encountered was:

error: failed to select a version for the requirement `bindgen = "^0.54.1"`

On master-86bba39, and after successfully running:

cd submodules/tensorflow
make -f tensorflow/lite/micro/tools/make/Makefile test_micro_speech_test
cd ../..

When I then try to run cargo test the following error occurs

cargo test
    Updating crates.io index
error: failed to select a version for the requirement `bindgen = "^0.54.1"`
candidate versions found which didn't match: 0.55.1, 0.55.0, 0.54.0, ...
location searched: crates.io index
required by package `tfmicro v0.1.0 (/Users/tallamjr/github/EDGE22/forks/tfmicro)`

Making a change in Cargo.toml:57 seems to resolve this issue

diff --git a/Cargo.toml b/Cargo.toml
index e75ccdb..c52bd88 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -62,7 +62,7 @@ ordered-float = { version = "2.0.0", default-features = false }
 
 [build-dependencies]
 cc = { version = "1.0.58", features = ["parallel"] }
-bindgen = "0.54.1"
+bindgen = "0.55.0"
 cpp_build = "0.5.5"
 glob = "0.3.0"
 fs_extra = "1.1.0"

However, when running cargo test again, I am faced with another error:

error[E0063]: missing field `content_only` in initializer of `fs_extra::dir::CopyOptions`
  --> build.rs:57:20
   |
57 |     let copy_dir = fs_extra::dir::CopyOptions {
   |                    ^^^^^^^^^^^^^^^^^^^^^^^^^^ missing `content_only`

error: aborting due to previous error

For more information about this error, try `rustc --explain E0063`.
error: could not compile `tfmicro`

Caused by:
  process didn't exit successfully: `rustc --crate-name build_script_build --edition=2018 build.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debuginfo=2 -C metadata=4b1014e20b828738 -C extra-filename=-4b1014e20b828738 --out-dir /Users/tallamjr/github/EDGE22/forks/tfmicro/target/debug/build/tfmicro-4b1014e20b828738 -C incremental=/Users/tallamjr/github/EDGE22/forks/tfmicro/target/debug/incremental -L dependency=/Users/tallamjr/github/EDGE22/forks/tfmicro/target/debug/deps --extern bindgen=/Users/tallamjr/github/EDGE22/forks/tfmicro/target/debug/deps/libbindgen-14c4122017cc841c.rlib --extern cc=/Users/tallamjr/github/EDGE22/forks/tfmicro/target/debug/deps/libcc-cee51b73f7a0431a.rlib --extern cpp_build=/Users/tallamjr/github/EDGE22/forks/tfmicro/target/debug/deps/libcpp_build-4b9fa5e7028dbb8c.rlib --extern error_chain=/Users/tallamjr/github/EDGE22/forks/tfmicro/target/debug/deps/liberror_chain-f2d7932f0828b8fd.rlib --extern fs_extra=/Users/tallamjr/github/EDGE22/forks/tfmicro/target/debug/deps/libfs_extra-87bdfbdbacb8e667.rlib --extern glob=/Users/tallamjr/github/EDGE22/forks/tfmicro/target/debug/deps/libglob-deca07ad86c3d9ca.rlib` (exit code: 1)

Finding similar issues here and solution here, I was able to correct this with this change to build.rs:

diff --git a/build.rs b/build.rs
index 2098620..204bd4e 100644
--- a/build.rs
+++ b/build.rs
@@ -54,13 +54,12 @@ fn prepare_tensorflow_source() -> PathBuf {
     let tf_src_dir = out_dir.join("tensorflow/tensorflow");
     let submodules = submodules();
 
-    let copy_dir = fs_extra::dir::CopyOptions {
-        overwrite: true,
-        skip_exist: false,
-        buffer_size: 65536,
-        copy_inside: false,
-        depth: 0,
-    };
+    let mut copy_dir = fs_extra::dir::CopyOptions::new();
+    copy_dir.overwrite =  true;
+    copy_dir.skip_exist = false;
+    copy_dir.buffer_size =  65536;
+    copy_dir.copy_inside = false;
+    copy_dir.depth = 0;
 
     if !tf_src_dir.exists() || cfg!(feature = "build") {
         // Copy directory

Finally, I after incorporating the above change I am faced with a hurdle I am not sure how to get around as it is linked to tensorflow/tensorflow#23440 , namely: "fatal error: 'NEON_2_SSE.h' file not found"

...
  cargo:warning=In file included from /Users/tallamjr/github/EDGE22/forks/tfmicro/target/debug/build/tfmicro-950f68498590f496/out/tensorflow/tensorflow/lite/micro/kernels/activations.cc:18:
  cargo:warning=In file included from /Users/tallamjr/github/EDGE22/forks/tfmicro/target/debug/build/tfmicro-950f68498590f496/out/tensorflow/tensorflow/lite/kernels/internal/common.h:28:
  cargo:warning=/Users/tallamjr/github/EDGE22/forks/tfmicro/target/debug/build/tfmicro-950f68498590f496/out/tensorflow/tensorflow/lite/kernels/internal/optimized/neon_check.h:25:10: fatal error: 'NEON_2_SSE.h' file not found
  cargo:warning=#include "NEON_2_SSE.h"
  cargo:warning=         ^~~~~~~~~~~~~~
  cargo:warning=In file included from /Users/tallamjr/github/EDGE22/forks/tfmicro/target/debug/build/tfmicro-950f68498590f496/out/tensorflow/tensorflow/lite/micro/kernels/add.cc:16:
  cargo:warning=In file included from /Users/tallamjr/github/EDGE22/forks/tfmicro/target/debug/build/tfmicro-950f68498590f496/out/tensorflow/tensorflow/lite/kernels/internal/reference/add.h:19:
  cargo:warning=In file included from /Users/tallamjr/github/EDGE22/forks/tfmicro/target/debug/build/tfmicro-950f68498590f496/out/tensorflow/tensorflow/lite/kernels/internal/common.h:28:
  cargo:warning=/Users/tallamjr/github/EDGE22/forks/tfmicro/target/debug/build/tfmicro-950f68498590f496/out/tensorflow/tensorflow/lite/kernels/internal/optimized/neon_check.h:25:10: fatal error: 'NEON_2_SSE.h' file not found
  cargo:warning=#include "NEON_2_SSE.h"
  cargo:warning=         ^~~~~~~~~~~~~~
  cargo:warning=1 error generated.
  exit code: 1

Perhaps the initial fixes I made were not necessary (and may have even caused the above error) but as I am quite new to rust and tensorflow/lite/micro world I am not sure what is best at this point. Please do let me know if there are any further information that would be helpful.

[tallamjr]

Currently working on a fix at https://github.com/EDGE22/tfmicro/tree/issues/12/neon-2-sse

Confusingly this comment suggests it should be fixed with:

$ cd submodules/tensorflow && git status
HEAD detached at 33689c48ad
nothing to commit, working tree clean

$ cat tensorflow/lite/micro/tools/make/targets/osx_x86_64_makefile.inc
# Settings for x86 on Mac
ifeq ($(TARGET), osx)
  ifeq ($(TARGET_ARCH), x86_64)
    PLATFORM_FLAGS = \
      -DTF_LITE_DISABLE_X86_NEON
    CXXFLAGS += $(PLATFORM_FLAGS)
    CCFLAGS += $(PLATFORM_FLAGS)
  endif
endif

But that does not seem to be the case.

[richardeoin]

Hey @tallamjr, I just merged #10 which fixed the yanked bindgen and fs_extra changing the API. #13 should fix the issue permanently by using tilde requirements.

I haven't looked into the issue with building on OS X. It would be great to have a fix for that, please do make a PR if you find a solution!

[tallamjr]

Hey @tallamjr, I just merged #10 which fixed the yanked bindgen and fs_extra changing the API. #13 should fix the issue permanently by using tilde requirements.

I haven't looked into the issue with building on OS X. It would be great to have a fix for that, please do make a PR if you find a solution!

Ah I see, sorry, I hadn't seen those changes had already been taken care of in that PR. Thanks for the updates, I'll rebase any of my changes to master now.

Certainly, hopefully I can stumble upon a solution soon, and will open a PR when I do 👍