Fix cross-compilation issues for aws-lc-sys

[samuel40791765]

Issues:

Addresses CryptoAlg-1422

Description of changes:

• created a generation rust crate
• Moved bindgen out of aws-lc-sysinto the generation rust crate. Moved the original bindgen in aws-lc-sys to a separate file under a feature flag as a fallback mechanism
• created scripts that will pre-generate the bindings on platforms we care about
• added sparse registry dependency in ARM docker image: cargo build during a docker build takes forever and blows up memory on "Updating crates.io index" rust-lang/cargo#10781
• backported changes in af25856 and added code to turn off options in cmake

Call-outs:

Most of the cmake changes were backported from af25856. Added four lines of code to turn off build options

Testing:

Running the script ./generate.sh. Tested the new aws-lc-sys crate with cross. Tests pass and call the correct bindings on the corresponding platform.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and
the ISC license.

[samuel40791765]

Verified against CMake3.0.0 and CMake3.10 that adding build options after the path work. AWS-LC requires a cmake version of at least 3.0.0 to build.

root@d0aa4f57add0:/home/ubuntu/sachiang/workplace/repo/aws-lc# rm -rf test_build_dir && mkdir test_build_dir
root@d0aa4f57add0:/home/ubuntu/sachiang/workplace/repo/aws-lc# cmake ../aws-lc -GNinja "-B`pwd`/test_build_dir" "-DCMAKE_INSTALL_PREFIX=`pwd`/test_build_dir/" -DCMAKE_BUILD_TYPE=Release -DDISABLE_PERL=ON -DDISABLE_GO=ON
-- The C compiler identification is Clang 10.0.0
-- Check for working C compiler using: Ninja
-- Check for working C compiler using: Ninja -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- The CXX compiler identification is Clang 10.0.0
-- Check for working CXX compiler using: Ninja
-- Check for working CXX compiler using: Ninja -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
pkgconfig not found. Disabling unwind tests.
-- Go not found. Disabling some code generation and using pre-generated code in generated-src/
-- Perl not found. Disabling some code generation and using pre-generated code in generated-src/
-- Test stdalign_check.c passed, enabling AWS_LC_STDALIGN_AVAILABLE
-- Test builtin_swap_check.c passed, enabling AWS_LC_BUILTIN_SWAP_SUPPORTED
-- Check if the system is big endian
-- Searching 16 bit integer
-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for stdint.h
-- Looking for stdint.h - found
-- Looking for stddef.h
-- Looking for stddef.h - found
-- Check size of unsigned short
-- Check size of unsigned short - done
-- Using unsigned short
-- Check if the system is big endian - little endian
-- The ASM compiler identification is Clang
-- Found assembler: /usr/bin/clang-10
-- Disabling debug symbols for Clang internal assembler
Copying platform assembly files from /home/ubuntu/sachiang/workplace/repo/aws-lc/generated-src/linux-x86_64/crypto/ to /home/ubuntu/sachiang/workplace/repo/aws-lc/test_build_dir/crypto
-- Looking for include file pthread.h
-- Looking for include file pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - not found
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - yes
-- Found Threads: TRUE  
-- Generating test executable mem_test.
-- Generating test executable mem_set_test.
-- Configuring done
-- Generating done
-- Build files have been written to: /home/ubuntu/sachiang/workplace/repo/aws-lc/test_build_dir
root@d0aa4f57add0:/home/ubuntu/sachiang/workplace/repo/aws-lc# cmake ../aws-lc -GNinja "-B`pwd`/test_build_dir" "-DCMAKE_INSTALL_PREFIX=`pwd`/test_build_dir/" -DCMAKE_BUILD_TYPE=Release -DDISABLE_PERL=ON -DDISABLE_GO=ON -DFIPS=1
pkgconfig not found. Disabling unwind tests.
-- Go not found. Disabling some code generation and using pre-generated code in generated-src/
-- Perl not found. Disabling some code generation and using pre-generated code in generated-src/
-- Test stdalign_check.c passed, enabling AWS_LC_STDALIGN_AVAILABLE
-- Test builtin_swap_check.c passed, enabling AWS_LC_BUILTIN_SWAP_SUPPORTED
-- Test linux_u32.c passed, enabling AWS_LC_URANDOM_U32
CMake Error at CMakeLists.txt:567 (message):
  Building AWS-LC for FIPS requires Go and Perl


-- Configuring incomplete, errors occurred!
See also "/home/ubuntu/sachiang/workplace/repo/aws-lc/test_build_dir/CMakeFiles/CMakeOutput.log".
See also "/home/ubuntu/sachiang/workplace/repo/aws-lc/test_build_dir/CMakeFiles/CMakeError.log".
root@d0aa4f57add0:/home/ubuntu/sachiang/workplace/repo/aws-lc# cmake --version
cmake version 3.1.0

CMake suite maintained and supported by Kitware (kitware.com/cmake).

[justsmth]

• For the feature that customers see, I think a name like "generate_bindings" would be more accurate. (The bindings will be "native" regardless.)
• Could we simply remove this "generate" feature from Cargo.toml prior to publishing the crate? Would that prevent its use?
• If we must keep this feature after publication, could we name it in a way that's more clearly internal-use-only -- like "internal_generate"?

[samuel40791765]

• Renamed to generate_bindings and internal_generate.
• I guess we could use a removal script to remove mentions of the generate feature? That would provide additional complications though.
  The best alternative IMO would be to place the internal_generate feature in a separate non-published generate crate, like we were initially doing? Although there will be a bit of duplicate code, that would prevent us exposing this feature externally.