Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot compile rust-htslib with x86_64-unknown-linux-musl (needed for AWS Lambda runtimes) #179

Closed
brainstorm opened this issue Jan 29, 2020 · 3 comments

Comments

@brainstorm
Copy link
Member

Hello rust-htslib,

I'm currently using this crate to simply read a BAM header. My crate builds fine on my machine (running OSX Catalina, also tested on a regular Linux machine):

$ cargo build
    Finished dev [unoptimized + debuginfo] target(s) in 1.76s

The problem comes when I try to cross-compile it to run on AWS Lambda... it looks like the linking phase fails for the few rust-htslib functions I need to parse that BAM header:

  = note: /code/target/lambda/release/deps/librust_htslib-fc86895fccfb6420.rlib(rust_htslib-fc86895fccfb6420.rust_htslib.25lsx2fx-cgu.11.rcgu.o): In function `rust_htslib::bam::Reader::new::h7805fb2441edac2e':
          rust_htslib.25lsx2fx-cgu.11:(.text._ZN11rust_htslib3bam6Reader3new17h7805fb2441edac2eE+0x4c): undefined reference to `sam_hdr_read'
          /code/target/lambda/release/deps/librust_htslib-fc86895fccfb6420.rlib(rust_htslib-fc86895fccfb6420.rust_htslib.25lsx2fx-cgu.11.rcgu.o): In function `_$LT$rust_htslib..bam..Reader$u20$as$u20$core..ops..drop..Drop$GT$::drop::hc749ab5efb9aea68':
          rust_htslib.25lsx2fx-cgu.11:(.text._ZN66_$LT$rust_htslib..bam..Reader$u20$as$u20$core..ops..drop..Drop$GT$4drop17hc749ab5efb9aea68E+0x5): undefined reference to `hts_close'
          /code/target/lambda/release/deps/librust_htslib-fc86895fccfb6420.rlib(rust_htslib-fc86895fccfb6420.rust_htslib.25lsx2fx-cgu.11.rcgu.o): In function `rust_htslib::bam::hts_open::hac1f0f64c9261566':
          rust_htslib.25lsx2fx-cgu.11:(.text._ZN11rust_htslib3bam8hts_open17hac1f0f64c9261566E+0xc3): undefined reference to `hts_open'
          /code/target/lambda/release/deps/librust_htslib-fc86895fccfb6420.rlib(rust_htslib-fc86895fccfb6420.rust_htslib.25lsx2fx-cgu.11.rcgu.o): In function `_$LT$rust_htslib..bam..HeaderView$u20$as$u20$core..ops..drop..Drop$GT$::drop::h06ec3494656e04e9':
          rust_htslib.25lsx2fx-cgu.11:(.text._ZN70_$LT$rust_htslib..bam..HeaderView$u20$as$u20$core..ops..drop..Drop$GT$4drop17h06ec3494656e04e9E+0xb): undefined reference to `bam_hdr_destroy'
          collect2: error: ld returned 1 exit status

Here's the full output while using serverless.com (lambda) deployment tool:

$ time npx serverless invoke local -f reads --path tests/rest/apigw_proxy_request.json
Serverless: Building native Rust reads func...
   Compiling reads v0.1.0 (/code/reads)
error: linking with `cc` failed: exit code: 1
  |
  = note: "cc" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-m64" "-L" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "/code/target/lambda/release/deps/reads-c59a3e25c753bcfc.reads.2spb1sze-cgu.0.rcgu.o" "/code/target/lambda/release/deps/reads-c59a3e25c753bcfc.reads.2spb1sze-cgu.1.rcgu.o" "/code/target/lambda/release/deps/reads-c59a3e25c753bcfc.reads.2spb1sze-cgu.10.rcgu.o" "/code/target/lambda/release/deps/reads-c59a3e25c753bcfc.reads.2spb1sze-cgu.11.rcgu.o" "/code/target/lambda/release/deps/reads-c59a3e25c753bcfc.reads.2spb1sze-cgu.12.rcgu.o" "/code/target/lambda/release/deps/reads-c59a3e25c753bcfc.reads.2spb1sze-cgu.13.rcgu.o" "/code/target/lambda/release/deps/reads-c59a3e25c753bcfc.reads.2spb1sze-cgu.14.rcgu.o" "/code/target/lambda/release/deps/reads-c59a3e25c753bcfc.reads.2spb1sze-cgu.15.rcgu.o" "/code/target/lambda/release/deps/reads-c59a3e25c753bcfc.reads.2spb1sze-cgu.2.rcgu.o" "/code/target/lambda/release/deps/reads-c59a3e25c753bcfc.reads.2spb1sze-cgu.3.rcgu.o" "/code/target/lambda/release/deps/reads-c59a3e25c753bcfc.reads.2spb1sze-cgu.4.rcgu.o" "/code/target/lambda/release/deps/reads-c59a3e25c753bcfc.reads.2spb1sze-cgu.5.rcgu.o" "/code/target/lambda/release/deps/reads-c59a3e25c753bcfc.reads.2spb1sze-cgu.6.rcgu.o" "/code/target/lambda/release/deps/reads-c59a3e25c753bcfc.reads.2spb1sze-cgu.7.rcgu.o" "/code/target/lambda/release/deps/reads-c59a3e25c753bcfc.reads.2spb1sze-cgu.8.rcgu.o" "/code/target/lambda/release/deps/reads-c59a3e25c753bcfc.reads.2spb1sze-cgu.9.rcgu.o" "-o" "/code/target/lambda/release/deps/reads-c59a3e25c753bcfc" "/code/target/lambda/release/deps/reads-c59a3e25c753bcfc.q66hosipaldt9qk.rcgu.o" "-Wl,--gc-sections" "-pie" "-Wl,-zrelro" "-Wl,-znow" "-Wl,-O1" "-nodefaultlibs" "-L" "/code/target/lambda/release/deps" "-L" "/code/target/lambda/release/build/rust-htslib-799c0bade2369d9f/out" "-L" "/code/target/lambda/release/build/bzip2-sys-5dfe5490da2f86dc/out/lib" "-L" "/usr/lib64" "-L" "/usr/lib64" "-L" "/code/target/lambda/release/build/backtrace-sys-535b6335aaa7b508/out" "-L" "/code/target/lambda/release/build/ring-b23e7a0b1eabf848/out" "-L" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-Bstatic" "/code/target/lambda/release/deps/libenv_logger-1c4a8fe09d99d6ce.rlib" "/code/target/lambda/release/deps/libtermcolor-2e1d9c18e8508b2c.rlib" "/code/target/lambda/release/deps/libatty-bbfde0bac2726585.rlib" "/code/target/lambda/release/deps/libhumantime-f6da84e640158326.rlib" "/code/target/lambda/release/deps/libreads-bc643fc5b3e937ba.rlib" "/code/target/lambda/release/deps/librusoto_s3-401facb5057844c9.rlib" "/code/target/lambda/release/deps/librusoto_core-8701ac61d4c1d223.rlib" "/code/target/lambda/release/deps/librusoto_signature-71355f46ad7b8805.rlib" "/code/target/lambda/release/deps/libsha2-bc802680d23010d3.rlib" "/code/target/lambda/release/deps/libopaque_debug-21cdd034a7501d3b.rlib" "/code/target/lambda/release/deps/libfake_simd-3d9cfe582744e094.rlib" "/code/target/lambda/release/deps/libblock_buffer-6e610641b8cd296a.rlib" "/code/target/lambda/release/deps/libblock_padding-d009ce113c0dc642.rlib" "/code/target/lambda/release/deps/libbyte_tools-dead233fc5d5d69f.rlib" "/code/target/lambda/release/deps/libmd5-4e8e4fcd18b2caa4.rlib" "/code/target/lambda/release/deps/libhmac-a3181935e8f75ea0.rlib" "/code/target/lambda/release/deps/libcrypto_mac-a2628570ae054160.rlib" "/code/target/lambda/release/deps/libsubtle-0ec90516f40cfc7d.rlib" "/code/target/lambda/release/deps/libdigest-9108291b95da666e.rlib" "/code/target/lambda/release/deps/libgeneric_array-b732771d4701424f.rlib" "/code/target/lambda/release/deps/libtypenum-0e70489e57426234.rlib" "/code/target/lambda/release/deps/libhex-9bcef11d2a9d8fdb.rlib" "/code/target/lambda/release/deps/libxml-6ba16929dd6c929e.rlib" "/code/target/lambda/release/deps/librusoto_credential-a1acb71a2a976b7f.rlib" "/code/target/lambda/release/deps/libtokio_process-e8c69168fdd98383.rlib" "/code/target/lambda/release/deps/libtokio_signal-7642daf1d7fb4535.rlib" "/code/target/lambda/release/deps/libsignal_hook-66111ed85addbc1f.rlib" "/code/target/lambda/release/deps/libsignal_hook_registry-5872f68d13fa9064.rlib" "/code/target/lambda/release/deps/libarc_swap-6e85afef0db53f36.rlib" "/code/target/lambda/release/deps/libshlex-5756f7ae206641fc.rlib" "/code/target/lambda/release/deps/libdirs-fb4c40eec9c64f1a.rlib" "/code/target/lambda/release/deps/libbase64-171cf550deb21045.rlib" "/code/target/lambda/release/deps/libhyper_rustls-8296aad5217adcdf.rlib" "/code/target/lambda/release/deps/libwebpki_roots-8f2f4be287387b9d.rlib" "/code/target/lambda/release/deps/libtokio_rustls-55f078686cd20e1d.rlib" "/code/target/lambda/release/deps/librustls-49f2857ea172cf4c.rlib" "/code/target/lambda/release/deps/libwebpki-59e0ffd29d8392ec.rlib" "/code/target/lambda/release/deps/libct_logs-033a8f1d3fd33e7e.rlib" "/code/target/lambda/release/deps/libsct-39fa75e23df9971e.rlib" "/code/target/lambda/release/deps/libring-80e7720f9c681234.rlib" "/code/target/lambda/release/deps/libspin-d84e03c9d5f99498.rlib" "/code/target/lambda/release/deps/libuntrusted-3e073aea3169f18f.rlib" "/code/target/lambda/release/deps/librust_htslib-fc86895fccfb6420.rlib" "/code/target/lambda/release/deps/libitertools-b4d00f4afdf9482c.rlib" "/code/target/lambda/release/deps/libieee754-31e358538504232c.rlib" "/code/target/lambda/release/deps/liburl-bf7ca148d5801d2d.rlib" "/code/target/lambda/release/deps/libpercent_encoding-d985814ec59e21ba.rlib" "/code/target/lambda/release/deps/libidna-8324e99afecc7625.rlib" "/code/target/lambda/release/deps/libbio_types-39802ccaf566d40a.rlib" "/code/target/lambda/release/deps/libquick_error-742a8a3c66b09d99.rlib" "/code/target/lambda/release/deps/libregex-ad20702983a00147.rlib" "/code/target/lambda/release/deps/libthread_local-fbdfc22547376d59.rlib" "/code/target/lambda/release/deps/libregex_syntax-bb131d70322970c6.rlib" "/code/target/lambda/release/deps/libaho_corasick-9242d4850ca1a516.rlib" "/code/target/lambda/release/deps/liblinear_map-1f78a0ead911ef1d.rlib" "/code/target/lambda/release/deps/libsnafu-e20f471e32757143.rlib" "/code/target/lambda/release/deps/libdoc_comment-c167071d06a64bad.rlib" "/code/target/lambda/release/deps/liblzma_sys-de8d3acab2d421ca.rlib" "/code/target/lambda/release/deps/libbzip2_sys-72db201261a30650.rlib" "/code/target/lambda/release/deps/liblibz_sys-f94e16d72d2a2728.rlib" "/code/target/lambda/release/deps/libnewtype_derive-3845a05c4ec64475.rlib" "/code/target/lambda/release/deps/libcustom_derive-ef5f5fe80dbc7d21.rlib" "/code/target/lambda/release/deps/libbio_index_formats-9a0b0efdc5052ebe.rlib" "/code/target/lambda/release/deps/libnom-9a57c05958f39efc.rlib" "/code/target/lambda/release/deps/liblexical_core-57660f3fd3f13ce1.rlib" "/code/target/lambda/release/deps/libstatic_assertions-956021c138bce1d0.rlib" "/code/target/lambda/release/deps/libmemchr-0e058df740fee616.rlib" "/code/target/lambda/release/deps/liblambda_http-b53f2ef9ecee8097.rlib" "/code/target/lambda/release/deps/libserde_urlencoded-00811dfcccc77fb5.rlib" "/code/target/lambda/release/deps/liburl-415f1c5bd79fc8b4.rlib" "/code/target/lambda/release/deps/libpercent_encoding-f6efc28bdf3e3b8b.rlib" "/code/target/lambda/release/deps/libidna-67d71c0abc3a4292.rlib" "/code/target/lambda/release/deps/libunicode_normalization-0b43df8177c5285a.rlib" "/code/target/lambda/release/deps/libunicode_bidi-127b6dea613aa070.rlib" "/code/target/lambda/release/deps/libmatches-656cd79ac1f4dbf3.rlib" "/code/target/lambda/release/deps/libdtoa-472f5f3f00a7735d.rlib" "/code/target/lambda/release/deps/libbase64-2ed305ffdd3ae9ce.rlib" "/code/target/lambda/release/deps/liblambda_runtime-794a2e51151d5f82.rlib" "/code/target/lambda/release/deps/liblambda_runtime_core-6eefdfd89ee7e561.rlib" "/code/target/lambda/release/deps/liblambda_runtime_client-87bf1eb260a4eaf3.rlib" "/code/target/lambda/release/deps/liblambda_runtime_errors-d64afb19e0dbdd5c.rlib" "/code/target/lambda/release/deps/libserde_json-c8a351a1c05e9114.rlib" "/code/target/lambda/release/deps/libryu-4ee08cd2532a545c.rlib" "/code/target/lambda/release/deps/libhyper-cf93579cb192ec79.rlib" "/code/target/lambda/release/deps/libwant-15b092e4a3d6a406.rlib" "/code/target/lambda/release/deps/libtry_lock-c3ab9ae9b8d10427.rlib" "/code/target/lambda/release/deps/libtokio-13b44038747fd62f.rlib" "/code/target/lambda/release/deps/libtokio_uds-1494cc16fafcd85f.rlib" "/code/target/lambda/release/deps/libmio_uds-a5228904c2869e8b.rlib" "/code/target/lambda/release/deps/libtokio_udp-561e74841b75bbac.rlib" "/code/target/lambda/release/deps/libtokio_timer-5b97631b8eb96162.rlib" "/code/target/lambda/release/deps/libtokio_tcp-995e65fd9a2000ff.rlib" "/code/target/lambda/release/deps/libtokio_reactor-339239976facad32.rlib" "/code/target/lambda/release/deps/libtokio_sync-f485e5f095e03bec.rlib" "/code/target/lambda/release/deps/libparking_lot-159d76519bc01e87.rlib" "/code/target/lambda/release/deps/libparking_lot_core-467e99363d50e2f7.rlib" "/code/target/lambda/release/deps/libsmallvec-5b05b57becb06706.rlib" "/code/target/lambda/release/deps/liblock_api-606b6eac909b2c11.rlib" "/code/target/lambda/release/deps/libtokio_fs-6a3a93cb8fdc7c9b.rlib" "/code/target/lambda/release/deps/libtokio_threadpool-88f4ea135c22fcb3.rlib" "/code/target/lambda/release/deps/libcrossbeam_queue-55b75d73ff737e2c.rlib" "/code/target/lambda/release/deps/libcrossbeam_deque-112850f4e0e976bf.rlib" "/code/target/lambda/release/deps/libcrossbeam_epoch-fddda5ae3a49d8dd.rlib" "/code/target/lambda/release/deps/libscopeguard-fbfb32c1cb4c3e17.rlib" "/code/target/lambda/release/deps/libmemoffset-fb5d6f6521a0c490.rlib" "/code/target/lambda/release/deps/libarrayvec-a45440c67092cb7a.rlib" "/code/target/lambda/release/deps/libnodrop-bdc84fbe608ce13e.rlib" "/code/target/lambda/release/deps/libtokio_current_thread-fe92cb9121a42232.rlib" "/code/target/lambda/release/deps/libtokio_executor-fcbd3244dcc6fd85.rlib" "/code/target/lambda/release/deps/libcrossbeam_utils-a3643d7f61cb12cf.rlib" "/code/target/lambda/release/deps/liblazy_static-9927bdb5a51d8866.rlib" "/code/target/lambda/release/deps/libtokio_codec-17dc43c924dd6243.rlib" "/code/target/lambda/release/deps/libmio-7201b2b7387f7aa3.rlib" "/code/target/lambda/release/deps/libnet2-4644b054ec10f0f1.rlib" "/code/target/lambda/release/deps/libhttparse-b16dabd259454e4e.rlib" "/code/target/lambda/release/deps/libhttp_body-6aaf78b60f341d5d.rlib" "/code/target/lambda/release/deps/libtokio_buf-9d4b34bec93a1efa.rlib" "/code/target/lambda/release/deps/libh2-906f495e546f7d18.rlib" "/code/target/lambda/release/deps/libindexmap-b77f8a21b11eb0e9.rlib" "/code/target/lambda/release/deps/libstring-1ca39da16cef2a4d.rlib" "/code/target/lambda/release/deps/libslab-aca7457c7f2adc8c.rlib" "/code/target/lambda/release/deps/libtokio_io-a43ce87c775afd43.rlib" "/code/target/lambda/release/deps/liblog-5f075c2f94d34499.rlib" "/code/target/lambda/release/deps/libfutures_cpupool-0dce3024d5bb313f.rlib" "/code/target/lambda/release/deps/libnum_cpus-b351c5920ba4ec23.rlib" "/code/target/lambda/release/deps/libfutures-d6d90b26a1db9345.rlib" "/code/target/lambda/release/deps/libchrono-da675f9399653e59.rlib" "/code/target/lambda/release/deps/libserde-76418f02191bd7eb.rlib" "/code/target/lambda/release/deps/libnum_integer-daa3b551460da5a5.rlib" "/code/target/lambda/release/deps/libnum_traits-a9b752e1c8cb274d.rlib" "/code/target/lambda/release/deps/libtime-5c0346007023d7b7.rlib" "/code/target/lambda/release/deps/libfailure-5f92cd4325d7f6ec.rlib" "/code/target/lambda/release/deps/libbacktrace-45eba65ba84e9c3f.rlib" "/code/target/lambda/release/deps/libbacktrace_sys-22c3e4d94d53b8b3.rlib" "/code/target/lambda/release/deps/libcfg_if-f51889ee12a5a31e.rlib" "/code/target/lambda/release/deps/librustc_demangle-2518b9cc69d3ccce.rlib" "/code/target/lambda/release/deps/libhttp-d0f67467b0300bf9.rlib" "/code/target/lambda/release/deps/libitoa-8b90c0416aaa1224.rlib" "/code/target/lambda/release/deps/libfnv-e24d8f06a947b9e8.rlib" "/code/target/lambda/release/deps/libbytes-9b9a39b2b28a2835.rlib" "/code/target/lambda/release/deps/libeither-49105ea47ad0d133.rlib" "/code/target/lambda/release/deps/libiovec-bb59257854a157b9.rlib" "/code/target/lambda/release/deps/liblibc-cc7ff7776d163983.rlib" "/code/target/lambda/release/deps/libbyteorder-8dbacdfd1cfce70a.rlib" "-Wl,--start-group" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-fae576517123aa4e.rlib" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libpanic_unwind-a72070139220275e.rlib" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libhashbrown-093434daf7d99801.rlib" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_alloc-24daf38551b7a03b.rlib" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libbacktrace-36d70d9746402ce9.rlib" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libbacktrace_sys-7acfc843240167a8.rlib" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_demangle-eb2e0f5fe057b8b3.rlib" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libunwind-75e9ddd83715a368.rlib" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcfg_if-af51e7c6fd7d1248.rlib" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-27f2a77b2995d98c.rlib" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-ad10152c26711a1e.rlib" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_core-291bd2456cb6c9fe.rlib" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-fc6e9071307a3016.rlib" "-Wl,--end-group" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-ebe4001ded7f33e7.rlib" "-Wl,-Bdynamic" "-llzma" "-lz" "-lutil" "-lutil" "-ldl" "-lrt" "-lpthread" "-lgcc_s" "-lc" "-lm" "-lrt" "-lpthread" "-lutil" "-lutil"
  = note: /code/target/lambda/release/deps/librust_htslib-fc86895fccfb6420.rlib(rust_htslib-fc86895fccfb6420.rust_htslib.25lsx2fx-cgu.11.rcgu.o): In function `rust_htslib::bam::Reader::new::h7805fb2441edac2e':
          rust_htslib.25lsx2fx-cgu.11:(.text._ZN11rust_htslib3bam6Reader3new17h7805fb2441edac2eE+0x4c): undefined reference to `sam_hdr_read'
          /code/target/lambda/release/deps/librust_htslib-fc86895fccfb6420.rlib(rust_htslib-fc86895fccfb6420.rust_htslib.25lsx2fx-cgu.11.rcgu.o): In function `_$LT$rust_htslib..bam..Reader$u20$as$u20$core..ops..drop..Drop$GT$::drop::hc749ab5efb9aea68':
          rust_htslib.25lsx2fx-cgu.11:(.text._ZN66_$LT$rust_htslib..bam..Reader$u20$as$u20$core..ops..drop..Drop$GT$4drop17hc749ab5efb9aea68E+0x5): undefined reference to `hts_close'
          /code/target/lambda/release/deps/librust_htslib-fc86895fccfb6420.rlib(rust_htslib-fc86895fccfb6420.rust_htslib.25lsx2fx-cgu.11.rcgu.o): In function `rust_htslib::bam::hts_open::hac1f0f64c9261566':
          rust_htslib.25lsx2fx-cgu.11:(.text._ZN11rust_htslib3bam8hts_open17hac1f0f64c9261566E+0xc3): undefined reference to `hts_open'
          /code/target/lambda/release/deps/librust_htslib-fc86895fccfb6420.rlib(rust_htslib-fc86895fccfb6420.rust_htslib.25lsx2fx-cgu.11.rcgu.o): In function `_$LT$rust_htslib..bam..HeaderView$u20$as$u20$core..ops..drop..Drop$GT$::drop::h06ec3494656e04e9':
          rust_htslib.25lsx2fx-cgu.11:(.text._ZN70_$LT$rust_htslib..bam..HeaderView$u20$as$u20$core..ops..drop..Drop$GT$4drop17h06ec3494656e04e9E+0xb): undefined reference to `bam_hdr_destroy'
          collect2: error: ld returned 1 exit status


error: aborting due to previous error

error: could not compile `reads`.

To learn more, run the command again with --verbose.
Serverless: Dockerized Rust build encountered an error: undefined 1.

  Error --------------------------------------------------

  Error:
      at /Users/romanvg/dev/umccr/htsget/htsget-aws/node_modules/serverless-rust/index.js:129:15
      at Array.forEach (<anonymous>)
      at RustPlugin.build (/Users/romanvg/dev/umccr/htsget/htsget-aws/node_modules/serverless-rust/index.js:110:22)
      at /Users/romanvg/dev/umccr/htsget/htsget-aws/node_modules/serverless/lib/classes/PluginManager.js:490:55
      at tryCatcher (/Users/romanvg/dev/umccr/htsget/htsget-aws/node_modules/bluebird/js/release/util.js:16:23)
      at Object.gotValue (/Users/romanvg/dev/umccr/htsget/htsget-aws/node_modules/bluebird/js/release/reduce.js:168:18)
      at Object.gotAccum (/Users/romanvg/dev/umccr/htsget/htsget-aws/node_modules/bluebird/js/release/reduce.js:155:25)
      at Object.tryCatcher (/Users/romanvg/dev/umccr/htsget/htsget-aws/node_modules/bluebird/js/release/util.js:16:23)
      at Promise._settlePromiseFromHandler (/Users/romanvg/dev/umccr/htsget/htsget-aws/node_modules/bluebird/js/release/promise.js:547:31)
      at Promise._settlePromise (/Users/romanvg/dev/umccr/htsget/htsget-aws/node_modules/bluebird/js/release/promise.js:604:18)
      at Promise._settlePromise0 (/Users/romanvg/dev/umccr/htsget/htsget-aws/node_modules/bluebird/js/release/promise.js:649:10)
      at Promise._settlePromises (/Users/romanvg/dev/umccr/htsget/htsget-aws/node_modules/bluebird/js/release/promise.js:729:18)
      at _drainQueueStep (/Users/romanvg/dev/umccr/htsget/htsget-aws/node_modules/bluebird/js/release/async.js:93:12)
      at _drainQueue (/Users/romanvg/dev/umccr/htsget/htsget-aws/node_modules/bluebird/js/release/async.js:86:9)
      at Async._drainQueues (/Users/romanvg/dev/umccr/htsget/htsget-aws/node_modules/bluebird/js/release/async.js:102:5)
      at Immediate.Async.drainQueues [as _onImmediate] (/Users/romanvg/dev/umccr/htsget/htsget-aws/node_modules/bluebird/js/release/async.js:15:14)
      at processImmediate (internal/timers.js:439:21)
      at process.topLevelDomainCallback (domain.js:130:23)

     For debugging logs, run again after setting the "SLS_DEBUG=*" environment variable.

  Get Support --------------------------------------------
     Docs:          docs.serverless.com
     Bugs:          github.com/serverless/serverless/issues
     Issues:        forum.serverless.com

  Your Environment Information ---------------------------
     Operating System:          darwin
     Node Version:              12.14.1
     Framework Version:         1.61.3
     Plugin Version:            3.2.7
     SDK Version:               2.3.0
     Components Core Version:   1.1.2
     Components CLI Version:    1.4.0


real	0m45.065s
user	0m3.069s
sys	0m1.341s

I don't have much experience with the rust-htslib approach to cbindgen & co, but if you could provide some pointers on what you think could be wrong here, I'd be happy to investigate further.

In order to reproduce this issue, assuming you are on Linux:

$ git clone https://github.com/brainstorm/htsget-aws && cd htsget-aws
$ brew install filosottile/musl-cross/musl-cross (if using OSX and brew)
$ rustup target add x86_64-unknown-linux-musl
$ npm i
$ time npx serverless invoke local -f reads --path tests/rest/apigw_proxy_request.json

If something fails on your machine, please take a look at this blogpost which contains additional directions should you encounter some difficulty with the cross-compiling toolchain.

I would like to fix this before jumping into another rust implementation to just read the first BGZF block of a BAM file to get the header :-S

@luizirber
Copy link

I think there are issues even before you get to rust-htslib:

  • I don't have a macOS machine around, but cargo build --release --target x86_64-unknown-linux-musl worked on Linux for me, so I don't think that is the issue.
  • You're installing musl-cross with homebrew, but serverless is running the build commands inside Docker...
  • ... and I don't think the docker image is using musl at all. It is showing 1.39.0-x86_64-unknown-linux-gnu as rust version and target in your logs, and it's also what I see in my machine.

@brainstorm
Copy link
Member Author

True to all you said. I'll be digging on the serverless-rust and docker container innards soon, closing as INVALID ;)

@brainstorm
Copy link
Member Author

brainstorm commented May 21, 2020

Back to this after a while... that was a not-so-INVALID issue, the path for MUSL + C bindings in Rust is pretty tortuous these days, see:

https://users.rust-lang.org/t/solved-musl-cross-compilation-how-to-reuse-curl-sys-libz-sys-bzip2-sys-lzma-sys-and-other-dependency-includes-under-target/42817/8

I'm planning to file an idea/pre-RFC to Rust internals, inquiring why *-sys crates objects and headers are not properly reused... in my view that'll solve a lot of problems and even get rid of the need to use Cross/Docker at all, but I might be missing several intricacies of Cargo, we'll see. Track this PR #193 if anybody requires more gory details.

Thanks @luizirber for reviewing this back in January ;)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants