Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failed to profile rust application #91

Closed
aregng opened this issue Sep 19, 2022 · 5 comments
Closed

Failed to profile rust application #91

aregng opened this issue Sep 19, 2022 · 5 comments

Comments

@aregng
Copy link

aregng commented Sep 19, 2022

Failed to profile rust application due to error

thread '<unnamed>' panicked at 'not implemented: 'aligned_alloc' is unimplemented!', preload/src/api.rs:907:5
stack backtrace:
   0:     0x7f9661783cea - std::backtrace_rs::backtrace::libunwind::trace::h972caad916e73545
                               at /rustc/d68e7ebc38cb42b8b237392b28045edeec761503/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   1:     0x7f9661783cea - std::backtrace_rs::backtrace::trace_unsynchronized::he59049878fe5a05d
                               at /rustc/d68e7ebc38cb42b8b237392b28045edeec761503/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   2:     0x7f9661783cea - std::sys_common::backtrace::_print_fmt::he4a91f9bcfad9b40
                               at /rustc/d68e7ebc38cb42b8b237392b28045edeec761503/library/std/src/sys_common/backtrace.rs:66:5
   3:     0x7f9661783cea - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h51433dc001920472
                               at /rustc/d68e7ebc38cb42b8b237392b28045edeec761503/library/std/src/sys_common/backtrace.rs:45:22
   4:     0x7f966172d10c - core::fmt::write::hc9dbd37d69b2c204
                               at /rustc/d68e7ebc38cb42b8b237392b28045edeec761503/library/core/src/fmt/mod.rs:1198:17
   5:     0x7f9661762b14 - std::io::Write::write_fmt::h6b2550ce8adb9e04
                               at /rustc/d68e7ebc38cb42b8b237392b28045edeec761503/library/std/src/io/mod.rs:1672:15
   6:     0x7f9661784af5 - std::sys_common::backtrace::_print::h006829bd22a5a4ee
                               at /rustc/d68e7ebc38cb42b8b237392b28045edeec761503/library/std/src/sys_common/backtrace.rs:48:5
   7:     0x7f9661784af5 - std::sys_common::backtrace::print::h0f4d319136ab4456
                               at /rustc/d68e7ebc38cb42b8b237392b28045edeec761503/library/std/src/sys_common/backtrace.rs:35:9
   8:     0x7f9661784af5 - std::panicking::default_hook::{{closure}}::h5b3cdff51fbe7401
                               at /rustc/d68e7ebc38cb42b8b237392b28045edeec761503/library/std/src/panicking.rs:295:22
   9:     0x7f9661785075 - std::panicking::default_hook::hdc1d8baf28b4ffd7
                               at /rustc/d68e7ebc38cb42b8b237392b28045edeec761503/library/std/src/panicking.rs:314:9
  10:     0x7f9661785075 - std::panicking::rust_panic_with_hook::h80e138cc00203db9
                               at /rustc/d68e7ebc38cb42b8b237392b28045edeec761503/library/std/src/panicking.rs:698:17
  11:     0x7f96617f9221 - nwind_ret_trampoline_start
                               at /home/user/.cargo/git/checkouts/not-perf-af1a46759dd83df9/18bd8d3/nwind/src/arch/amd64_trampoline.s:17
  12:                0x0 - <unknown>

used command

LD_PRELOAD=./libbytehound.so <rust_application> <args>

Tried with local build from the latest master, and with pre-built versions
Target OS: Ubuntu 20.04

Please let me know if any option is missing to run tool properly, or rust applications are not support for now.
Thanks in advance.

@koute
Copy link
Owner

koute commented Sep 20, 2022

This is expected. Your application (or one of your dependencies) is using the aligned_alloc function, which is currently unimplemented in Bytehound.

Can you rerun your application with MEMORY_PROFILER_USE_SHADOW_STACK=0 set? This won't fix it, but should make the panic backtrace show us what exactly is using aligned_alloc in your application.

@aregng
Copy link
Author

aregng commented Sep 20, 2022

Thanks for the hint.
After running with MEMORY_PROFILER_USE_SHADOW_STACK=0 it turned out that "aligned_alloc" is used by rocksdb implementation:

12:     0x7f135f75353c - std::sys_common::backtrace::__rust_end_short_backtrace::h444863965d46291e
                               at /rustc/a37499ae66ec5fc52a93d71493b78fb141c32f6b/library/std/src/sys_common/backtrace.rs:138:18
  13:     0x7f135f753511 - rust_begin_unwind
                               at /rustc/a37499ae66ec5fc52a93d71493b78fb141c32f6b/library/std/src/panicking.rs:584:5
  14:     0x7f135f6b0ea2 - core::panicking::panic_fmt::h3fce1d62b588c5f8
                               at /rustc/a37499ae66ec5fc52a93d71493b78fb141c32f6b/library/core/src/panicking.rs:142:14
  15:     0x7f135f6f2baf - aligned_alloc
                               at /workspace/supra/examples/bytehound/preload/src/api.rs:907:5
  16:     0x7f135f546bf6 - _ZnwmSt11align_val_t
  17:     0x5639119112c6 - _ZN7rocksdb6DBImpl4OpenERKNS_9DBOptionsERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERKSt6vectorINS_22ColumnFamilyDescriptorESaISD_EEPSC_IPNS_18ColumnFamilyHandleESaISJ_EEPPNS_2DBEbb
                               at /home/areg/.cargo/registry/src/github.com-1ecc6299db9ec823/librocksdb-sys-0.8.0+7.4.4/rocksdb/db/db_impl/db_impl_open.cc:1752:77
  18:     0x5639119105ad - _ZN7rocksdb2DB4OpenERKNS_9DBOptionsERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERKSt6vectorINS_22ColumnFamilyDescriptorESaISD_EEPSC_IPNS_18ColumnFamilyHandleESaISJ_EEPPS0_
                               at /home/areg/.cargo/registry/src/github.com-1ecc6299db9ec823/librocksdb-sys-0.8.0+7.4.4/rocksdb/db/db_impl/db_impl_open.cc:1613:50
  19:     0x56391182354c - rocksdb_open_column_families
                               at /home/areg/.cargo/registry/src/github.com-1ecc6299db9ec823/librocksdb-sys-0.8.0+7.4.4/rocksdb/db/c.cc:873:33
  20:     0x56391174c002 - rocksdb::db::DBCommon<T,rocksdb::db::DBWithThreadModeInner>::open_cf_raw::h3610a311cf8d9c40
                               at /home/areg/.cargo/registry/src/github.com-1ecc6299db9ec823/rocksdb-0.19.0/src/db.rs:717:42
  21:     0x56391174ccee - rocksdb::db::DBCommon<T,rocksdb::db::DBWithThreadModeInner>::open_cf_descriptors_internal::h037e47976a5d926e
                               at /home/areg/.cargo/registry/src/github.com-1ecc6299db9ec823/rocksdb-0.19.0/src/db.rs:626:18
  22:     0x56391174d93e - rocksdb::db::DBCommon<T,rocksdb::db::DBWithThreadModeInner>::open_cf::hc28b03a86869aba8
                               at /home/areg/.cargo/registry/src/github.com-1ecc6299db9ec823/rocksdb-0.19.0/src/db.rs:436:9
  23:     0x56391179ffa5 - storage::db::rocksdb::RocksDb::open::h594d1a948b126357

@koute
Copy link
Owner

koute commented Sep 20, 2022

This is very interesting; I regularly profile programs using rocksdb and I've never seen it calling aligned_alloc. From the stack trace is looks like it's called by C++'s new, so it's probably either due to the newer version of rocksdb suddenly allocating aligned memory where it previously didn't, or due to the particular version of the C++'s standard library and/or the compiler you're using.

Thanks for the report; I'll fix this.

@aregng
Copy link
Author

aregng commented Sep 20, 2022

It might be the case. rocksdb version used by application is "0.19.0".

@koute
Copy link
Owner

koute commented Oct 6, 2022

Fixed on master.

@koute koute closed this as completed Oct 6, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants