Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Linux Perf Support + Causal Profiling Updates (#276)
* causal backtrace updates - fix initial causal sampling period value * causal delay updates - tweak handling of sleep_for_overhead * Fix experiment global scaling for prog pts - results in drastically improved predictions * pthread_mutex_gotcha updates - disable all wrappers during causal profiling * validate-causal-json.py updates - support decimal stddev - fix setting stddev from command-line * causal perform_experiment_impl update - handle start failing because finalizing * deprecate causal::component::sample_rate - appears to not help at all * Rework sample info * Increase causal unwind_depth - use OMNITRACE_MAX_UNWIND_DEPTH * validate-causal-json updates - min experiments - exclude reporting predictions with less than X experiments at a given speedup - percent samples - only print samples within X% of the peak (default: 95%) * Update timemory submodule - extensions to sampling for signals delivered via non-timer method - e.g. via HW counter overflow * dwarf_entry::operator< updates - sort via file * causal profiling docs updates - info about backends - info about installing/enabling perf * config updates: causal backend - CausalBackend enum - OMNITRACE_CAUSAL_BACKEND: perf, timer, auto - omnitrace-causal option: --backend * debug update - use spin_mutex instead of std::mutex * address_range::contains update - range from 0-100 contains range from 10-100 but was returning false because high was == 100 not < 100 * symbol::operator< update - handle load address differences * sampling updates (non-causal) - update get_timer to get_trigger + dynamic_cast * container::static_vector updates - support construction from container::c_array - update_size private member func for handling atomic m_size * Move perf files - moved library/causal/perf.{hpp,cpp} to library/perf.{hpp,cpp} * causal example update - created impl.hpp (forward decls) - renamed {cpu,rng}_func_impl to {cpu,rng}_impl_func - only create two threads which run N iterations instead of two threads each iteration * Update timemory submodule - updates to unwind::processed_entry - updates to procfs::maps * Updated causal documentation - fixed line numbers changed by modifications to causal example * omnitrace-causal exe updates - set OMNITRACE_THREAD_POOL_SIZE to zero by default * core/containers updates - static_vector: provide data() member function - c_array pop_front() and pop_back() member functions * core: config and argparse updates + perf - core/perf.{hpp,cpp} - forward decl of enums - config-related capabilities - argparse: --sample-overflow - renamed some config functions - e.g. get_sampling_cpu_freq -> get_sampling_cputime_freq - added config settings related to overflow sampling via perf - added timer_sampling and overflow_sampling categories * Update timemory submodule - sampling allocator flushing * binary updates - lookup_ipaddr_entry - use bfd_find_nearest_line instead of bfd_find_nearest_line_discriminator - discriminators are not used - explicit instantiations of inlined_symbol::serialize * Bump VERSION to 1.10.0 * sampling and perf updates - support overflow sampling via Linux Perf - update perf namespace - update perf::perf_event - update record ctor: pointer instead of const ref - update open member func: return optional string - add m_batch_size member variable - sampling updates - support overflow sampling - flush allocators - increase buffer size from 1024 to 2048 - restructure post-processing in light of perf overflow supports - improve offload memory usage only load buffers for thread - load_offload_buffer(tid) uses thread-specific filepos - component updates - backtrace_metrics::operator-= - backtrace_metrics::operator- - backtrace::sample does not record for overflow signal - callchain: perf overflow sample * core updates - component::sampling_percent does not report self + uses_percent_units * causal updates - tweak get_line_info - overloads for set_current_selection (uint64_t, c_array, std::array) - delay - use sampling::pause/sampling::resume - experiment - experiment::sample derives from unwind::processed_entry - experiment::samples is vector instead of set - fixed samples - overloads for is_selected (uint64_t, c_array, std::array) - scaling factor defaults to 100 instead of 50 - serialize updates follow change to experiment::sample - modify algorithm for increasing/decreasing experiment length - sample_data - use map<uintptr, uint64_t> instead of set<sample_data> - get_samples returns vector<sample_data> instead of set<sample_data> - sampling - support overflow via Linux Perf - update causal_offload_buffer - flush sampling allocator - backtrace - overflow component * libomnitrace-dl updates - handle dl::InstrumentMode::PythonProfile * testing updates (causal) - causal line 155 -> causal line 100 - causal line 165 -> causal line 110 * formatting * exit_gotcha updates - exit_info for abort() - message about non-zero exit code * testing updates - fail regex for causal tests - validate-causal-json: >= min_experiments instead of > min_experiments - handle OMNITRACE_DEBUG_SETTINGS in omnitrace_write_test_config * causal sampling updates - add new lines where appropriate * causal data updates - reorder diagnostic info when experiment fails to start * binary updates - symbol address range from address to address + symsize + 1 - add 1 based on debug info * causal data updates - sample_selection wait_ns defaults to 1,000 instead of 10,000 - sample_selection wait scaled by iteration number - save_line_info_impl verbosity - print latest_eligible_pc when experiment does not start * causal sampling + component updates - perf backend disables component::backtrace - ensure get_sampling_(realtime|cputime|overflow)_signal do not malloc * causal: remove period stats * validate-causal-json update - fix --help * causal data updates - improve eligible pc history reporting when experiment fails to start * causal data updates - fix compute_eligible_lines_impl - eligible address ranges returning too many ranges - occasionally, overwrite all *true* eligible address ranges * causal data updates - reduce scoped ranges to symbol ranges - is_eligible_address() returns true contains (not just coarse) - revert some sample_selection behavior * binary address_multirange updates - make coarse_range private - fix operator+=(pair<coarse, uintptr_t>) * causal example update - fix nsync to default to once per iteration * binary analysis updates - tweak header file includes * causal updates - remove factoring in sleep_for_overhead - invoke delay::process() even if experiment is not active * causal data updates - update latest_eligible_pc structure * update omnitrace-install.py.in - fix support for fedora - /etc/os-release does not have ID_LIKE - fallback to RHEL 8.7 if version not specified * update omnitrace-install.py.in - fix support for debian - /etc/os-release does not have ID_LIKE - version mapping * Update documentation - update docs on installation * causal data and experiment updates - data: reset_sample_selection * causal set_current_selection debugging - debug messages for failed e2e runs * causal data and backtrace component updates - data: set_current_selection returns the number of eligible addresses added - backtrace: if cputime signal has selected zero IPs > 5x, then realtime signal starts contributing call-stacks * core library updates - move config::parse_numeric_range to utility namespace - add core/utility.cpp - support range:increment, e.g. 5-25:10 expands to '5 15 25' instead of '5 10 15 20 25' * omnitrace-causal update - end-to-end expands all speedups - support range:increment in speedups * causal backtrace updates - remove select_ival (realtime signal always contributes when select_count == 0) * containers: static_vector update - explicit c_array constructor - explicit std::array constructor * causal data updates - remove set_current_selection(uint64_t) - remove set_current_selection(std::array) - sample_selection increase default wait time - report eligible PC candidates - move reset_sample_selection to perform_experiment_impl - decrease latest_eligible_pc array size - set_current_selection does not guard for experiment::active * core debug updates - OMNITRACE_PRINT_COLOR macros * causal data updates - tweak to experiment never started message * causal gotcha updates - remove unused code * critical trace updates - remove unused code * omnitrace-causal - OMNITRACE_LAUNCHER * causal data updates - don't fail on end-to-end + omnitrace-causal * causal backtrace updates - reintroduce select_ival behavior * causal data updates - tweak verbose messages about number of PC candidates * core mproc updates - utilities for waiting on child PID and diagnosing status - omnitrace::mproc::wait_pid - omnitrace::mproc::diagnose_status * omnitrace-run updates - support --fork argument for executing via fork in current process + execvpe on child instead of execvpe in current process * omnitrace-causal updates - wait_pid and diagnose_status just call equivalent functions in omnitrace::mproc * ubuntu-focal workflow update - attempt to launch ubuntu-focal-codecov job with CAP_SYS_ADMIN and use perf backend * tests reorg and updates - remove binary-rewrite-sampling and runtime-instrument-sampling tests - rename *-preload tests (which use omnitrace-sample exe) to *-sampling - split tests/CMakeLists.txt into several tests/omnitrace-<category>-tests.cmake files - tweak to causal-both-omni-func test - add args: -n 2 -b timer * update validate-causal-json.py - better reasoning info for adjusting tolerance - always apply tolerance adjustments in CI mode * causal e2e tests update - add label "causal-e2e" label - tweak params - old: 80 12 432525 500000000 - new: 80 50 432525 100000000 - disable processor affinity for slow-func/line-100 tests - artificially inflates some speedups with perf * unblocking_gotcha updates - overload operator() according to gotcha function index * blocking_gotcha updates - overload operator() according to gotcha function index - fix bug where potentially post block functors (e.g. pthread_mutex_trylock) throw error if lock is not acquired. * parse_numeric_range update - support unordered_set * config update - OMNITRACE_DEBUG_{TIDS,PIDS} use parse_numeric_range
- Loading branch information