This repository was archived by the owner on Feb 2, 2024. It is now read-only.
Merge from master 2021_w5 #956
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Adding debug decorator to print Numba compile stats This adds a decorator to print Numba compilation stats for a decorated function and all nested functions (literally all compiled overloads). Filtering can be made using decorator arguments. NOTE: this may show a bit lesser times than compile time calculated using common approach via first_run_exec_time - second_run_exec_time. * Adding tests for compile time log format * Remove check for CPUDispatcher to work with numba DPPL
Motivation: before this change column names were passed to DF ctor as arguments of LiteralString types (each name of it's own type), which seems to add to linear dependency of LLVM IR size and hence impact DF ctor compile time. Since this information is saved into DF type itself and can be captured in any of DF methods on typing it's proposed to remove columns from DF model struct as redundant.
Motivation: init_dataframe was implemented via Numba intrinsic taking *args, which seems to generate redundant extractvalue/insertvalue LLVM instructions, producing quadratic IR when number of DF columns grows and affecting total compilation time of function that create large DFs. This PR replaces singe init_dataframe with multiple functions basing on number of columns in a DF which are generated at compile time, thus avoiding use of *args.
Motivation: df[idx] when idx is a slice produces DataFrame with the same internal structure as original one (only index type may change). Hence it can be copied and filled with column[idx] slices, which improves compilation time.
* Fix segfault in NRT_dealloc for str_arr getitem Motivation: legacy SDC implementation copied Numba internals (MemInfo struct and API functions), which if changed on the Numba side (as it's done in IntelPython/Numba where external_allocator is added) may cause segfaults. This PR removes duplication of Numba internals and moves hstr extension to NRT API functions * Updating conda-recipe with build requirement on Numba
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.