[BUG] Fixed problems with generation parquet files (#93) #1
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Owner
kozlov-alexey
commented
Jul 18, 2019
kozlov-alexey
commented
- enabled tests with parquet files problem
- fixed Spark data generation
- new functions to generate test-data 'on-the-fly'
* enabled tests with parquet files problem * fixed Spark data generation * new functions to generate test-data 'on-the-fly'
kozlov-alexey
added a commit
that referenced
this pull request
Sep 7, 2020
kozlov-alexey
added a commit
that referenced
this pull request
Sep 22, 2020
* Changing csv_reader_py impl to return df from objmode Motivation: returning Tuple of columns read from csv file with pyarrow csv reader from objmode and further calling init_dataframe ctor to create native DF turned out to be inneficient in sense of LLVM IR size and compilation time. With this PR we now rely on DF unboxing and return py DF from objmode. * Capture dtype dict instead of building in objmode * Applying comments #1
kozlov-alexey
added a commit
that referenced
this pull request
Jan 18, 2021
kozlov-alexey
added a commit
that referenced
this pull request
Jan 19, 2021
kozlov-alexey
added a commit
that referenced
this pull request
Jan 27, 2021
kozlov-alexey
added a commit
that referenced
this pull request
Feb 1, 2021
…elPython#950) * Adds Int64Index type and updates Series and DF methods to use it Motivation: as part of the work on supporting common pandas indexes a new type (Int64IndexType) representing pandas.Int64Index is added. Boxing/unboxing of Series and DataFrames as well as common numpy-like functions are changed accordingly to handle it. * Fixing DateTime tests and PEP remarks * Fixing review comments #1
kozlov-alexey
added a commit
that referenced
this pull request
Feb 24, 2021
* Adds Int64Index type and updates Series and DF methods to use it (IntelPython#950) * Adds Int64Index type and updates Series and DF methods to use it Motivation: as part of the work on supporting common pandas indexes a new type (Int64IndexType) representing pandas.Int64Index is added. Boxing/unboxing of Series and DataFrames as well as common numpy-like functions are changed accordingly to handle it. * Fixing DateTime tests and PEP remarks * Fixing review comments #1 * Move to Numba 0.52 (IntelPython#939) * Taking numba from master * Moving to Numba 0.52 commit 3182540b127268ace11cf4042cd87f044875d9fa Author: Kozlov, Alexey <alexey.kozlov@intel.com> Date: Wed Oct 21 19:49:58 2020 +0300 Cleaning up before squash commit 895668116542fe3057f73fcb276c441cbde66747 Author: Kozlov, Alexey <alexey.kozlov@intel.com> Date: Tue Oct 13 17:31:34 2020 +0300 Workaround for set from str_arr problem * Fixing correct NUMBA_VERSION * Remove intel/label/beta channel from Azure CI builds * Move to pandas=1.2.0 (IntelPython#959) * Move to pandas=1.2.0 Motivation: use latest versions of dependencies. * More failed tests are fixed * Fixing doc build * Fixing bug in stability of mergesort impl for StringArray (IntelPython#961) Motivation: for StringArray type legacy implementation of stable sort computed result when sorting with ascending=False by reversing the result of argsorting with ascending=True, which produces wrong order in groups of elements with the same value. Implemented solution adds new function argument 'ascening' and uses it when calling native function impl via serial stable_sort.
kozlov-alexey
added a commit
that referenced
this pull request
May 5, 2021
kozlov-alexey
added a commit
that referenced
this pull request
May 18, 2021
…hon#972) * Initial version of ConcurrentDict container via TBB hashmap Motivation: SDC relies on typed.Dict implementation in many core pandas algorithms, and it doesn't support concurrent read/writes. To fill this gap we add ConcurrentDict type which will be used if threading layer is TBB. * Fixing PEP and updating failing import * Fixing builds, warnings and complying to C++11 syntax * Fixing PEP and review comments #1 * Fixing remarks #2 * Applying remarks #3
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.