-
-
Notifications
You must be signed in to change notification settings - Fork 213
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix client.insert_dataframe() for tables with tuple columns with acceptance of numpy.ndarray
s
#426
fix client.insert_dataframe() for tables with tuple columns with acceptance of numpy.ndarray
s
#426
Conversation
@xzkostyan , does it look better now? |
numpy.ndarray
s
@xzkostyan , the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good job.
Let's keep non-numpy/numpy things separated.
CHECK_NUMPY_TYPES = False | ||
|
||
|
||
def _check_sequence_to_be_an_expected_iterable(seq): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Numpy/pandas helpers should be located in clickhouse_driver/numpy/helpers.py
.
I'd keep two different versions of chunks: one for non-numpy and another one for numpy.
column_values = dataframe[column].values | ||
for idx, col_vals in enumerate(column_values): | ||
if isinstance(col_vals, dict): | ||
column_values[idx] = tuple(col_vals.values()) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What about speed and memory consumption here? Did you test the new code with large dataframes?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This part is to be refined later.
9bf3ad8
to
7f0cc87
Compare
from tests.numpy.testcase import NumpyBaseTestCase | ||
|
||
|
||
@skipIf(not PANDAS_IMPORTED, reason="pandas cannot be imported") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please see how simple numpy/pandas tests are implemented: tests/numpy/test_generic.py. Skipping logic (@skipIf
) is hidden under the hood. Variable PANDAS_IMPORTED
is redundant in this case.
fixes
reconsiders the PR fix
client.insert_dataframe()
for tuple columns in tables #425Packages:
pytest.mark.parametrize
)Checklist:
flake8
and fix issues;pytest
no tests failed. See the dev docs.