Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Date of null value cannot be inserted with using pandas.dataframe.to_sql #21

Closed
wuhuanyan opened this issue Jul 12, 2018 · 0 comments
Closed

Comments

@wuhuanyan
Copy link

If I use pandas.dataframe.to_sql, insert the empty date value in dataframe to Clickhouse,There will be a mistake here.

code

def main():
    d2 = pd.DataFrame({
        'date': [pd.to_datetime('2018-01-01'), None, pd.to_datetime('2018-01-03')]
    })
    print(d2)
    """
            date
    0 2018-01-01
    1        NaT
    2 2018-01-03
    """
    ch_engine = sqla.create_engine(Const.CH_URI)
    d2.to_sql(name='t_date', con=ch_engine,
              if_exists='append', index=False)

error message

.............
  File "/anaconda3/envs/study/lib/python3.6/site-packages/clickhouse_driver/columns/service.py", line 80, in write_column
    column.write_data(items, buf)
  File "/anaconda3/envs/study/lib/python3.6/site-packages/clickhouse_driver/columns/base.py", line 79, in write_data
    self._write_data(items, buf)
  File "/anaconda3/envs/study/lib/python3.6/site-packages/clickhouse_driver/columns/base.py", line 82, in _write_data
    prepared = self.prepare_items(items)
  File "/anaconda3/envs/study/lib/python3.6/site-packages/clickhouse_driver/columns/base.py", line 69, in prepare_items
    x = before_write(x)
  File "/anaconda3/envs/study/lib/python3.6/site-packages/clickhouse_driver/columns/datetimecolumn.py", line 26, in before_write_item
    if value.tzinfo is None:
AttributeError: 'NoneType' object has no attribute 'tzinfo'

so,I edit 'clickhouse_driver/columns/datetimecolumn.py',such as:

    def before_write_item(self, value):
        if self.timezone:
            # Set server's timezone for offset-naive datetime.
            # if value.tzinfo is None:
            #     value = self.timezone.localize(value)
            # value = value.astimezone(utc)
            # return int(timegm(value.timetuple()))
            """
            my code
            """
            if hasattr(value, 'tzinfo'):
                if value.tzinfo is None:
                    value = self.timezone.localize(value)
                value = value.astimezone(utc)
                return int(timegm(value.timetuple()))
            else:
                return 0
        else:
            # If datetime is offset-aware use it's timezone.
            if value.tzinfo is not None:
                value = value.astimezone(utc)
                return int(timegm(value.timetuple()))

            return int(mktime(value.timetuple()))

Null date can also be inserted into Clickhouse (0000-00-00 00:00:00).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant