Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Timestamp which dtype is Float64 are treated with timezone #335

Open
zhouwuji opened this issue Sep 7, 2022 · 0 comments
Open

Timestamp which dtype is Float64 are treated with timezone #335

zhouwuji opened this issue Sep 7, 2022 · 0 comments

Comments

@zhouwuji
Copy link

zhouwuji commented Sep 7, 2022

Describe the bug
When I insert a df with a timestamp into a table, if the dtype of that timestamp column is Float64, it is considered to be local_time, and then the value inserted into the clickhouse produces a wrong offset, which is offset for me by 8 hours.

To Reproduce

from clickhouse_driver import Client
import pandas as pd

port = 9000
client = Client(
    host='192.168.1.160',
    settings={
        'use_numpy': True},
    port=9000,
    user='default',
    password='')

sql1 = '''

drop table if exists dt;
'''
sql2 = '''
CREATE TABLE dt
(
    `timestamp` DateTime64(9),
    `event_id` UInt8
)
ENGINE = TinyLog;
'''
client.execute(sql1)
client.execute(sql2)
data = [
    (1659099540588999936, 1),
    (1659099540588999936, 2),
    # ('2022-07-29 20:59:00.588999936', 3),
    (1659099540588999936.0, 2),

]

df = pd.DataFrame(data)
df.columns= ['timestamp', 'event_id']
df.info()
res = client.insert_dataframe(f'INSERT INTO dt VALUES', df)

df2 = client.query_dataframe('SELECT *,toUnixTimestamp64Nano(timestamp) FROM dt;')
print(df2)

output:
                   timestamp  event_id  toUnixTimestamp64Nano_timestamp_
0 2022-07-29 12:59:00.588998         1               1659070740588998912
1 2022-07-29 12:59:00.588998         2               1659070740588998912
2 2022-07-29 12:59:00.588998         2               1659070740588998912

Expected behavior
I hope the value of timestamp do not minus 8hours.

Apparently, it treated my UTC time as UTC+8

                   timestamp  event_id  toUnixTimestamp64Nano_timestamp_
0 2022-07-29 20:59:00.588999         1               1659099540588999936
1 2022-07-29 20:59:00.588999         2               1659099540588999936
2 2022-07-29 20:59:00.588999         2               1659099540588999936

(1659099540588999936-1659070740588998912)/1e9/3600
Out[28]: 8.000000000284444

Versions

  • numpy==1.19.0
  • pandas==1.0.5
  • clickhouse-driver==0.2.2
  • ClickHouse server version==21.8.4.51
  • Python version==3.7
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant