Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pandas: Support datetime64[ns] dataframe index as designated timestamp. #35

Open
amunra opened this issue Feb 6, 2023 · 3 comments
Open

Comments

@amunra
Copy link
Collaborator

amunra commented Feb 6, 2023

It's common to use a datetime64[ns] df.index in Pandas when dealing with timeseries.
In such case our API should just be:

buffer.dataframe(df, table_name="some_name")

This means changing the default logic of the at argument to also accept two new singleton types:

buffer.dataframe(df, ..., at=Server)  # timestamps are set by the server -- the current default.
buffer.dataframe(df, ..., at=Index)  # Use the index.

The new behaviour for the at=None default would be to:

  • Use at=Index logic if the index column is a datetime64,
  • or use at=Server logic if the index is any other type.

Whilst technically a breaking change, the feature change is minor and is very unlikely to affect any of our users, thus this feature will not require a new major software release number.

@amunra
Copy link
Collaborator Author

amunra commented Feb 6, 2023

What is also neat is that since we can already pluck the table name off the index, we can end up in a situation where we can fully ingest a pandas dataframe with no additional args.

I.e.:

buffer.dataframe(df)

Short and sweet :-)

@amunra amunra changed the title Dataframe: Support datetime64[ns] as designated timestamp. Pandas: Support datetime64[ns] dataframe index as designated timestamp. Feb 6, 2023
@javier
Copy link

javier commented Feb 6, 2023

For context, the pandas docs on indices for time-series. I see them very often specially when doing downsampling or filling gaps in your data (equivalent to questdb's FILL)

@javier
Copy link

javier commented Feb 6, 2023

Also, when using a named index, I would expect the designated timestamp column in QuestDB to retain the name. Otherwise when I have to use the column in a select statement it is confusing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants