Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(memtable): handle castable inputs from date-like strings to date #9245

Open
1 task done
NickCrews opened this issue May 24, 2024 · 3 comments
Open
1 task done
Labels
feature Features or general enhancements

Comments

@NickCrews
Copy link
Contributor

What happened?

import ibis

# errors
ibis.memtable({"d": ["1970-01-01"]}, schema={"d": "date"}).execute()
# works
# ibis.memtable({"d": ["1970-01-01"]}).cast({"d": "date"}).execute()

What version of ibis are you using?

main

What backend(s) are you using, if any?

duckdb

Relevant log output

No response

Code of Conduct

  • I agree to follow this project's Code of Conduct
@NickCrews NickCrews added the bug Incorrect behavior inside of ibis label May 24, 2024
@gforsyth
Copy link
Member

gforsyth commented May 24, 2024

Ok, so pyarrow won't handle casts from stringy dates to date-types, but DuckDB will.

there's pyarrow.compute.strptime, but we'd have to think about how many date format strings to try before bailing out

@NickCrews
Copy link
Contributor Author

I'd be fine with literally just "yyyy-mm-dd" format. I am only running into this in testing, when constructing literals. If someone else has real data they are working with, I think it wouldn't be too much to expect them to either format it that way, or convert it to a numpy datetime or a python datetime or something less ambiguous.

Related to this, but I think worth thinking about, is ideally I want to be able to create all ibis datatypes without needing to import anything. eg using the same logic as used in ibis.date(str), etc:

import ibis

ibis.memtable(
    {
        "uuid": ["9ff13914-a718-48b3-a746-5114cff95d56"],
        # "uuid": [1234],  # would be awesome to specify as an int too
        # "date": ["1970-01-01"],  # currently errors
        # "time": ["12:34:56"],  # currently errors
        # "timestamp": ["2023-01-02T03:04:05"],  # currently errors
    },
    schema={
        "uuid": "uuid",
        # "date": "date",
        # "time": "time",
        # "timestamp": "timestamp",
    },
).cache().execute()

This would make testing much easier.

@cpcloud
Copy link
Member

cpcloud commented Jun 2, 2024

Removing the bug label. This isn't broken functionality, it's a feature request for specific new behavior.

@cpcloud cpcloud added bug Incorrect behavior inside of ibis feature Features or general enhancements and removed bug Incorrect behavior inside of ibis labels Jun 2, 2024
@cpcloud cpcloud changed the title bug: memtable(..."1970-01-01"..., schema="date") errors feat(memtable): handle castable inputs from date-like strings to date Jun 2, 2024
@cpcloud cpcloud removed the bug Incorrect behavior inside of ibis label Jun 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature Features or general enhancements
Projects
Status: backlog
Development

No branches or pull requests

3 participants