Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Series.set() results in NotImplementedType #15287

Open
2 tasks done
MarcusJellinghaus opened this issue Mar 25, 2024 · 4 comments
Open
2 tasks done

Series.set() results in NotImplementedType #15287

MarcusJellinghaus opened this issue Mar 25, 2024 · 4 comments
Labels
bug Something isn't working needs triage Awaiting prioritization by a maintainer python Related to Python Polars

Comments

@MarcusJellinghaus
Copy link

Checks

  • I have checked that this issue has not already been reported.
  • I have confirmed this bug exists on the latest version of Polars.

Reproducible example

import polars as pl
import datetime

import os
os.environ["POLARS_VERBOSE"] = "1"

pl_series_datetime = pl.Series("date", [datetime.date(1751, 1, 1), datetime.date(1754, 1, 1)])
pl_series_datetime_sql_compliant = pl_series_datetime.set(    pl_series_datetime.dt.year() < 1753, pl.Null)
pl2 = pl_series_datetime_sql_compliant.abs()

results in NotImplementedType

Log output

Traceback (most recent call last):
  File "c:\Users\Marcus\Documents\Development\Polars_bug\bug_test.py", line 9, in <module>
    pl2 = pl_series_datetime_sql_compliant.abs()
AttributeError: 'NotImplementedType' object has no attribute 'abs'

Issue description

pl_series_datetime.set( pl_series_datetime.dt.year() < 1753, pl.Null) should return a series

Expected behavior

The result should match

expected_result = pl.Series("date", [None, datetime.date(1754, 1, 1)])
print(expected_result)

shape: (2,)
Series: 'date' [date]
[
        null
        1754-01-01
]

Installed versions

--------Version info---------
Polars:               0.20.16
Index type:           UInt32
Platform:             Windows-10-10.0.19041-SP0
Python:               3.9.0 (tags/v3.9.0:9cf6752, Oct  5 2020, 15:34:40) [MSC v.1927 64 bit (AMD64)]

----Optional dependencies----
adbc_driver_manager:  <not installed>
cloudpickle:          <not installed>
connectorx:           <not installed>
deltalake:            <not installed>
fastexcel:            <not installed>
fsspec:               <not installed>
gevent:               <not installed>
hvplot:               <not installed>
matplotlib:           <not installed>
numpy:                <not installed>
openpyxl:             <not installed>
pandas:               <not installed>
pyarrow:              <not installed>
pydantic:             <not installed>
pyiceberg:            <not installed>
pyxlsb:               <not installed>
sqlalchemy:           <not installed>
xlsx2csv:             <not installed>
xlsxwriter:           <not installed>
@MarcusJellinghaus MarcusJellinghaus added bug Something isn't working needs triage Awaiting prioritization by a maintainer python Related to Python Polars labels Mar 25, 2024
@MarcusJellinghaus
Copy link
Author

Or is it an issue with the code?

@cmdlineluser
Copy link
Contributor

(pl.Null in the code is "wrong" but not the problem.)

It just seems .set() is not yet supported for Date/Datetime Series.

pl_series_datetime.set(pl_series_datetime.dt.year() < 1753, None)
# NotImplemented

If we test Int:

pl_series_datetime.cast(int).set(pl_series_datetime.dt.year() < 1753, None)
# shape: (2,)
# Series: 'date' [i64]
# [
#	null
#	-78892
# ]

@MarcusJellinghaus
Copy link
Author

Thank you for the clarification :-)

Would it make sense to support .set() for Date/DateTime Series?
As an enhancement idea?

@reswqa
Copy link
Collaborator

reswqa commented Mar 26, 2024

You can shift the paradigm to when-then-otherwise as set is a little anti-pattern, and will block optimizations like predicate pushdown...

pl_series_datetime = pl.Series("date", [datetime.date(1751, 1, 1), datetime.date(1754, 1, 1)])
pl.select(pl.when(pl_series_datetime.dt.year() < 1753).then(None).otherwise(pl_series_datetime))
shape: (2, 1)
┌────────────┐
│ literal    │
│ ---        │
│ date       │
╞════════════╡
│ null       │
│ 1754-01-01 │
└────────────┘

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working needs triage Awaiting prioritization by a maintainer python Related to Python Polars
Projects
None yet
Development

No branches or pull requests

3 participants