Datetime localization support

[ap-Codkelden]

Description

By using Pandas, I can get localized strings from dates with pandas.Series.dt.strftime, e.g.:

import polars as pl
import pandas as pd
from datetime import datetime
import locale

locale.setlocale(locale.LC_TIME, "uk_UA.UTF-8")

date_ = pd.Series([datetime(2023, 11, 8, 19, 15, 7)])
date_.dt.strftime("%A, %d %B %Y")  # Wednesday, 08 November 2023 in Ukrainian

# 0    середа, 08 листопада 2023
# dtype: object

However, this feature is currently absent in Polars (as of ver. 0.19.12).

Certainly, I can use lambda or a regular custom function to do that,

date_1 = pl.Series("", [datetime(2023, 11, 8, 19, 15, 7)])
date_1.map_elements(lambda x: x.strftime("%A, %d %B %Y"))

but I'd like to use a more native mechanism.

[cmdlineluser]

chrono does support this via features = ["unstable-locales"]

use chrono::prelude::*;

fn main() {
    let dt = Utc.with_ymd_and_hms(2023, 11, 8, 19, 5, 7).unwrap();
    let out = dt
        .format_localized("%A, %d %B %Y", Locale::uk_UA)
        .to_string();

    dbg!(out);
}

[src/main.rs:7] out = "середа, 08 листопада 2023"

I'm not sure if enabling it has been previously discussed?

polars/Cargo.toml

Line 34 in 30e4471

chrono = { version = "0.4.31", default-features = false, features = ["std"] }

[MarcoGorelli]

I think this is a reasonable request

In the meantime, rather than map_elements (sloooow 🐌 ), maybe something like

In [43]: def localize_ukraine(expr):
    ...:     return (
    ...:         expr
    ...:         .str.replace('Wednesday', 'середа')
    ...:         .str.replace('November', 'листопада')  # put other replacements below
    ...:     )
    ...:

In [44]: localize_ukraine(date_1.dt.strftime('%A, %d %B %Y'))
Out[44]:
shape: (1,)
Series: '' [str]
[
        "середа, 08 листопада 2023"
]

would work better for you

[deanm0000]

Instead of trying to do string replacements to work around the locale issue, an alternate approach would be to use pyarrow's strftime which I put in the SO answer

[ap-Codkelden]

@cmdlineluser I comlpiled py-polars with the
chrono = { version = "0.4.31", default-features = false, features = ["std", "unstable-locales"] }
in the polars/Cargo.toml, but nothing changed with dt.strftime(...) behavior.

Where am I wrong?

[cmdlineluser]

Building chrono with that feature gives access to the .format_localized() function.

A way to call .format_localized() from Polars would also need to be added.

[MarcoGorelli]

Hi @ap-Codkelden - I'm adding this to polars-xdt, as I think it's a bit out-of-scope for the main Polars library

In [1]: import polars as pl
   ...: import polars_xdt  # noqa: F401
   ...: 
   ...: from datetime import datetime
   ...: 
   ...: df = pl.DataFrame(
   ...:     {
   ...:         "date_col": [datetime(2024, 8, 24), datetime(2024, 10, 1)],
   ...:     }
   ...: )
   ...: df.with_columns(result=pl.col("date_col").xdt.format_localized("%A, %d %B %Y", 'uk_UA'))
Out[1]: 
shape: (2, 2)
┌─────────────────────┬──────────────────────────┐
│ date_col            ┆ result                   │
│ ---                 ┆ ---                      │
│ datetime[μs]        ┆ str                      │
╞═════════════════════╪══════════════════════════╡
│ 2024-08-24 00:00:00 ┆ субота, 24 серпня 2024   │
│ 2024-10-01 00:00:00 ┆ вівторок, 01 жовтня 2024 │
└─────────────────────┴──────────────────────────┘

Just fixing up a couple of things, hoping to make a new release with this today

[MarcoGorelli]

Released, polars-xdt now supports this

Closing then, but thanks for the request! And thanks @cmdlineluser for finding format_localized in Chrono