fix!: POSIXct without timezone and naive time handling

[eitsupi]

Fix #875

The latest version (0.14.1):

Sys.setenv(TZ = "America/New_York")

rdf = data.frame(
  non_exsitent_time = as.POSIXct("2020-03-08 02:00:00"),
  ambiguous_time = as.POSIXct("2020-11-01 01:00:00")
)
rdf
#>     non_exsitent_time      ambiguous_time
#> 1 2020-03-08 01:00:00 2020-11-01 01:00:00

pldf = polars::as_polars_df(rdf)
pldf
#> shape: (1, 2)
#> ┌─────────────────────┬─────────────────────┐
#> │ non_exsitent_time   ┆ ambiguous_time      │
#> │ ---                 ┆ ---                 │
#> │ datetime[ms]        ┆ datetime[ms]        │
#> ╞═════════════════════╪═════════════════════╡
#> │ 2020-03-08 06:00:00 ┆ 2020-11-01 05:00:00 │
#> └─────────────────────┴─────────────────────┘

as.data.frame(pldf)
#>     non_exsitent_time      ambiguous_time
#> 1 2020-03-08 01:00:00 2020-11-01 01:00:00

This PR:

Sys.setenv(TZ = "America/New_York")

rdf = data.frame(
  non_exsitent_time = as.POSIXct("2020-03-08 02:00:00"),
  ambiguous_time = as.POSIXct("2020-11-01 01:00:00")
)
rdf
#>     non_exsitent_time      ambiguous_time
#> 1 2020-03-08 01:00:00 2020-11-01 01:00:00

pldf = polars::as_polars_df(rdf)
pldf
#> shape: (1, 2)
#> ┌─────────────────────┬─────────────────────┐
#> │ non_exsitent_time   ┆ ambiguous_time      │
#> │ ---                 ┆ ---                 │
#> │ datetime[ms]        ┆ datetime[ms]        │
#> ╞═════════════════════╪═════════════════════╡
#> │ 2020-03-08 01:00:00 ┆ 2020-11-01 01:00:00 │
#> └─────────────────────┴─────────────────────┘

In the above example, an error occurs when converting Polars to R.
Perhaps something similar to the Int64 type conversion option should be added to the arguments and global options.

[eitsupi]

Perhaps the POSIXct to Polars conversion also needs to be modified.
Currently (0.14.1) I am seeing some rather strange behavior when specifying a time that does not exist.

Sys.setenv(TZ = "America/New_York")
datetime = as.POSIXct("2020-03-08 02:00:00")
datetime
#> [1] "2020-03-08 01:00:00 EST"

clock::as_naive_time(datetime)
#> <naive_time<second>[1]>
#> [1] "2020-03-08T01:00:00"

s = polars::as_polars_series(datetime)
s
#> polars Series: shape: (1,)
#> Series: '' [datetime[ms]]
#> [
#>  2020-03-08 06:00:00
#> ]

as.vector(s)
#> [1] "2020-03-08 01:00:00 EST"

^{Created on 2024-03-02 with reprex v2.1.0}

Perhaps it needs to behave like clock::as_naive_time() here.

[eitsupi]

Perhaps it needs to behave like clock::as_naive_time() here.

I think the following process is needed.

polars::as_polars_series(datetime)$dt$replace_time_zone(
  "UTC"
)$dt$convert_time_zone(
  Sys.timezone()
)$dt$replace_time_zone(NULL)

[etiennebacher]

Note that for the example in your first post, this is what I get with this PR:

Sys.setenv(TZ = "America/New_York")

rdf = data.frame(
  non_exsitent_time = as.POSIXct("2020-03-08 02:00:00"),
  ambiguous_time = as.POSIXct("2020-11-01 01:00:00")
)
rdf

  non_exsitent_time      ambiguous_time
1        2020-03-08 2020-11-01 01:00:00

pldf = polars::as_polars_df(rdf)
pldf

shape: (1, 2)
┌─────────────────────┬─────────────────────┐
│ non_exsitent_time   ┆ ambiguous_time      │
│ ---                 ┆ ---                 │
│ datetime[ms]        ┆ datetime[ms]        │
╞═════════════════════╪═════════════════════╡
│ 2020-03-08 00:00:00 ┆ 2020-11-01 01:00:00 │
└─────────────────────┴─────────────────────┘

The column ambiguous_time is the same but not non_exsitent_time.

[etiennebacher]

The error message following as.data.frame(pldf) says Please use `ambiguous` to tell how it should be localized but it's not clear to me where I should specify this arg

[eitsupi]

The column ambiguous_time is the same but not non_exsitent_time.

Interesting. Perhaps the environment changes the display of non-existent time?

[eitsupi]

The error message following as.data.frame(pldf) says Please use `ambiguous` to tell how it should be localized but it's not clear to me where I should specify this arg

As I mentioned in the PR description, I would like to do this in a follow-up PR. It is also related to updates in other parts, for example, there is no option in as.vector() to instruct how to convert Int64.

[etiennebacher]

Perhaps the environment changes the display of non-existent time?

How can I test that? The timezone is manually set so what else could change the display?

[eitsupi]

How can I test that? The timezone is manually set so what else could change the display?

It may not be possible to get consistent results on different platforms as this behavior varies from platform to platform.
https://stackoverflow.com/questions/39526527/is-there-a-reliable-way-to-detect-posixlt-objects-representing-a-time-which-does

[etiennebacher]

Thanks, just one last comment to clarify why this datetime doesn't exist

[eitsupi]

Thanks for the detailed review!