Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

to.daily producing duplicates #277

Closed
ggrothendieck opened this issue Oct 14, 2018 · 3 comments
Closed

to.daily producing duplicates #277

ggrothendieck opened this issue Oct 14, 2018 · 3 comments
Labels
bug
Milestone

Comments

@ggrothendieck
Copy link

@ggrothendieck ggrothendieck commented Oct 14, 2018

The code below results in duplicate date values in zd for 2015-10-23. to.daily should not have produced duplicates.

Lines <- "localminute,dataid,meter_value
2015-10-03 09:51:53,6578,157806
2015-10-13 13:41:49,6578,158086
2015-10-13 17:41:49,6578,158088
2015-10-13 20:40:49,6578,158100
2015-10-13 22:19:50,6578,158102
2015-10-15 18:41:48,6578,158170
2015-10-16 19:51:49,6578,158180
2015-10-18 22:16:48,6578,158202
2015-10-19 01:14:47,6578,158204
2015-10-19 01:20:47,6578,158204
2015-10-19 02:45:48,6578,158204
2015-10-19 17:49:48,6578,158210
2015-10-19 19:56:48,6578,158212
2015-10-19 20:14:47,6578,158212
2015-10-19 20:56:48,6578,158212
2015-10-19 23:34:47,6578,158212
2015-10-20 02:21:47,6578,158214
2015-10-20 21:46:48,6578,158222
2015-10-20 21:51:47,6578,158222
2015-10-20 22:19:47,6578,158222
2015-10-21 20:24:46,6578,158232
2015-10-22 21:50:46,6578,158262
2015-10-23 05:50:46,6578,158276
2015-10-23 07:16:46,6578,158276
2015-10-23 11:01:45,6578,158278
2015-10-23 11:11:46,6578,158278
2015-10-23 17:01:46,6578,158290
2015-10-24 04:31:46,6578,158300
2015-10-24 10:46:45,6578,158304
2015-10-24 23:19:46,6578,158314
2015-10-24 23:41:45,6578,158314"

library(xts)
library(zoo)
z <- read.csv.zoo(text = Lines)
zd <- to.daily(z)

giving:

> zd
           z.Open z.High z.Low z.Close
2015-10-03   6578   6578  6578    6578
2015-10-14   6578   6578  6578    6578
2015-10-15   6578   6578  6578    6578
2015-10-16   6578   6578  6578    6578
2015-10-19   6578   6578  6578    6578
2015-10-20   6578   6578  6578    6578
2015-10-21   6578   6578  6578    6578
2015-10-22   6578   6578  6578    6578
2015-10-23   6578   6578  6578    6578
2015-10-23   6578   6578  6578    6578
2015-10-25   6578   6578  6578    6578
@joshuaulrich
Copy link
Owner

@joshuaulrich joshuaulrich commented Oct 16, 2018

Thanks for the report! I can replicate. This is ultimately caused by xts:::.drop.time() not accounting for the local timezone when converting to Date.

@joshuaulrich joshuaulrich added the bug label Oct 16, 2018
@joshuaulrich
Copy link
Owner

@joshuaulrich joshuaulrich commented Oct 20, 2018

This also looks like it may be a duplicate of #53.

joshuaulrich added a commit that referenced this issue Nov 3, 2018
Both the indexClass<-.xts() call and the indexTZ<-.xts() calls set the
index timezone to UTC, but does not change the actual index values.
The call to 'index(x) <- index(x)' updates the values, but the dates
will calculated as if the local time is UTC, not the local time of the
user-specified xts object. This can cause the output to be different
than if the local timezone is used.

Convert the index to Date using the local timezone before updating the
index timezone.

Fixes #53, #277.
@joshuaulrich
Copy link
Owner

@joshuaulrich joshuaulrich commented Nov 3, 2018

Closing because I verified this is a duplicate, and is now fixed.

joshuaulrich added a commit that referenced this issue Nov 4, 2018
as.Date.POSIXct() does not respect the tzone attribute of the POSIXct
object, so the Date is a UTC date. Convert to POSIXlt first, because
as.Date.POSIXlt() does use the timezone and therefore returns a local
date.

See #53, see #277.
joshuaulrich added a commit that referenced this issue May 12, 2019
Conflicts in R/toperiod.R related to:
 - #53 time-based to Date-based index causes duplicate index values)
 - #277 to.daily producing duplicates

Conflicts in R/xts.methods.R related to:
 - #193 time-of-day subset performance

Fixes #245.
@joshuaulrich joshuaulrich added this to the 0.11-2 milestone Jul 4, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants