New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

to.daily producing duplicates #277

Closed
ggrothendieck opened this Issue Oct 14, 2018 · 3 comments

Comments

Projects
None yet
2 participants
@ggrothendieck

ggrothendieck commented Oct 14, 2018

The code below results in duplicate date values in zd for 2015-10-23. to.daily should not have produced duplicates.

Lines <- "localminute,dataid,meter_value
2015-10-03 09:51:53,6578,157806
2015-10-13 13:41:49,6578,158086
2015-10-13 17:41:49,6578,158088
2015-10-13 20:40:49,6578,158100
2015-10-13 22:19:50,6578,158102
2015-10-15 18:41:48,6578,158170
2015-10-16 19:51:49,6578,158180
2015-10-18 22:16:48,6578,158202
2015-10-19 01:14:47,6578,158204
2015-10-19 01:20:47,6578,158204
2015-10-19 02:45:48,6578,158204
2015-10-19 17:49:48,6578,158210
2015-10-19 19:56:48,6578,158212
2015-10-19 20:14:47,6578,158212
2015-10-19 20:56:48,6578,158212
2015-10-19 23:34:47,6578,158212
2015-10-20 02:21:47,6578,158214
2015-10-20 21:46:48,6578,158222
2015-10-20 21:51:47,6578,158222
2015-10-20 22:19:47,6578,158222
2015-10-21 20:24:46,6578,158232
2015-10-22 21:50:46,6578,158262
2015-10-23 05:50:46,6578,158276
2015-10-23 07:16:46,6578,158276
2015-10-23 11:01:45,6578,158278
2015-10-23 11:11:46,6578,158278
2015-10-23 17:01:46,6578,158290
2015-10-24 04:31:46,6578,158300
2015-10-24 10:46:45,6578,158304
2015-10-24 23:19:46,6578,158314
2015-10-24 23:41:45,6578,158314"

library(xts)
library(zoo)
z <- read.csv.zoo(text = Lines)
zd <- to.daily(z)

giving:

> zd
           z.Open z.High z.Low z.Close
2015-10-03   6578   6578  6578    6578
2015-10-14   6578   6578  6578    6578
2015-10-15   6578   6578  6578    6578
2015-10-16   6578   6578  6578    6578
2015-10-19   6578   6578  6578    6578
2015-10-20   6578   6578  6578    6578
2015-10-21   6578   6578  6578    6578
2015-10-22   6578   6578  6578    6578
2015-10-23   6578   6578  6578    6578
2015-10-23   6578   6578  6578    6578
2015-10-25   6578   6578  6578    6578
@joshuaulrich

This comment has been minimized.

Owner

joshuaulrich commented Oct 16, 2018

Thanks for the report! I can replicate. This is ultimately caused by xts:::.drop.time() not accounting for the local timezone when converting to Date.

@joshuaulrich joshuaulrich added the bug label Oct 16, 2018

@joshuaulrich

This comment has been minimized.

Owner

joshuaulrich commented Oct 20, 2018

This also looks like it may be a duplicate of #53.

joshuaulrich added a commit that referenced this issue Nov 3, 2018

Convert index to Date using index timezone
Both the indexClass<-.xts() call and the indexTZ<-.xts() calls set the
index timezone to UTC, but does not change the actual index values.
The call to 'index(x) <- index(x)' updates the values, but the dates
will calculated as if the local time is UTC, not the local time of the
user-specified xts object. This can cause the output to be different
than if the local timezone is used.

Convert the index to Date using the local timezone before updating the
index timezone.

Fixes #53, #277.
@joshuaulrich

This comment has been minimized.

Owner

joshuaulrich commented Nov 3, 2018

Closing because I verified this is a duplicate, and is now fixed.

joshuaulrich added a commit that referenced this issue Nov 4, 2018

Convert to POSIXlt before Date
as.Date.POSIXct() does not respect the tzone attribute of the POSIXct
object, so the Date is a UTC date. Convert to POSIXlt first, because
as.Date.POSIXlt() does use the timezone and therefore returns a local
date.

See #53, see #277.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment