Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

to.daily producing duplicates #277

Closed
ggrothendieck opened this issue Oct 14, 2018 · 3 comments
Closed

to.daily producing duplicates #277

ggrothendieck opened this issue Oct 14, 2018 · 3 comments
Labels
bug
Milestone
0.11-2

Comments

@ggrothendieck
Copy link

The code below results in duplicate date values in zd for 2015-10-23. to.daily should not have produced duplicates.

Lines <- "localminute,dataid,meter_value
2015-10-03 09:51:53,6578,157806
2015-10-13 13:41:49,6578,158086
2015-10-13 17:41:49,6578,158088
2015-10-13 20:40:49,6578,158100
2015-10-13 22:19:50,6578,158102
2015-10-15 18:41:48,6578,158170
2015-10-16 19:51:49,6578,158180
2015-10-18 22:16:48,6578,158202
2015-10-19 01:14:47,6578,158204
2015-10-19 01:20:47,6578,158204
2015-10-19 02:45:48,6578,158204
2015-10-19 17:49:48,6578,158210
2015-10-19 19:56:48,6578,158212
2015-10-19 20:14:47,6578,158212
2015-10-19 20:56:48,6578,158212
2015-10-19 23:34:47,6578,158212
2015-10-20 02:21:47,6578,158214
2015-10-20 21:46:48,6578,158222
2015-10-20 21:51:47,6578,158222
2015-10-20 22:19:47,6578,158222
2015-10-21 20:24:46,6578,158232
2015-10-22 21:50:46,6578,158262
2015-10-23 05:50:46,6578,158276
2015-10-23 07:16:46,6578,158276
2015-10-23 11:01:45,6578,158278
2015-10-23 11:11:46,6578,158278
2015-10-23 17:01:46,6578,158290
2015-10-24 04:31:46,6578,158300
2015-10-24 10:46:45,6578,158304
2015-10-24 23:19:46,6578,158314
2015-10-24 23:41:45,6578,158314"

library(xts)
library(zoo)
z <- read.csv.zoo(text = Lines)
zd <- to.daily(z)

giving:

> zd
           z.Open z.High z.Low z.Close
2015-10-03   6578   6578  6578    6578
2015-10-14   6578   6578  6578    6578
2015-10-15   6578   6578  6578    6578
2015-10-16   6578   6578  6578    6578
2015-10-19   6578   6578  6578    6578
2015-10-20   6578   6578  6578    6578
2015-10-21   6578   6578  6578    6578
2015-10-22   6578   6578  6578    6578
2015-10-23   6578   6578  6578    6578
2015-10-23   6578   6578  6578    6578
2015-10-25   6578   6578  6578    6578
@joshuaulrich
Copy link
Owner

Thanks for the report! I can replicate. This is ultimately caused by xts:::.drop.time() not accounting for the local timezone when converting to Date.

@joshuaulrich
Copy link
Owner

This also looks like it may be a duplicate of #53.

joshuaulrich added a commit that referenced this issue Nov 3, 2018
@joshuaulrich
Convert index to Date using index timezone
bfcbe50 
Both the indexClass<-.xts() call and the indexTZ<-.xts() calls set the
index timezone to UTC, but does not change the actual index values.
The call to 'index(x) <- index(x)' updates the values, but the dates
will calculated as if the local time is UTC, not the local time of the
user-specified xts object. This can cause the output to be different
than if the local timezone is used.

Convert the index to Date using the local timezone before updating the
index timezone.

Fixes #53, #277.
@joshuaulrich
Copy link
Owner

Closing because I verified this is a duplicate, and is now fixed.

joshuaulrich added a commit that referenced this issue Nov 4, 2018
@joshuaulrich
Convert to POSIXlt before Date
cbacd5d 
as.Date.POSIXct() does not respect the tzone attribute of the POSIXct
object, so the Date is a UTC date. Convert to POSIXlt first, because
as.Date.POSIXlt() does use the timezone and therefore returns a local
date.

See #53, see #277.
joshuaulrich added a commit that referenced this issue May 12, 2019
@joshuaulrich
Merge branch '245-move-index-attr'
ea22577 
Conflicts in R/toperiod.R related to:
 - #53 time-based to Date-based index causes duplicate index values)
 - #277 to.daily producing duplicates

Conflicts in R/xts.methods.R related to:
 - #193 time-of-day subset performance

Fixes #245.
@joshuaulrich joshuaulrich added this to the 0.11-2 milestone Jul 4, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
0.11-2
Development

No branches or pull requests
2 participants
@joshuaulrich @ggrothendieck