New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Daily time series with zeros #11

Open
lmkirvan opened this Issue Sep 5, 2017 · 6 comments

Comments

Projects
None yet
3 participants
@lmkirvan

lmkirvan commented Sep 5, 2017

I have an issue that I'm having trouble debugging, I'm not convinced its a bug, but thought that I should post it here anyway. I have a fair number (10+) of daily time series that I'm using as control series. Some of the time-series contain 0 entries when daily. When I run the model using daily time series I get the following error message:

> impact <- CausalImpact(
+   final_series, 
+   pre.period = first_last(pre_index),
+   post.period = first_last(post_index),
+   model.args = list(nseasons = 7),
+   alpha = .2
+ )
Error in data.frame(y.model, cum.y.model, point.pred, cum.pred) : 
  arguments imply differing number of rows: 979, 1459

I've tried dropping out control time series one by one without luck.

When I convert the same time series to weekly, most (but not all) of the zero entries are removed and the model runs without any problems.

I'm at a bit of a loss as to why this error is occurring. I'm using the latest version of the package.

> packageVersion("CausalImpact")
[1] ‘1.2.1’

@jbao

This comment has been minimized.

jbao commented Sep 6, 2017

Do you happen to have a reproducible example?

@lmkirvan

This comment has been minimized.

lmkirvan commented Sep 7, 2017

I'll try to post one tomorrow. Thanks!

@jbao

This comment has been minimized.

jbao commented Oct 5, 2017

Actually I encountered the same problem as @lmkirvan , see the reproducible example below

library(zoo)
library(CausalImpact)

ts <- zoo(c(5L, 5L, 2L, 2L, 2L, 2L, 3L, 1L, 1L, 2L, 0L, 0L, 0L, 0L, 2L, 
  0L, 2L, 3L, 2L, 5L, 4L, 7L, 6L, 4L, 9L, 6L, 2L, 3L, 3L, 0L, 5L, 
  1L, 4L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 1L, 2L, 4L, 6L, 1L, 3L, 
  5L, 2L, 3L, 1L, 4L, 2L, 2L, 2L, 0L, 1L, 2L, 0L, 1L, 1L, 0L, 0L, 
  1L, 1L, 1L, 1L, 3L, 2L, 0L, 1L, 2L, 0L, 0L, 1L, 3L, 3L, 0L, 2L, 
  0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 1L, 1L, 2L, 2L, 2L, 
  1L, 0L, 2L, 1L, 0L, 2L, 2L, 6L, 1L, 5L, 1L, 0L, 0L, 1L, 0L, 0L, 
  4L, 0L, 4L, 7L, 4L, 6L, 9L, 4L, 7L, 3L, 6L),
  structure(c(1505304000, 1505307600, 1505311200, 1505314800, 1505318400, 
          1505322000, 1505325600, 1505329200, 1505332800, 1505336400, 1505340000, 
          1505343600, 1505347200, 1505350800, 1505354400, 1505358000, 1505361600, 
          1505365200, 1505368800, 1505372400, 1505376000, 1505379600, 1505383200, 
          1505386800, 1505390400, 1505394000, 1505397600, 1505401200, 1505404800, 
          1505408400, 1505412000, 1505415600, 1505419200, 1505422800, 1505426400, 
          1505430000, 1505433600, 1505437200, 1505440800, 1505444400, 1505448000, 
          1505451600, 1505455200, 1505458800, 1505462400, 1505466000, 1505469600, 
          1505473200, 1505476800, 1505480400, 1505484000, 1505487600, 1505491200, 
          1505494800, 1505498400, 1505502000, 1505505600, 1505512800, 1505516400, 
          1505520000, 1505523600, 1505527200, 1505530800, 1505534400, 1505538000, 
          1505541600, 1505545200, 1505548800, 1505552400, 1505556000, 1505559600, 
          1505563200, 1505566800, 1505570400, 1505574000, 1505577600, 1505581200, 
          1505584800, 1505588400, 1505592000, 1505595600, 1505599200, 1505602800, 
          1505606400, 1505610000, 1505613600, 1505617200, 1505620800, 1505624400, 
          1505628000, 1505631600, 1505635200, 1505638800, 1505642400, 1505646000, 
          1505649600, 1505653200, 1505656800, 1505660400, 1505664000, 1505667600, 
          1505671200, 1505674800, 1505678400, 1505682000, 1505685600, 1505689200, 
          1505692800, 1505696400, 1505700000, 1505703600, 1505707200, 1505710800, 
          1505714400, 1505718000, 1505721600, 1505725200, 1505728800, 1505732400, 
          1505736000, 1505739600, 1505743200), class = c("POSIXct", "POSIXt"
          ), tzone = "GMT")
)

pre.period <- structure(c(1505304000, 1505732400), class = c("POSIXct", "POSIXt"))
post.period <- structure(c(1505736000, 1505743200), class = c("POSIXct", "POSIXt"))
impact <- CausalImpact(ts, pre.period, post.period, model.args = list(niter = 5000, nseasons = 24))

I'm not really sure if the issue is caused by the 0s, though.

@lmkirvan

This comment has been minimized.

lmkirvan commented Oct 5, 2017

Thanks for posting, I've been meaning to get around to recreating the error.

@alhauser

This comment has been minimized.

Collaborator

alhauser commented Oct 10, 2017

Thanks for sharing the reproducible example.

The problem is caused by the fact that the times are irregular:

diff(time(ts))

shows the granularity is mostly hourly, but there is one 2-hour gap.

No fix ready yet (working on it), but as a workaround, you can regularize the time series before fitting the model:

times <- seq(start(ts), end(ts), by = "hour")
ts.regularized <- merge(ts, zoo(, times), all = TRUE)
diff(time(ts.regularized))

Now that the granularity is always 1 hour (having value NA at the time point that was missing before), the model fit does not fail anymore:

impact <- CausalImpact(ts.regularized, pre.period, post.period, 
                       model.args = list(niter = 5000, nseasons = 24))
@jbao

This comment has been minimized.

jbao commented Oct 11, 2017

Hi @alhauser , thanks, that's indeed the case!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment