Milliseconds data detection and plotting

[danielkrizian]

Detect the dataset resolution and automagically switch between the parser modes.

JavaScript millisecond parsing here (WIP) : danielkrizian/dygraphs#12

Other resources:
https://groups.google.com/forum/#!topic/dygraphs-users/qAKQndJvkRE
http://dygraphs.com/data.html

[TonyDIRL]

Hi,

Apologies for the delay. After guidance from the dygraph users group there is a working jfiddle of using millisecond plotting:
http://jsfiddle.net/kaliatech/DfWNz/3/

So to accomplish this in R, I think we'd first need to convert POSIXlt objects to milliseconds since epoch and enclose within "#! !#". As a reference here is an example of millisecond plotting in highcharts:

require(rCharts);require(xts)
npoints<-500
Data_xts=xts(1:npoints, as.POSIXlt(1366039619, tz="GMT", origin="1970-01-01") + round(runif(npoints,1,50)))
names(Data_xts)<-'RandomData'
DataDF=data.frame(Date=index(Data_xts),coredata(Data_xts$RandomData))
DataDF$Date=paste("#!", as.numeric(strptime((DataDF$Date), format="%Y-%m-%d %H:%M:%OS"))*1000, "!#")

HCGraph <- Highcharts$new()
HCGraph$yAxis(list(title = list(text = 'Data1') ) )
HCGraph$series(data = toJSONArray2(DataDF[,c('Date','RandomData')], json = F, names = F),enableMouseTracking=TRUE,shadow=FALSE,name = "Data1",type = "line")
HCGraph$xAxis(type = "datetime"); HCGraph$chart(zoomType = "x")
HCGraph$plotOptions(column=list(animation=FALSE),shadow=FALSE,line=list(marker=list(enabled=FALSE)));
HCGraph

Then based on the jfiddle we need to ass custom axisLabelFormatter and valueFormatter functions.

myChart$setOpts(axisLabelFormatter = "#! function (d, gran) {
                  return Dygraph.zeropad(d.getHours()) + ":" + Dygraph.zeropad(d.getMinutes()) + ":" + Dygraph.zeropad(d.getSeconds()) + "." + Dygraph.zeropad(d.getMilliseconds());
              } !#")
myChart$setOpts(valueFormatter = "#! function (ms) {
                  var d = new Date(ms);
                  return Dygraph.zeropad(d.getHours()) + ":" + Dygraph.zeropad(d.getMinutes()) + ":" + Dygraph.zeropad(d.getSeconds()) + "." + Dygraph.zeropad(d.getMilliseconds());
              }!#")

Unforuntately, I was unable to get this to work - could you take a look please.

Cheers

[danielkrizian]

After I edited your comment with adding ```r blocks around the code, I noticed that " in `":"` interfere with `"#! !#"`. Replace inside `"` quotes with equivalent `'`.

Thanks for providing example data and jsfiddle. Reproducible code of what you have tried so far with Dygraph would be also nice.

I was able to produce something like this:

npoints<-500
# second data:
# Data_xts=xts(1:npoints, as.POSIXlt(1366039619, tz="GMT", origin="1970-01-01") + round(runif(npoints,1,50)))
# millisecond data:
Data_xts=xts(1:npoints, as.POSIXlt(1366039619, tz="GMT", origin="1970-01-01") + rnorm(npoints, 500000, 250000)/1000000)
names(Data_xts)<-'RandomData'

myChart = dygraph(Data_xts)
myChart$setOpts(axes=list(
  x=list(
    ticker="#! Dygraph.dateTicker !#",
    axisLabelFormatter = "#! function (d, gran) {
                              return Dygraph.zeropad(d.getHours()) + ':' + 
                                     Dygraph.zeropad(d.getMinutes()) + ':' + 
                                     Dygraph.zeropad(d.getSeconds()) + '.' + 
                                     Dygraph.zeropad(d.getMilliseconds());
                             } !#",
    valueFormatter = "#! function (ms) {
                          var d = new Date(ms);
                          return Dygraph.zeropad(d.getHours()) + ':' + 
                          Dygraph.zeropad(d.getMinutes()) + ':' + 
                          Dygraph.zeropad(d.getSeconds()) + '.' + 
                          Dygraph.zeropad(d.getMilliseconds());
                         } !#"
    )
  )
)

myChart

Second data:

Millisecond data:

Note that time is off by exactly 1 hour, but that's problem in general with daylight saving offset I think and evident with daily frequency data too.
Let me know how does that work for your case. Once this use case is mature, I will implement automatic formatting based on data frequency detection in the Dygraph ReferenceClass.

[TonyDIRL]

Thanks for catching the double versus single quotes parsing error.
Apologies, I provided second rather than millisecond level data yesterday, updated as:

Data_xts=xts(1:npoints, as.POSIXlt(1366039619, tz="GMT", origin="1970-01-01") + rnorm(npoints, 500000, 250000)/1000000)

I confirm your suggested changes work with the exception of how the data is passed to dygraph.
Passing the data.frame directly:

 myChart = dygraph(DataDF)

Result in POSIXlt complaining about the format - so I assume it is still trying to obtain a date object. Where passing in the xts object directly worked with issue

 myChart = dygraph(Data_xts)

Out of interest how do you plan to automatically detect data frequency?

[danielkrizian]

Yes sorry, was typing that fast, I meant myChart = dygraph(Data_xts). Thanks for the millisecond update, edited my prev comment accordingly.

Out of interest how do you plan to automatically detect data frequency?

Don't know yet, focusing on other parts for the time being. Any suggestions, contributions welcome! Feel free to fork, develop and I will make sure to answer your specific questions, review and accept your pull request :)

[TonyDIRL]

Sure, I'll fork and have a go.

[hallegue]

Hi I tried your code in order to display milliseconds in a dygraph plot but when I run this code:

myChart$setOpts(axes=list(
    x=list(
        ticker="#! Dygraph.dateTicker !#",
        axisLabelFormatter = "#! function (d, gran) {
        return Dygraph.zeropad(d.getHours()) + ':' + 
        Dygraph.zeropad(d.getMinutes()) + ':' + 
        Dygraph.zeropad(d.getSeconds()) + '.' + 
        Dygraph.zeropad(d.getMilliseconds());
        } !#",
        valueFormatter = "#! function (ms) {
        var d = new Date(ms);
        return Dygraph.zeropad(d.getHours()) + ':' + 
        Dygraph.zeropad(d.getMinutes()) + ':' + 
        Dygraph.zeropad(d.getSeconds()) + '.' + 
        Dygraph.zeropad(d.getMilliseconds());
        } !#"
    )
)
)

I got this error:

Error: attempt to apply non-function

Do you know what to fix it?

[danielkrizian]

@Haycen please note that the active development has moved to rstudio/dygraphs repository and they track the sub-second handling issue.