adding preprocessing of dates and values

docs/make.jl

@@ -18,8 +18,8 @@ makedocs(
        "tutorial/tsclassifier.md"
     ],
     "Manual" => Any[
-      "Value Processing" => "man/valueproc.md",
-      "Date Processing" => "man/dateproc.md",
+      "Value PreProcessing" => "man/valueproc.md",
+      "Date PreProcessing" => "man/dateproc.md",
       "Aggregation" => "man/aggregation.md",
       "Imputation" => "man/imputation.md",
       "Monotonic Detection" => "man/monotonic.md",

docs/src/man/dateproc.md

@@ -3,3 +3,41 @@ Author = "Paulito P. Palmes"
 ```
 
 # Date Preprocessing
+Extracting the Date features in a Date, Value table follows
+similar workflow with the Value preprocessing in the previous
+section. The main difference is we are only interested on the
+date corresponding to the last column of the values generated
+by the `Matrifier`. This last column contains the values before 
+the prediction happens and the dates corresponding to these
+values carry significant information based on recency compared
+to the other dates.
+
+Let us start by creating a Date,Value dataframe similar to the previous section.
+
+```@example dateifier
+using Dates
+using TSML, TSML.Utils, TSML.TSMLTypes
+using TSML.TSMLTransformers
+using DataFrames
+
+lower = DateTime(2017,1,1)
+upper = DateTime(2018,1,31)
+dat=lower:Dates.Day(1):upper |> collect
+vals = rand(length(dat))
+x = DataFrame(Date=dat,Value=vals)
+first(x,5)
+```
+
+Let us create an instance of `Dateifier` passing the size of row,
+stride, and steps ahead to predict:
+
+```@example dateifier
+mtr = Dateifier(Dict(:ahead=>24,:size=>24,:stride=>5))
+fit!(mtr,x)
+res = transform!(mtr,x)
+first(res,5)
+```
+
+The output extract automatically several date features
+such as year, month, day, hour, week, day of the week, 
+day of quarter, quarter of year.

docs/src/man/valueproc.md

@@ -29,30 +29,31 @@ upper = DateTime(2017,1,5)
 dat=lower:Dates.Hour(1):upper |> collect
 vals = 1:length(dat)
 x = DataFrame(Date=dat,Value=vals)
-first(x,5)
+last(x,5)
 ```
 
 Let us create an instance of Matrifier passing the size of row,
 stride, and steps ahead to predict:
 
 ```@example matrify
-mtr = Matrifier(Dict(:ahead=>24,:size=>24,:stride=>5))
+mtr = Matrifier(Dict(:ahead=>6,:size=>6,:stride=>3))
 fit!(mtr,x)
 res = transform!(mtr,x)
+first(res,5)
 ```
 
 In this example, we have hourly values. We indicated in the 
 `Matrifier` to generate a matrix where the size of each row
-is 24 hours, steps ahead for prediction is 24 hours and the
-stride of 5 hours.
+is 6 hours, steps ahead for prediction is 6 hours and the
+stride of 5 hours. There are 7 columns because the last column
+indicates the value indicated by the steps `ahead` argument.
 
-Let us try to make a matrix with the size of 12 hours, ahead of
-1 hour, and stride of 6 hours:
+Let us try to make a matrix with the size of 6 hours, ahead of
+2 hour, and stride of 3 hours:
 
 ```@example matrify
-mtr = Matrifier(Dict(:ahead=>1,:size=>12,:stride=>6))
+mtr = Matrifier(Dict(:ahead=>2,:size=>6,:stride=>3))
 fit!(mtr,x)
 res = transform!(mtr,x)
+first(res,5)
 ```
-
-

src/valdate.jl

@@ -93,7 +93,8 @@ function transform!(dtr::Dateifier,xx::T) where {T<:Union{Matrix,Vector,DataFram
   dt[:doq]=Dates.dayofquarter.(endpoints)
   dt[:qoy]=Dates.quarterofyear.(endpoints)
   dtr.args[:header] = names(dt)
-  convert(Matrix{Int64},dt)
+  #convert(Matrix{Int64},dt)
+  return dt
 end