Add functional data section #101

pfistfl · 2017-03-14T09:03:40Z

This is a first draft for the tutorial on functional data.
The section depends on several pull-requests that are not merged into mlr yet.

The Text and Structure can be reviewed though.

can not build without merged branch in mlr.

feature extraction, as it is not in the PR yet.

SteveBronder · 2017-03-19T18:46:53Z

src/functional_data.Rmd

@@ -0,0 +1,177 @@
+# Functional Data
+
+Functional data provides information about curves varying over a continuum, such as time.


How well known is functional data analysis? I would give a bit more of a primer on how, when, and why functional analysis is used. Like a paragraph or so

Also, for all of my comments keep in mind I am coming from the perspective of someone who knows little about FDA. I may not be your target audience

Agree that there should be just a paragraph of explanation, especially on the how and why of FDA

Done, see next commit.

I may not be your target audience ->You are exactly the target audience!

SteveBronder · 2017-03-19T18:47:32Z

src/functional_data.Rmd

+Functional data provides information about curves varying over a continuum, such as time.
+There are two commonly used approaches for analysing functional data.
+
+* Directly analyse the functional data using a [learner](&Learner.md) that is suitable for functional data on a [FDATask](&makeFDATask). Those learners have the prefixes __FDAClassif.__ and __FDARegr.__.


analyse should be analyze? I think?

SteveBronder · 2017-03-19T18:50:23Z

src/functional_data.Rmd

+* Directly analyse the functional data using a [learner](&Learner.md) that is suitable for functional data on a [FDATask](&makeFDATask). Those learners have the prefixes __FDAClassif.__ and __FDARegr.__.
+
+* Transform the task into a format suitable for standard __classification__ or __regression__ [learners](&Learner.md). 
+This is done by [extracting](functional_data.Rmd#feature-extraction) non-temporal/non-functional features from the curves.


I would define what you mean by temporal and functional features

SteveBronder · 2017-03-19T18:58:33Z

src/functional_data.Rmd

+## Creating a FDATask
+
+The first step is to get the data in the right format. [%mlr] expects a  [data.frame](&base::data.frame) which consists of the functional features and the target variable as input. The rows are considered independent, while column-ranges, such as $3:136$ in the example below correspond to a functional feature.
+After that a [FDATask](&makeFDATask) that contains the data in a well-defined format is created. [FDATasks](&makeFDATask), just like normal [Tasks](&Task.md) come in different flavours, such as [FDAClassifTask](&makeFDAClassifTask) and [FDARegrTask](&makeFDARegrTask), which can be used according to the class of the target variable.


that is ambiguous. I would explain what is the well-defined format that you use. Also try to use present tense

After coalescing the features to a data.frame, the data object needs to pass into an FDATask, where the data is parsed into ____ format. This structure of data is used inside an FDATask because ________

SteveBronder · 2017-03-19T18:59:07Z

src/functional_data.Rmd

+## Creating a FDATask
+
+The first step is to get the data in the right format. [%mlr] expects a  [data.frame](&base::data.frame) which consists of the functional features and the target variable as input. The rows are considered independent, while column-ranges, such as $3:136$ in the example below correspond to a functional feature.
+After that a [FDATask](&makeFDATask) that contains the data in a well-defined format is created. [FDATasks](&makeFDATask), just like normal [Tasks](&Task.md) come in different flavours, such as [FDAClassifTask](&makeFDAClassifTask) and [FDARegrTask](&makeFDARegrTask), which can be used according to the class of the target variable.


After that a FDATask that contains the data in a well-defined format is created. FDATasks, just like normal Tasks come in different flavours, such as FDAClassifTask and FDARegrTask, which can be used according to the class of the target variable.

Break into multiple sentences

SteveBronder · 2017-03-19T18:59:26Z

src/functional_data.Rmd

+The first step is to get the data in the right format. [%mlr] expects a  [data.frame](&base::data.frame) which consists of the functional features and the target variable as input. The rows are considered independent, while column-ranges, such as $3:136$ in the example below correspond to a functional feature.
+After that a [FDATask](&makeFDATask) that contains the data in a well-defined format is created. [FDATasks](&makeFDATask), just like normal [Tasks](&Task.md) come in different flavours, such as [FDAClassifTask](&makeFDAClassifTask) and [FDARegrTask](&makeFDARegrTask), which can be used according to the class of the target variable.
+
+In the following example, this is done for the [fuelSubset](fuelSubset.task) data from package [%FDboost].


this is ambiguous

SteveBronder · 2017-03-19T19:04:50Z

src/functional_data.Rmd

+```{r}
+## Put all values into a data.frame
+df = data.frame(fuelSubset[c("heatan", "h2o", "UVVIS", "NIR")])
+## Change row names to V1 to V367


Are you not changing the columns? Also is this necessary? Kind of odd

Ah, now I get it...
Xudong suspedted a bug when not renaming, when I wrote the tutorial.
This is now fixed.

OLD:
So the original structure is a list.
It has one list element for each feature, such as
h2o which is a vector of length NObs
NIR and UVVIS which are matrices of Size NObs x P1, NObs x P2
NIR.lambda and UVVIS.lambda which are vectors of Size P1 and P2.
-> What I do is to actually cbind those vectors, matricies. from a list of objects that do not have
equal dimensions.
If you have a better idea feel free to tell me :)

SteveBronder · 2017-03-19T19:06:13Z

src/functional_data.Rmd

+`makeLearner("<fdaclassif.<R_method_name>")` or 
+`makeLearner("<fdaregr.<R_method_name>")` depending on the target variable.
+
+Applying learners to a [FDATask](&makeFDATask) works in two ways:


List both ways here, then explain each

Done, I think

SteveBronder · 2017-03-19T20:11:52Z

src/functional_data.Rmd

+non-functional domain, e.g by extracting wavelets.
+
+The currently supported preprocessing functions are:
+* discrete wavelet transform


note: unrelated to tutorial
It would be really nice if these preprocessing methods could be used in forecasting as well. Maybe we could construct a sub preprocessing method like createWaveletFeatures() that can be used on TimeTasks?

We kinda have that.
My proposal for the API is contained in the fda_pull1_task_featExtract branch.
I am not entirely sure, this is how it is going to be, but we can build upon that.

I think it makes sense to keep
The currently supported preprocessing functions are:
as we can extend this list along with an example whenever a new method is added.
Or do we want a getFDAFeatureExtractors() or getFDAFeaturePreprocessingMethods()

pfistfl · 2017-07-24T17:02:17Z

Update:
The branch now contains all relevant features from the current master branch.
Maybe someone could review?

SteveBronder · 2017-11-13T22:19:05Z

Did you guys rebase this yet to be up to date with gh-pages? travis works on gh-pages now

…to fda

pfistfl · 2017-11-13T23:20:15Z

I do not get why travis fails here and yet works on gh-pages....

...
Knitting file 'nested_resampling.Rmd' ...
Quitting from lines 312-338 () 
Error in requirePackages(package, why = stri_paste("learner", id, sep = " "),  : 
  For learner classif.kknn please install the following packages: kknn
Calls: lapply ... addClasses -> makeRLearnerInternal -> requirePackages -> stopf

SteveBronder · 2017-11-14T02:50:06Z

Not sure, I did a rebase on mine and was able to get it to work

pfistfl · 2017-11-14T10:24:56Z

Continued in #125

Lars Kotthoff and others added 10 commits June 17, 2015 15:01

travis button in readme

4a0696c

Update README.md

4c5c3a3

didn't see it there

876f2be

Put some links here [ci skip]

adf5a27

Update paths (#2) in master Readme [ci skip]

135a07a

revert tutorial links for now

fe5ee3b

tutorial links

9e7f1b2

Added basic tutorial for functional data. Not ready for merge yet, as it

4c355bc

can not build without merged branch in mlr.

Adapted suggestions by Laura.

a0e603e

Updated according to suggestions, eval = FALSE for

da5c3e4

feature extraction, as it is not in the PR yet.

pfistfl added the pr-work in progress - not done label Mar 14, 2017

SteveBronder reviewed Mar 19, 2017

View reviewed changes

pfistfl and others added 6 commits March 20, 2017 15:20

Changed according to proposals by Mason and Steve

2d60038

Removed column renaming

9a3f056

Update functional data section

56229c3

Merge origin

b217854

Update Tutorial 2/3

a70c8a6

Finished fda tutorial

7aabdb8

pfistfl mentioned this pull request Jul 24, 2017

Better integrate new(ish) learner properties into the learner tables #110

Closed

Updated tutorial to reflect master version

d98bd57

pfistfl added pr-please review pr-ready for merge (?) and removed pr-work in progress - not done labels Jul 24, 2017

pfistfl added 5 commits October 2, 2017 10:56

Update, fix travis

c0d3d85

Fix README

00cda6e

Update rmd, works now

7aa2c7a

Fix missing underscore

c22b656

Update tutorial

bef8456

Merge branch 'gh-pages' of https://github.com/mlr-org/mlr-tutorial in…

794eeab

…to fda

pfistfl mentioned this pull request Nov 14, 2017

Add functional data section #125

Closed

pfistfl closed this Nov 14, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add functional data section #101

Add functional data section #101

pfistfl commented Mar 14, 2017

SteveBronder Mar 19, 2017

SteveBronder Mar 19, 2017

masongallo Mar 19, 2017

pfistfl Mar 20, 2017 •

edited

Loading

SteveBronder Mar 19, 2017

SteveBronder Mar 19, 2017

pfistfl Mar 20, 2017

SteveBronder Mar 19, 2017

SteveBronder Mar 19, 2017

SteveBronder Mar 19, 2017

SteveBronder Mar 19, 2017

pfistfl Mar 20, 2017 •

edited

Loading

SteveBronder Mar 19, 2017

pfistfl Mar 20, 2017

SteveBronder Mar 19, 2017

pfistfl Mar 20, 2017

pfistfl Mar 20, 2017 •

edited

Loading

pfistfl commented Jul 24, 2017

SteveBronder commented Nov 13, 2017

pfistfl commented Nov 13, 2017 •

edited

Loading

SteveBronder commented Nov 14, 2017

pfistfl commented Nov 14, 2017

		@@ -0,0 +1,177 @@
		# Functional Data

		Functional data provides information about curves varying over a continuum, such as time.

Add functional data section #101

Add functional data section #101

Conversation

pfistfl commented Mar 14, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pfistfl Mar 20, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pfistfl Mar 20, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pfistfl Mar 20, 2017 • edited Loading

Choose a reason for hiding this comment

pfistfl commented Jul 24, 2017

SteveBronder commented Nov 13, 2017

pfistfl commented Nov 13, 2017 • edited Loading

SteveBronder commented Nov 14, 2017

pfistfl commented Nov 14, 2017

pfistfl Mar 20, 2017 •

edited

Loading

pfistfl Mar 20, 2017 •

edited

Loading

pfistfl Mar 20, 2017 •

edited

Loading

pfistfl commented Nov 13, 2017 •

edited

Loading