Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
WIP: Cross-validation #235
Another bit of WIP. The idea here is to use the nascent prediction API on the tensor model (and hopefully also on other models soon enough...) to perform k-fold cross validation.
The scheme is that in each iteration, a certain number of random diffusion-weighted directions is dropped. A model is fit to the rest of the directions and then a prediction is made for the left out directions. This is performed so that every direction is dropped in one iteration, so we get a complete prediction of out-of-sample measurements, which in the end comprise all the measurements in the original sample.
Questions at this point: this function relies on a rather uniform API: it requires that the Fit object derived from the model have a
Maybe we should implement a base-class from which all the models inherit that enforces these assumptions? Are there any models that are not initialized with a
Also - I created a separate module for this under
Comments and ideas are most welcome at this point. I am still trying to hammer this out in the best possible way.
@arokem, this is good stuff. Just a few comments regarding design. All models will at some point need information from some GradientTable, and I believe that currently all models do this in a way that is compatible with your implementation. Adding an abstract model class wouldn't really help here, except to serve as an example, because each child class would need to override the abstract init. At that point a developer could choose to change the signature of init in a non-compatible way. My suggestion would be to add these requirements to our Model API and document them well for future developers. The other option would be to have a method like
The idea with having a base-class is to enforce certain uniformity in the
You would of course write a new init for every child-class of the
On Fri, Sep 6, 2013 at 4:17 PM, MrBago firstname.lastname@example.org wrote:
More specifically, DTI has
On Sat, Sep 7, 2013 at 11:12 AM, Ariel Rokem email@example.com wrote: