Generic types/methods for normalizing input data

[tbreloff]

No description provided.

[tbreloff]

I'm creating a Normalized class with functions

normalize(o::Normalized, x::Float64) = ...  # normalizes and returns float
normalize!(o::Normalized, x::Float64) = ... # calls update! on underlying Var and normalizes
denormalize(...) = ...
...

Is there standard notation/lettering for signifying raw vs normalized variables? what should statenames return?

[joshday]

Sorry for answering almost none of your questions and added a few more questions. Typically notation is X and Z (z-score), but should state/statenames just return mean and standard deviation?

Side note: Should we change Var to MeanVar to be more clear on what it estimates?

Could we do something along these lines? Does Normalized store any fields other than Var objects?

normalize(o::Var, x::Float64) = (x - mean(o)) / std(o))
normalize!(o::Var, x::Float64) = (update!(o, x); normalize(o, x))

[joshday]

I also made a Vars type last night, which may be easier to use than a vector of Var types.

[joshday]

How about something like

type Normalized{W <: Weighting}
    xdata::Vars{W}
    ydata::Var{W}
end

normalize() could return a tuple of standardized x and standardized y

xnew, ynew = normalize!(o.Normalized, x::VecF, y::Float64)

[tbreloff]

• Could we just add normalize/denormalize to Var? Yes I think so. I'll trash my normalize.jl file.
• I don't think we need to change Var to MeanVar because it also tracks the mean... however I do think that it might improve readability to call it Variance??
• Side question... does it make sense to return stddev instead of variance in the state? Or both? Which do you find is more useful?

[tbreloff]

I don't think we need that normalized class at all... Var is plenty as you suggested

[joshday]

• I do like Variance a bit better
• stddev is more useful, but is it strange if the Variance type returns mean and standard deviation?

[tbreloff]

Yes... that is a bit strange I suppose. I'll leave it up to you, since I can always map(sqrt, ...)

[joshday]

Also, if you use the Vars type, it has a std() method which returns a vector of stddevs.

[tbreloff]

Did you add Vars to the repo? I see means.jl, but no vars.jl

[joshday]

Sorry, just pushed it now.