Gini operate on subspaces that than “features”

[falkben]

@jovo @MrAE need a little more info/explanation

[jovo]

currently, -1*x and +1*x are treated differently, but they are effectively identical, so they should be treated the same. similarly, [-1 +1]* [x1, x2] is the same as [+1 -1]*[x1 x2]. see what i mean?

…

On Mon, Nov 19, 2018 at 10:58 AM Ben Falk ***@***.***> wrote: @jovo <https://github.com/jovo> @MrAE <https://github.com/MrAE> need a little more info/explanation — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#91>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AACjcnt4m4pFyEASJ6zUdo8gf9B5z7Wiks5uwtU0gaJpZM4YpZTq> .

-- the glass is all full: half water, half air. neurodata.io

[MrAE]

So, would we have the following features as equivalent?

[-1 +1]* [x1, x2] == [+1 -1]*[x1 x2] =?= [-1 -1]* [x1, x2] == [+1 +1]*[x1 x2]

?

[jovo]

if the decision boundary is the same, than the features are the same. for example, [-1]*[x1] == [+1][x1] so, the first two are not equal to the last two. we need to think more about the others....

…

On Tue, Jan 15, 2019 at 4:18 PM Jesse Leigh Patsolic < ***@***.***> wrote: So, would we have the following features as equivalent? [-1 +1]* [x1, x2] == [+1 -1]*[x1 x2] =?= [-1 -1]* [x1, x2] == [+1 +1]*[x1 x2] ? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#91 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AACjckK3hVv2Hvlot7UwV4YQxtA2R72Zks5vDkWPgaJpZM4YpZTq> .

-- the glass is all full: half water, half air. neurodata.io

[falkben]

How do you want to handle randmat continuous? It seems like the feature importance as calculated right now would count every linear combination as different for continuous. We could tackle that at some other point though. My naive suggestion for continuous would be to just count how many times a particular feature was used in the entire forest (and not worry at all about the weights).

[jovo]

oh, i never think about continuous. let's just count that they exist, as you propose....

…

On Tue, Jan 15, 2019 at 4:30 PM Ben Falk ***@***.***> wrote: How do you want to handle randmat continuous? It seems like the feature importance as calculated right now would count every linear combination as different for continuous. We could tackle that at some other point though. My naive suggestion for continuous would be to just count how many times a particular feature was used in the entire forest (and not worry at all about the weights). — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#91 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AACjcmQRhwq-UYnKzAgTUbvFMQOc63vFks5vDkhggaJpZM4YpZTq> .

-- the glass is all full: half water, half air. neurodata.io

[MrAE]

Ok, so disregard my previous statement.

Projection weights are equivalent if they parametrize the same line in $\mathbb{R}^p$ e.g for two projection vectors $w$ and $v$ if $w = av$ for some $a \in \mathbb{R}$, then we should count them as the same thing when computing feature importance.

This is easy to check if the weights are binary {1,-1} or are sampled from some finite discrete set.

If the weights are sampled continuously, I'm not sure it would be efficient to check this or if we would even have a high enough probability of sampling equivalent vectors.

As Ben suggested, we could just count the number of times a unique combination of the original features was used disregarding the projection weights.

[jovo]

agreed

…

On Wed, Jan 16, 2019 at 12:03 PM Jesse Leigh Patsolic < ***@***.***> wrote: Ok, so disregard my previous statement. Projection weights are equivalent if they parametrize the same line in $\mathbb{R}^p$ e.g for two projection vectors $w$ and $v$ if $w = av$ for some $a \in \mathbb{R}$, then we should count them as the same thing when computing feature importance. This is easy to check if the weights are binary {1,-1} or are sampled from some finite discrete set. If the weights are sampled continuously, I'm not sure it would be efficient to check this or if we would even have a high enough probability of sampling equivalent vectors. As Ben suggested, we could just count the number of times a unique combination of the original features was used disregarding the projection weights. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#91 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AACjcgBro3LhSR8UGIZM_FkaxY_KAvB7ks5vD1rZgaJpZM4YpZTq> .

-- the glass is all full: half water, half air. neurodata.io