RReliefFix

[kernc]

No description provided.

[codecov-io]

Current coverage is 90.42%

Merging #919 into master will not affect coverage as of 4de4335

@@            master    #919   diff @@
======================================
  Files          143     143       
  Stmts        14690   14696     +6
  Branches         0       0       
  Methods          0       0       
======================================
+ Hit          13283   13289     +6
  Partial          0       0       
  Missed        1407    1407       

Review entire Coverage Diff as of 4de4335

Powered by Codecov. Updated on successful CI builds.

[BlazZupan]

The same problem persist, nothing has changed (in terms of the ouput). Try xor.tab data set (a and b should be informative) and housing.tab (there, all the scores are still around 0).

[kernc]

You need to python setup.py develop as there are Cython updates that need to be rebuilt. I added tests for those two datasets so it would be pretty weird if they didn't work.

[BlazZupan]

python setup.py develop helped :-). But the results are still very different from those in Orange2. In xor.tab Orange2 Rank gives (n=10, k=20):

0.220 0.192 0.076 0.026 0.006

while the scores for Orange3 with same parameters is too high for attributes other than a, b:

0.213 0.215 0.157 0.150 0.147

Also, the difference of scores is not large enough, considering that attributes c, d, e are all irrelevant.

[janezd]

I checked the differences between this code and my old code, and I can't find anything that would explain the difference in results. My code follows Kononenko's "Strojno učenje" (the white book from 1997) and hence does not use weighting by distance. But removing this from your code (I removed dist[j] entirely) does not change the results.

I've now studied my old code enough that I guess I haven't missed any details. It takes care of a few subtleties (if n_iter is larger than data size, every row should be taken as the reference once ... etc), but none of them should play a role here.

Could it be that Orange 2 was just lucky? Try increasing the number of reference examples (not neighbours) by one at a time and see how the results change. At a certain number, the scores suddenly multiply by ten (reaching values between 2 and 4 -- is that theoretically possible, given the normalizations?!) and then drop again. With such instability, it's not inconceivable that it's all just luck.

Also, if you increase the number of examples in Orange 2 to, say, 50, results are comparable to what the new code gives, whereas in theory a larger number of reference examples should increase robustness.

[BlazZupan]

Jernej, I was just about to merge this request. Why is it in WIP now?

[kernc]

Because it's not ready yet. Please postpone merging until tomorrow.

[kernc]

Yes, some features in the small XOR example get overvalued. But the order of features by importance matches that from Orange 2 reasonably well for all datasets I tried.

Merge at convenience. For the brave journeymen, there is a block comment in the source.