bad stats

[garfieldnate]

The example used to demonstrate #34 gives weird output (but didn't with AM::Parallel):

    +-----------------------------+-------------+------------+
    | Class                       | Score       | Percentage |
    +-----------------------------+-------------+------------+
    | alternarialeaf-spot         |        2180 |   0.000%   |
    | anthracnose                 |       49856 |   0.000%   |
    | bacterial-blight            |       61952 |   0.000%   |
    | brown-spot                  |      364608 |   0.001%   |
    | brown-stem-rot              |     1060652 |   0.002%   |
    | charcoal-rot                |  8964860640 |  18.824%   |
    | diaporthe-pod-&-stem-blight |         140 |   0.000%   |
    | diaporthe-stem-canker       |       51840 |   0.000%   |
    | downy-mildew                |       50688 |   0.000%   |
    | frog-eye-leaf-spot          |      104448 |   0.000%   |
    | phytophthora-rot            |       62912 |   0.000%   |
    | powdery-mildew              |        7008 |   0.000%   |
    | purple-seed-stain           |       21664 |   0.000%   |
    +-----------------------------+-------------+------------+
    | Total                       | 47623698012 |            |
    +-----------------------------+-------------+------------+

It does get basically the right idea, with charcoal-rot outdoing everything else, and the count for diaporthe-pod-&-stem-blight is correct, but everything else is wacky. Did I screwed up the biginteger calculations? This is all awful. Here's the AM::Parallel output, which matches that of Weka AM:

alternarialeaf-spot               3016836    0.001%
anthracnose                       5358272    0.002%
bacterial-blight                  2880000    0.001%
brown-spot                        2134080    0.001%
brown-stem-rot                  976826156    0.289%
charcoal-rot                 337300810464   99.700%
diaporthe-pod-&-stem-blight           140    0.000%
diaporthe-stem-canker            10013312    0.003%
downy-mildew                        50688    0.000%
frog-eye-leaf-spot                 890880    0.000%
phytophthora-rot                  2028992    0.001%
powdery-mildew                   11869024    0.004%
purple-seed-stain                 1463456    0.000%
                             ------------
                             338317342300

[garfieldnate]

Minimal breaking example. Data set:

purple-seed-stain , normal gt-norm upper-areas 80-89 abnorm no-yellow-halos w-s-marg lt-1/8 absent absent absent norm no absent tan absent absent absent none absent diseased colored absent norm absent norm , num_501
purple-seed-stain , normal lt-norm whole-field lt-80 abnorm no-yellow-halos w-s-marg lt-1/8 absent absent absent abnorm yes absent tan absent absent absent none absent diseased colored absent norm absent norm , num_502

Test set:

charcoal-rot , normal gt-norm whole-field lt-80 abnorm absent dna dna absent absent absent abnorm yes absent tan absent absent absent black present norm dna absent norm absent norm , test_item

Bad output:

Analogical Set
Total Frequency = 196608
+-------------------+---------+--------+------------+
| Class             | Item    | Score  | Percentage |
+-------------------+---------+--------+------------+
| purple-seed-stain | num_501 |  49152 |  25.000%   |
| purple-seed-stain | num_502 | 147456 |  75.000%   |
+-------------------+---------+--------+------------+

Statistical Summary
+-------------------+--------+------------+
| Class             | Score  | Percentage |
+-------------------+--------+------------+
| purple-seed-stain | 131072 |  66.667%   |
+-------------------+--------+------------+
| Total             | 196608 |            |
+-------------------+--------+------------+

Expected result:

Statistical Summary

purple-seed-stain  327680  100.000%
                   ------
                   327680


Analogical Set

Total Frequency = 327680

purple-seed-stain  num_501   49152   15.000%
purple-seed-stain  num_502  278528   85.000%

[garfieldnate]

Broke before 2.38. Obviously I need to add extensive testing for accuracy. I always accepted that matching accuracy of the finnverb dataset was good enough, but that's really laziness.

[garfieldnate]

Bad commit:

commit 336cbde30e7c82f1fa8d0655b026a7591a725bf4
Author: Nathan Glenn <garfieldnate@gmail.com>
Date:   Sun Feb 9 17:51:46 2014 -0800

    add carry_pointer macro

    I don't really like how this macro looks; it's not very transparent to
    me. However, this allows us to gather all of the carrying logic into
    one place.