data/para/msr/README.md: Update with new results

brmson · Aug 11, 2016 · 05ae789 · 05ae789
1 parent 47a67a2
commit 05ae789
Showing 1 changed file with 14 additions and 13 deletions.
diff --git a/data/para/msr/README.md b/data/para/msr/README.md
@@ -15,8 +15,6 @@ Model Comparison
 
 For randomized models, 95% confidence intervals (t-distribution) are reported.
 
-Also NOTE THAT THESE RESULTS ARE OBSOLETE because they predate the f/bigvocab port.
-
 | Model                    | train acc | train F1 | val acc  | val F1   | test acc | test F1  | settings
 |--------------------------|-----------|----------|----------|----------|----------|----------|---------
 | always y=1               | 0.673378  | 0.804748 | 0.694000 | 0.818935 | 0.665507 | 0.799025 | (defaults)
@@ -25,19 +23,22 @@ Also NOTE THAT THESE RESULTS ARE OBSOLETE because they predate the f/bigvocab po
 | Ji and Eisenstein (2013) |           |          |          |          | 0.804    | 0.859    | Matrix factorization with supervised reweighting
 | He et al. (2015)         |           |          |          |          | 0.786    | 0.847    | Multi-perspective Convolutional NNs and structured similarity layer
 |--------------------------|-----------|----------|----------|----------|----------|----------|---------
-| avg                      | 0.752237  | 0.835484 | 0.714875 | 0.817603 | 0.701775 | 0.803806 | (defaults)
-|                          |±0.016065  |±0.008362 |±0.006655 |±0.006592 |±0.003495 |±0.005670 |
+| avg                      | 0.726702  | 0.817746 | 0.734750 | 0.828196 | 0.707319 | 0.804215 | (defaults)
+|                          |±0.005720  |±0.005384 |±0.003011 |±0.002984 |±0.002614 |±0.003876 |
+|--------------------------|-----------|----------|----------|----------|----------|----------|---------
+| rnn                      | 0.713035  | 0.808908 | 0.720250 | 0.819535 | 0.703007 | 0.801361 | (defaults)
+|                          |±0.004878  |±0.004163 |±0.003282 |±0.003952 |±0.004594 |±0.005121 |
+| cnn                      | 0.848819  | 0.903752 | 0.732875 | 0.829464 | 0.698188 | 0.801707 | (defaults)
+|                          |±0.067693  |±0.039620 |±0.015090 |±0.005002 |±0.012668 |±0.004526 |
+| attn1511                 | 0.856946  | 0.899338 | 0.764750 | 0.841120 | 0.726993 | 0.810355 | (defaults)
+|                          |±0.030108  |±0.020247 |±0.007894 |±0.004343 |±0.005860 |±0.004620 |
+
+rnncnn does not converge in our experiments.
+
+THE RESULTS BELOW ARE OBSOLETE because they predate the f/bigvocab port.
+
 | DAN                      | 0.751258  | 0.830429 | 0.719750 | 0.815998 | 0.703478 | 0.799095 | ``inp_e_dropout=0`` ``inp_w_dropout=1/3`` ``deep=2`` ``pact='relu'``
 |                          |±0.021642  |±0.014033 |±0.005811 |±0.006513 |±0.003917 |±0.006118 |
-|--------------------------|-----------|----------|----------|----------|----------|----------|---------
-| rnn                      | 0.717806  | 0.816834 | 0.705750 | 0.812771 | 0.691920 | 0.797981 | (defaults)
-|                          |±0.016668  |±0.006131 |±0.004277 |±0.006366 |±0.007403 |±0.007051 |
-| cnn                      | 0.746487  | 0.827205 | 0.704125 | 0.805049 | 0.702029 | 0.798059 | ``inp_e_dropout=1/2`` ``dropout=1/2`` (FIXME)
-|                          |±0.015070  |±0.009841 |±0.006288 |±0.006411 |±0.003744 |±0.005246 |
-| rnncnn                   | 0.782316  | 0.854726 | 0.712750 | 0.811653 | 0.704167 | 0.799246 | (defaults)
-|                          |±0.028483  |±0.015017 |±0.006946 |±0.008944 |±0.006232 |±0.009749 |
-| attn1511                 | 0.741401  | 0.821830 | 0.702250 | 0.801453 | 0.699891 | 0.791798 | (defaults)
-|                          |±0.012435  |±0.005271 |±0.004882 |±0.007168 |±0.004946 |±0.008456 |
 
 These results are obtained like this: