* Exclude clusterings for words only seen 1 or 2 times, as their clus…

…ters are unreliable
explosion · Apr 17, 2015 · 693c5a1 · 693c5a1
1 parent cc4e395
commit 693c5a1
Showing 1 changed file with 4 additions and 1 deletion.
diff --git a/bin/init_model.py b/bin/init_model.py
@@ -46,7 +46,10 @@ def _read_clusters(loc):
             cluster, word, freq = line.split()
         except ValueError:
             continue
-        clusters[word] = cluster
+        # If the clusterer has only seen the word a few times, its cluster is
+        # unreliable.
+        if int(freq) >= 3:
+            clusters[word] = cluster
     return clusters