Skip to content

Commit

Permalink
flat-* has to return a java.util.Iterator now uugghhh
Browse files Browse the repository at this point in the history
  • Loading branch information
sorenmacbeth committed Jul 28, 2016
1 parent 456875d commit 8edda47
Showing 1 changed file with 3 additions and 3 deletions.
6 changes: 3 additions & 3 deletions test/flambo/example/tfidf.clj
Expand Up @@ -19,8 +19,8 @@
(clojure.string/split content #" "))
doc-terms-count (count terms)
term-frequencies (frequencies terms)]
(map (fn [term] (ft/tuple doc-id [term (term-frequencies term) doc-terms-count]))
(distinct terms))))
(.iterator (map (fn [term] (ft/tuple doc-id [term (term-frequencies term) doc-terms-count]))
(distinct terms)))))

(defn calc-idf [doc-count]
(f/fn [term tuple-seq]
Expand Down Expand Up @@ -56,7 +56,7 @@
;; where tf is per document, that is, tf(term, document)
tf-by-doc (-> doc-term-seq
(f/map-to-pair (ft/key-val-fn (f/fn [doc-id [term term-freq doc-terms-count]]
(ft/tuple term [doc-id (double (/ term-freq doc-terms-count))]))))
(ft/tuple term [doc-id (double (/ term-freq doc-terms-count))]))))
(inspect "tf-by-doc")
f/cache)

Expand Down

0 comments on commit 8edda47

Please sign in to comment.