No description, website, or topics provided.
Clojure Ruby
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.


Tiny word segmenter written in clojure. Although the attached model is trained with the Japanese corpora, you can train your own model with Chinese or Korean corpora.


Install Leiningen

First of all, you have to install (Leiningen)[]. To do this:

cd ~/bin
chmod +x lein
lein self-install
# or you can use macports 
sudo port install leiningen

How to train a model

You have to fetch this repository from github.

git clone

TinyWordSegmenter depends on several libraries. You can install these libraries:

cd TinyWordSegmenter
lein deps

You can train a model file for SVM like this:

lein run --file KyotoUniv.txt --mode train --eta 1.0 --lambda 1.0 --max-iter 100 --model-file weight.model --model fobos_clj.svm.SVM

This will take a long time. If you prefer logistic regression rather than SVM, then type

lein run --file KyotoUniv.txt --mode train --eta 1.0 --lambda 1.0 --max-iter 100 --model-file weight.model --model fobos_clj.logistic.Logistic

How to decode characters using your trained model

After training the model, you can evaluate its model like this:

lein trampoline run --mode decode --model-file weight.model


Copyright (C) 2012 Yasuhisa Yoshida

Distributed under the Eclipse Public License, the same as Clojure.