Scripts for natural language processing, mostly machine translation
Ruby Perl Python Shell Smalltalk NewLisp Other
Switch branches/tags
Nothing to show
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
nonbreaking_prefixes steal tokenizer from moses' scripts Jun 14, 2014
test rename, remove non nlp stuff Nov 4, 2016
.gitmodules memusg Feb 4, 2014
LICENSE make use of nlp_ruby, LICENSE Jan 29, 2014
README.md README Nov 12, 2015
add-ln mv Jul 5, 2016
add-seg mv Jul 5, 2016
add-start-end mv Jul 5, 2016
avg undo unfortunate variable naming: cfg -> conf! Jun 10, 2015
avg-seg-len avg-seg-len Aug 4, 2017
avg-weights mv Jul 5, 2016
bishuf bishuf: proper fixed source of randomness Dec 5, 2017
bitext-filter-length bitext-filter-length Dec 14, 2017
cdec-hg-to-json mv Jul 5, 2016
cmp cmp Aug 4, 2017
compound-splitter.perl compound-splitter.perl (taken from moses v2.1.1) Jul 22, 2014
cumul cumul Aug 4, 2017
de-bpe de-bpe Aug 4, 2017
de-sgm de-sgm: use egrep instead of grep for compat. Nov 8, 2017
detruecase.perl add moses' truecaser Nov 12, 2015
div div Jan 25, 2015
dot alles neu macht der mai Oct 9, 2014
even init Dec 5, 2013
feature-dict mv Jul 5, 2016
filter-illegal filter-illegal Jun 21, 2017
filter-len filter-len Dec 3, 2017
filter-tokens filter-tokens Dec 13, 2017
first-lower mv Jul 5, 2016
fix-utf-8-pua script to remove private use area chars Nov 12, 2015
gigaword-collapse-tags mv Jul 5, 2016
hadoop-uniq mv Jul 5, 2016
hist-tok hist-tok: +x Dec 3, 2017
htmlentities make use of nlp_ruby, LICENSE Jan 29, 2014
kbest-bleu-oracles mv Jul 5, 2016
kendalls-tau mv Jul 5, 2016
key-count mv Jul 5, 2016
kmeans undo unfortunate variable naming: cfg -> conf! Jun 10, 2015
lang lang Dec 5, 2017
length-ratio length-ratio Dec 14, 2017
lin-reg mv Jul 5, 2016
log-reg mv Jul 5, 2016
lowercase.perl steal tokenizer from moses' scripts Jun 14, 2014
ltok map lines to number of token they contain Nov 12, 2015
make-rule-features mv Jul 5, 2016
max alles neu macht der mai Oct 9, 2014
max-len mv Jul 5, 2016
median alles neu macht der mai Oct 9, 2014
merge-files mv Jul 5, 2016
merge-ttable mv Jul 5, 2016
min alles neu macht der mai Oct 9, 2014
min-max mv Jul 5, 2016
moses-1best mv Jul 5, 2016
mult alles neu macht der mai Oct 9, 2014
multi-bleu.perl moses' multi-bleu.perl Aug 4, 2017
ng undo unfortunate variable naming: cfg -> conf! Jun 10, 2015
nn init Dec 5, 2013
no-empty mv Jul 5, 2016
no-non-printables mv Jul 5, 2016
norm norm May 13, 2015
norm-german mv Jul 5, 2016
norm-hyphens mv Jul 5, 2016
normalize-punctuation mv Jul 5, 2016
normchr normalize on char level Nov 12, 2015
num-tok mv Jul 5, 2016
odd alles neu macht der mai Oct 9, 2014
overlap overlap Jul 5, 2017
paste-pairs mv Jul 5, 2016
per-sentence-bleu per-sentence-bleu: fix Aug 4, 2017
per-sentence-bleu-kbest mv Jul 5, 2016
per-sentence-ter mv Jul 5, 2016
pot alles neu macht der mai Oct 9, 2014
preprocess mv Jul 5, 2016
preprocess-no-lower mv Jul 5, 2016
pt-bloom mv Jul 5, 2016
push-rules mv Jul 5, 2016
repetition-rate repetition rate Nov 11, 2017
round alles neu macht der mai Oct 9, 2014
rule-shapes mv Jul 5, 2016
sample sample: tab as separator Nov 12, 2015
select add select Sep 21, 2014
select-from select-from: fix Dec 3, 2017
shard alles neu macht der mai Oct 9, 2014
sort-features mv Jul 5, 2016
source-sides mv Jul 5, 2016
split-kbest mv Jul 5, 2016
split-lines mv Jul 5, 2016
split-pipes mv Jul 5, 2016
sqrt pot sqrt Oct 3, 2014
stanford-parser-run mv Jul 5, 2016
stddev corrected stddev Dec 18, 2015
strips make use of nlp_ruby, LICENSE Jan 29, 2014
sum alles neu macht der mai Oct 9, 2014
tc alles neu macht der mai Oct 9, 2014
tf-idf undo unfortunate variable naming: cfg -> conf! Jun 10, 2015
to-ascii mv Jul 5, 2016
tokenizer-no-escape.perl alles neu macht der mai Oct 9, 2014
toks alles neu macht der mai Oct 9, 2014
toks-per-line mv Jul 5, 2016
train-test-split mv Jul 5, 2016
train-truecaser.perl add moses' truecaser Nov 12, 2015
truecase.perl add moses' truecaser Nov 12, 2015
var undo unfortunate variable naming: cfg -> conf! Jun 10, 2015
vocab init Dec 5, 2013
vocab2 vocab2 Nov 28, 2017

README.md