Skip to content
This repository has been archived by the owner on Dec 13, 2021. It is now read-only.

Commit

Permalink
Updated documentation for 5.0 release
Browse files Browse the repository at this point in the history
  • Loading branch information
mjpost committed Aug 16, 2013
1 parent f0e12dd commit ce4f53e
Show file tree
Hide file tree
Showing 6 changed files with 107 additions and 203 deletions.
7 changes: 7 additions & 0 deletions 5.0/advanced.md
@@ -0,0 +1,7 @@
---
layout: default
category: links
title: Advanced features
---


164 changes: 86 additions & 78 deletions 5.0/pipeline.md

Large diffs are not rendered by default.

115 changes: 0 additions & 115 deletions 5.0/scale2013-tutorial.md

This file was deleted.

16 changes: 9 additions & 7 deletions 5.0/tutorial.md
Expand Up @@ -50,11 +50,11 @@ versions, and preventing the pipeline from retokenizing using the `--no-prepare-
In `$INDIAN/bn-en/tok`, you should see the following files:

$ ls $INDIAN/bn-en/tok
dev.bn-en.bn devtest.bn-en.bn dict.bn-en.bn test.bn-en.en.2
dev.bn-en.en.0 devtest.bn-en.en.0 dict.bn-en.en test.bn-en.en.3
dev.bn-en.en.1 devtest.bn-en.en.1 test.bn-en.bn training.bn-en.bn
dev.bn-en.en.2 devtest.bn-en.en.2 test.bn-en.en.0 training.bn-en.en
dev.bn-en.en.3 devtest.bn-en.en.3 test.bn-en.en.1
dev.bn-en.bn devtest.bn-en.bn dict.bn-en.bn test.bn-en.en.2
dev.bn-en.en.0 devtest.bn-en.en.0 dict.bn-en.en test.bn-en.en.3
dev.bn-en.en.1 devtest.bn-en.en.1 test.bn-en.bn training.bn-en.bn
dev.bn-en.en.2 devtest.bn-en.en.2 test.bn-en.en.0 training.bn-en.en
dev.bn-en.en.3 devtest.bn-en.en.3 test.bn-en.en.1

We will now use this data to test the complete pipeline with a single command.

Expand All @@ -71,8 +71,8 @@ related experiments.

cd ~/expts/joshua
$JOSHUA/bin/pipeline.pl \
--readme "Baseline Hiero run" \
--rundir 1 \
--readme "Baseline Hiero run" \
--source bn \
--target en \
--corpus $INDIAN/bn-en/tok/training.bn-en \
Expand Down Expand Up @@ -145,7 +145,7 @@ directory, tell the pipeline to start at the tuning step, and provide the needed
pipeline always builds an LM on the target side of the training data, if provided, but we are
supplying the language model that was already built. We could equivalently have removed the
`--corpus` line.

## Changing the model type

Let's compare the Hiero model we've already built to an SAMT model. We have to reextract the
Expand All @@ -164,6 +164,8 @@ grammar, but can reuse the alignments and the language model:
--no-build-lm \
--lmfile 1/lm.gz

See [the pipeline script page](pipeline.html#steps) for a list of all the steps.

## Analyzing the results

We now have three runs, in subdirectories 1, 2, and 3. We can display summary results from them
Expand Down
6 changes: 4 additions & 2 deletions _layouts/default.html
Expand Up @@ -86,12 +86,14 @@
<a class="brand" href="/">Joshua</a>
<div class="nav-collapse collapse">
<ul class="nav">
<li class="active"><a href="/">Home</a></li>
<li><a href="index.html">Documentation</a></li>
<li><a href="pipeline.html">Pipeline</a></li>
<li><a href="tutorial.html">Tutorial</a></li>
<li><a href="decoder.html">Decoder</a></li>
<li><a href="thrax.html">Thrax</a></li>
<li><a href="file-formats.html">File formats</a></li>
<li><a href="advanced.html">Advanced</a></li>
<!-- <li><a href="advanced.html">Advanced</a></li> -->
<li><a href="faq.html">FAQ</a></li>
</ul>
</div><!--/.nav-collapse -->
</div>
Expand Down
2 changes: 1 addition & 1 deletion joshua.css
Expand Up @@ -39,6 +39,6 @@ a.bibtex {
}

img.sponsor {
height: 120px;
width: 120px;
margin: 5px;
}

0 comments on commit ce4f53e

Please sign in to comment.