Home

Latent Tree Language Model (LTLM)

LTLM discovers the tree structure of word roles hidden in the text.

Training

Inference

To infer the tree for a given sentence you need to follow these steps:

1. Load the model:

   for bigram model use: LTLM2sides2gram ltlm = IOUtils.load2Gram("model.bin");

   for trigram model use: LTLM2sides3gram ltlm = IOUtils.load3Gram("model.bin");

2. Create instace of inferencer:

   There are two types of inference. The first exact inference guarantees to find the most probable tree, but is quite slow. The second pseudo inference finds sub-optimal solution, but is much faster. Note that inferencer needs to know max. possible length of sentence in advance.

   for bigram exact inference use: Inferencer inferencer = new ExactInferencer2gram(ltlm, SENTENCE_MAX_LENGTH);

   for bigram pseudo inference use: Inferencer inferencer = new PseudoInferencer2gram(ltlm, SENTENCE_MAX_LENGTH);

   for trigram exact inference use: Inferencer inferencer = new ExactInferencer3gram(ltlm, SENTENCE_MAX_LENGTH);

   for trigram pseudo inference use: Inferencer inferencer = new PseudoInferencer3gram(ltlm, SENTENCE_MAX_LENGTH);

3. Infer the tree for a given sentence:

   The algorithm requires already tokenized text. Note that the models available here are trained on lowercased text.

   String[] words = "Everything has beauty , but not everyone sees it .".toLowerCase().split(" ");

   Sentence sentence = new Sentence(ltlm.getVocabulary().getWordKey(words));

   inferencer.infer(sentence, false, false);