A flexible and general-purpose ngrams library written in Ruby. Raingrams supports ngram sizes greater than 1, text/non-text grams, multiple parsing styles and open/closed vocabulary models.
Ruby
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
lib Oops again. Feb 21, 2011
spec fixing specs to use should be_within since exact floating point match… Feb 20, 2011
.document Dropped Jeweler and Bundler for Ore. Oct 27, 2010
.gemtest Opting into `gem test`. Jan 20, 2011
.gitignore Dropped Jeweler and Bundler for Ore. Oct 27, 2010
.rspec Upgraded to jeweler ~> 1.5.0.pre and rspec 2.0.0. Oct 11, 2010
.yardopts Use RDoc for formatting. Jan 24, 2011
ChangeLog.md
LICENSE.txt Migrate off of Hoe and onto Jeweler. Apr 17, 2010
README.md Renamed License to Copyright. Oct 27, 2010
Rakefile
gemspec.yml Use rubygems-tasks ~> 0.1. Apr 25, 2012
raingrams.gemspec Use the new pure-Ruby gemspec from Ore 0.8.0. Jun 19, 2011

README.md

Raingrams

Description

Raingrams is a flexible and general-purpose ngrams library written in Ruby. Raingrams supports ngram sizes greater than 1, text/non-text grams, multiple parsing styles and open/closed vocabulary models.

Features

  • Supports ngram sizes greater than 1.
  • Supports text and non-text grams.
  • Supports Open and Closed vocabulary models.
  • Supports calculating the similarity and commonality of sample text against specified models.
  • Supports generating random text from models.

Install

$ sudo gem install raingrams

Examples

Train a model with ycombinator comments:

require 'raingrams'
require 'nokogiri'
require 'open-uri'

include Raingrams

model = BigramModel.build do |model|
  doc = Nokogiri::HTML(open('http://news.ycombinator.org/newcomments'))
  doc.search('span.comment') do |span|
    model.train_with_text(span.inner_text)
  end
end

Update a trained model:

model.train_with_text %{Interesting videos. Anders talks about
  functional support on .net, concurrency, immutability. Guy Steele
  talks about Fortress on JVM. Too bad they are afraid of macros
  (access to AST), though Steele does say Fortress has some support.}

model.refresh

Generate a random sentence:

model.random_sentence
# => "OTOOH if you use slicehost even offer to bash Apple makes it will
exit and its 38 month ago based configuration of little networks
created."

Dump a model to a file, to be marshaled later:

model.save('path/for/model')

Load a model from a file:

Model.open('path/for/model')

Copyright

Copyright (c) 2010 Hal Brodigan

See {file:LICENSE.txt} for license information.