Skip to content
Browse files

Merge pull request #1 from arbox/master

Working with files and documentation.
  • Loading branch information...
2 parents fbc8f3b + b404956 commit eb8305124c0688a392381eeba497670c2b51c192 @raypereda committed Jan 24, 2012
Showing with 37 additions and 10 deletions.
  1. +1 −0 .gitignore
  2. +17 −6 README.rdoc
  3. +10 −0 Rakefile
  4. +4 −3 bin/stem
  5. +3 −0 lib/stemmify.rb
  6. +2 −1 stemmify.gemspec
View
1 .gitignore
@@ -0,0 +1 @@
+rdoc
View
23 README.rdoc
@@ -1,5 +1,6 @@
= Stemmify
+== Description
This is a Rails gem for reducing words to their roots. For example, all the
following words to are stemmed to "observ", which is not a real word
in this case:
@@ -30,16 +31,16 @@ Martin Porter explains:
== Install
- sudo gem install stemmify
- or
- gem install stemmify
+ $ sudo gem install stemmify
+or
+ $ gem install stemmify
-Note that while the source code is hosted github, the actual gem
-is hosted on RubyGem.org
+Note that while the source code is hosted on {github}[https://github.com/raypereda/stemmify],
+the actual gem is hosted on {RubyGem.org}[https://rubygems.org/gems/stemmify].
== Usage
-Let's say you building some sort of search tool. You want
+Let's say you are building some sort of search tool. You want
searches for "observations" and "observer" to all bring up
the same items. When you are building you index, you can
map all the words to their roots using the stem method.
@@ -49,6 +50,16 @@ Here's an example usage:
require 'stemmify'
print("observations".stem) # ==> "observ"
+You can use <tt>stemmify</tt> as a command line tool:
+ $ stem input_file.txt
+ $ printf "I\nwalked\nin\nthe\nforest\n.\n" | stem
+
+Both the file based input and the input taken from the stdin should be tokenized,
+i.e. provide one token per line, tokens should be delimeted e.g. by a new line feed.
+
+For tokenization consider using a tokenizer, for instance the
+{tokenizer}[https://rubygems.org/gems/tokenizer] gem.
+
== Test Suite
This test is based on the sample input and output text from Martin Porter
View
10 Rakefile 100755 → 100644
@@ -6,3 +6,13 @@ end
desc "Run tests"
task :default => :test
+
+# Generate documentation.
+require 'rdoc/task'
+RDoc::Task.new do |rdoc|
+ rdoc.rdoc_files.include('README.rdoc',
+ 'lib/**/*',
+ 'bin/**/*')
+ rdoc.options = ['--main', 'README.rdoc']
+ rdoc.rdoc_dir = 'rdoc'
+end
View
7 bin/stem
@@ -1,8 +1,9 @@
-#!/usr/bin/ruby
+#!/usr/bin/env ruby
+
require 'stemmify'
-$stdin.each do |word|
- puts word.chomp.stem + "\n"
+while line = gets
+ puts line.chomp.stem
end
View
3 lib/stemmify.rb
@@ -10,6 +10,9 @@
# See also http://www.tartarus.org/~martin/PorterStemmer
#
# Send comments to raypereda (at) gmail.com
+# RDoc stuff.
+# :title: Stemmify - Ruby based Porter Stemmer
+
module Stemmify
View
3 stemmify.gemspec 100755 → 100644
@@ -11,7 +11,8 @@ Gem::Specification.new do |s|
s.test_files = ["test/test_stemmify.rb", "test/input.txt", "test/output.txt"]
s.homepage = "http://github.com/raypereda/stemmify"
s.require_paths = ["lib"]
- s.rubygems_version = "0.0.1"
+# s.rubygems_version = "0.0.1"
s.summary = "Stemming reducing an inflected word to its root form (approximately). For example, running reduces to run."
+ s.add_development_dependency('rdoc', '>=3.9.1')
end

0 comments on commit eb83051

Please sign in to comment.
Something went wrong with that request. Please try again.