Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Browse files

Added mapper and reducer examples.

  • Loading branch information...
commit 42dd937011afa361e8d8bc2dd7a1babe7ede42a0 1 parent 6cbba64
@bendiken bendiken authored
Showing with 56 additions and 2 deletions.
  1. +36 −2 README.md
  2. +10 −0 doc/examples/mapper.rb
  3. +10 −0 doc/examples/reducer.rb
View
38 README.md
@@ -1,11 +1,45 @@
RDFgrid: Map/Reduce-based Linked Data Processing with Hadoop
============================================================
-RDFgrid is a framework for [map/reduce][MapReduce]-based batch-processing of
-[RDF][] data with [Hadoop][] and [Amazon Elastic MapReduce][AWS EMR].
+RDFgrid is a simple framework for [map/reduce][MapReduce]-based
+batch-processing of [RDF][] data with [Hadoop][] and [Amazon Elastic
+MapReduce][AWS EMR].
* <http://github.com/datagraph/rdfgrid>
+Examples
+--------
+
+### A mapper for counting RDF predicate usage (`doc/examples/mapper.rb`)
+
+ #!/usr/bin/ruby -Ilib
+ require 'rdfgrid'
+
+ class PredicateCounter < RDFgrid::Mapper::StatementMapper
+ def process(statement)
+ yield statement.predicate, 1
+ end
+ end
+
+ PredicateCounter.process!
+
+### A reducer for summing up RDF predicate usage (`doc/examples/reducer.rb`)
+
+ #!/usr/bin/ruby -Ilib
+ require 'rdfgrid'
+
+ class PredicateSummer < RDFgrid::Reducer
+ def process(values)
+ yield values.inject(0) { |sum, value| sum + value.to_i }
+ end
+ end
+
+ PredicateSummer.process!
+
+### Running the mapper and reducer pipeline with a local N-Triples dataset
+
+ $ cat data.nt | ruby mapper.rb | sort | ruby reducer.rb
+
Documentation
-------------
View
10 doc/examples/mapper.rb
@@ -0,0 +1,10 @@
+#!/usr/bin/ruby -Ilib
+require 'rdfgrid'
+
+class PredicateCounter < RDFgrid::Mapper::StatementMapper
+ def process(statement)
+ yield statement.predicate, 1
+ end
+end
+
+PredicateCounter.process!
View
10 doc/examples/reducer.rb
@@ -0,0 +1,10 @@
+#!/usr/bin/ruby -Ilib
+require 'rdfgrid'
+
+class PredicateSummer < RDFgrid::Reducer
+ def process(values)
+ yield values.inject(0) { |sum, value| sum + value.to_i }
+ end
+end
+
+PredicateSummer.process!
Please sign in to comment.
Something went wrong with that request. Please try again.