Skip to content
This repository
Browse code

merging in new implementation

  • Loading branch information...
commit 96056fbf5f4445a9c1d2919d5337584d9b6c8fd4 1 parent 717e752
authored September 27, 2006
1  .bzrignore
... ...
@@ -1,2 +1,3 @@
1 1
 .project
2 2
 doc
  3
+coverage
24  active_rdf.rb
... ...
@@ -0,0 +1,24 @@
  1
+# Loader of ActiveRDF library
  2
+#
  3
+# (c) 2005-2006 by Eyal Oren and Renaud Delbru - All Rights Reserved
  4
+
  5
+# adding active_rdf subdirectory to the ruby loadpath
  6
+file = 
  7
+	if File.symlink?(__FILE__)
  8
+		File.readlink(__FILE__)	
  9
+	else
  10
+		__FILE__
  11
+	end
  12
+
  13
+$: << File.dirname(File.expand_path(file)) + '/src'
  14
+$: << File.dirname(File.expand_path(file))
  15
+
  16
+class ActiveRdfError < StandardError
  17
+end
  18
+
  19
+# load standard classes that need to be loaded at startup
  20
+require 'objectmanager/resource'
  21
+require 'objectmanager/namespace'
  22
+require 'federation/connection_pool'
  23
+require 'queryengine/query'
  24
+
53  doc/approach.txt
... ...
@@ -0,0 +1,53 @@
  1
+all objects are completely empty studs: they carry no state at all
  2
+	* data lookup -> database select query
  3
+	* data setting -> database update query
  4
+	* method lookup (eyal.methods) -> select ?p where eyal ?t ?c and ?p domain ?c
  5
+	* class lookup (eyal.class) -> select ?c where eyal ?t ?c
  6
+ 
  7
+challenge:
  8
+	* how to cast object into (one-more) class(es) after class lookup
  9
+	* how to enable class-based code, e.g. Person.to_s
  10
+	* allowing people to describe and extend certain classes:
  11
+		class Person
  12
+		  class_uri http://foaf/Person
  13
+		  def birthday
  14
+		   "don't know"
  15
+		  end
  16
+	  end
  17
+  * enable object invocation based on their class, e.g. eyal.birthday
  18
+  	- figure out that eyal is a Person
  19
+  	- use birthday from Person
  20
+  	solution:
  21
+  	
  22
+  	(in Resource) 
  23
+  	def method_missing(method, *args)
  24
+  		# we are invoked with 'method' but we don't provide that method (yet)
  25
+  		# simple approach: look in all classes of which we are member, 
  26
+  		# and use the first one to provide the method
  27
+  		# which classes do we belong to?
  28
+  		klasses = lookup_own_types
  29
+  		klasses.each do |klass|
  30
+  			# if any of our klasses includes the method, add it and invoke it
  31
+  			if klass.methods.include?(method)
  32
+  				# TODO: incorrect syntax, figure out how to inject code into existing ruby object
  33
+  				self.inject(method)
  34
+  				self.send(method,*args)
  35
+  			end
  36
+  		end
  37
+  		
  38
+  		# method not found in any of our classes
  39
+  		super
  40
+  	end
  41
+  	
  42
+	* construct Ruby class for RDFS class (instead of just Resource), 
  43
+	e.g. Resource.lookup(http://foaf/Person) should return the Person class
  44
+	and after that the Person class should be known
  45
+	
  46
+	* allow Person.find_by_firstname
  47
+	challenge: when do we construct the Person class?
  48
+	1. at program startup we construct all classes (from RDFS class)
  49
+	- not transparent empty proxy, now we do maintain state (we know which classes exist), if changes in database we're out-of-date
  50
+	2. when needed we figure out which classes exist
  51
+	- difficult, need to catch some ClassUnknown error, and divert it
  52
+	- difficult to integrate with requirement to allow people to actually write and extend the Person class
  53
+	
15  doc/assumptions.txt
... ...
@@ -0,0 +1,15 @@
  1
+* all activerdf ruby objects are empty proxies (studs)
  2
+
  3
+* object property values can change on the fly ---> no caching
  4
+* objects can belong to multiple classes ---> roll our own class mechanism
  5
+* object membership can change on-the-fly (do we?) ---> no class-membership caching
  6
+
  7
+* want to use dynamic method typing, e.g.
  8
+class Person
  9
+  def to_s
  10
+    firstname + ' ' + lastname
  11
+  end
  12
+end
  13
+  so we need real class membership, e.g. eyal.class ---> Person 
  14
+  (but figure out when Ruby determines class membership, maybe we can
  15
+  overwrite Resource.class to look into the database all the time)
156  doc/new-design.rb
... ...
@@ -0,0 +1,156 @@
  1
+# scenario's: 
  2
+# - construct class model: renaud
  3
+# - lookup resource: renaud
  4
+# - find resource: renaud
  5
+# - read attr. value: eyal
  6
+# - write attr. value: eyal
  7
+#
  8
+#
  9
+# Object logic 
  10
+#  instance mngr | class mngr | namespace manager
  11
+# Caching Graph 
  12
+#  lookup & update | complete graph
  13
+# Query Engine
  14
+# Federation
  15
+# Adapter
  16
+#  connector | translator | triples
  17
+#
  18
+# graph model is accessible from everywhere (sidebar)
  19
+#
  20
+# dependencies:
  21
+# Class: Graph, QE
  22
+# QE: Graph (construct query), Federation (execute query)
  23
+# Federation: Adapter (execute query), Graph (merge results)
  24
+# Adapter: Connector (execute query on datasource), Translator (transform triples/graph)
  25
+# Translator: Graph, Triple (translate up/down)
  26
+
  27
+# federation manager: 
  28
+# 1. ensure single connection to single datasource (REUSE)
  29
+# 2. retrieve right datasource for some task, e.g. writable (SELECT)
  30
+# 3. distribute query and aggregate results (FEDERATE)
  31
+
  32
+
  33
+class RedlandAdapter
  34
+	writable = true
  35
+	down = Graph2SparqlTranslator.new
  36
+	up  = RedlandRubyTranslator.new
  37
+	connector = RedlandConnector.new
  38
+
  39
+	def query(graph)
  40
+		result = connector.query(down(graph))
  41
+		return up(results)
  42
+	end
  43
+end
  44
+
  45
+# managing graph in memory cache
  46
+# caching options
  47
+# 1. RDF(S)::Core (builtin to ActiveRDF 
  48
+# 2. schema caching (classes and properties)
  49
+# 3. instance caching (deletion slightly inefficient)
  50
+
  51
+# deletion scenario: QE builds delete
  52
+# QE: federation.delete(graph)
  53
+# federation: adapter.delete(graph)
  54
+# adapter figures out which subgraph will be deleted (same subgraph will be 
  55
+# deleted in memory)
  56
+class Adapter
  57
+	def delete(graph)
  58
+		deleted = query(graph)
  59
+		delete(graph)
  60
+		return deleted
  61
+	end
  62
+end
  63
+
  64
+class Federation
  65
+	def delete(graph)
  66
+		pool.writable.delete(graph)
  67
+	end
  68
+end
  69
+
  70
+class QueryEngine
  71
+	def delete(graph)
  72
+		deleted = federation.delete(graph)
  73
+		graph.remove_graph(subgraph)
  74
+	end
  75
+end
  76
+
  77
+class Graph
  78
+	def add(source, edge, target)
  79
+	end
  80
+
  81
+	def remove(source, edge, target)
  82
+	end
  83
+
  84
+	def add_graph(datagraph)
  85
+	end
  86
+
  87
+	def remove_graph(datagraph)
  88
+	end
  89
+
  90
+	def nodes
  91
+	end
  92
+
  93
+	def lookup(content)
  94
+	end
  95
+end
  96
+
  97
+# code below now CHANGED: adapter receives and returns graphs
  98
+
  99
+# in adapter:
  100
+def query(qs)
  101
+	results = query(qs)
  102
+	# an example, actual code would depend on number of result bindings (in this 
  103
+	# case, we only consider triples)
  104
+	results.each do |s,p,o|
  105
+		# an example, actual code would depend on parse-type of s, p, and o
  106
+		yield URI.new(s), URI.new(p), Literal.new(o)
  107
+	end
  108
+end
  109
+
  110
+# in QueryEngine (including translator)
  111
+# before this we have code building the query... such as select, condition, 
  112
+# keyword
  113
+def query
  114
+	query_graph = build_graph # builds a query graph from the current query options
  115
+	result_graph = Graph.new
  116
+
  117
+	datasource.query(translator.query_string(query_graph)).each do |s,p,o|
  118
+		$graph.add(s,p,o)
  119
+	end
  120
+	return result_graph
  121
+end
  122
+
  123
+# in class logic (static API)
  124
+def get(property)
  125
+	qe.select :o
  126
+	qe.condition self, property, :o
  127
+	nodes = qe.execute
  128
+	nodes.collect do |n|
  129
+		# instance manager returns objects, such as Strings or Persons
  130
+		InstanceManager.retrieve(n)
  131
+	end
  132
+end
  133
+
  134
+# in instance manager (part of OMM)
  135
+def retrieve(node)
  136
+	if cache.contains? node
  137
+		cache[node]
  138
+	else
  139
+		node_type = node.outgoing(RDF::type)
  140
+		node_class = retrieve(type)
  141
+		cache[node] = node_class.send(:create, node)
  142
+	end
  143
+end
  144
+
  145
+# in Resource (or Person, virtually)
  146
+def create(uri)
  147
+	node = graph.lookup(uri)
  148
+	OMM.retrieve(node)
  149
+end
  150
+
  151
+# OMM manages cache if cache enabled
  152
+#
  153
+# OMM builds class model (need to work out further)
  154
+# def add_namespace(prefix, symbol)
  155
+# def add_class(URI, symbol)
  156
+# def add_instance(URI, symbol)
1  rcov.sh
... ...
@@ -0,0 +1 @@
  1
+rcov -Isrc test/**/ts_*.rb
126  src/adapter/redland.rb
... ...
@@ -0,0 +1,126 @@
  1
+# (read-only) adapter to Redland databse
  2
+# uses SPARQL for querying
  3
+require 'federation/connection_pool'
  4
+require 'queryengine/query2sparql'
  5
+require 'rdf/redland'
  6
+
  7
+class RedlandAdapter
  8
+	ConnectionPool.instance.register_adapter(:redland,self)
  9
+	
  10
+	# TODO: manage context? 
  11
+	# attr_reader :model, :store, :query_language, :context
  12
+
  13
+	# instantiate connection to Redland database
  14
+	def initialize(params = {})
  15
+
  16
+		if params[:location] and params[:location] != :memory
  17
+			# setup file locations for redland database
  18
+			path, file = File.split(params[:location])
  19
+			type = 'bdb'
  20
+		else
  21
+			# fall back to in-memory redland 	
  22
+			type = 'memory'; path = '';	file = '.'
  23
+		end
  24
+		
  25
+		@store = Redland::HashStore.new(type, file, path, false)
  26
+		@model = Redland::Model.new @store
  27
+	end	
  28
+	
  29
+	# yields query results (as many as requested in select clauses) executed on data source
  30
+	def query(query)
  31
+		qs = Query2SPARQL.instance.translate(query)
  32
+		clauses = query.select_clauses.size
  33
+		redland_query = Redland::Query.new(qs, 'sparql')
  34
+		query_results = @model.query_execute(redland_query)
  35
+
  36
+		# verify if the query has failed
  37
+		return false if query_results.nil?
  38
+		return false unless query_results.is_bindings?
  39
+
  40
+		# convert the result to array
  41
+		results = query_result_to_array(query_results) 
  42
+			
  43
+		# we want to write here: 
  44
+		# results.each do |c1, c2, c3, ...| 
  45
+		# 	yield c1, c2, c3, ...
  46
+		# end
  47
+		# for as many clauses as have been defined in the query
  48
+		# 
  49
+		# so something like:
  50
+		# results.each do |clauses.times|
  51
+		# 	yield i, j, k, ...
  52
+		# end
  53
+		if block_given?
  54
+			results.each do |clauses|
  55
+				yield(*clauses)
  56
+			end
  57
+		else
  58
+			results
  59
+		end
  60
+	end
  61
+	
  62
+	# add triple to datamodel
  63
+	# TODO: rewrite, just copied from old code
  64
+	def add(s, p, o)
  65
+		# verify input
  66
+		return false if s.nil? or p.nil? or o.nil?
  67
+		return false if !s.kind_of?(RDFS::Resource) or !p.kind_of?(RDFS::Resource)
  68
+	
  69
+		begin
  70
+		  # TODO: disabled context temporarily, does not work properly in Redland
  71
+			@model.add(wrap(s), wrap(p), wrap(o))
  72
+		  
  73
+		rescue Redland::RedlandError => e
  74
+		  return false
  75
+		end		
  76
+	end
  77
+	
  78
+	def reads?
  79
+		true
  80
+	end
  81
+	
  82
+	def writes?
  83
+		true
  84
+	end
  85
+	
  86
+	################ helper methods ####################
  87
+	def query_result_to_array(query_results)
  88
+	 	results = []
  89
+	 	number_bindings = query_results.binding_names.size
  90
+	 	
  91
+	 	# walk through query results, and construct results array
  92
+	 	# by looking up each result (if it is a resource) and adding it to the result-array
  93
+	 	# for literals we only add the values
  94
+	 	
  95
+	 	# redland results are set that needs to be iterated
  96
+	 	while not query_results.finished?
  97
+	 		# we collect the bindings in each row and add them to results
  98
+	 		results << (0..number_bindings-1).collect do |i|
  99
+	 		
  100
+	 			# node is the query result for one binding
  101
+	 			node = query_results.binding_value(i)
  102
+
  103
+				# we determine the node type
  104
+ 				if node.literal?
  105
+ 					# for literal nodes we just return the value
  106
+ 					node.to_s
  107
+ 				else
  108
+ 				 	# TODO manage blank nodes 				
  109
+ 					RDFS::Resource.lookup(node.uri.to_s)
  110
+	 			end
  111
+	 		end
  112
+	 		# iterate through result set
  113
+	 		query_results.next
  114
+	 	end
  115
+	 	results
  116
+	end	 	
  117
+	
  118
+	def wrap node
  119
+		case node
  120
+		when RDFS::Resource
  121
+			Redland::Uri.new(node.uri)
  122
+		else
  123
+			Redland::Literal.new(node)
  124
+		end
  125
+	end
  126
+end
145  src/adapter/sparql.rb
... ...
@@ -0,0 +1,145 @@
  1
+# sparql adapter
  2
+require 'active_rdf'
  3
+require 'queryengine/query2sparql'
  4
+
  5
+require 'net/http'
  6
+require 'cgi'
  7
+require 'active_rdf'
  8
+
  9
+class SparqlAdapter
  10
+	ConnectionPool.instance.register_adapter(:sparql, self)
  11
+
  12
+	def reads?
  13
+		true
  14
+	end
  15
+	
  16
+	def writes?
  17
+		false
  18
+	end
  19
+	
  20
+	# Instantiate the connection with the SPARQL Endpoint.
  21
+	def initialize(params = {})
  22
+		raise(ActiveRdfError, 'SPARQL adapter initialised with nil parameters') if params.nil?
  23
+		
  24
+		@host = params[:host] || 'm3pe.org'
  25
+		@port = params[:port] || 2020
  26
+		@context = params[:context] || 'books'
  27
+		@result_format = params[:result_format] || :json
  28
+		
  29
+		raise ActiveRdfError, "Result format unsupported" unless (@result_format == :xml or @result_format == :json)
  30
+		
  31
+		# We don't open the connection yet but let each HTTP method open and close 
  32
+		# it individually. It would be more efficient to pipeline methods, and keep 
  33
+		# the connection open continuously, but then we would need to close it 
  34
+		# manually at some point in time, which I do not want to do.
  35
+		
  36
+		@sparql = Net::HTTP.new(@host,@port)
  37
+	end
  38
+	
  39
+	# query datastore with query string (SPARQL), returns array with query results
  40
+	def query(query)
  41
+		qs = Query2SPARQL.instance.translate(query)
  42
+		clauses = query.select_clauses.size
  43
+		
  44
+		# initialising HTTP header
  45
+		case @result_format
  46
+		when :json
  47
+			header = { 'accept' => 'application/sparql-results+json' }
  48
+		when :xml
  49
+			header = { 'accept' => 'application/rdf+xml' }
  50
+		end
  51
+		
  52
+		response = @sparql.get("/#{@context}?query=#{CGI.escape(qs)}", header)
  53
+		# If no content, we return an empty array
  54
+		return Array.new if response.is_a?(Net::HTTPNoContent)
  55
+		return false unless response.is_a?(Net::HTTPOK)
  56
+		response = response.body
  57
+		
  58
+		results = case @result_format
  59
+		when :json
  60
+			parse_sparql_query_result_json response
  61
+		when :xml
  62
+			parse_sparql_query_result_xml response
  63
+		end
  64
+		
  65
+		if block_given?
  66
+			results.each do |clauses|
  67
+				yield(*clauses)
  68
+			end
  69
+		else
  70
+			results
  71
+		end
  72
+	end
  73
+
  74
+	def parse_sparql_query_result_json(query_result)
  75
+    require 'json'
  76
+    
  77
+    parsed_object = JSON.parse(query_result)
  78
+    return [] if parsed_object.nil?
  79
+    
  80
+    results = []    
  81
+    vars = parsed_object['head']['vars']
  82
+    objects = parsed_object['results']['bindings']
  83
+    if vars.length > 1
  84
+      objects.each do |obj|
  85
+        result = []
  86
+        vars.each do |v|
  87
+          result << create_node( obj[v]['type'], obj[v]['value'])
  88
+        end
  89
+        results << result
  90
+      end
  91
+    else
  92
+      objects.each do |obj| 
  93
+        obj.each_value do |e|
  94
+          results << create_node(e['type'], e['value'])
  95
+        end
  96
+      end
  97
+    end
  98
+    return results
  99
+  end
  100
+  
  101
+  def parse_sparql_query_result_xml(query_result)
  102
+    require 'rexml/document'
  103
+    results = []
  104
+    vars = []
  105
+    objects = []
  106
+    doc = REXML::Document.new query_result
  107
+    doc.elements.each("*/head/variable") {|v| vars << v.attributes["name"]}
  108
+    doc.elements.each("*/results/result") {|o| objects << o}
  109
+    if vars.length > 1
  110
+      objects.each do |result|
  111
+        myResult = []
  112
+        vars.each do |v|
  113
+          result.each_element_with_attribute('name', v) do |binding|
  114
+            binding.elements.each do |e|
  115
+              myResult << create_node(e.name, e.text)
  116
+            end            
  117
+          end          
  118
+        end
  119
+        results << myResult
  120
+      end
  121
+      
  122
+    else
  123
+      objects.each do |bs| 
  124
+        bs.elements.each("binding") do |b|
  125
+          b.elements.each do |e|
  126
+						results << create_node(e.name, e.text)
  127
+					end
  128
+        end
  129
+      end
  130
+    end
  131
+    return results
  132
+  end
  133
+  
  134
+  def create_node(type, value)
  135
+    case type
  136
+    when 'uri'
  137
+      RDFS::Resource.lookup(value)
  138
+    when 'bnode'
  139
+      RDFS::Resource.lookup(value)
  140
+      #raise(ActiveRdfError, "blank node not implemented.")
  141
+    when 'literal','typed-literal'
  142
+      value.to_s
  143
+    end
  144
+  end
  145
+end
82  src/federation/connection_pool.rb
... ...
@@ -0,0 +1,82 @@
  1
+# maintains pool of adapter instances that are connected to datasources
  2
+# returns right adapter for a given datasource, by either reusing 
  3
+# existing adapter-instance or creating new adapter-instance
  4
+require 'singleton'
  5
+class ConnectionPool
  6
+	include Singleton
  7
+	
  8
+	# adapters-classes known to the pool, registered by the adapter-class 
  9
+	# itself using register_adapter method, used to select new 
  10
+	# adapter-instance for requested connection type
  11
+	@@registered_adapter_types = Hash.new
  12
+
  13
+	def initialize
  14
+		clear
  15
+	end
  16
+
  17
+	# clears the pool: removes all registered data sources
  18
+	def clear
  19
+	 # pool of all adapters
  20
+		@@adapter_pool = []
  21
+
  22
+    # pool of connection parameters to all adapter
  23
+		@@adapter_parameters = []
  24
+		@write_adapter = nil
  25
+	end
  26
+	
  27
+	# returns the set of currently registered read-access datasources
  28
+	def read_adapters
  29
+		reads = @@adapter_pool.select {|adapter| adapter.reads? }
  30
+	end
  31
+	
  32
+	# returns the currently selected data source for write-access
  33
+	def write_adapter
  34
+		@@write_adapter
  35
+	end
  36
+	
  37
+	# sets the given adapter as currently selected writeable adapter
  38
+	# (sets currently selected writer to nil if given adapter does not support writing)
  39
+	def write_adapter= adapter
  40
+		@@write_adapter = adapter.writes? ? adapter : nil
  41
+	end
  42
+
  43
+	# returns adapter-instance for given parameters (either existing or new)
  44
+	def add_data_source(connection_params)
  45
+		# either get the adapter-instance from the pool 
  46
+		# or create new one (and add it to the pool)
  47
+		index = @@adapter_parameters.index(connection_params)
  48
+		if index.nil?
  49
+		  # adapter not in the pool yet: create it,
  50
+		  # register its connection parameters in parameters-array
  51
+		  # and add it to the pool (at same index-position as parameters)
  52
+		  adapter = create_adapter(connection_params)
  53
+		  @@adapter_parameters << connection_params
  54
+		  @@adapter_pool << adapter
  55
+      adapter
  56
+		else
  57
+		  # if adapter parametrs registered already,
  58
+		  # then adapter must be in the pool, at the same index-position as its parameters
  59
+		  @@adapter_pool[index]
  60
+		end
  61
+	end
  62
+	
  63
+	# adapter-types can register themselves with connection pool by 
  64
+	# indicating which adapter-type they are
  65
+	# TODO: necessary?
  66
+	def register_adapter(type, klass)
  67
+		@@registered_adapter_types[type] = klass
  68
+	end
  69
+	
  70
+	private
  71
+	# create new adapter from connection parameters
  72
+	def create_adapter connection_params
  73
+		# lookup registered adapter klass
  74
+		klass = @@registered_adapter_types[connection_params[:type]]
  75
+		
  76
+		# raise error if adapter type unknown
  77
+		raise(ActiveRdfError, "unknown adapter type #{connection_params[:type]}") if klass.nil?
  78
+		
  79
+		# create new adapter-instance
  80
+		klass.send(:new,connection_params)
  81
+	end
  82
+end
54  src/federation/federation_manager.rb
... ...
@@ -0,0 +1,54 @@
  1
+# manages the federation of datasources
  2
+# distributes queries to right datasources and merges their results
  3
+require 'set'
  4
+require 'federation/connection_pool'
  5
+class FederationManager
  6
+	include Singleton
  7
+	
  8
+	def initialize
  9
+		@@pool = ConnectionPool.instance
  10
+	end
  11
+		
  12
+	# executes read-only queries
  13
+	# by distributing query over complete read-pool
  14
+	# and aggregating the results
  15
+	def query(q, options={:flatten => true})
  16
+		# TODO: manage update queries
  17
+		
  18
+		# ask each adapter for query results
  19
+		# and yield them consequtively
  20
+		if block_given? 
  21
+			@@pool.read_adapters.each do |source| 
  22
+				source.query(q) do |*clauses|
  23
+					yield(*clauses)
  24
+				end
  25
+			end
  26
+		else
  27
+			# build Array of results from all sources
  28
+			results = @@pool.read_adapters.collect { |source| source.query(q) }
  29
+
  30
+			# filter the empty results
  31
+			results.reject {|ary| ary.empty? }
  32
+			
  33
+			# give the union of results
  34
+			union = []
  35
+			results.each { |res| union |= res }
  36
+			
  37
+			# flatten results array if only one select clause
  38
+			# to prevent unnecessarily nested array [[eyal],[renaud],...]
  39
+			union.flatten! if q.select_clauses.size == 1
  40
+			
  41
+			# and remove array (return single value) unless asked not to 
  42
+			if options[:flatten]
  43
+  			case union.size
  44
+  			when 0
  45
+  			 nil
  46
+  			when 1
  47
+  			 union.first
  48
+  			else
  49
+  			 union			
  50
+  			end
  51
+  		end
  52
+		end
  53
+	end
  54
+end
68  src/objectmanager/namespace.rb
... ...
@@ -0,0 +1,68 @@
  1
+# manages namespace abbreviations and expansions
  2
+
  3
+require 'singleton'
  4
+class Namespace
  5
+  include Singleton
  6
+  
  7
+  @@namespaces = Hash.new 
  8
+	@@inverted_namespaces = Hash.new
  9
+
  10
+	# registers a namespace prefix and its associated expansion (full URI)
  11
+	# e.g. :rdf and 'http://www.w3.org/1999/02/22-rdf-syntax-ns#'
  12
+	def register(prefix, fullURI)
  13
+		@@namespaces[prefix.to_sym] = fullURI.to_s
  14
+		@@inverted_namespaces[fullURI.to_s] = prefix.to_sym
  15
+	end
  16
+	
  17
+	# returns a resource whose URI is formed by concatenation of prefix and localname
  18
+  def lookup(prefix, localname)
  19
+    RDFS::Resource.lookup(expand(prefix, localname))
  20
+	end
  21
+	
  22
+	# returns URI (string) formed by concatenation of prefix and localname
  23
+	def expand(prefix, localname)
  24
+		@@namespaces[prefix.to_sym].to_s + localname.to_s
  25
+	end
  26
+
  27
+	# returns prefix (if known) for the non-local part of the URI, 
  28
+	# or nil if prefix not registered
  29
+	def prefix(resource)
  30
+		# get string representation of resource uri
  31
+		uri = case resource
  32
+			when RDFS::Resource: resource.uri
  33
+			else resource.to_s
  34
+		end
  35
+
  36
+		# uri.to_s gives us the uri of the resource (if resource given)
  37
+		# then we find the last occurrence of # or / (heuristical namespace 
  38
+		# delimitor)
  39
+		delimiter = uri.rindex(/#|\//)
  40
+
  41
+		# if delimiter not found, URI cannot be split into (non)local-part
  42
+		return uri if delimiter.nil?
  43
+
  44
+		# extract non-local part (including delimiter)
  45
+		nonlocal = uri[0..delimiter]
  46
+
  47
+		@@inverted_namespaces[nonlocal]
  48
+	end
  49
+	
  50
+	# returns local-part of URI
  51
+	def localname(resource)
  52
+	  # get string representation of resource uri
  53
+		uri = case resource
  54
+			when RDFS::Resource: resource.uri
  55
+			else resource.to_s
  56
+		end
  57
+	 	
  58
+		delimiter = uri.rindex(/#|\//)
  59
+		if delimiter.nil?
  60
+			uri
  61
+		else
  62
+			uri[delimiter+1..-1]
  63
+		end
  64
+	end
  65
+end
  66
+
  67
+Namespace.instance.register(:rdf, 'http://www.w3.org/1999/02/22-rdf-syntax-ns#')
  68
+Namespace.instance.register(:rdfs, 'http://www.w3.org/2000/01/rdf-schema#')
7  src/objectmanager/object_manager.rb
... ...
@@ -0,0 +1,7 @@
  1
+# ObjectManager maps each RDF resource (identified by URI) to a Ruby object.
  2
+# Resources can also be RDFS:Classes, in which case they will be mapped to a Ruby class
  3
+# The ObjectManager returns the right Ruby object given a URI identifying a resource.
  4
+require 'singleton'
  5
+class ObjectManager < Hash
  6
+	include Singleton
  7
+end
221  src/objectmanager/resource.rb
... ...
@@ -0,0 +1,221 @@
  1
+# represents an RDF resource and manages manipulations of that resource, 
  2
+# including data lookup (e.g. eyal.age), data updates (e.g. eyal.age=20),
  3
+# class-level lookup (Person.find_by_name 'eyal'), and class-membership 
  4
+# (eyal.class ...Person)
  5
+
  6
+# TODO: load classes at user's request (or when adding data_source)
  7
+# TODO: factor out class/module construction
  8
+# TODO: when constructing new classes, set their own @@class_uri
  9
+# TODO: add unit test to validate class construction and queries on them
  10
+# TODO: add Person.find_all
  11
+
  12
+require 'objectmanager/object_manager' 
  13
+require 'objectmanager/namespace'
  14
+require 'queryengine/query'
  15
+	   
  16
+module RDFS
  17
+	class Resource
  18
+		# class uri is the uri of the rdf resource being represented by this class
  19
+		@@class_uri = Namespace.instance.expand(:rdfs, :Resource)
  20
+
  21
+		# you can only create new resources through Resource.lookup
  22
+		private_class_method :new
  23
+
  24
+		# uri of the resource
  25
+		attr_reader :uri
  26
+
  27
+		def initialize uri
  28
+			@uri = uri
  29
+		end
  30
+
  31
+		#####                   	#####
  32
+		##### class level methods	#####
  33
+		#####                    	#####
  34
+		
  35
+		# returns Ruby object represting the RDF resource with the given URI (or nil)
  36
+		# (customised initialiser method, Resource.new is forbidden)
  37
+		def Resource.lookup(uri)
  38
+			ObjectManager.instance[uri] ||= new(uri)
  39
+		end
  40
+
  41
+		# returns the predicates that have this resource as their domain (applicable 
  42
+		# predicates for this resource)
  43
+		def Resource.predicates
  44
+			domain = Namespace.instance.lookup(:rdfs, 'domain')			
  45
+			Query.new.select(:p).where(:p, domain, lookup(@@class_uri)).execute || []
  46
+		end
  47
+
  48
+		# manages invocations such as Person.find_by_name
  49
+		def Resource.method_missing(method, *args)
  50
+			method_name = method.to_s
  51
+			
  52
+			# extract predicates on which to match
  53
+			# e.g. find_by_name, find_by_name_and_age
  54
+			if match = /find_by_(.+)/.match(method_name)
  55
+				# find searched attributes, e.g. name, age
  56
+				attributes = match[1].split('_and_')
  57
+
  58
+				# get list of possible predicates for this class
  59
+				possible_predicates = self.predicates
  60
+
  61
+				# build query looking for all resources with the given parameters
  62
+				query = Query.new.select(:s)
  63
+
  64
+				# add where clause for each attribute-value pair,
  65
+				# looking into possible_predicates to figure out
  66
+				# which full-URI to use for each given parameter (heuristic)
  67
+				
  68
+				attributes.each_with_index do |atr,i|
  69
+					possible_predicates.each do |pred|
  70
+						query.where(:s, pred, args[i]) if Namespace.instance.localname(pred) == atr
  71
+					end
  72
+				end
  73
+
  74
+				# execute query
  75
+				return query.execute
  76
+			end
  77
+
  78
+			# otherwise, if no match found, raise NoMethodError (in superclass)
  79
+			super
  80
+		end
  81
+			
  82
+		#####                         #####
  83
+		##### instance level methods	#####
  84
+		#####                         #####
  85
+		
  86
+		# manages invocations such as eyal.age
  87
+		def method_missing(method, *args)
  88
+			# TODO:  method_missing (eyal.age)
  89
+			# 
  90
+			# possibilities:
  91
+			# 1. eyal.age is a property of eyal (triple exists <eyal> <age> "30")
  92
+			# evidence: eyal age ?a, ?a is not nil (only if value exists)
  93
+			# action: return ?a
  94
+			# 
  95
+			# 2. eyal's class is in domain of age, but does not have value for eyal
  96
+			# explain: eyal is a person and some other person (not eyal) has an age
  97
+			# evidence: eyal type ?c, age domain ?c
  98
+			# action: return nil
  99
+			# 
  100
+			# 3. eyal.age is a custom-written method in class Person
  101
+			# evidence: eyal type ?c, ?c.methods includes age
  102
+			# action: inject age into eyal and invoke
  103
+			
  104
+			# maybe change order in which to check these, checking (3) is probably 
  105
+			# cheaper than (1)-(2) but (1) and (2) are probably more probable (getting 
  106
+			# attribute values over executing custom methods)
  107
+			
  108
+			
  109
+			# checking possibility (1) and (2)
  110
+			predicates.each do |pred|
  111
+				if Namespace.instance.localname(pred) == method.to_s
  112
+					# found a property invocation of eyal: option 1) or 2)
  113
+					# query execution will return either the value for the predicate (1)
  114
+					# or nil (2)
  115
+					return Query.new.select(:o).where(self,pred,:o).execute
  116
+				end
  117
+			end
  118
+
  119
+			# checking possibility (3)
  120
+#			self.class.each do |klass| 
  121
+#				if klass.instance_methods.include?(method.to_s) 
  122
+#				  p "should invoke #{method} on #{klass} now"
  123
+#				  return
  124
+#				  #extend(klass)
  125
+#				end
  126
+#			end
  127
+			
  128
+			# if none of the three possibilities work out,
  129
+			# we don't know this method invocation, so we throw NoMethodError (in 
  130
+			# superclass)
  131
+			super
  132
+		end
  133
+
  134
+		# returns classes to which this resource belongs (according to rdf:type)
  135
+		def class
  136
+			types.collect do |type| 
  137
+			
  138
+				# get prefix abbreviation and localname from type
  139
+				# e.g. :foaf and Person
  140
+				localname = Namespace.instance.localname(type)
  141
+				prefix = Namespace.instance.prefix(type)
  142
+
  143
+				# find (ruby-acceptable) names for the module and class 
  144
+				# e.g. FOAF and Person
  145
+				modulename = prefix_to_module(prefix)
  146
+				klassname = localname_to_class(localname)
  147
+
  148
+				# look whether module defined
  149
+				# else: create it
  150
+				_module = if Object.const_defined?(modulename.to_sym)
  151
+										Object.const_get(modulename.to_sym)
  152
+									else
  153
+										Object.const_set(modulename, Module.new)
  154
+									end
  155
+
  156
+				# look whether class defined in that module
  157
+				# if not: define the class insinde that module
  158
+				# and return the class
  159
+				
  160
+				_class = if _module.const_defined?(klassname.to_sym)
  161
+									 _module.const_get(klassname.to_sym)
  162
+								 else
  163
+									 _module.module_eval("#{klassname} = Class.new")
  164
+								 end
  165
+
  166
+				# collect the found/created _class for return
  167
+				_class
  168
+			end
  169
+		end
  170
+
  171
+		# overrides built-in instance_of? to use rdf:type definitions
  172
+		def instance_of?(klass)
  173
+			self.class.include?(klass)
  174
+		end
  175
+
  176
+		# returns all predicates that fall into the domain of the rdf:type of this 
  177
+		# resource
  178
+		def predicates
  179
+			type = Namespace.instance.lookup(:rdf, 'type')
  180
+			domain = Namespace.instance.lookup(:rdfs, 'domain')
  181
+			Query.new.select(:p).where(self,type,:t).where(:p, domain, :t).execute || []
  182
+		end
  183
+
  184
+		# returns all rdf:types of this resource
  185
+		def types
  186
+			type = Namespace.instance.lookup(:rdf, 'type')
  187
+			
  188
+			# we lookup the type in the database
  189
+			# if we dont know it, we return Resource (as toplevel)
  190
+			# this should in theory actually never happen (since any node is a rdfs:Resource)
  191
+			# but could happen if the subject is unknown to the database
  192
+			Query.new.select(:t).where(self,type,:t).execute(:flatten => false) || [Namespace.instance.lookup(:rdfs,"Resource")]
  193
+		end	
  194
+
  195
+		# returns uri of resource, can be overridden in subclasses
  196
+		def to_s
  197
+			'<' + uri + '>'
  198
+		end
  199
+				
  200
+		def label
  201
+		  Namespace.instance.localname(self)
  202
+		end
  203
+		
  204
+		#####                         #####
  205
+		##### private methods         #####
  206
+		#####                         #####
  207
+		
  208
+		private
  209
+		def prefix_to_module(prefix)
  210
+		  # TODO: replace illegal characters
  211
+		  raise ActiveRdfError, 'bug 62491' if prefix.to_s.empty?
  212
+		  prefix.to_s.upcase
  213
+		end
  214
+		
  215
+		def localname_to_class(localname)
  216
+		  # TODO: replace illegal characters (numbers,#<(*, etc)
  217
+		  # replace spaces by _
  218
+		  localname.to_s
  219
+		end		
  220
+	end
  221
+end
52  src/queryengine/query.rb
... ...
@@ -0,0 +1,52 @@
  1
+# represent a query on a datasource, abstract representation of SPARQL features
  2
+# is passed to federation/adapter for execution on data
  3
+require 'federation/federation_manager'
  4
+class Query	
  5
+	attr_reader :select_clauses, :where_clauses
  6
+	def initialize
  7
+		@select_clauses = []
  8
+		@where_clauses = []
  9
+	end
  10
+	
  11
+	def select *s
  12
+		s.each do |e|
  13
+			@select_clauses << parametrise(e)
  14
+		end
  15
+		self
  16
+	end
  17
+	
  18
+	def where s,p,o
  19
+		@where_clauses << [s,p,o].collect{|arg| parametrise(arg)}
  20
+		self
  21
+	end	
  22
+	
  23
+	# execute query on data sources
  24
+	# either returns result as array 
  25
+	# (flattened into single value unless specified otherwise) 
  26
+	# or executes a block (number of block variables should be 
  27
+	# same as number of select variables)
  28
+	# 
  29
+	# usage: results = query.execute
  30
+	# usage: query.execute do |s,p,o| ... end
  31
+	def execute(options={:flatten => true}, &block)
  32
+		if block_given?
  33
+			FederationManager.instance.query(self) do |*clauses|
  34
+				block.call(*clauses)
  35
+			end
  36
+		else
  37
+			FederationManager.instance.query(self, options)
  38
+		end
  39
+	end
  40
+	
  41
+	private
  42
+	def parametrise s		
  43
+		case s
  44
+		when Symbol
  45
+			'?' + s.to_s
  46
+		when RDFS::Resource
  47
+		  s
  48
+		else
  49
+			'"' + s.to_s + '"'
  50
+		end
  51
+	end
  52
+end
14  src/queryengine/query2sparql.rb
... ...
@@ -0,0 +1,14 @@
  1
+# translates abstract query into SPARQL that can be executed on SPARQL-compliant data source
  2
+require 'singleton'
  3
+
  4
+class Query2SPARQL
  5
+	include Singleton
  6
+	def translate(query)
  7
+		str = ""
  8
+		str << "SELECT DISTINCT #{query.select_clauses.join(' ')} "
  9
+		
  10
+		# concatenate each where clause using space (e.g. 's p o')
  11
+		# and concatenate the clauses using dot, e.g. 's p o . s2 p2 o2 .'		
  12
+		str << "WHERE { #{query.where_clauses.collect{|w| w.join(' ')}.join('. ')} .}"
  13
+	end
  14
+end
92  test/adapter/ts_redland_adapter.rb
... ...
@@ -0,0 +1,92 @@
  1
+require 'test/unit'
  2
+require 'active_rdf'
  3
+require 'adapter/redland_adapter'
  4
+require 'federation/federation_manager'
  5
+require 'queryengine/query'
  6
+# require 'active_rdf/test/common'
  7
+
  8
+class TestObjectCreation < Test::Unit::TestCase
  9
+	def setup
  10
+		ConnectionPool.instance.clear
  11
+	end
  12
+	
  13
+	def teardown
  14
+	end
  15
+	
  16
+	def test_registration
  17
+		adapter = ConnectionPool.instance.add_data_source(:type => :redland)
  18
+		assert_instance_of RedlandAdapter, adapter
  19
+	end
  20
+	
  21
+	def test_redland_connections
  22
+		adapter = RedlandAdapter.new({})
  23
+		assert_instance_of RedlandAdapter, adapter
  24
+	end
  25
+	
  26
+	def test_simple_query
  27
+		adapter = ConnectionPool.instance.add_data_source(:type => :redland)
  28
+
  29
+		eyal = RDFS::Resource.lookup 'eyaloren.org'
  30
+		age = RDFS::Resource.lookup 'foaf:age'
  31
+		test = RDFS::Resource.lookup 'test'
  32
+		
  33
+		adapter.add(eyal, age, test)
  34
+		result = Query.new.select(:s).where(:s, :p, :o).execute
  35
+
  36
+		assert_instance_of RDFS::Resource, result
  37
+		assert_equal 'eyaloren.org', result.uri
  38
+	end
  39
+	
  40
+	def test_federated_query
  41
+		adapter1 = ConnectionPool.instance.add_data_source(:type => :redland)
  42
+		adapter2 = ConnectionPool.instance.add_data_source(:type => :redland, :fake_symbol_to_get_unique_adapter => true)
  43
+		
  44
+		eyal = RDFS::Resource.lookup 'eyaloren.org'
  45
+		age = RDFS::Resource.lookup 'foaf:age'
  46
+		test = RDFS::Resource.lookup 'test'
  47
+		test2 = RDFS::Resource.lookup 'test2'
  48
+		
  49
+		adapter1.add(eyal, age, test)
  50
+		adapter2.add(eyal, age, test2)
  51
+		
  52
+		results = Query.new.select(:s, :p, :o).where(:s, :p, :o).execute
  53
+		assert_equal 2, results.size
  54
+		assert_instance_of RDFS::Resource, results[0][0]
  55
+		
  56
+		literals = [results[0][2].uri, results[1][2].uri]
  57
+		assert literals.include?('test')
  58
+		assert literals.include?('test2')
  59
+	end
  60
+