Skip to content

A library for taking an XML stream and persisting objects from it to a database, based on @pauldix's sax-machine

Notifications You must be signed in to change notification settings

MikeSofaer/sax-mapper

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

19 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

SAXMapper is a database persistence extension to SAXMachine.
It uses DataObjects/MySQL going in, and DataMapper coming back out.

SAXual Replication supports getting multiple instances of an object without an
explicit wrapper class, just by specifying the wrapper tag, using parse_multiple.

You can also set a column as a remote primary key, and it will overwrite rather
than add records where that key is a duplicate (you need to separately set
the column to be have a unique index, since it uses ON DUPLICATE KEY UPDATE)

You can specify a batch size to save in, and whether to wrap the save in a transaction.

Finally, you can mark fields as required, and it will raise an expection if the
XML is missing that field.

Many thanks to Paul Dix, who helped me integrate this with SAXMachine, and Dan
Kubb, who helped me with DataObjects and gave hints on memory footprint.
All bugs and memory leaks are mine, though!

Example:

require 'sax-mapper'
class Person
  include SaxMapper
  element :sourced_id, :required => true
  element :given, :as => :given_name, :required => true
  element :family, :as => :family_name, :required => true
  element :email, :required => true

  table "people"
  tag :person
  key_column :sourced_id
end

Person.parse_multiple(xml) will return an array of Person objects, found inside
<person></person> tags.  You can save them to the DB with Person.save(array)

Person.save(array, :batch_size => 3000, :transaction => true) will try to save them in batches of 3000, and roll everything back if any save fails.

Multiple values with the same sourced_id will replace each other in the DB.

gem install MikeSofaer-sax-mapper

Enjoy!

SAXMapper is written by Michael Sofaer, July 2009, and MIT licenced.

About

A library for taking an XML stream and persisting objects from it to a database, based on @pauldix's sax-machine

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages