-
Notifications
You must be signed in to change notification settings - Fork 33
Tutorial: Storing persistent data
This document is a tutorial on how to use Autumn’s DataMapper integration to add persistent storage to your leaf. If you’d like an overview of DataMapper, please visit its website. If you’d like an overview of DataMapper integration in Autumn, consult the README.
To use DataMapper you may need to edit the Gemfile and change the DataMapper adapter to one suited for your database (by default it’s dm-sqlite-adapter).
Autumn already includes an example leaf, Scorekeeper, complete with a full data model. However, I realize it can be a bit daunting to jump right into the middle of that, especially since Scorekeeper is not a simple leaf… So let’s start from the beginning.
In this tutorial, I’ll show you how to build a very simple bot that remembers the last time each channel member said something. People will be able to query it to figure out if their friends were recently talking in IRC.
Let’s generate our leaf first by typing script/generate leaf last_online. A new leaf directory, last_online, will be created in the leaves directory. We need to add this leaf to our stem, so open the config/seasons/[season]/leaves.yml file (if you are using a leaves.yml file) and add an entry for your leaf:
LastOnline:
class: LastOnline
We’ve named the leaf after its class, to keep things simple. Now we need to configure the database connection. In the database.yml file, add an entry for our new leaf “LastOnline”. In this example I will use SQLite 3, but you can set up the database for MySQL or PostgreSQL if you prefer.
LastOnline: sqlite:leaves/last_online/data/development.db
I’d then create the database with sqlite3 leaves/last_online/data/development.db.
I’ve named my database connection after my leaf so they’ll be associated automatically. See the README for a more in-depth discussion on naming database connections.
To keep things extra-simple, we’re going to have only one table and therefore only one model object. This is therefore not a spectacular way to learn about DataMapper’s many features, like associations and lazy loading and what-have-you. Again, visit the DataMapper website to learn about DataMapper’s features; this tutorial just shows you how to use DataMapper in Autumn.
Anyway, we need three pieces of information to form a complete picture on when someone last talked in IRC: The time they talked, their nick, and the channel they were in. That’s the minimum, but for good fun, we’ll throw in a fourth column: what they said. (Maybe it’s something helpful like, “Going to the gym, be back at 7.”)
DataMapper is an active-record ORM (like Ruby on Rails’s ActiveRecord class), so in general, one model object corresponds to one table and one attribute of that object corresponds to one column in the table.
We’re going to define the table by creating a new mode object called Event. This will automatically correspond to a table called events. So, first off, create a file at leaves/last_online/models/event.rb. In it, create your class, and add the four properties specified above:
class Event
include DataMapper::Resource
property :nick, String, :nullable => false, :limit => 16
property :channel, String, :nullable => false, :limit => 16
property :message, String, :nullable => false
property :updated_at, DateTime, :default => 'NOW()'
end
(Different IRC servers have different length limits on nick and channel names; make sure 16 characters is enough for your server.)
Unlike with ActiveRecord, the updated_at column will not be set automatically; we will have to do it manually. (Alternatively you could use the dm-timestamps gem to get it automagically.)
With our leaf’s models defined we can now populate the database. Type DB=LastOnline rake db:migrate to turn the model objects into database tables. Verify that the table has been added to your database and its columns are correct.
Our leaf will respond to the “!seen” command. The command will be followed with a nickname. The leaf will then return the last thing that person said in the sender’s channel, along with a timestamp. Here’s an example:
(in channel #flowers)
<Someone> !seen Sancho <LastOnline> [10/19/07 10:15 PM] <Sancho> have fun!
(in channel #honeybees)
<Someone> !seen Sancho <LastOnline> Sancho has not been seen in this channel.
So, let’s write this “!seen” command:
def seen_command(stem, sender, reply_to, msg)
unless stem.nick? msg
render :invalid_nick
return
end
event = Event.first(:channel.eql => reply_to, :nick.eql => msg)
var :name => msg if event.nil?
var :event => event
end
This method finds the first row in the events table that has a matching channel and nick combination, then passes it to the format method, which we will write next. It only accepts valid nicks. Now let’s write a view in the views/seen.txt.erb file:
<% if var(:event) %>
[<%= var(:event).updated_at.strftime '%m/%d/%y %I:%M %p' %>] <<%= var(:event).nick %>> <%= var(:event).message %>
<% else %>
<%= var :name %> has not been seen in this channel.
<% end %>
And we’ll need to add the invalid_nick view as well, in views/invalid_nick.txt.erb:
I need a nick to look up.
Now that we’ve handled reading these events out from the database, we of course need to deal with writing them into the database. To do this, we will override the did_receive_channel_message method to write into the database each time a new message is received.
def did_receive_channel_message(stem, sender, channel, msg)
event = Event.find_or_create :channel => channel, :nick => sender[:nick]
event.message = msg
event.updated_at = Time.now
event.save
end
I realize the database-savvy among you are cringing at the thought of both reading from and writing to the database (up to twice) each and every time a message is received, but this is a K.I.S.S. example. The creation of an in-memory cache that periodically flushes to DB is left as as exercise for the reader.
Anyway, that Event#find_or_create call either finds us or makes for us a row containing the channel and nick of the sender. That’s all we need to uniquely identify a row in our table schema. Given that row, we set the other fields, and save the record.
And that’s it! Our leaf is complete!
Although for the sake of example, this leaf is finished, it actually has a long way to go.
First off, it’s not very smart about nicks. Sancho could change his nick to “Sancho|eating”, and the bot would think it’s a completely different person. I’d recommend checking the hostmask or stripping the “|eating” part out.
Secondly, while the database schema is (mostly) solid, our usage of the database is pretty bad. As I mentioned with regards to the did_receive_channel_message method, it would probably be a good idea to store these new records in memory, only writing them to database periodically (say, every 2 minutes).
Even without doing that, the did_receive_channel_message method can be optimized. The find_or_create call is one DB lookup and possibly one insert; the save call is yet another insert. That means we are doing three DB operations when we really should be doing one or two.
Finally, we should probably spawn off a separate thread to purge old entries from the database. If someone logs into the IRC server once, and never again; or if someone changes their nick to something like “sdgjsfdhie” to say one thing, they will get an entry in the database. That entry will sit there for all eternity. It’s easy to see how your DB could grow very large over time. A reaper thread would delete rows with updated_at columns earlier than, say, a couple weeks. After all, if you want to know when your friend was last online, beyond a couple of weeks, what difference does it make? He was online “a long time ago,” for all intents and purposes.
These are of course the standard concerns you must address when you bring your leaf wide-eyed into the world of relational databases. DataMapper will help you get your persistent storage off the ground quickly and painlessly.