Document best practices for ensure_index

[kylev]

Right now I'm putting "ensure_index" statements right in my models, but this seems likely to cause problems (or at least a lot of no-ops). How are people using ensure_index? Do they call it from a rake task post-deploy? Put it in an initializer and hope everyone remembers to set ":background => true" or suffer blocking deploy hell?

It seems like a pretty important topic for it to be utterly absent from the docs.

[jnunemaker]

Indexing should not be tied to your application's boot process. I do them in a rake task and run it separately whenever it is needed.

On Tuesday, November 1, 2011 at 9:06 PM, Kyle VanderBeek wrote:

Right now I'm putting "ensure_index" statements right in my models, but this seems likely to cause problems (or at least a lot of no-ops). How are people using ensure_index? Do they call it from a rake task post-deploy? Put it in an initializer and hope everyone remembers to set ":background => true" or suffer blocking deploy hell?

It seems like a pretty important topic for it to be utterly absent from the docs.

Reply to this email directly or view it on GitHub:
#337

[kylev]

Excellent, that's what I thought. I'll just put in this ticket, in case it gets searched for, that I am now writing a db:mongo:index rake task to do the work. It looks something like this:

namespace :db do
  namespace :mongo do

    desc "Create mongo_mapper indexes"
    task :index => :environment do
      Pants.ensure_index :color

      Outfit.ensure_index :nickname, :unique => true
      Outfit.ensure_index [[:created_at, Mongo::DESCENDING]], :background => true
    end

    desc "Remove old MongoDB indexes"
    task :deindex => :environment do
      # If you find out an index is un-needed, you should add an appropriate Model.collection.drop_index here.
      # MongoMapper doesn't yet have an inverse of ensure_index, so you have to do it via the index name and Mongo driver.
    end
  end
end

No: I do not own enough clothes to warrant MongoDB-backed tracking. It's just an example.

[bkeepers]

@kylev you want to take a crack at documenting it? You just need to check out the gh-pages branch and make the changes there.

[hamin]

@jnunemaker @bkeepers @kylev i've actually been looking at ensure_index lately too. One thing that I came across is that mongo mapper is actually NOT doing ensure_index per se the ensure_index method for a collection provided by the mongo ruby driver. The mongomapper method is using create_index under the hood. As far as mongo and the ruby driver are concerned, ensure_index and create_index are different, so for the MM method to be called ensure_index and actually just use create_index is misleading imho. @jnunemaker is there another reason why MM's ensure_index map's to the ruby driver's create_index ?

I'd be happy to send a pull-request to make sure that MM's ensure_index actually uses the ruby driver's ensure_index method under the hood. I was also thinking of mapping, these ruby driver methods to the MM indexes plugin: drop_index, drop_indexes, create_index, and index_information. I know that one can easily just drop to the ruby driver to call those methods, but it would be nice to have them in the indexes plugin. What do you guys think?

[bkeepers]

@hamin sounds reasonable. I don't know that I would map the other methods, simply because I think it's really unlikely that they'll be used, but I'm not opposed.

One idea I've been throwing around lately is a way to define syntaxes in a non-ruby syntax. I'd love to have a file like db/indexes.yml (yaml is not a great format for this, I tried) that a built-in rake task can look for and call to load the indexes. I'm thinking non-ruby so the rake task doesn't have to load all the models and everything. But if Ruby makes the most sense, then that's fine to. But basically, it'd be like db/seeds.rb.

thoughts?

[hamin]

so the reason i though the other methods would be good to have is coz i was thinking of something similiar to what you were saying, something like db/seeds.rb for indexes. I thought that if the other methods were mapped too it would make it convenient to provide rake tasks to reindex and drop_index . But then again it would have to load the models.

and just so i'm clear, you guys think its fine to have MM ensure_index call the ruby driver's ensure_index method and MM create_index method call the ruby driver's create_index method? Just feel like we need to be consistent with the driver
@bkeepers @jnunemaker thoughts?

[bkeepers]

@hamin that sounds good to me.

[kylev]

@bkeepers Do you have a methodology for previewing the textile contents? I started on additions to the docs, but have no idea if they'll render correctly. Do you have a little textile web server or a Rakefile to generate them?

[brianhempel]

@kylev check out http://mongomapper.com/documentation/contributing.html I think that has the answer to your question.

[bkeepers]

I made the changes we talked about, along with adding a rake task. See 4529a72 and 7a8d0dc

[hamin]

@bkeepers damn you stole my commits :) nice job, what's Forwadable? I'm assuming it just forwards those named methods to the Ruby driver ?

[hamin]

@bkeepers feel like we should definitely update the docs for indexes to reveal that change

[bkeepers]

@hamin Sorry, I was working on some other stuff and just did it. Forwardable is like delegation in ActiveSupport.

I'll take your docs request as a volunteering to do it. :)

[hamin]

@bkeepers you bastard :)

[bkeepers]

@hamin hey man, I don't use the docs, so I don't care about them.

[hamin]

@bkeepers i was just jesting...no i'd be more than happy to contribute to the docs again...i always end up learning something new when i do :)

[SaschaKonietzke]

Wouldn't it be nicer to keep the index definitions next to the keys in the models, but use a rake task to actually create the indexes?

Meaning "ensure_index :name" in a model just marks the key for indexing, and the rake task creates the actual index?

[bkeepers]

I could go either way. Feel free to submit a patch to make it work like you describe.

[JonKernPA]

Thanks for the inspiration... I documented my own progression through the forest of Indexing... Pretty sure there is lots more bushwhacking left, but I did arrive at a little clearing.

http://technicaldebt.com/?p=1089