October 09, 2009

Posted by John

Tagged mongodb and mongomapper

Older: Lookin' on Up...To the East Side

Newer: Know When to Fold 'Em

More MongoMapper Awesomeness

September was a month of craziness and for the first month in quite a while I did not post here. I promise it hurt me as much as it hurt you. In an effort to get back in the rhythm, I am going to start with an easy article. MongoMapper has been getting a lot of love lately and I thought I would mention some of the awesomeness.

Dynamic Finders

Dynamic finders are so darn handy in ActiveRecord. How many times have you used User.find_by_email and the like? Thankfully David Cuadrado took a stab at it. I took what he started, tested it a bit harder and added it onto document associations as well. This means when you have a document with a many documents association, you can now use dynamic finders that are scoped to that association.

class User
  include MongoMapper::Document

  many :posts
end

class Post
  include MongoMapper::Document
  key :user_id, ObjectId
  key :title, String
end

user = User.create
user.posts.create(:title => 'Foo')

# would return post we just created
user.posts.find_by_title('Foo')

Document associations now also have all the normal Rails association methods such as build, create, find, etc.

Logging

The mongo ruby driver added logging support so a few days ago, I added some basic support for accessing and using that logger from within MongoMapper. When you pass a logger instance to the ruby driver, you can access that connections logger instance from MongoMapper.logger like so:

logger = Logger.new('test.log')
MongoMapper.connection = Mongo::Connection.new('127.0.0.1', 27017, :logger => logger)
MongoMapper.logger # would be equal to logger

Tailing the log would give you output like the following:

MONGODB db.$cmd.find({"count"=>"statuses", "query"=>{"project_id"=>"4aceaabed072c4745f0003ca"}, "fields"=>nil})
MONGODB db.$cmd.find({"count"=>"statuses", "query"=>{"project_id"=>"4aceaabed072c4745f0003ce"}, "fields"=>nil})

The nifty part about this is you can setup your Mongo::Connection to use Rails.logger and then all your mongo queries show up in your Rails logs if you have your log level set low enough. This has been very handy for me working on MongoMapper because I can see exactly what MM is sending to Mongo behind the scenes.

Because of this addition, I noticed that every find(:first) was using :order => ‘$natural’ which doesn’t allow using indexes and leads to slow queries. I removed the default order so instead it is just a find with a limit of 1, which should help make a few parts perform better.

Dirty Attributes

ActiveRecord’s dirty attributes is such a cool feature that yesterday, I spent a few hours porting it to MongoMapper::Document. Now you can do things like:

class Foo
  include MongoMapper::Document
  key :phrase, String
end

foo = Foo.new
foo.changed? # false
foo.phrase_changed? # false

foo.phrase = 'Dirty!'

foo.changed? # true
foo.phrase_changed? # true
foo.phrase_change # [nil, 'Dirty!']

I’m sure there will be edge cases, but as we find them we can fortify the tests and go from there.

Custom Data Types

With the 0.4 release came the transition from typecasting to custom data types. Now, instead of natively defining typecasting for “allowed” data types, you can have any data type that you like. You just have to do the conversion to and from mongo yourself. Making your own data types is as simple as:

class Foo
  def self.to_mongo(value)
    # convert value to a mongo safe data type
  end
  
  def self.from_mongo(value)
    # convert value from a mongo safe data type to your custom data type
  end
end

class Thing
  include MongoMapper::Document
  key :name, Foo
end

This means each time the name of Thing is saved to mongo or pulled out of mongo it will be ran through the Foo#to_mongo and Foo#from_mongo to make sure it is exactly what you want it to be.

Out of the box, MongoMapper supports Array, Binary, Boolean, Date, Float, Hash, Integer, ObjectId, String, and Time. You can check out the support file and tests to see how this works.

Time Zones

One not on times, since I mentioned it above is that all times are stored in the datbase as utc now. Also, if you have Time.zone set, all times are converted to the current time zone going to and from the database. This actually turned out to be really easy. We’ll see if I did it all correctly once people start pounding on it I guess. :)

Lazy Loading

One thing that I’ve been working on in between other features is making MongoMapper more lazy. I have already made connection, database and collection lazy so MM doesn’t actually create the connection or connection to the database until needed which makes MM work a lot better with Rails.

I still need to make indexes lazy, so that is the next thing to tackle. I’m thinking once that is in, I’ll have something like MongoMapper.ensure_indexes!, similar to DataMapper.auto_migrate!, which actually ensures the indexes exist rather than doing that the second a class loads.

Internal Improvements

Along with all the public features, I have been working on the internals of MM whenever I get a chance. They still need cleaning up, but things are getting better. Along with some refactoring, I did some work to speed the tests up.

The tests were starting to creep up to around 40 seconds which was driving me nuts. I did a bit of work and realized that clearing every collection before every test was causing most of the slowdown so I pruned the functional tests to only clear the collections that were actually used in that test. This cut the time from around 40 seconds to 10. Yep, huge!

Conclusion

There are still rough parts and I would recommend MongoMapper for beginners, but if you can troubleshoot not only your own code but others, MM is in a good place for you. Up until now, I’ve been working on adding features that I needed similar to ActiveRecord, but I am almost to a place where I am going to start adding features to MM that can literally only exist because of MongoDB.

The next month is going to see some really cool things like upserts, modifiers ($set, $inc, $dec, $push, $pull, etc.) and the like make their way into MM. I also have some plans for an identity map implementation. Oooohs and aaaaaahs abound!

8 Comments

  1. Awesome work, John! The logging is definitely something I’m looking forward to using.

  2. This is really nice! I’m already testing it :)

    I’ve found one bug by the way:

    
    NoMethodError (undefined method `try' for Tue Oct 06 06:25:25 UTC 2009:Time):
        /Library/Ruby/Gems/1.8/gems/mongo_mapper-0.5.2/lib/mongo_mapper/support.rb:146:in `to_utc_time'
    

    I replaced the code by

    to_local_time(value).utc

  3. @Bruno You must have a pretty old version of active support. Update to the latest and you’ll be fine. I’ll make a note to require at least a specific version of active support.

  4. Damn… John you’re on fire!

    Great work on all the changes (and the help of the contributors). MM is really coming into it’s own. I was always keen to see a Datamapper adaptor for mongo but perhaps not anymore, MM is at such a point that it does the important stuff without the complexity.

    Identity mapping is sounding really exciting along with all those mongo specific things.

    If recall someone in a branch had done some work on named scopes, do you intend to add something like that to MM?

  5. Wow…..this MongoMapper thing ist really great!

  6. Hi John

    Thanks for your great work on mongo_mapper.
    I’m using it to try a new web app with mongoDB as the database.

    I wonder what is the best way to map nested structure for a Collection in mongo_mapper.
    Here is an example of a structure I’d like to have in mongoDB :

    
      {
        "_id": "",
        "name": "",
        "address" : {
          "address1" : "",
          "address2" : "",
          "address3" : "",
          "zip" : "",
          "city" : "",
          "country" : {"name" : "", "code" : ""},
          "state" : "",
          "province" : "",
          "continent" : {"name" : "", "code" : ""},
          "airport" : {"name" : "", "code" : ""},
        },
        "price" : { "min" : "", "max" : "", "currency_code" : ""},
        "position" : {"lat" : "", "lng" : "", "accuracy" : ""},
        "contacts" : {"tel" : "", "email" : "", "web" : ""},
        "descriptions" : {"fr" : "", "en" : ""},
      }
    

    Do I have to create a new Class for price/position/contacts/… ? I tend to think that I do.
    If I need to, how can I make the equivalent of “has_one” relations ? I only have one price. I see 2 options : nesting attributes (like in my example) or declare basic keys : price_min, price_max, … which is best ?

    Thanks for any help.

  7. Transparent getters/setters would be very cool to have in MM, like we have in ActiveRecord.

    I know it may be tricky or even impossible because attributes are not (or not easily) predictable, but i’d be very good.

  8. good job guys! Thanks for yours big work!

Sorry, comments are closed for this article to ease the burden of pruning spam.

About

Authored by John Nunemaker (Noo-neh-maker), a programmer who has fallen deeply in love with Ruby. Learn More.

Projects

Flipper
Release your software more often with fewer problems.
Flip your features.