July 13, 2010

Posted by John

Tagged mongodb and scout

Older: MongoMapper 0.8: Goodies Galore

Newer: Caching With Mongo

Mongo Scout Plugins

As I have stated before, Harmony is happily hosted with the fine folks at Railsmachine. They use and recommend Scout. While I have been quite happily using several scout plugins to keep track of Harmony, including the MongoDB Slow Query Plugin, I felt like a few things were missing. Last night and tonight, I did a bit of hacking and created a few more plugins to help round out our view into Mongo.

Mongo Stats

First, in order to learn how scout plugins work, I started with something simple: db stats. You may or may not know that you can grab stats for your Mongo databases quite easily from the ruby driver. Using this along with scout’s plugin API, I wrote a Mongo Stats plugin that reports a few things about your database:

  • The number of objects
  • The size of your data
  • The storage size of your data
  • The number of indexes
  • The size of your indexes

Nothing fancy, but a good start and since keeping your indexes in RAM is important, having the index size reported continually will be quite handy in the future.

Mongo Server Status

Next, I did some digging on mongodb.org and found the server status command. I remembered that Kyle had written about how to call any Mongo command from the Ruby driver and a few minutes later, I had even more mongo information to peruse. The server status plugin keeps track of:

  • btree accesses, hits, misses, resets, and miss ratio
  • global lock total time, lock time, and lock ratio
  • background flushes total, total ms, average ms and last ms
  • memory bits, resident, virtual, and mapped (if your Mongo version is up to date)

Most of this information is not super helpful now, but down the road I know I will be happy I was logging it, especially when things go wrong (they always do).

If you are curious as to how you can call the server status command in ruby, here is the money shot:

db = Mongo::Connection.new.db('testing')
db.command('serverStatus' => 1)

It returns a hash with all kinds of information. Obviously, you would have to setup your connection correctly to do this.

Mongo Ops

But wait! There is more! The server status command also brings back operation counts. You can get totals for commands, deletes, get mores, inserts, queries, and updates. With this information and scout’s handy remember/memory, I can keep track of the numbers for these values from the last run and compare them with the current run to get up to date values for the mongo ops plugin. If you aren’t giggling right now, well, there is nothing I can do for you.

Harmony’s numbers are not going to impress anyone right now, but I will post a few screenshots so you can get some idea of what the plugins look like before running off to install them.

With the addition of these three plugins, in cooperation with the slow query plugin, and entirely thanks to Scout, I now have lots of numbers and graphs that allow me to watch growth. Fun stuff!

What are you using to keep track of Mongo?

7 Comments

  1. Great article, John, as usual!

  2. Rodrigo Dellacqua Rodrigo Dellacqua

    Jul 14, 2010

    Good stuff, that plugin structure from scout seems very helpful into integrating your needs into a single app.

  3. Really nice work on these, John. Let’s get these incorporated into the plugin directory — I’ll be in touch.

  4. That’s the first time I’ve heard “money shot” and “Scout” together.

    Why does something so wrong feel so right?

  5. Great stuff! Can’t wait to get home and back to “work” and try it out!

  6. The Server Density guys have MongoDB monitoring plugin and they (at least supposed to :) ) know what they’re doing since they’re using MongoDB internally.

    Here are the pretty graphs Since I’m new to mongo I have no idea what some of these metrics mean :)

    Most cryptic are Asserts (what are these) and Connections (why do I have 20 “Current Connections” on idle server?)

  7. jnunemaker: thanks for the mention, we were glad to have you using our beta :) We have added alot more to our admin since then, but its good to see others keeping pace.

    Evgeniy: The current connections are controlled by the number of threads you have open * the connection pool size per thread. This is a configurable option with the Mongo Ruby Driver.

Thoughts? Do Tell...


textile enabled, preview above, please be nice
use <pre><code class="ruby"></code></pre> for code blocks

About

Authored by John Nunemaker (Noo-neh-maker), a web developer and programmer who has fallen deeply in love with Ruby. More about John.

Syndication

Feed IconRailsTips Articles - An assortment of howto's and thoughts on Ruby and Rails.

Feed IconRails Quick Tips - Ruby and Rails related links.