December 01, 2011

Posted by John

Tagged api and gauges

Older: Stupid Simple Debugging

Newer: Acquired

Creating an API

A few weeks back, we publicly released the Gauges API. Despite building Gauges from the ground up as an API, it was a lot of work. You really have to cross your t’s and dot your i’s when releasing an API.

1. Document as You Build

We made the mistake of documenting after most of the build was done. The problem is documenting sucks. Leaving that pain until the end, when you are excited to release it, makes doing the work twice as hard. Thankfully, we have a closer on our team who powered through it.

2. Be Consistent

As we documented the API, we noticed a lot of inconsistencies. For example, in some places we return a hash and in others we returned an array. Upon realizing these issues, we started making some rules.

To solve the array/hash issue, we elected that every response should return a hash. This is the most flexible solution going forward. It allows us to inject new keys without having to convert the response or release a whole new version of the API.

Changing from an array to a hash meant that we needed to namespace the array with a key. We then noticed that some places were name-spaced and others weren’t. Again, we decided on a rule. In this case, all top level objects should be name-spaced, but objects referenced from a top level object or a collection of several objects did not require name-spacing.

{users:[{user:{...}}, {user:{...}}]} // nope
{users:[{...}, {...}]} // yep
{username: 'jnunemaker'} // nope
{user: {username:'jnunemaker'}} // yep 

You get the idea. Consistency is important. It is not so much how you do it as that you always do it the same.

3. Provide the URLs

Most of my initial open source work was wrapping APIs. The one thing that always annoyed me was having to generate urls. Each resource should know the URLs that matter. For example, a user resource in Gauges has a few URLs that can be called to get various data:

{
  "user": {
    "name": "John Doe",
    "urls": {
      "self": "https://secure.gaug.es/me",
      "gauges": "https://secure.gaug.es/gauges",
      "clients": "https://secure.gaug.es/clients"
    },
    "id": "4e206261e5947c1d38000001",
    "last_name": "Doe",
    "email": "john@doe.com",
    "first_name": "John"
  }
}

The previous JSON is the response of the resource /me. /me returns data about the authenticated user and the URLs to update itself (self), get all gauges (/gauges), and get all API clients (/clients). Let’s say next you request /gauges. Each gauge returned has the URLs to get more data about the gauge.

{
  "gauges": [
    {
      // various attributes
      "urls": {
        "self":"https://secure.gaug.es/gauges/4ea97a8be5947ccda1000001",
        "referrers":"https://secure.gaug.es/gauges/4ea97a8be5947ccda1000001/referrers",
        "technology":"https://secure.gaug.es/gauges/4ea97a8be5947ccda1000001/technology",
        // ... etc
      },
    }
  ]
}

We thought this would prove helpful. We’ll see in the long run if it turns out to work well.

4. Present the Data

Finally, never ever use to_json and friends from a controller or sinatra get/post/put block. At least as a bare minimum rule, the second you start calling to_json with :methods, :except, :only, or any of the other options, you probably want to move it to a separate class.

For Gauges, we call these classes presenters. For example, here is a simplified version of the UserPresenter.

class UserPresenter
  def initialize(user)
    @user = user
  end

  def as_json(*)
    {
      'id'          => @user.id,
      'email'       => @user.email,
      'name'        => @user.name,
      'first_name'  => @user.first_name,
      'last_name'   => @user.last_name,
      'urls'        => {
        'self'    => "#{Gauges.api_url}/me",
        'gauges'  => "#{Gauges.api_url}/gauges",
        'clients' => "#{Gauges.api_url}/clients",
      }
    }
  end
end

Nothing fancy. Just a simple ruby class that sits in app/presenters. Here is an example of the the /me route looks like in our Sinatra app.

get('/me') do
  content_type(:json)
  sign_in_required
  {:user => UserPresenter.new(current_user)}.to_json
end

This simple presentation layer makes it really easy to test the responses in detail using unit tests and then just have a single integration test that makes sure overall things look good. I’ve found this tiny layer a breath of fresh air.

I am sure that nothing above was shocking or awe-inspiring, but I hope that it saves you some time on your next public API.

24 Comments

  1. Fredrik Björk Fredrik Björk

    Dec 01, 2011

    Great article.

    I’ve seen great value in presenting the JSON with tilt (https://github.com/rtomayko/tilt) and Yajl JSON templates. Makes it more readable and sticks to the MVC pattern.

  2. Did you think about versioning your API in case you have to introduce breaking changes in the future?

  3. @John: Yep, versioning will be done through a header. I don’t really believe in versioning through the URL.

  4. Fredrik Björk Fredrik Björk

    Dec 01, 2011

    @John Nunemaker: I tend to agree that versioning should be done in a header, but sometimes clients can’t pass headers, i.e JSONP

  5. @Fredrik Björk: That is a good point. Though I’m sure we could also allow for a query param in that instance if necessary.

  6. The other thing I like about the Presenter approach for public api’s is that it’s makes API versioning much easier to implement.

  7. @Duff OMelia: Yep, inside your app you can certainly have directories of version numbers and presenters. You can also inherit from v1 for v2 or whatever to share responses that are the same. Moving to presenter classes opens up a lot of possibilities.

  8. Any reason for not using Link headers (http://tools.ietf.org/html/rfc5988) for URLs?

  9. @bai: interesting idea. Had forgot about those.

  10. Many of the API’s I have worked with allow you to specify certain requirements in either the URL query string or the HTTP headers. This makes it easy to construct with a URL or to put the HTTP request together behind the scenes.

    The presenter approach is what we used a few years back when building an API. They made the most sense, and I still think they make the most sense.

  11. Not sure if you’ve already seen these articles, but Tribesports wrote a couple of informative posts on versioning their API using a custom MIME type in Rails and the draper gem to implement the decorator pattern.

  12. @Ben: Yep, I think I can see a custom MIME making sense. Thanks for linking.

  13. very interesting, do you have any other good resource I can read about the best ways on creating an API? in few months I’ll have to add them to an existing platform (my bad, next time I’ll start with APIs from the beginning)

  14. @ivan you might look into https://github.com/intridea/grape

    It’s a DSL for APIs. You can see Michael Bleigh discuss it in a RubyConf talk here http://confreaks.net/videos/475-rubyconf2010-the-grapes-of-rapid

  15. Consistency IS hard – but I’d say it’s more than important – it’s essential. Sometimes new rules present themselves along the way. Refactoring those rules into completed API work takes courage and perseverance. But the time up front really pays dividends for both users and future development.

  16. Ivan K. Ivan K.

    Dec 02, 2011

    translated to russian http://habrahabr.ru/blogs/webdev/133821/

  17. Thanks Noah, why use an extra library like Grape when Rails 3 provides already a way to respond based on the requested format? With Grape all the controller logic (get all the records based on the current user, and things like that) would be duplicated in the Grape dsl too

  18. zimbatm zimbatm

    Dec 02, 2011

    @bai: if you start adding header informations like that it means that http header and body can’t be disassociated. Usually I would try to keep header information strictly for the clientserver communication.

  19. Cameron Cameron

    Dec 02, 2011

    How does as_json(*) work. I have been playing around with the json gem in irb, but maybe I am missing something here. Does calling to_json on the has invite as_json on the presenter?

  20. @Cameron: Pretty sure that is part of Active Support or something. to_json calls as_json.

  21. Bernd Ahlers Bernd Ahlers

    Dec 02, 2011

    John, thanks for the article!

    I have a somewhat unrelated question. How do you guys handle the sharing of stuff like database models for separate API apps?

    Thanks!

  22. @Bernd: Your API should provide all the functionality other apps need. If it doesn’t, then add it to the API.

  23. Tyler Stromberg Tyler Stromberg

    Jan 18, 2013

    I know this post is a bit old, but I was wondering how you handle presenters where you need to return a collection of objects. For example, does the ‘/gauges’ route look something like this?

    get('/gauges') do
      content_type(:json)
      sign_in_required
      {:gauges => current_user.gauges.collect {|gauge| GaugePresenter.new(gauge)}.to_json
    end
    

    Or is there a cleaner way to handle this (e.g. the presenter itself knows how to render a collection)?

  24. @Tyler: Yep, either that or make a presenter for the collection (ie: GaugesPresenter, note the pluralization).

Sorry, comments are closed for this article to ease the burden of pruning spam.

About

Authored by John Nunemaker (Noo-neh-maker), a programmer who has fallen deeply in love with Ruby. Learn More.

Projects

Flipper
Release your software more often with fewer problems.
Flip your features.