Too Hard to Pronounce

What I’m Using for Behavior Driven API Development.

September 27th, 2008

As I mentioned in my previous post, I’ve been enjoying using an API-first methodology for developing my current project. Combining this with a BDD and “less software” approach powered by Cucumber, RSpec, Halcyon, and Datamapper seems to be flowing nicely for me. Given that I’d already had an initial sense of what I wanted the app to be, here’s what my process looked like:

* Cut features out of the app API until it’s ‘barely releasable’
* Write automated user stories to drive the API in a feature driven manner
* Implement each feature as part of a pure JSON API
* Write clients, each of which are only allowed to interact with domain objects through the API

Cutting Features

My natural tendency has always been to plan releases with tons of features. In the spirit of adopting a more agile approach of short release cycles with rapid iterations, I’ve been taking 37Signals’ advice to “release half a product, not a half-assed product” and trim release feature expectations down until the resulting product would be ‘barely releasable.’

Executable User Stories

Given a manageable set of features that I want the API to have, I circle back and refine those features as executable user stories. By “user stories”, I mean somewhat formal scenarios for how the API will be used and what business value is being gained. By “executable,” I mean that the text of user stories is hooked into an actual automated test suite so that I can use the written behavior to concretely drive the code development.

The concept of executable user stories is a really neat one. Thankfully the testing tools I’m using, Cucumber and RSpec, really make it pleasurable to integrate user stories with the testing. While most of you have heard of the popular RSpec test tool, Cucumber may be newer to you.

Cucumber is Aslak Hellesøy’s very impressive Treetop-powered rewrite of the RSpec Story Runner (due to replace Story Runner in RSpec 1.1.5). Having watched the progression from the earliest incarnations of Story Runner to the current RSpec version to Cucumber, I’m very impressed. Cucumber gets the syntax just about perfect.

Without getting too much into the weeds, here’s a quick example of how executable user stories work with Cucumber and RSpec. First, I write a text based feature formatted as a user story:

 1 Feature: Openings for a Specific Time, but Unspecified Activity
 2 
 3 In order to alleviate boredom
 4 As someone who is free to go out, but doesn't have a specific event in mind
 5 I want to be able to post a new opening in my schedule
 6 
 7 Scenario: Create a new foobar for a specific quuz_status, for my favorite baz
 8   Given I have a user account already
 9   When I tell the app to create a foobar for baz $baz with status $quux_status
10   Then the result should be an OK HTTP response
11   And the response should include the ID of the newly created foobar

By formalizing each of my features into this user story format, I ensure that my each of my planned features is actually well-defined and derived from a business need. It may not be obvious, but this user story is actually using some very specific keywords like Scenario, Given, When, Then, and And to allow Cucumber to parse the story and provide hooks for my tests (or, in RSpec parlance, "specs").

In order to take advantage of the hooks provided by these keywords, I define a “steps” file that bridges the user story to a concrete test. For example:

 1 require 'spec'
 2 $:.unshift(File.dirname(__FILE__) + '/../../lib')
 3 require 'client'
 4  
 5 Before do
 6   @client = FooBarBazApp::Client.new('http://localhost:4647/')
 7 end
 8  
 9 Given "I have a user account already" do
10   true #haven't implemented authentication yet
11 end
12 
13 Then "the result should be an HTTP $status response" do |status|
14   @response[:status].to_i.should == status.to_i
15 end
16 
17 Then "the response should include the ID of the newly created foobar" do
18   @response[:body].should_not == nil
19   @response[:body].class.should == Fixnum
20 end
21 
22 When "I tell the app to create an foobar for baz $baz with status $quux_status" do |baz_id, quux_status|
23   @response = @client.post('/foobars', :baz_id => baz_id, :quux_status => quux_status)
24 end

This code sample is by no means meant to represent best practice tests or even particularly good ones; I just want to give a gestalt of what the setup looks like. I think if you inspect the code, you can get a sense of the fact that Cucumber is providing a pretty neat syntax for transforming the English language user story into an automated test suite. Now, with a simple “rake features” at the command line, I can run my test suite, see what step is failing, and then implement it quickly. This way, my code stays clean and minimalistic; I only add code that directly contributes value.

Implement the API

It’s entirely possible to power the API with a traditional web application framework like Rails or Merb. In fact, I’d definitely recommend it for some cases. For this particular situation, though, it would mostly serve as a distraction. I simply don’t need the functionality of a full stack framework right now, at least for the API work I’m doing. Rather than weigh the API down with a lot of infrastructure I don’t need, I’d rather work closer to the metal so that I can keep resource requirements light and maintain focus on the core functionality I’m providing. Should my needs change, I’ll either integrate the API with another framework via RESTful HTTP calls or simply port the code over to Merb directly.

The framework I’ve been using to accomplish this is Halcyon, a microframework designed by Matt Todd just for this sort of application. There were a number of other attractive choices, but I was drawn to Halcyon primarily because of its focus on providing all responses in JSON . . . no standard HTML views whatsoever. The framework is intended exactly for the kind of API endpoint work I’m doing.

The following code sample depicts a simple (but probably not production ready) Halcyon Controller.

 1 #A Typical Halcyon Controller
 2 class Foobars < Application 
 3   def new
 4     ok %w(baz_id quux_status name)
 5   end
 6   
 7   def create
 8     f = Foobar.create(:baz_id => params[:baz_id], 
 9                        :quux_status => params[:quux_status], 
10                        :name => params[:name])
11     f.save!
12     ok f.id
13   end
14   
15   def show
16     ok Foobar.get(params[:id])
17   end
18   
19   def delete
20     Foobar.get(params[:id]).destroy
21     ok
22   end
23   
24   def index
25     ok Foobar.all.collect {|f| f.to_json}
26   end
27 end

As you can probably guess, this interacts with a Foobar DataMapper model that you might imagine looks something like this:

 1 class Foobar
 2  include DataMapper::Resource
 3  belongs_to :baz 
 4   
 5  before :save, :verify_foobar_will_be_awesome
 6 
 7  property :id,         Serial, :serial => true    # primary serial key
 8  property :created_at, DateTime
 9  property :updated_at, DateTime
10 
11  property :baz_id, Integer, :nullable => false
12  property :name, String
13  property :quux_status, String
14 
15  def verify_foobar_will_be_awesome
16   #TODO: implement
17  end
18 end

DataMapper is a really impressive ORM that I prefer to the Rails popularized ActiveRecord these days. Rather than go into its advantages here, I’ll refer you to Yehuda Katz’s MountainWest RubyConf presentation on DataMapper that really got me to switch for Ruby, non-Rails projects.

Write Clients

As described earlier, when it comes to actually interacting with the API, whether we’re talking about our web site, a phone app, or some other type of interface, they’re all just clients. One really convenient feature of Halcyon is that it automatically generates a skeleton client for you! The client does all of the RESTful heavy lifting for you, making it easy to customize for your application.

 1 %w(rubygems halcyon).each{|dep|require dep}
 2 
 3 module FooBarBazApp
 4   
 5   # = Client
 6   # 
 7   # Barebones client to a Halcyon powered API.
 8   class Client < Halcyon::Client
 9       def list
10        if (msgs = get('/foobars'))[:status] == 200
11          # success
12          msgs[:body] # return message
13        else
14          # failure
15          msgs # return status and error message
16        end
17       end
18       
19       # get a single message
20        def show(id)
21         if (msg = get('/foobars/'+id.to_s))[:status] == 200
22        # success
23         msg[:body] # return message
24       else
25        # failure
26         msg # return status and error message
27         end
28       end
29 end

This is really useful because it can make accessing your API trivial from any Ruby app just by requiring the client file. Though it’s convenient, one potential pitfall of the autogenerated client skeleton is that it’s tempting to customize it to the extent that you end up creating an ad-hoc, poorly specified abstraction over your well-thought API. It’s a good idea to use discretion with how you use the client. Remember that a specialized Ruby client like this is just for convenience . . . your Halcyon app, if well-designed, already gives you a RESTful HTTP/JSON API that’s easy to access with any number of pre-existing HTTP libraries (John Nunemaker’s HTTParty being one of my favorites).

Without going on too much of a tangent, I do want to show a quick example of the sort of integration this API approach allows. Right now I’m working on creating a voice app that interacts with this API. I’m using the excellent CloudVox voice service with Adhearsion to build and deploy the app. Here’s what an ultra-basic Adhearsion dialplan file would look like using our Halcyon client:

 1 
 2 require '../lib/client'
 3 @client = FooBarBazApp::Client.new('http://localhost:4647')
 4 @num = @client.list.size
 5 puts "There are #{@num} foobars in the database right now."
 6 default {
 7   say_digits @num.to_i
 8 }
 9 

What that will do is tell callers the digits in the number of foobars in our system. Obviously this is a toy app, but note what we’ve done. With practically no work, we’ve just integrated telephony with our little Foobar API! Not bad at all. :)

Conclusion

That’s just a little taste of what I’ve been up to with the API-first web app development approach. I encourage you to give it a try and let me know what you think! If there’s enough interest, I may spinoff some actual tutorials on aspects of what I’ve done with the API when I’m a little closer to launch.