Gray Soft / Rubies in the Rough / Riding the Testrocket

11

APR
2012

Riding the Testrocket

I say it a lot, but programming is about ideas. More specifically, it's about not running out of ideas.

Along these lines, I read the results of an interesting study recently. It was a study about how we think and solve problems. When I was in school, the belief was that we needed to cram our brain full of facts. We were pushed to memorize, memorize, memorize.

The more modern view of learning is that facts don't matter as much. Nowadays we just try to teach children how to think. The assumption is that they can find the facts they need, because information is so universally available, and their thinking skills will allow them to work their way to other bits of knowledge.

The study looked at people taught using both of these techniques and found some surprising results: us memorizers can often out think the thinkers. A leading theory about why that's the case is that the memorizers have more of a foundation to build their ideas off of. Thinkers may be starting closer to scratch each time. If so, they have further to go to get to the needed solution. Memorizers, on the other hand, may already have a lot of knowledge that puts them closer to the solution before they even need to start thinking.

I think a lot of our habits in programming are about this phenomenon. We read lots of books, priming the pump with language details and sample code. We listen to conference talks about techniques other programmers have used. We program in pairs, to double our knowledge foundation before we even try anything.

You should also be reading code. That's important because it let's you see the ideas of others, introducing external data points you might not divine on your own. It's also different from books and conference talks, which tend to stick to best practices. Some code is wild and the value of it may be hard to judge. Still, it can give you ideas and that's what counts.

Today, I want to read some fun code together.

The Results of a Brawl

It's well known that I like code challenges, another great source of before-I-really-need-them ideas. That's why I ran the Ruby Quiz for years.

A more recent source of such contests is the Codebrawl site. They throw up a topic and programmers "fight" it out to come up with the best solution. One of the early brawls centered around testing libraries. Peter Cooper won that fight by inventing a neat little library called Testrocket.

The primary source of Testrocket is 18 lines of code, but there's a surprising amount of features packed in there. It also uses Ruby in some fairly surprising ways. Even the interface it achieves can be surprising. Let's pull back the curtain and see how it's done.

Rockets Away

The Testrocket library allows us to write tests for our code. It's named for the syntax used in those tests. In Ruby, we call the Hash pairing operator => the "Hashrocket." This library creates some pseudo-operators for testing, like +-> and -->. Those are the "Testrockets."

Here's what a simple set of tests looks like:

require "testrocket"

!-> { "Basic Math Tests" }

+-> { 40 + 2 == 42          }
--> { 4 - 2 == 1            }
--> { 5 / 0                 }
~-> { "test multiplication" }

!-> { "A Failing Test" }

+-> { 2 + 2 == 5 }

Running that code gives this output:

   FIRE 'Basic Math Tests'!
     OK
     OK
     OK
PENDING 'test multiplication' @ math_test.rb:8
   FIRE 'A Failing Test'!
   FAIL @ math_test.rb:12

This is obviously very simple, but it's interesting to look at just how this syntax is accomplished. Before we dive straight into the code though, let's discuss some parts of the Ruby language that will help us understand it.

The Arrow-to-the-Knee Operator

A new operator was added to Ruby 1.9 and it has a pretty interesting history. Rubyists have long complained that we wanted block arguments to be more like full method arguments. We wanted to be able to set defaults and such, but, most importantly, we wanted blocks that could take blocks.

Metaprogramming was the main motivator here. It's common to dynamically define methods with tools like define_method(), which essentially turns a block into a method. If we wanted that newly created method to be able to take a block, blocks had to be able to take blocks.

It was thought that it would be too hard to modify Ruby's parser to work this way, so a compromise was made. A new syntax for lambda() was introduced that could accept a normal range of arguments. You could convert that to a block, using the & prefix, when needed. Plus it could serve in other areas.

In it's simplest form, the new operator looks like this:

>> -> { }
=> #<Proc:0x000001010da6b8@(pry):1 (lambda)>

Matz says the arrow was chosen because it kind of looks like a lambda (λ) if you lay it on it's side… and squint. Most people have taken to calling it the "stabby lambda," but it also looks like it could be someone taking an arrow to the knee, if you ask me.

Of course, the new operator can take various arguments. That was the whole point:

>> ->(n) { n * 2 }.call(21)
=> 42
>> ->(n, offset = 2) { n + offset }.call(40)
=> 42
>> ->(&block) { block[20] + block[1] }.call { |n| n * 2 }
=> 42

This feature wasn't well liked by the community. Many people thought it looked more like messy Perl (which the idea was borrowed from) and less like beautiful Ruby. The community begged for a different solution and many ideas were offered up.

One Rubyist even worked out a wickedly complex patch that enabled method style arguments for normal blocks. Aside from some rare edge cases, it all just worked as expected. There was a lot of debate over that patch, but it was eventually accepted. It too made it into Ruby 1.9:

>> def m(&b) b.call(20) { |n| n * 2 } end
=> nil
>> m { |n, offset = 1, &bib| bib[n] + bib[offset] }
=> 42

With the block problem properly solved, a lynch mob formed and went after stabby lambda's head. It was too late though. Matz was in love and the new syntax stayed.

Given this history, I don't see the stabby lambda used in the wild much. But it was there for Peter Cooper to abuse in his testing library.

Like it or not, I do think it holds some interesting possibilities for methods that need to take multiple blocks (especially when combined with the new Hash syntax). For example:

UserAgent.get url, success: ->(response) { handle_response(response) },
                   failure: ->(error)    { fail error.message        }

Plus the Other Half of the Puzzle

Stabby lambda is half of the story of how Peter created the Testrocket syntax. Remember that the library uses sequences like +-> and -->. The last two characters in those are the start of a stabby lambda, but the leading +/- isn't.

Most Ruby operators are just syntactic sugar for calling certain methods on objects. Ruby allows us to define those methods to support the operators. For example, Fixnum has an unary minus method. It negates the following number. This is typically used to created negative numbers in our code:

$ ruby -e 'n = 13; p -n'
-13

But it doesn't have to function like that. We could be evil and make it double numbers instead:

$ ruby -e 'class Fixnum; def -@; self * 2 end end; n = 13; p -n'
26

Note that I overrode the -@ method there. That's the unary minus, whereas the - method is the binary operator that normally performs subtraction.

Now a Proc, the object created by the stabby lambda, doesn't normally respond to the math operators. We can't add lambdas, for example:

$ ruby -e '-> { } + -> { }'
-e:1:in `<main>': undefined method `+' for #<Proc:…> (NoMethodError)

However, if we could think up a reasonable meaning for adding two Proc objects together, we could make it work:

$ ruby -e 'class Proc; def +(o) self.class.new { call; o.call } end end;
           (-> { puts 1 } + -> { puts 2 }).call'
1
2

Peter chose to define the unary plus and minus on Proc, plus other methods. This gave them new capabilities and turned them into his Testrockets.

If you're still with me, you are finally ready to see Peter's code:

A Tiny Library

Here's all 18 lines of Peter's code:

module TestRocket
  extend Module.new { attr_accessor :out }

  def _test(a, b); send((call rescue()) ? a : b); end

  def +@; _show _test :_pass, :_fail end
  def -@; _show _test :_fail, :_pass end
  def ~@; _show _pend;               end
  def !@; _show _desc;               end

  def _show(r); (TestRocket.out || $>) << r; r end
  def _pass; "     OK\n"; end
  def _fail; "   FAIL @ #{source_location * ':'}\n"; end
  def _pend; "PENDING '#{call}' @ #{source_location * ':'}\n"; end
  def _desc; "   FIRE '#{call}'!\n"; end
end

Proc.send :include, TestRocket

OK, so this code defines a Module called Testrocket (the first line) that is mixed into all Proc objects (the last line). In other words, it adds some methods to Proc.

In the middle chunk of code, you'll see definitions for +@, -@, and other operators. That's how you get the Testrocket interface that we've been discussing. These methods hand their work off to various helper methods, but essentially they do some combination of call() to run the code in the block and printing some results.

Those helper methods are the rest of the method definitions above and below the operators we just looked at. Peter prefixed them all with underscores as a convention to say that they are just for internal use. This should keep them from conflicting with normal methods defined on Proc.

The _test() method is where the main action happens. It runs the block and then calls a different method based on whether or not the code returned a true value. It also turns any StandardError raised into a normal failure.

Skip over _show() for now and the rest of the helpers are pretty simple stuff. They just construct the various messages that get printed to tell the tester what happened with the code. There is an interesting trick used by _fail() and _pend(). They call the source_location() method on Proc to find out where the code is. That method usually returns a two element Array containing the file name and line number. Peter then uses an operator alias for join() (*) to convert them to a single String. Here's the process spelled out:

>> -> { }.source_location
=> ["(pry)", 1]
>> _ * ":"
=> "(pry):1"

The extend() line is the hardest thing in the whole piece to figure out, if you ask me. It's obvious that the methods out() and out=() are being defined. It took me a while to figure out where though, since this is a pretty tricky way to define them.

When you get stuck like that, try to see where the method gets used. That will usually give you the needed hint. Peeking inside _show() taught me that it's just a fancy way to define a getter and setter on a Module or Class. Normally the attr_accessor() helper affects the instance level, not the Module level. So, Peter dynamically defined a mix-in using Module.new() and a block to define it, stuck the attr_accessor() in there, then mixed that in at the Module level with extend().

It's worth thinking about that a little more, because it has advantages. Normally, programmers will cheat the attr_accessor() level with code like this:

class << self
  attr_accessor :out
end

But there's a difference between these two approaches. The latter technique puts the methods in the singleton class directly. Peter's technique puts the methods in a Module mixed into the singleton class. This means that Peter's methods are easier to override: you can just mix-in another module, or define the override in the singleton class.

Hopefully that's enough explanation to help you make sense of the code. I recommend you keep fiddling with it until you fully understand how it works. It's worth the investment of time. It will level-up your brain.