Gray Soft / Tags / APIs

Delaying Decisions

2014-04-29T03:29:32Z

I love playing with Ruby's Hash. I think it has a neat API and experimenting with it can actually help you understand how to write good Ruby. Let's dig into this idea to see what I mean.

The nil Problem

In Destroy All Software #9 Gary chooses to show an example in Python because, unlike Ruby's Hash, it will raise an error for a non-existent key. Ruby just returns nil, he explains.

What Gary said isn't really true, but I'm guessing he just didn't know that at the time. He was in the process of switching to Ruby from Python and I'm guessing he just didn't have a deep enough understanding of Ruby's Hash yet. I bet he does know how it works now.

But assume he was right. What's he saying and why does it matter? Consider some code like this:

class SearchesController < ApplicationController
  def show
    terms = params[:terms]
    SomeModel.search(terms)
    # ...
  end
end

This is what Gary doesn't like, and rightfully so. Because I indexed into params here with the []() method, I will indeed get a nil if the :terms key wasn't in params.

The problem with nil is that it's a ticking time bomb just waiting to go off. It has a very limited interface by design, so calling methods on it may raise errors. It's probably not the interface your code is expecting. For example, in the code above I likely intended to get a String in terms, but if I got nil instead and I later try to call String methods on it, it's going to blow up.

This gets worse because nil values tend to bounce around in the code a bit. This means the error may finally manifest far from the line where I actually assigned terms, which is the real problem that needs fixing. For example, let's say the model code I handed off to looks something like this:

class SomeModel < ActiveRecord::Base
  def self.search(terms)
    words = terms.scan(/\S+/)
    # ...
  end
end

This is where the error would show up. The call to scan() would toss a NoMethodError since nil doesn't have scan(). The stack trace would lead us here, but, as I said before, this isn't what needs fixing.

Now, we could fix it here. One way to do that would be to insert a conversion before the scan() call:

words = terms.to_s.scan(/\S+/)

That's probably not a good idea though. We've now said that calling SomeModel.search(nil) is OK and that doesn't really make much sense.

A slightly better fix would be to add a guard before the call to scan()

fail ArgumentError, "You must pass search terms" if terms.nil?
words = terms.scan(/\S+/)

This would get us a better error message that actually tells us what went wrong. But the stack trace still wouldn't be ideal. It's going to bring us here first even though the real issue is in the controller.

My Favorite Method to the Rescue

This whole problem started because I allowed the Hash to give me a nil. I didn't want a nil, so I shouldn't have said it was OK.

That brings us to fetch(). Watch this:

>> hash = { }
=> {}
>> hash[:missing]
=> nil
>> hash.fetch(:missing)
KeyError: key not found: :missing
from (pry):3:in `fetch'

You can think of []() as saying give me some key, if it exists. You get a nil if it doesn't. Now fetch(), on the other hand, is more like saying I need this key, no substitutions allowed. This forces fetch() to raise a KeyError when the key is missing, since nil isn't allowed.

The best fix to the problem I started with would be to replace the initial assignment in the controller with this:

terms = params.fetch(:terms)

You may also want to add some code to handle the KeyError, but the important thing is that the problem now triggers from the proper place. This is the most helpful error we could work with in this scenario.

I Don't Know What You Need

Let's take a step back for a moment to the overall Hash. What exactly is a Hash in your code? The only right answer is that I have no idea.

You might be creating a typical key-to-value mapping:

>> name = {first: "James", last: "Gray"}
=> {:first=>"James", :last=>"Gray"}
>> name[:last]
=> "Gray"

Alternately, you could be tracking some counts:

>> counts = Hash.new(0)
=> {}
>> counts[:one] += 1
=> 1
>> 3.times do counts[:three] += 1 end
=> 3
>> counts.values_at(:zero, :one, :three)
=> [0, 1, 3]

Or you could be keeping some named buckets for data:

>> buckets = Hash.new { |hash, key| hash[key] = [ ] }
=> {}
>> buckets[:one] << 1
=> [1]
>> buckets[:three] << 1 << 2 << 3
=> [1, 2, 3]
>> buckets
=> {:one=>[1], :three=>[1, 2, 3]}

Perhaps it's even a deeply nested structure that you wish to track:

>> tree = Hash.new { |hash, key| hash[key] = Hash.new(&hash.default_proc) }
=> {}
>> tree[:deeply][:nested][:structure] = 42
=> 42
>> tree[:deeply][:nested][:branch] = 1
=> 1
>> tree
=> {:deeply=>{:nested=>{:structure=>42, :branch=>1}}}

These are all very different, but valid, uses of a Hash in Ruby to model various data structures and there are other options too. Ruby can't know what you intend to do. But it was wisely designed so that it doesn't have to know.

Note the progression above that makes this possible. You can create a normal Hash. You can pass a default object, like an initialization value for counters. Or you can go all the way to defining the default behavior with custom code inside of a block as I did in the last two examples.

Ruby delays the decision of default behavior so that it can leave that decision to you. You know what you need better than Ruby does.

And it doesn't stop there.

Back to My Favorite

Being able to set some default behavior at Hash construction time is great, but what if I don't know everything I need to know even then? What if I need to delay the decision even more, until key lookup time? What if what I actually need is different at different times? The answer is that fetch() can handle those cases too.

>> hash = { }
=> {}
>> hash.fetch(:missing, 0)
=> 0
>> hash.fetch(:missing) { |key| key.to_s.split("").sort.join }
=> "giimnss"

You can see that fetch() also supports default return values for a missing keys and it will even allow you to provide custom code to handle the situation however you need to.

If you only get one thing out of this article, make it Hash#fetch(). It's a powerhouse tool that enables you to do all kinds of fancy tricks. (It's worth noting that Hash#delete() is similarly flexible. Look it up.) But that's not really why I'm showing you these methods.

The Pattern is the Thing

The real reason to play with these methods is that they represent a great tactic for Ruby programming in general. It's important to remember that you won't know exactly what the users of your code will need. Delay making that decision whenever you can.

In fact, delay it as long as you can, ideally just pushing the decision off on the user when they finally need to make it. They will know what they need better than you do. Let them raise an error, do a conversion, or whatever else makes sense for them.

In a lot of ways, this discussion is about Ruby's blocks. That's the great tool Ruby gives us to allow us to delay these decisions. Consider even a simple method like this:

def do_some_io_operations_for_me
  # ... IO code here...
rescue IOError, Errno::EACCES #, ...
  block_given? ? yield : raise
end

This is a pretty perfect setup, in my opinion. This code tries to push through some IO operations. Those could fail for any number of reasons, so the code intelligently traps the errors that apply. If a users gives a block, they can control what happens on error without even needing to know which errors the IO code could trigger. However, the decision of exactly what to do is left to the caller.

Of course, the biggest limitation on this strategy is that we only get one block in Ruby. Once you delay one decision with it, you won't have it for other purposes. Try not to let that stop you though! Ruby doesn't.

Consider the find() iterator. With find() the block is already tied up for the test, but Ruby wants to let you decide what happens when nothing is found. Returning nil is not enough for those cases, because nil could legitimately be what the block found. Because of that, find() cheats to expose kind of a second block:

>> (0..9).find(-> { fail ArgumentError, "Not found" }) { |n| n >= 10 }
ArgumentError: Not found
from (pry):28:in `block in <main>'

I think these kind of tricks come out a little cleaner with the new "stabby lambda()" (->) syntax that I used above.

Further Investigation

Fully getting the hang of when and where to use blocks in Ruby seems to take almost as long to get a feel for as the rest of the language does. Here are some sources that might shave a little time off of the journey though:

I wrote an article on blocks a long time ago. I think it has aged fairly well and still provides some insight into why blocks exist and what they really are.
Rake is a great example of the delayed decisions I am advocating in this article in many ways. When you think about it, Rakes is really just an executable dependency graph of delayed decisions. For a more concrete example though, check out Rake's FileTask. It's a delayed decision about how to build a file from one or more dependencies any time the content is no longer fresh.
The 2.10.0 release of RSpec brings with it some matchers for yielded control to a block. This can make testing all of those block methods I'm asking you to write a little easier.

Dual Interface Modules

2014-04-11T19:28:44Z

I'm guessing we've all seen Ruby's Math Module. I'm sure you know that you can call methods in it as "module (or class) methods:"

Math.sqrt(4)  # => 2.0

That's just one way to use the Math Module though. Another is to treat it as a mixin and call the same methods as instance methods:

module MyMathyThing
  extend Math

  def self.my_sqrt(*args)
    sqrt(*args)
  end
end

MyMathyThing.my_sqrt(4)  # => 2.0

Ruby ships with a few Modules that work like this, including the mighty Kernel.

How is this dual interface accomplished? With the seldom seen module_function() method. You use this much like you would private(), to affect all following method definitions:

module Greeter
  module_function

  def hello
    "Hello!"
  end
end

module MyGreeter
  extend Greeter

  def self.my_hello
    hello
  end
end

Greeter.hello       # => "Hello!"
MyGreeter.my_hello  # => "Hello!"

As you can see, it magically gives us the dual interface for the methods beneath it. You can also affect specific methods by name, just as you could with private(). This is equivalent to my definition above:

module Greeter
  def hello
    "Hello!"
  end
  module_function :hello
end

What this helper actually does is to make a copy of the method and move it up to the module interface level. Once the copy is made, they can be affected separately:

module Copies
  def copy
    "Copied!"
  end
  module_function :copy

  alias_method :copier, :copy
  public       :copier
  undef        :copy
end

Copies.copy  # => "Copied!"
c = Object.new.extend(Copies)
c.copier     # => "Copied!"
c.copy
# ~> -:16: undefined method `copy' for #<Object:0x26e6c> (NoMethodError)

This process also marks the instance method version private(), which is why I needed the call to public() in the last example. This means that methods you mixin to another object do not add to its external interface:

module Selfish
  module_function

  def mine
    "mine"
  end
end

module Sharing
  extend Selfish

  def self.yours_and_mine
    "yours and #{mine}"
  end
end

Sharing.yours_and_mine  # => "yours and mine"
Sharing.mine
# ~> -:18: private method `mine' called for Sharing:Module (NoMethodError)

As we've seen there are some advantages to this interface, but there are some drawbacks too. For example, I first tried to write the MyGreeter example as:

module MyGreeter
  include Greeter

  module_function

  def my_hello
    hello
  end
end

MyGreeter.my_hello  # => 
# ~> -:15:in `my_hello': undefined local variable or method
# ~>      `hello' for MyGreeter:Module (NameError)
# ~>    from -:20

That didn't work because the Greeter functionality was not copied up with my method. You can fix that by using a different trick to duplicate the functionality:

module MyGreeter
  include Greeter

  extend self  # mixin functionality to our own Module interface

  def my_hello
    hello
  end
end

MyGreeter.my_hello  # => "Hello!"

This provides a similar dual interface, but there are important differences. First, we've changed the ancestors() of MyGreeter, not copied methods:

MyGreeter.ancestors  # => [MyGreeter, Greeter]

There's just one method and changing it affects everywhere it is used.

We also didn't magically make the mixin interface private() and it will bleed through:

module MyNestedGreeter
  extend MyGreeter

  def self.my_nested_hello
    my_hello
  end
end

MyNestedGreeter.my_nested_hello  # => "Hello!"
MyNestedGreeter.my_hello         # => "Hello!"

It's all tradeoffs of course, but knowing our options allows us to make informed choices about what is best for our needs.

Readable Booleans

2014-04-11T19:19:30Z

There's a great little trick you can do to improve the readability of your code. A common problem is dealing with methods that have a boolean flag arguments. Here's an example I ran into just today in a Rails application:

def rating_stars(..., clickable = false)
  # ...
end

The problem with this is that you typically see calls like this scattered around the application:

<%= rating_stars(..., true) %>

Would you know what true did there if I hadn't shown you the name of the variable first? I didn't. I had to go hunting for that method definition.

Ironically the opposite problem, a magical dangling false, is much more rare in my experience. That's typically the default for these kind of arguments and it just makes more sense and reads better to leave it out.

Anyway, the point is that we can typically improve the ease of understanding the common case. Remember that in Ruby false and nil are false while everything else is true. That means that truth is very loosely defined and we can pass a lot of things for our boolean flag value. For example, after looking up the method and understanding what was needed, I chose to call it like this:

<%= rating_stars(..., :clickable) %>

My hope is that might save a future maintainer a trip to the method definition to understand the call.

I've used that trick in quite a few situations now. To give another example, FasterCSV's documentation encourages user code to pass :headers => :first_row instead of just :headers => true. That has the advantage of being more self-documenting as well as leaving me room for possible expansion to the API.

Conversion Methods

2014-04-11T19:05:08Z

I want to take a step back from all the syntax I've been covering lately and just talk about some simple methods in Ruby's core. Ruby ships with so many great helpers, it's often hard to keep track of what everything can do. Specifically, let's talk about the type conversion methods.

I assume we all make calls to to_s() and to_i() regularly:

255.to_s    # => "255"
"255".to_i  # => 255

There shouldn't be any surprises there. Even these two simple methods can do more though. They make it possible to convert to and from various numeric bases. For example, here are the same conversions into and out of base 16 (hexadecimal):

255.to_s(16)   # => "ff"
"ff".to_i(16)  # => 255

Ruby has other ways to do these same conversions. Here are two unusual methods (beginning with capital letters) that are similar:

String(255)     # => "255"
Integer("255")  # => 255

I'll be honest and tell you that I don't really find String() useful as it just calls to_s() for you, but Integer() is a different story. First of all, to_i() is very lenient about what it converts while Integer() is more strict:

p "C#".to_i
# >> 0
Integer("C#")
# ~> -:5:in `Integer': invalid value for Integer: "C#" (ArgumentError)

That can be handy when you want to be sure you you have a number.

Now while Integer() must be passed a number, it understands numbers in all of the native formats Ruby does: decimal, octal, hexadecimal, and binary. In comparison, to_i() just understands decimal integers:

255                    # => 255
0377                   # => 255
0xFF                   # => 255
0b11111111             # => 255

Integer("255")         # => 255
Integer("0377")        # => 255
Integer("0xFF")        # => 255
Integer("0b11111111")  # => 255

"255".to_i             # => 255
"0377".to_i            # => 377
"0xFF".to_i            # => 0
"0b11111111".to_i      # => 0

If you want to be sure you are passing in safe content to either method, you should probably verify the String contents with a regular expression before you make the call.

There is a Float() method as well, but it doesn't give you much over to_f(). The only significant difference is how they handle nil:

p nil.to_f
# >> 0.0
Float(nil)
# ~> -:10:in `Float': can't convert nil into Float (TypeError)

Numbers aren't the only thing we can convert in Ruby though. Let's talk about Array(). Ruby use to have a direct to_a() method on all objects, but it has been deprecated in Ruby 1.8:

$ ruby -ve 'p 5.to_a'
ruby 1.8.6 (2008-08-11 patchlevel 287) [i686-darwin9.4.0]
-e:1: warning: default `to_a' will be obsolete
[5]

And it's finally gone in Ruby 1.9:

$ ruby_dev -ve 'p 5.to_a'
ruby 1.9.0 (2008-09-27 revision 0) [i386-darwin9.5.0]
-e:1:in `<main>': undefined method `to_a' for 5:Fixnum (NoMethodError)

That leaves us with just Array(), but that's not a bad thing at all since it is very handy. What makes Array() great is the three different behaviors it has on the objects you pass:

It has no effect on an Array which is return unchanged
It returns an empty Array when passed nil
All other objects are returned wrapped in an Array

Here are examples of each of those:

Array([1, 2, 3])  # => [1, 2, 3]
Array(nil)        # => []
Array(5)          # => [5]

These simple rules mean that you can count on an Array being returned from Array(), no matter what you pass it. Thus it is safe to call any Array methods on the result, including iterators.

To give a more practical example, let's say you have some parameter your Rails application is prepared to receive. You may receive any number of items in this parameter: zero, one, or many. You can process them with a simple:

Array(params[:choices]).each do |choice|
  # process choice here...
end

There's one last conversion method that I make regular use of and it is Hash[]. Note the brackets there as they aren't parentheses like the other methods we've been talking about. This one is a class method on Hash.

This method is very simple in function. It takes an even number of arguments and creates a Hash:

Hash["a", 1, "b", 2]  # => {"a"=>1, "b"=>2}

At first glance, that doesn't seem to gain us much over the normal Hash literal syntax of { … }. However, this is a method call we can use to build a Hash and that means we can apply the normal tricks of method calls to it, like splatting an Array of arguments:

Hash[*%w[a 1 b 2]]  # => {"a"=>"1", "b"=>"2"}

That opens up all kinds of possibilities for converting from Array objects to Hash objects or for building iterations that result in a Hash:

h = Hash[*%w[a 1 b 2 c 3].map { |f| Integer(f) rescue f }]
p h
# >> {"a"=>1, "b"=>2, "c"=>3}
p Hash[*h.select { |_, n| (n % 2).nonzero? }.flatten]
# >> {"a"=>1, "c"=>3}

Notice how we get a Hash out of the last line there when Ruby 1.8 would usually give us an Array of Array objects. Ruby 1.9 modifies most Hash iterators to return a Hash though:

$ ruby_dev -ve 'p({"a"=>1, "b"=>2, "c"=>3}.select { |_, n| (n % 2).nonzero? })'
ruby 1.9.0 (2008-09-27 revision 0) [i386-darwin9.5.0]
{"a"=>1, "c"=>3}

Hopefully that gives you some fresh ideas about how you might handle simple conversions in the future.

The One Method Config

2014-04-11T14:49:50Z

I've used this technique a couple of times now for dirt-simple configurations. The idea is to provide a trivial way to read and write configuration values with just a single method. Let me show you what I mean:

module Configurable
  module_function

  def config(new_config = nil)
    if new_config.nil?
      @config ||= { }
    else
      config.merge!(new_config)
    end
  end
end

include Configurable

config                    # => {}

config :a => 1, :b => 2
config                    # => {:a=>1, :b=>2}
config[:a]                # => 1

config :a => -1, :c => 3
config                    # => {:a=>-1, :b=>2, :c=>3}

config.clear
config                    # => {}

There's no deep magic here, obviously. The method has two function: read and write for the configuration. Read is handled with what I like refer to as Ruby's "caching operator" (||=). The first time that line is triggered, it will cache an empty Hash in the variable. Thereafter, the same call is just a cache hit to get the same Hash back.

We could stop there, of course. Access to the Hash allows us to use any methods we need on it. However, one more nicety really pays off, in my opinion. Give an optional argument and, when it's available, use it as a shortcut for one of the write methods. In this case, I chose merge!() just because it's a well rounded tool for adding and/or editing multiple Hash entries at once.

The examples show how this plays out. Note that I mix both styles of the method calls with some standard Hash tools ([] and clear()). I feel like that gives a whole lot of cool interface with barely any effort.

One more example, just to get your brain spinning in other directions:

module Sortable
  module_function

  def sortable_fields(*fields)
    if fields.empty?
      @sortable_fields ||= [ ]
    else
      sortable_fields.push(*fields.flatten)
    end
  end
end

include Sortable

sortable_fields  # => []

sortable_fields :a, :b, :c
sortable_fields  # => [:a, :b, :c]

sortable_fields %w[d e f]
sortable_fields  # => [:a, :b, :c, "d", "e", "f"]

This method wraps an Array instead of a Hash, as you can seen. With that there's one other trick I sometimes like to throw in that should be called something like "slurp-flatten()-splat." By taking any number of arguments, calling flatten() to avoid worrying about whether the caller used an Array (say for the convenient syntax used in the examples), and splitting the fields back out, we again get a smooth interface out of minimal code.

As I said earlier there's no deep magic here, but don't underestimate how nice the simple tricks can be in the right circumstances. Of course, purists who strongly hold to the "a method should only do one thing" philosophy will need to comfort themselves to sleep by repeating, "It just manages configuration!"