Gray Soft / Key-Value Stores

10
JAN
2010

Tokyo Cabinet's Key-Value Database Types

We've taken a good look at Tokyo Cabinet's Hash Database, but there's a lot more to the library than just that. Tokyo Cabinet supports three other kinds of databases. In addition, each database type accepts various tuning parameters that can be used to change its behavior. Each database type and setting involves different tradeoffs so you really have a lot of options for turning Tokyo Cabinet into exactly what you need. Let's look into some of those options now.

The B+Tree Database

Tokyo Cabinet's B+Tree Database is a little slower than the Hash Database we looked at before. That's its downside. However, giving up a little speed gains you several extra features that may just allow you to work smarter instead of faster.

The B+Tree Database is a more advanced form of the Hash Database. What that means is that all of the stuff I showed you in the last article still applies. You can set, read, and remove values by keys, iteration is supported, and you still have access to the neat options like adding to counters. With a B+Tree Database you get all of that and more.
Read more…

In: Key-Value Stores | Tags: Tokyo Cabinet | 4 Comments

1

JAN
2010

Tokyo Cabinet as a Key-Value Store

Like most key-value stores, Tokyo Cabinet has a very Hash-like interface from Ruby (assuming you use Oklahoma Mixer). You can almost think of a Tokyo Cabinet database as a Hash that just happens to be stored in a file instead of memory. The advantage of that is that your data doesn't have to fit into memory. Luckily, you don't have to pay a big speed penalty to get this disk-backed storage. Tokyo Cabinet is pretty darn fast.

Getting and Setting Keys

Let's have a look at the normal Hash-like methods as well as the file storage aspect:

#!/usr/bin/env ruby -KU

require "oklahoma_mixer"

OklahomaMixer.open("data.tch") do |db|
  if db.size.zero?
    puts "Loading the database.  Rerun to read back the data."
    db[:one] = 1
    db[:two] = 2
    db.update(:three => 3, :four => 4)
    db["users:1"] = "James"
    db["users:2"] = "Ruby"
  else
    puts "Reading data."
    %w[ db[:one]
        db["users:2"]
        -
        db.keys
        db.keys(:prefix\ =>\ "users:")
        db.keys(:limit\ =>\ 2)
        db.values
        -
        db.values_at(:one,\ :two) ].each do |command|
      puts(command == "-" ? "" : "#{command} = %p" % [eval(command)])
    end
  end
end

16

SEP
2009

Lists and Sets in Redis

[Update: though all of the techniques I show here still apply, many methods of the Redis gem have changed names to match the actual Redis commands they call. There are also easier and more powerful ways to do some of what I show in here, thanks to additions to Redis.]

Redis adds one huge twist to traditional key-value storage: collections. Supporting both lists and sets through some very powerful atomic operations allows for advanced key-value usage.

Lists

Redis allows a single key to hold a list of values. This is your typical ordered list with the operations you would expect: appending, indexed access, and access to a range of values.

This has many potential uses. I'll cover two that I think will be very common. First, if you are going to use Redis as a full database, you store things that are naturally a list of items, like comments, in a real list. Let's look at some code:

#!/usr/bin/env ruby -wKU

require "redis"

CLEAR = `clear`

# create an article to comment on
db                = Redis.new
article_id        = db.incr("global:next_article_id")
article           = "article:#{article_id}"
class << article
  def method_missing(field, *args, &blk)
    return super unless field.to_s !~ /[!?=]\z/ && args.empty? && blk.nil?
    "#{self}:#{field}"
  end
end
db[article.title] = "My Favorite Language"
db[article.body]  = "I love Ruby!"
# initialize some session details
comments_per_page = 2
comment_page      = 1
login             = ARGV.shift || "JEG2"

loop do
  # show article
  print CLEAR
  puts "#{db[article.title]}:"
  puts "  #{db[article.body]}"

  # paginate comments
  start      =  comments_per_page * (comment_page - 1)
  finish     =  start + comments_per_page - 1
  comments   =  db.list_range(article.comments, start, finish)
  pagination =  Array(start.zero? ? nil : "(p)revious")
  pagination << "(n)ext" if db.list_length(article.comments) - 1 > finish
  # show comments
  comments.each do |comment|
    posted, user, body = comment.split("|", 3)
    puts "----"
    puts "  #{body}"
    puts "  posted by #{user} on #{posted}"
  end

  # handle commands
  puts
  print "Command? [#{(%w[(c)omment (q)uit] + pagination).join(', ')}]  "
  case (command = gets)
  when /\Ac(?:omment)?\Z/i  # add a comment
    print "Your comment?  "
    comment = gets or break
    posted  = Time.now.strftime('%m/%d/%Y at %H:%M:%S')
    db.push_tail(article.comments, "#{posted}|#{login}|#{comment.strip}")
  when /\Ap(?:revious)?\Z/i  # view previous page of comments
    if pagination.first =~ /\A\(p\)/
      comment_page -= 1
    else
      puts "You are on the first page of comments."
      gets or break
    end
  when /\An(?:ext)?\Z/i  # view next page of comments
    if pagination.last =~ /\A\(n\)/
      comment_page += 1
    else
      puts "You are on the last page of comments."
      gets or break
    end
  when /\Aq(?:uit)?\Z/i, nil  # exit program
    break
  end
end