Key-Value Stores

Notes from my learning about simple NoSQL storage solutions.

17

SEP
2009

Where Redis is a Good Fit

Like any system, Redis has strengths and weaknesses. Some of the biggest positives with Redis are:

  • It's wicked fast. In fact, it may just be the fastest key-value store.
  • The collection types and the atomic operations that work on them allow you to model some moderately complex data scenarios. This makes Redis fit some higher order problems where a simple key-value store wouldn't quite be enough.
  • The snapshot data dumping model can be an asset. You get persistence with Redis, but you pay a minimal penalty for it.

Of course, there are always some minuses. These are the two I consider the most important:

  • Redis is an in-memory data store, first and foremost. That means your entire dataset must fit completely in RAM and leave enough breathing room for anything else the server must do.
  • Snapshot backups are not perfect. If Redis fails between snapshots, you can lose data. You need to make sure that's acceptable for any application you use it in.

It may seem weird to call snapshots both a pro and a con, but it does work for you in some ways and against you in others. You have to decide where the trade-off is worth it.

Given the above breakdown, I will list three places where I think Redis can be the right tool for the job. This is not meant to be an exhaustive list. I'm sure there are many other places Redis could be well used. Instead, it's better to look at why I've chosen these areas and try Redis with problems that have similar needs.

  • Redis makes an excellent cache or, more specifically, a memcached replacement. The reasoning for this is simple: you pretty much get memcached's features, plus lists and sets. You can use those collections to answer some queries, saving you trips to more expensive databases. That means you can use your cache more and increase the benefits of having it. You also get some persistence, which isn't critical in this scenario but is a nice value add.
  • Redis is an ideal realtime statistics tracker. If you are tracking stats in realtime, there are three things you really care about: speed, speed, and speed. Some nice atomic operations don't hurt either. From simple counters, to audit logs, to sets of unique IP addresses, and much more, Redis really rocks this kind of problem domain.
  • Redis can be the primary database for some Web applications. This one isn't as much of a given as the other two, obviously, and you can see that I've switched to using words like can and some. However, when your database needs are simple, Redis may be enough tool for the job.

    If you are going to try Redis in this role, you need to stay very aware of its minuses. For example, will it be OK if you lose some keys here and there? As you consider that though, do remember that you can force a save when needed. Perhaps it would be enough to force saves after a new user account is created and play things a little looser with the rest of the data, for example. It's also worth noting that Redis supports master-slave replication which can help reduce this risk.

    When considering if the entire database can be in memory at once, consider how fast the data will grow as well. Are you going to have time to monitor usage and react long before you run into a nasty limit?

    That said, there are plenty of applications that fit into those criteria. Take this blog for example. I'm sure the contents of it fit into a reasonable chunk of memory, it changes so little I could afford a hard save after every single write, and the rest of the time Redis would pay me back with crazy great reading speed.

    There's even a nice object mapping library for Redis: Ohm. I do encourage you to play with Redis at a lower level before resorting to such shortcuts though.

Hopefully this series has given you some ideas of how you might use Redis. It's not right for every application, but it can really be a big win where it fits. You should now know enough to watch for such opportunities and take advantage of them.

Comments (6)
  1. Janos
    Janos September 24th, 2009 Reply Link

    Thanks for the write up and excellent intro to Redis, James! I hope you'll continue the adventures in key-value land :) Keep up the good work!

    1. Reply (using GitHub Flavored Markdown)

      Comments on this blog are moderated. Spam is removed, formatting is fixed, and there's a zero tolerance policy on intolerance.

      Ajax loader
  2. Nemanja
    Nemanja January 19th, 2010 Reply Link

    James, thank you for this, and upcoming Tokyo Cabinet, series. Keep up the good work!

    1. Reply (using GitHub Flavored Markdown)

      Comments on this blog are moderated. Spam is removed, formatting is fixed, and there's a zero tolerance policy on intolerance.

      Ajax loader
  3. Paul Carey
    Paul Carey April 20th, 2010 Reply Link

    I'm not sure Redis does make an excellent memcached replacement, at least, not a drop-in one. As I understand it, memcached will evict the least recently read entries when it hits its memory limit. By contrast, Redis will only look at the TTL—which I don't believe is reset on a read.

    If this is the case, using Redis as a cache would require quite a bit more consideration than memcached. Maybe the list and set operations make that a worthwhile trade-off.

    1. Reply (using GitHub Flavored Markdown)

      Comments on this blog are moderated. Spam is removed, formatting is fixed, and there's a zero tolerance policy on intolerance.

      Ajax loader
  4. Toby Hede
    Toby Hede August 21st, 2010 Reply Link

    Do you think that now redis supports a fully persistent mode this would change some of your recommendations and advice?

    1. Reply (using GitHub Flavored Markdown)

      Comments on this blog are moderated. Spam is removed, formatting is fixed, and there's a zero tolerance policy on intolerance.

      Ajax loader
    2. James Edward Gray II
      James Edward Gray II August 21st, 2010 Reply Link

      I think it definitely gives more weight to the possibility of using it as a full database, in some cases. I don't think that will ever be the primary focus of Redis though.

      1. Reply (using GitHub Flavored Markdown)

        Comments on this blog are moderated. Spam is removed, formatting is fixed, and there's a zero tolerance policy on intolerance.

        Ajax loader
  5. DX
    DX May 7th, 2012 Reply Link

    Very useful, many thanks! To me the most convincing usage scenario is the second (real-time statistics) one.

    1. Reply (using GitHub Flavored Markdown)

      Comments on this blog are moderated. Spam is removed, formatting is fixed, and there's a zero tolerance policy on intolerance.

      Ajax loader
Leave a Comment (using GitHub Flavored Markdown)

Comments on this blog are moderated. Spam is removed, formatting is fixed, and there's a zero tolerance policy on intolerance.

Ajax loader