Terminal Tricks

Basic Curses

In my last article, I showed off some of the various magic commands a Unix terminal recognizes and how you can use those commands to move the cursor around, change the colors of your output, and read input in a character by character fashion. It's not very common to manipulate things manually as I did in those examples. There are libraries that wrap these mechanisms and add abstractions of their own. Using one of them can be easier and less error prone.

Probably the most famous of these higher abstraction libraries is curses. However, your version won't be called that. curses was the original library for System V UNIX. These days you are far more likely to have ncurses which a replacement library that emulates the original. The truth is, you probably don't have exactly that library either. Instead, you may have ncursesw, which is the same library with wide character (read: non-ASCII) support. Also, curses is often discussed with several add on libraries: panel, menu, and form. Documentation often covers these separate units together.

Speaking of documentation, coverage of using curses in Ruby is pretty non-existent. That's the main reason I'm writing these articles. Most of the Ruby wrappers are almost direct copies of the C APIs, so you can mostly use that documentation with minor changes. (I'm aware of at least one higher level wrapper, but the documentation story isn't much better there.) My three best sources for learning curses have been this very thorough How To from The Linux Documentation Project, the man pages, and the examples directory in the ffi-ncurses gem. Hopefully I can save you the trouble of reading all of that with these articles.

The previously mentioned ffi-ncurses gem is one of your choices for wrappers in Ruby. It's the library I'll be covering, because:

• It worked for me without additional installs on Mac OS X (Yosemite)
• It does some reasonable setup for you to make using Unicode characters pretty painless
• It supports JRuby

According to The Ruby Toolbox, the more popular library is ncursesw. It doesn't quite meet the advantages I listed above, but it does wrap menu and form (which ffi-ncurses does not).

Again though, both of these libraries a very nearly direct copies of the C API, so using either is pretty similar.

Enough background. Let's play with some curses code. I'll redo the examples from my last article below, using ffi-ncurses and point out the differences as we go. It may be handy to have those previous examples open at the same time so you can compare the two versions as you read this.

Output

Last time, my first example was how to move the cursor around relative to it's current position. I don't think this kind of treatment is as common when using curses for reasons I'll get into in a bit, but it can still be done.

require "ffi-ncurses"  # gem install ffi-ncurses

begin
  # setup
  stdscr = FFI::NCurses.initscr  # start curses
  FFI::NCurses.cbreak
  FFI::NCurses.noecho

  # write some content
  FFI::NCurses.waddstr(stdscr, "onez\n")
  FFI::NCurses.waddstr(stdscr, "twos\n")
  FFI::NCurses.waddstr(stdscr, "threes\n")
  FFI::NCurses.wrefresh(stdscr)

  FFI::NCurses.getch  # pause waiting on a keypress

  y, x = FFI::NCurses.getyx(stdscr)         # find the cursor
  FFI::NCurses.wmove(stdscr, y - 3, x + 3)  # make a relative move
  FFI::NCurses.waddstr(stdscr, "s")         # fix the content
  FFI::NCurses.wmove(stdscr, y, x)          # go back to where we were
  FFI::NCurses.wrefresh(stdscr)             # update what the user is seeing

  FFI::NCurses.getch  # pause waiting on a keypress
ensure
  FFI::NCurses.endwin  # end curses
end

OK, so the first major difference you notice is that it's a lot more code and it's not too pretty. We're straight up using a C API here, so it's not going to be very Rubyish. I'll try to fix that in a later article, but I wanted to cover how this stuff works first. Also, I could have done an include FFI::NCurses and dropped a ton of references to the module here. I didn't do that for two reasons:

• So you could easily see what's part of the module's API
• Doing so would dump a ton of methods into Ruby's top-level scope, which is probably not a great practice

Another difference between the code in my last article and what you see above is the need to initialize curses and shut it down when we're done. You will pretty much always see this begin … ensure … end pattern with calls to initscr() and endwin() when working with curses in Ruby.

Ignore the calls to cbreak() and noecho() for now. I'll discuss those below.

Finally, the first chunk of waddstr() calls should start to look something like the example we're rewriting. waddstr() is one of about a gazillion ways that you can write to the screen using curses. It writes a String. I recommend pretty much always using this method for several reasons:

• It's easy to understand
• It will do the right thing if your String includes Unicode characters
• It allow you to ignore quite a bit of the curses API

This last point is a pretty big deal. curses is a massive collection of functions. A great many of them provide alternate ways to do similar things. Variety is nice, but Ruby already gives you so many useful tools. For example, there's a printf()-like variation of the String writing function I showed you. You don't need that. Use String#% in Ruby and hand the result to waddstr(). It's true that some of the curses functions I'm skipping probably perform a tiny bit better due to skipping some steps, but on a modern computer you are unlikely to need more drawing speed even with a full screen terminal. Instead, I recommend getting a handle on the API basics and adding the other stuff only when you find yourself with a genuine need.

At the end of the output, notice that I needed a call to wrefresh(). In curses adding content is a separate operation from drawing that content to the screen. We have to explicitly tell a window when to redraw itself, which is what wrefresh() does.

I haven't talked about what a curses window is yet and, to be honest, I'm going to save most of that for a future article. But here's all you need to know for now:

• The w prefix on most of these functions stands for window and it means they expect a window as their first argument
• A window is a place where you can draw output to the screen in curses code
• When curses starts up, it creates a default window, that is the full size of the terminal, called stdscr (for standard screen I assume)
• initscr() returns stdscr
• curses has a duplicate API without the w prefixes that mostly just assumes you want to work with stdscr (addstr(), refresh(), etc.)
• You can't do everything with that less verbose API, so I'm voting you ignore it for now

To sum up: curses will give you the stdscr window. Just pass that everywhere you need a window reference until you're ready to start making windows of your own.

OK, another change in this example is that I added two pauses using getch(). This just waits for you to push a key and it's the reason that I set the cbreak() and noecho() modes early on. (More on modes below.) It's worth noting that I had to do this. curses usually clears the screen as you enter and exit curses mode so, without the pauses, you wouldn't have seen any output.

We're finally ready to discuss the meat of this example, the moving and changing code. To refresh your memory, here are the lines I'm talking about:

# ...

  y, x = FFI::NCurses.getyx(stdscr)         # find the cursor
  FFI::NCurses.wmove(stdscr, y - 3, x + 3)  # make a relative move
  FFI::NCurses.waddstr(stdscr, "s")         # fix the content
  FFI::NCurses.wmove(stdscr, y, x)          # go back to where we were
  FFI::NCurses.wrefresh(stdscr)             # update what the user is seeing

# ...

getyx() is the reason I had to use the window referencing functions in this example. Though it doesn't have a w prefix, it does require a window to find the cursor in. It's also one of the few places the ffi-ncurses deviates from the C API, switching to Ruby's multiple-assignment friendly return of an Array of two Integers instead of forcing you to pass two pointer arguments.

It's worth noting that *yx() functions in the curses API are always in that order, Y then X. This can trip you up if your accustom to X then Y ordering.

The rest of this code should be pretty easy to grok as it just moves around using the known cursor coordinates. This is very similar to the original example, except that we didn't need to find the cursor last time.

I know that was a slog because I had to cover through so many curses-isms, but that should allow us to coast through these next few examples now that we have the basics. Here's the similar example from last time using absolute movement:

require "ffi-ncurses"

begin
  stdscr = FFI::NCurses.initscr
  FFI::NCurses.cbreak
  FFI::NCurses.noecho

  FFI::NCurses.waddstr(stdscr, "onez\n")
  FFI::NCurses.waddstr(stdscr, "twos\n")
  FFI::NCurses.waddstr(stdscr, "threes\n")
  FFI::NCurses.wrefresh(stdscr)

  FFI::NCurses.getch

  y, x = FFI::NCurses.getyx(stdscr)          # save cursor position
  FFI::NCurses.mvwaddstr(stdscr, 0, 3, "s")  # a combined move and add
  FFI::NCurses.wmove(stdscr, y, x)           # restore the cursor position
  FFI::NCurses.wrefresh(stdscr)              # update what the user sees

  FFI::NCurses.getch
ensure
  FFI::NCurses.endwin
end

Only the commented lines differ in this example and they correspond pretty closely to the matching example from the previous article. There's really just one surprise here. Remember when I said that terminals index rows and columns starting at 1? Well curses switches back to the more common 0 for maximum confusion. That's why 1;4 has become 0, 3 in this version.

I do believe this is a more common usage pattern with curses code. The reason is that we know the screen has been cleared and we know that we're drawing into some window at a known location. With those givens, absolute coordinates make a lot of sense.

The colors example has more changes:

require "ffi-ncurses"

def color_pairs
  @color_pairs ||= { }
end

def color_pair(foreground, background)
  foreground = foreground.to_s.sub(/\Abright_/, "").to_sym
  color      = [foreground, background]

  # add the color pair if this is the first time we've seen it
  unless color_pairs.include?(color)
    foreground_color   = FFI::NCurses::Color.const_get(foreground.upcase)
    background_color   = FFI::NCurses::Color.const_get(background.upcase)
    number             = color_pairs.size + 1
    color_pairs[color] = number
    FFI::NCurses.init_pair(number, foreground_color, background_color)
  end

  # lookup the color pair
  color_pairs[color]
end

def color(window, foreground, background)
  weight = foreground.to_s.start_with?("bright") ? FFI::NCurses::A_BOLD
                                                 : FFI::NCurses::A_NORMAL
  pair   = color_pair(foreground, background)
  FFI::NCurses.wattr_set(window, weight, pair, nil)  # change colors
  yield
ensure
  # change back to default colors
  FFI::NCurses.wattr_set(window, FFI::NCurses::A_NORMAL, 0, nil)
end

begin
  stdscr = FFI::NCurses.initscr
  FFI::NCurses.cbreak
  FFI::NCurses.noecho
  FFI::NCurses.start_color                    # turn colors on
  FFI::NCurses.assume_default_colors(-1, -1)  # make color pair 0 the defaults

  [ ["C", :red],
    ["O", :bright_red],
    ["L", :bright_yellow],
    ["O", :bright_green],
    ["R", :bright_blue],
    ["S", :blue],
    ["!", :bright_magenta] ].each do |char, color|
    color(stdscr, color, :white) do
      FFI::NCurses.waddstr(stdscr, char)
    end
  end
  FFI::NCurses.waddstr(stdscr, "\n")
  FFI::NCurses.wrefresh(stdscr)

  FFI::NCurses.getch
ensure
  FFI::NCurses.endwin
end

When you want to use colors in curses you need to set them up. First, that means that you should call start_color() as part of your initialization sequence. I would also recommend calling FFI::NCurses.assume_default_colors(-1, -1). Let me explain why I say that.

To use colors in curses, you create color pairs: numbered groupings of foreground and background colors. Pair 0 already exists. It's set to black on black, I believe. Not so helpful. (The documentation says you can't change it though I'm pretty sure I've been allowed to do so in the past.) Anyway, if you call FFI::NCurses.assume_default_colors(-1, -1), it switches pair 0 to being whatever the default foreground and background are for the terminal. That's a much more useful default, if you ask me.

Now that you know about color pairs, you can probably puzzle out my color_pair() method. It just pulls color pair numbers out of the color_pairs() Hash. If it's the first time a pair has been seen, it inserts it into the Hash in the next numbered slot before reading it back. It does this by looking up some predefined constants in FFI::NCurses::Color for named terminal colors.

color() works similar to how it did in the example from my last article: switch colors, write some content (by invoking the block in this case), and then switch back (using that useful pair 0 default that we setup earlier). Note that bright colors are handled here by setting the FFI::NCurses::A_BOLD attribute with the color pair.

The rest of the code is much like it was before.

The Status Line Trick from the previous article no longer applies when we've gone all in on loading curses. However, I did have a tiny example last time that showed how to get the screen size when I introduced io/console in the Input section. For completeness, here's the similar code for curses:

require "ffi-ncurses"

rows, cols = nil, nil
begin
  stdscr = FFI::NCurses.initscr
  rows   = FFI::NCurses.getmaxy(stdscr)
  cols   = FFI::NCurses.getmaxx(stdscr)
ensure
  FFI::NCurses.endwin
end
puts "Columns:  #{cols}, Rows:  #{rows}"

I prefer to do it in two steps like this because you don't have to worry about pointers. getmaxy() and getmaxx() are a set of functions classified as a legacy interface, but they've been available forever and I don't think there's any danger of them being removed.

If you prefer though, you can do it in one step by passing Arrays as the pointers to be filled in:

require "ffi-ncurses"

rows, cols = [ ], [ ]
begin
  stdscr = FFI::NCurses.initscr
  FFI::NCurses.getmaxyx(stdscr, rows, cols)
ensure
  FFI::NCurses.endwin
end
puts "Columns:  #{cols.first}, Rows:  #{rows.first}"

Note the added calls to first() in the last line to get the Integers back out of the Arrays.

Input

If you'll recall, the trick of getting terminal input right is all about using the right modes. Well, you just got to the good stuff! curses has a rich set of modes that you can mix and match to suite exactly your current needs. In my opinion, this is one of the biggest reasons to use curses.

Here's the simplest way to read individual characters as they are typed using curses:

require "ffi-ncurses"

begin
  stdscr = FFI::NCurses.initscr
  FFI::NCurses.cbreak  # like raw, but don't eat things like control-c
  FFI::NCurses.noecho  # don't echo what the user types

  loop do
    key = FFI::NCurses.getch.chr  # read and convert to a String
    FFI::NCurses.waddstr(stdscr, "You typed:  #{key.inspect}\n")
    FFI::NCurses.wrefresh(stdscr)
    break if key == ?q
  end
ensure
  FFI::NCurses.endwin
end

Most of the work here is handled by two of the most boring lines. The call to cbreak() puts us in a raw-like mode where we can read single keystrokes as they are made, but where we can still interrupt the program (with control-C) or suspend it (with control-Z).

noecho() turns off the automatic copying of what's typed to the screen. Notice how this is a separate concern from cbreak() mode in curses and I get to handle them independently.

Once your modes are set, getting input is as simple as calling getch(). Remember that it will return Integers though, so you'll usually want a call to chr() to get the String for what was typed.

The next example we looked at last time was how to read keys only if they are available. Again, curses modes to the rescue:

require "ffi-ncurses"

begin
  stdscr = FFI::NCurses.initscr
  FFI::NCurses.halfdelay(1)  # set a read delay of one tenth of a second
  FFI::NCurses.noecho

  last_read = Time.now
  prompted  = false
  loop do
    char = FFI::NCurses.getch     # this will wait for the read delay
    if char != FFI::NCurses::ERR  # ERR is returned when we hit the delay
      last_read = Time.now
      key       = char.chr
      FFI::NCurses.waddstr(stdscr, "You typed:  #{key.inspect}\n")
      FFI::NCurses.wrefresh(stdscr)
      break if key == ?q
    elsif !prompted && Time.now - last_read > 3
      FFI::NCurses.waddstr(stdscr, "Please type a character.\n")
      FFI::NCurses.wrefresh(stdscr)
      prompted = true
    end
  end
ensure
  FFI::NCurses.endwin
end

This new halfdelay() mode is like cbreak(), but with an added delay that you specify in tenths of a second. In this mode, calls to getch() will wait for input until the delay is exhausted, then return FFI::NCurses::ERR. This solves exactly the problem we had last time, doesn't require the use of additional libraries or cryptic low-level methods, and it even allows us to drop the call to sleep(). You just have to check to see if you got FFI::NCurses::ERR instead of an actual key press.

If you ever need a full non-breaking, zero wait read with curses you can have that too. The timeout()/wtimeout() functions are there when you need them.

It's also worth pointing out that we never ran into the newline translation problem with the curses. That's yet another mode. I never turned it off and we never has a problem.

Which brings us to the final example. Last time we noted that our code didn't properly handle special keys, like the arrow keys. Both examples above have the same issue. This probably won't surprise you by now, but curses has a mode for that too:

require "ffi-ncurses"

begin
  stdscr = FFI::NCurses.initscr
  FFI::NCurses.halfdelay(1)
  FFI::NCurses.noecho
  FFI::NCurses.keypad(stdscr, true)  # handle special keys

  last_read = Time.now
  prompted  = false
  loop do
    char = FFI::NCurses.getch
    if char != FFI::NCurses::ERR
      last_read = Time.now
      key       = nil
      # find key in special constants
      if (FFI::NCurses::KEY_CODE_YES..FFI::NCurses::KEY_MAX).include?(char)
        FFI::NCurses::KeyDefs.constants.each do |key_name|
          if key_name.to_s.start_with?("KEY_") &&
             FFI::NCurses::KeyDefs.const_get(key_name) == char
            key = key_name
          end
        end
      else
        key = char.chr
      end
      FFI::NCurses.waddstr(stdscr, "You typed:  #{key.inspect}\n")
      FFI::NCurses.wrefresh(stdscr)
      break if key == ?q
    elsif !prompted && Time.now - last_read > 3
      FFI::NCurses.waddstr(stdscr, "Please type a character.\n")
      FFI::NCurses.wrefresh(stdscr)
      prompted = true
    end
  end
ensure
  FFI::NCurses.endwin
end

The new call here is FFI::NCurses.keypad(stdscr, true) which turns on the handling of special characters. I can't think of very many cases where that isn't desirable, so I always turn it on.

I used a fancy scan of constants in the if statement to show the name of the pressed key in the output of this program, but the typical usage is much simpler. You often write code like this:

case char
when FFI::NCurses::KeyDefs::KEY_UP
  # ...
when FFI::NCurses::KeyDefs::KEY_RIGHT
  # ...
end

Anyway, when I run the code above and push some arrow keys, I see output like:

You typed:  :KEY_UP
You typed:  :KEY_RIGHT

I still have trouble with some keystrokes not being detected (like control-alt-o) and other being swallowed by my terminal for other purposes (some function keys). I believe some of this could be solved by tweaking my terminal settings, but I'm generally only ever after the arrow keys anyway and they work fine.

That concludes our adaptation of basic terminal manipulation code to similar operations using curses. This shows you some basics, but there's more to curses than what I've covered here. Next time I'll look at some of the abstractions curses adds beyond these basic tools.