The Ruby VM Interview

The Ruby VM: Episode I

Hello and thank you both for agreeing to answer my questions. To begin, would you please introduce yourselves and tell us about your role in Ruby's development?

Matz:

I am the designer and the first implementer of the Ruby language. My real name is Yukihiro Matsumoto, that sounds something like You-Key-Hero Matz-Motor in English. But it's too long to remember and pronounce, so just call me Matz.

I have been developing Ruby since 1993. It is now quite complicated and has performance problem. I have had vague plan of rewriting the interpreter for long time, but I have never been motivated enough to throw out the current interpreter and start developing new one.

Then Koichi came in with YARV that seemed to have much brighter future than my vaporware - it runs - so I asked him to take a role of the official implementer of the core. Although I enjoy both designing and implementation of the language, I don't think I am gifted for language implementation. So I thought that it might be the time to focus on designing when I saw YARV.

ko1:

Thank you for your interest in YARV and me. BTW, I'm thinking what "YARV" stand for. Because it is not Yet Another. Someone proposed that "YARV ain't RubyVM". If YARV means "YARV ain't RubyVM", what is YARV?

I'm Koichi Sasada. Koichi is given name, and "ichi" means "one" in Japanese. So I use "ko1" as my nick. I'm an assistant at Department (...snip...) of Tokyo. My research interest is systems software, especially operating system, programming language, parallel systems, and so on. And I'm a member of Nihon Ruby no Kai (Ruby Association in Japan). I plan(ed) some Ruby events like RubyKaigi and am an editor of Rubyist Magazine. I also develop(ed) Nadoka, Rava, Rucheme, and some projects. Say, I'm a developer of YARV: Yet Another RubyVM.

My role in Ruby's development? To steal VM hacking pleasure from Matz?

The point of this interview is to talk about the future of Ruby's interpreter. To start that, can you please explain what YARV/Rite is? How is it different in design from the old Ruby interpreter?

Matz:

I have always been more interested in designing the language than implementing it. So Ruby interpreter is always slower than it should be. I think I pruned all low-hanging fruits, so that it seemed to required to re-implement whole core to achieve performance boost. I planned a new interpreter code-named 'Rite' in 2001 or so, but I have never motivated enough to start the project. Maybe I had been too busy, or perhaps too lazy.

Then, Koichi came in, and showed us his YARV. Many had tried implementing Ruby interpreter in the past, but no one but Koichi reached that level of implemented feature set (at the time; now we have JRuby and RubyCLR both compatible with Ruby 1.8). So I asked him to take part in the development of the new core, and he agreed.

January 1st 2007, he checked in YARV in to the trunk of our repository, so it is now official core of the Ruby 1.9. I am still working on old implementation in matzruby branch. Since it is easier for me to experiment new language features on the old interpreter, but I will eventually switch to the new engine.

For YARV implementation detail, Koichi will explain.

Does this mean we are leaving the name Rite behind and keeping YARV? Or will YARV be renamed at some point?

The name Rite will not be used for this generation of the language, unless Koichi ask me. I am not sure Koichi is going to keep YARV, or not, since it already 'the VM' for Ruby.

ko1:

YARV is vanished :)

In fact, I'm removing "yarv" words from structure names, function names, and file names. YARV is only code name that not made by *Matz*. Now, YARV is not "Yet Another". In this article, I use "YARV" words as current Ruby trunk on official repository.

At first, YARV is simple stack machine which run pseudo sequential instructions. Old interpreter (matzruby) *traverses* abstract syntax tree (AST) naively. Obviously it's slow. YARV compile that AST to YARV bytecode and run it.

Secondly, YARV uses native thread (that supported by OS or so) to implement Ruby thread. It means that you can run *blocking* task in extension libraries. (On Ruby's spec, blocking task should be interrupted by Thread#raise. To know details, see [ruby-core:10252].) Because thread creation is slower than matzruby (green thread), you shouldn't make many threads at a time. Supporting native thread *does not* means that you can run Ruby scripts in *parallel* on parallel machine such as Multi-Core CPUs. Current implementation uses Giant VM Lock to avoid synchronization problems. (Many extension libraries doesn't care thread safety. See array.c, string.c, etc.)

Thirdly, I made many optimization like specialized instructions, etc. These features are my purpose of developing YARV. Toy benchmarks run fast because of these optimization techniques.

YARV doesn't change parser/syntax/specs (matz' hobbytask), GC (memory/object management), and extension libraries like String/Array/Hash/Regexp/etc. Therefore your script doesn't run fast on YARV if bottleneck is string processing, or so.

Congratulations to you both for completing Ruby/YARV merger recently. That must have been a lot of work, but I know it has the whole Ruby world very excited. Now that the merger has taken place, how do you see this changing the way Ruby is developed?

Matz:

Congrats should go to Koichi who has done a lot of work. I am moving my developing from matzruby (a branch for my old interpreter) to trunk (the yarv). Recently I have implemented some new features on the trunk, for example, class local instance variables and new local variable scope. The transition will complete pretty soon.

Since the trunk is originally Koichi's work, I need more help from others especially from Koichi than before. I know everything about the previous interpreter (well, most of them), but there are still mysteries in the new one. I am well satisfied with new one. It's clearer, well-formed, and faster.

ko1:

Thank you. I'm a newbie of Ruby developer (in fact, I didn't have CVS account to commit any ruby codes). So I can't say how change on ruby development :)

When will the first production release of Ruby running on YARV by available for all Rubyists to play with?

Matz:

Short answer: now.

Longer answer: the YARV is already publicly avaiblabe via our Subversion repository. You can fetch and play with it now. But the first public "release" from us will be Christmas 2007, if we are as diligent as we should be. Knowing how lazy I am, I will try not to be a stumbling block for the release. ;-)