Gray Soft / The Ruby VM Interview

The Ruby VM: Episode V

2014-04-04T21:36:08Z

You have told us before that one of the big reasons to move to a new Ruby VM was to provide new options for optimization. Can you talk a little about the optimizations you have added to the new Ruby VM thus far and what operations will likely be faster because of them?

ko1:

OK. At first, I write about basic of YARV instruction. YARV has two type instructions. First is primitive instruction. It's as written, primitive. Ruby code can be represented in these primitive instruction. Second is instructions for optimization. It's not needed to represent Ruby scripts, but they are added for optimization. Primitive instructions doesn't include _ in their name (like putobject), and optimize instructions do (like opt_plus). This policy helps you if you want to see VM instructions. Initially, you need to read primitive instructions.

The most easy and effective optimization is Specialized Instructions. This optimization replace method call with another VM instruction, such as Fixnum#+ to opt_plus. Current Ruby's numeric calculation is slow because all operations are method call. For example, 1 + 2 means 1.+(2). But numeric operations are more lightweight than Ruby's method invocation. So method call is only overhead for numeric operation. Specialized Instructions allow the VM to skip method call overhead.

But we can't know which expression is numeric operation or not at compile time. See this expression: a = c ? 1 : [:elem], a will be Fixnum or Array at runtime.

So, we can't replace + expression with numeric operation instruction. Specialized Instruction, for example opt_plus which is replaced with + method invocation will do following code:

def opt_plus(recv, val) # simple version
   if recv.class == Fixnum && val.class == Fixnum
     if Fixnum#+ is not redefined
       return calculate "recv + val" without method call
     end
   end
   # normal method invocation
   recv.+(val)
end

Check receiver and value are Fixnum or not, and check Fixnum#+ are not redefined. After these check, calculate them without method invocation. In fact, Float#+ are also checked. There are other specialized instructions.

YARV eases to implement such instructions with VM generator. You shouldn't write bothersome code such as stack manipulation. If you write VM instruction such as opt_plus in simple VM DSL, VM generator will translate it to C code.

Specialized Instruction is very simple, but effective for simple benchmark such as fib() or tak() and some calculate bound program.

One question I thought of while reading your previous answer was: will Ruby scripts be able to access these VM instructions, if desired?

ko1:

Simple answer is "yes".

On YARV, bytecode and other information are represented as the VM::InstructionSequence class. I often use the name "ISeq" to point that class. ISeq object contains a bytecode sequence, a catch table (to retrieve exception and other global escape such as break), a local variable name table and others.

ISeq object can be dumped in Ruby's primitive objects such as Array, Hash, Fixnum and so on. In the same way, ISeq can be built with such data with primitive objects. This means that you can built YARV bytecode without YARV compiler. Of course, this feature can be used for other purpose such as ruby script obfuscation (this is like Java class file).

(BTW, I use this feature on Ruby2C compiler. It is hard to translate Ruby program to C program directly. But from YARV instruction, translation is easy. If I finished it, I want to bundle this with Ruby.)

Therefore it is hard to write ISeq dumped data. So I had prepared lib/yasm.rb as YARV Assembler (this is not committed on current trunk). With YASM, you can write YARV bytecode sequence on Ruby program. Note that YARV/ISeq loader doesn't have the byte code verifier. So illegal bytecode sequence is loaded, YARV/Ruby will dumps core.

If I commit lib/yasm.rb, I'll write tutorial to use that.

Does the new Ruby VM optimize tail recursive methods? If no, are there any plans to add this optimization?

ko1:

YARV doesn't support "tail recursion optimization", but supports "tail call optimization".

See this program:

class C
   def foo
     foo # (A) tail recursive call
   end
end

class D < C
   def foo
     super
   end
end

D.new.foo

Can you replace goto with (A)? (A) should call D#foo so we eliminate tail method call. Yes, we can implement this optimization with following trick.

class C
   def foo
     if search_method(:foo) == C#foo
       goto first_of_foo
     else
       foo
     end
   end
end

But we must think of inter block tail recursion or so (inter block goto is not permitted) if implement tail recursion optimization.

BTW, YARV support tail call optimization, eliminate stack frame of caller. You can call method which at tail position without consuming VM stack like scheme language. So you can use method call to loop something. You can make state transition with method call.

Note that tail call optimization has some caution. First is backtrace elimination. You can't see caller method of tail method with backtrace. Second, this optimization does not speedup method call. Tail call process is almost same as process of normal method call. At end of normal method call process, check if tail call or not. If that method call is tail call, use current method frame to setup method frame instead of pushing new stack frame.

Current Ruby 1.9 (trunk) is not enabled this optimization. If you want to try this, please re-write that option in vm_opts.h (OPT_TAILCALL_OPTIMIZATION) and re-compile that. I think release version of Ruby 1.9 is enabled this optimization. I need more comments of it. Please teach me if you find out some critical problem.

Can you talk a little about some optimizations you would like to add to the new Ruby VM in the future?

ko1:

In near future, I'll release AOT, Ruby to C compiler. This translator will support all Ruby specification, so it's shouldn't be silver bullet for performance.

Keeping all Ruby spec means "can't achieve high performance". If I ignore some spec, I'll be able to do more drastic optimization. So C code translated from Ruby script will be slow (of course, faster than normal interpretation).

Ruby specification is enemy for compiler/VM developer. So I want to add a "pragma" syntax to add programer's knowledge. For example, "eval is not appear in this file" or "Fixnum methods are not re-defined". These information will help compiler to do more effective optimization.

And I'm planning to implement block inlining. I think it is very effective for Ruby. An experimental, incomplete version has been made. I need more research to realize it.

BTW, I will not touch JIT compilation. I think it is not reasonable (not worth the cost of implementation). Everyone love "JIT" words, but I think it's not effective on Ruby spec.

The Ruby VM: Episode IV

2014-04-04T21:27:35Z

We've talked about threads, so let's talk a little about character encodings. This is another big change planned for Ruby's future. Matz, you have stated that you plan to add m17n (multilingualization) support to Ruby. Can you talk a little about what that change actually means for Ruby users?

Matz:

Nothing much, except for some incompatibility in string manipulation, for example, "abc"[0] will give "a" instead of 97, and string indexing will be based on character instead of byte. I guess the biggest difference is that we can officially declare we support Unicode. ;-)

Unlike Perl nor Python, Ruby's M17N is not Unicode based (Universal Character Set or USC). It's character set independent (CSI). It will handle Unicode, along with other encoding schemes such as ISO8859 or EUC-JP etc. without converting them into Unicode.

Some misunderstand our motivation. We are no Unicode haters. Rather, I'd love to use Unicode if situation allows. We hate conversion between character sets. For historical reasons, there are many variety of character sets. For example, Shift_JIS character set has at least 5 variations, which differ each other in a few characters mapping. Unfortunately, we have no way to distinguish them. Thus conversion may cause information loss. If a language provide Unicode centric text manipulation, there's no way to avoid the problem, as long as we use that language.

ko1:

On my policy, I escape from this topic :)

With String being enhanced to be encoding aware, some worry that we will need to specify an encoding for every String we make. Can you talk a little about how this will work in practice? Is there a default encoding? Can we set an encoding for the entire program?

Matz:

You can specify the encoding for Ruby scripts by the coding pragma at the head of the script. For example, if your script is in UTF-8, try specify

# coding: utf-8

that makes all strings and regex literals in the script to be specified UTF-8. You can also specify the encoding for IO reading strings via open, e.g.

open(path, "r:utf-8") do |f|
  line = f.gets
end

or by binmode (ala Perl), e.g.

f = open(path, "r")
f.binmode(":utf-8")

The default encoding is binary for ordinary IO, and locale specified encoding for STDIN. It should be allowed that encoding conversion at the time of IO reading, but the API is not fixed yet. Maybe

open(path, "r:utf-8<euc-jp")

that should read EUC-JP data then convert it into UTF-8 and return the converted string.

Can you tell us how far along the m17n code is and how much still needs to be done? Is this change expected to be in the 1.9.1 release?

Matz:: You will see M17N in 1.9.1 coming out next Christmas, unless something bad happens. I have done almost everything for character treating, but things related to code conversion (String#encode method and code conversion for IO) are still left undone.

The Ruby VM: Episode III

2014-04-04T21:03:39Z

Let's talk a little about threading, since that's a significant change in the new VM. First, can you please explain the old threading model used in Ruby 1.8 and also the new threading model now used in Ruby 1.9?

Matz:

Old threading model is the green thread, to provide universal threading on every platform that Ruby runs. I think it was reasonable decision 14 years ago, when I started developing Ruby. Time goes by situation has changed. pthread or similar threading libraries are now available on almost every platform. Even on old platforms, pth library (a thread library which implements pthread API using setjmp etc.) can provide green thread implementation.

Koichi decided to use native thread for YARV. I honor his decision. Only regret I have is we couldn't have continuation support that used our green thread internal structure. Koichi once told me it's not impossible to implement continuation on YARV (with some restriction), so I expect to have it again in the future. Although it certainly has lower priority in 1.9 implementation.

ko1:

Matz explained old one, so I show you YARV's thread model.

As you know, YARV support native thread. It means that you can run each Ruby thread on each native thread concurrently.

It doesn't mean that every Ruby thread runs in parallel. YARV has global VM lock (global interpreter lock) which only one running Ruby thread has. This decision maybe makes us happy because we can run most of the extensions written in C without any modifications.

Why was this change made? What's wrong with green threads?

Matz:: Because green threads does not work well with libraries using native threads. For example, Ruby/Tk has made huge effort to live along with pthread.
ko1:: Ruby's green (userlevel) thread implementation was too naive to run fast. All machine stacks are copied when thread context switches. And more important point is it's not easy to re-implement green thread on YARV :)

What are the downsides to the native threads approach?

Matz:

It is pretty difficult to implement continuation. Besides that, even with native thread approach, no real concurrency can not be made due to the global interpreter lock. Koichi is going to address this issue by Multi-VM approach in the (near) future.

ko1:

Yes, it has several problems. First is Performance problem (as you know, I love to discuss about performance). Too create native thread is too pricey. So you may use thread pool or so. And current trunk (YARV) is not tuned on native thread, so I believe some unknown problems around threads.

Second problem is portability. If your environment has pthread library, but there are some difference from other pthread system in detail.

Third problem is absence of callcc (which is implemented with green thread scheme) ... for some people :)

Programming on native thread has own difficulty. For example, on MacOS X, exec() doesn't work (cause exception) if other threads are running (one of portability problem). If we find critical problems on native thread, I will make green thread version on trunk (YARV).

Are there plans to support other threading models in the future?

Matz:

Other threading model, no. Win32 threads and pthreads are enough burden for us to support. There might be other features to support parallelism in the future, for example light-weight process a la Erlang.

Koichi may have other idea(s) about supporting concurrency, such as Multi-VM since he is the expert on it.

ko1:

Parallel computing with Ruby is one of my main concern. There are some way to do it, but running Ruby threads in parallel (without Giant VM Lock) on a process is too difficult to support current C extension libraries because of their synchronization problems.

As matz say, if we have multiple VM instance on a process, these VMs can be run in parallel. I'll work on that theme in the near future (as my research topic).

BTW, I wrote on last question, if there are many many problems on native threads, I'll implement green thread. As you know, it's has some benefit against native thread (lightweight thread creation, etc). It will be lovely hack (FYI. my graduation thesis is to implement userlevel thread library on our specific SMT CPU).

... Does anyone have interest to implement it?

The Ruby VM: Episode II

2014-04-30T02:30:03Z

We started these talks because of the excitement around the alternate implementations, like JRuby and Rubinius. How do you feel about all of these new interpreters and how do you see them affecting the official development of Ruby?

Matz:

Alternate implementations mean maturity of Ruby language. I'm glad for the fact. But we have never had enough number of developers for core, so I think we need more cooperation between implementations. I had a good talk about future Ruby spec. with Charles Nutter recently. I expect occasion like this more often.

ko1:

I think having alternatives is very important. I want to know how to implement Ruby and apply these techniques to YARV.

In fact, implementing from scratch is very fun. YARV (official Ruby implementation) has many problems resulted from historical reasons (a biggest problem is compatibility to extension libraries).

Have you downloaded and installed any of the other interpreters?

Matz:

No, I just skimmed a few files from Rubinius, but not others. Mostly because I am not familiar with neither Java nor Parrot.

ko1:

I wanted to try these alternatives, but no time to do it (and no time to hack YARV ...).

Answer is: No. I'll try.

Is there a good exchange of ideas between the various implementation teams? Do you talk to the other teams, read their code, and/or discuss implementation details with them?

Matz:

Besides Koichi who works on YARV with me, Last month I met with Charles Nutter and exchanged very interesting idea about 2.0 behavior. Evan Phoenix also gave me inspiration. I am very glad to see more programmers with interest and knowledge in language implementation.

ko1:

Sometimes I talked with JRuby team on IRC. I want to discuss every Ruby implementation developers, especially performance of it.

BTW we need 3 things on this context:

Documents of specification
Good tests
Good benchmarks

Tests: Ruby trunk and 1.8 have test suits. But it's too difficult to test with it on early stage of implementation, because test/unit use many ruby's functions (RSpec has a same problem). Now, trunk has "bootstraptest" to solve it. I think it is good solution for this problem. And it's show a minimum ruby's specification.

Benchmark tests: Some people using YARV's benchmarks I wrote. But I didn't write these codes to measure "Ruby's general benchmark test", but to measure speed-up ratio on YARV. It's means that I wrote codes what YARV optimizes. We must prepare more suitable benchmarks for "Ruby implementations".

The Ruby VM: Episode I

2014-04-04T20:29:56Z

Hello and thank you both for agreeing to answer my questions. To begin, would you please introduce yourselves and tell us about your role in Ruby's development?

Matz:

I am the designer and the first implementer of the Ruby language. My real name is Yukihiro Matsumoto, that sounds something like You-Key-Hero Matz-Motor in English. But it's too long to remember and pronounce, so just call me Matz.

I have been developing Ruby since 1993. It is now quite complicated and has performance problem. I have had vague plan of rewriting the interpreter for long time, but I have never been motivated enough to throw out the current interpreter and start developing new one.

Then Koichi came in with YARV that seemed to have much brighter future than my vaporware - it runs - so I asked him to take a role of the official implementer of the core. Although I enjoy both designing and implementation of the language, I don't think I am gifted for language implementation. So I thought that it might be the time to focus on designing when I saw YARV.

ko1:

Thank you for your interest in YARV and me. BTW, I'm thinking what "YARV" stand for. Because it is not Yet Another. Someone proposed that "YARV ain't RubyVM". If YARV means "YARV ain't RubyVM", what is YARV?

I'm Koichi Sasada. Koichi is given name, and "ichi" means "one" in Japanese. So I use "ko1" as my nick. I'm an assistant at Department (...snip...) of Tokyo. My research interest is systems software, especially operating system, programming language, parallel systems, and so on. And I'm a member of Nihon Ruby no Kai (Ruby Association in Japan). I plan(ed) some Ruby events like RubyKaigi and am an editor of Rubyist Magazine. I also develop(ed) Nadoka, Rava, Rucheme, and some projects. Say, I'm a developer of YARV: Yet Another RubyVM.

My role in Ruby's development? To steal VM hacking pleasure from Matz?

The point of this interview is to talk about the future of Ruby's interpreter. To start that, can you please explain what YARV/Rite is? How is it different in design from the old Ruby interpreter?

Matz:

I have always been more interested in designing the language than implementing it. So Ruby interpreter is always slower than it should be. I think I pruned all low-hanging fruits, so that it seemed to required to re-implement whole core to achieve performance boost. I planned a new interpreter code-named 'Rite' in 2001 or so, but I have never motivated enough to start the project. Maybe I had been too busy, or perhaps too lazy.

Then, Koichi came in, and showed us his YARV. Many had tried implementing Ruby interpreter in the past, but no one but Koichi reached that level of implemented feature set (at the time; now we have JRuby and RubyCLR both compatible with Ruby 1.8). So I asked him to take part in the development of the new core, and he agreed.

January 1st 2007, he checked in YARV in to the trunk of our repository, so it is now official core of the Ruby 1.9. I am still working on old implementation in matzruby branch. Since it is easier for me to experiment new language features on the old interpreter, but I will eventually switch to the new engine.

For YARV implementation detail, Koichi will explain.

Does this mean we are leaving the name Rite behind and keeping YARV? Or will YARV be renamed at some point?

The name Rite will not be used for this generation of the language, unless Koichi ask me. I am not sure Koichi is going to keep YARV, or not, since it already 'the VM' for Ruby.

ko1:

YARV is vanished :)

In fact, I'm removing "yarv" words from structure names, function names, and file names. YARV is only code name that not made by *Matz*. Now, YARV is not "Yet Another". In this article, I use "YARV" words as current Ruby trunk on official repository.

At first, YARV is simple stack machine which run pseudo sequential instructions. Old interpreter (matzruby) *traverses* abstract syntax tree (AST) naively. Obviously it's slow. YARV compile that AST to YARV bytecode and run it.

Secondly, YARV uses native thread (that supported by OS or so) to implement Ruby thread. It means that you can run *blocking* task in extension libraries. (On Ruby's spec, blocking task should be interrupted by Thread#raise. To know details, see [ruby-core:10252].) Because thread creation is slower than matzruby (green thread), you shouldn't make many threads at a time. Supporting native thread *does not* means that you can run Ruby scripts in *parallel* on parallel machine such as Multi-Core CPUs. Current implementation uses Giant VM Lock to avoid synchronization problems. (Many extension libraries doesn't care thread safety. See array.c, string.c, etc.)

Thirdly, I made many optimization like specialized instructions, etc. These features are my purpose of developing YARV. Toy benchmarks run fast because of these optimization techniques.

YARV doesn't change parser/syntax/specs (matz' hobbytask), GC (memory/object management), and extension libraries like String/Array/Hash/Regexp/etc. Therefore your script doesn't run fast on YARV if bottleneck is string processing, or so.

Congratulations to you both for completing Ruby/YARV merger recently. That must have been a lot of work, but I know it has the whole Ruby world very excited. Now that the merger has taken place, how do you see this changing the way Ruby is developed?

Matz:

Congrats should go to Koichi who has done a lot of work. I am moving my developing from matzruby (a branch for my old interpreter) to trunk (the yarv). Recently I have implemented some new features on the trunk, for example, class local instance variables and new local variable scope. The transition will complete pretty soon.

Since the trunk is originally Koichi's work, I need more help from others especially from Koichi than before. I know everything about the previous interpreter (well, most of them), but there are still mysteries in the new one. I am well satisfied with new one. It's clearer, well-formed, and faster.

ko1:

Thank you. I'm a newbie of Ruby developer (in fact, I didn't have CVS account to commit any ruby codes). So I can't say how change on ruby development :)

When will the first production release of Ruby running on YARV by available for all Rubyists to play with?

Matz:

Short answer: now.

Longer answer: the YARV is already publicly avaiblabe via our Subversion repository. You can fetch and play with it now. But the first public "release" from us will be Christmas 2007, if we are as diligent as we should be. Knowing how lazy I am, I will try not to be a stumbling block for the release. ;-)

The Ruby VM Serial Interview

2014-04-04T21:40:57Z

I have really enjoyed reading Pat Eyler's Rubinius Serial Interview and Nick Sieger's spun-off JRuby Serial Interview. It's very educational to read what the developers have to say about their projects and ideas.

The more I read though, the more I wanted the equivalent content for the official Ruby VM. I asked Matz and Koichi if they would be willing to answer questions from me and they agreed to do so. We are now ready to share their responses with the community.

This will be a serial interview as Pat Eyler calls them. We will deliver regular episodes until I run out of good questions or Matz and Koichi get sick of me bothering them, whichever comes first. I will ask the questions in the interview, but feel free to make suggestions in the comments to this article.

One last note: we are not promising any kind of schedule for the episodes. Matz and Koichi are heroically providing their answers in English. We want to respect how much work that is and give them all the time they need to do that. Personally, I cannot thank them enough.

With that, I give you the episode index:

In this first episode I ask Matz and Koichi to introduce themselves and their roles as well as to give us an update on where we are with the Ruby VM.
In the second episode I ask Matz and Koichi about for their thoughts on the alternate Ruby implementations and how they see them changing Ruby's development.
In the third episode Matz and Koichi discuss the past, present and future of Ruby threading.
In the fourth episode Matz tells us a little about how m17n is shaping up.
In the fifth episode Koichi gives us the inside story on optimization in the new VM.