Gray Soft

The programming blog of James Edward Gray II (JEG2).
  • 7

    SEP
    2007

    Marcel at Play

    At the preconference charity event for the first Lone Star Rubyconf, Marcel Molina, Jr gave one of the best talks I've ever heard at a conference. The entire talk was Marcel showing examples of pathological (his word, not mine) Ruby code. Not only did he show all these examples to a room full of people new to language at an event titled Intro to Ruby, but he actually made a case for these examples being proof positive Ruby is a reliable language.

    This was a very unique approach to speeches in that we literally saw a Ruby master at play. While most speakers try to put our best foot forward, Marcel embraced the craziness and just had some fun. His attitude was infectious and really drove his point home.

    We should all try to have more fun with our speeches. It's so enlightening to watch, for any audience. I hope I have the courage to try a similar talk in the future.

    In: My Heroes | Tags: For Fun | 3 Comments
  • 23

    AUG
    2007

    Finding Serenity

    [Update: This blog no longer runs on Serenity. I've created yet another engine to power it.]

    I'll be nice and not name any names, but my old blog software was really struggling. I've dealt with numerous issues from it over the years I've run this blog and all of that extra maintenance finally wore me down.

    I'm sure you've noticed the theme change here by now. That's the surface result of me having replaced the entire backend. This blog now runs on a blogging engine I invented called Serenity. It's a young engine at this point, but I suspect it will improve as I fiddle with it and find what I like.

    I've moved all of the content over, so my hope is that nothing is missing. Unfortunately there have been some URL changes. I apologize for that and promise that I don't plan to change them again anytime in the near future.

    Thanks for being patient with me during this transition. I hope that it will lighten my maintenance load and give me more time to write articles.

  • 13

    AUG
    2007

    Erlang Message Passing

    Like many Pragmatic Programmer fans, I've been having a look at Erlang recently by working my way through Programming Erlang. In the book, the author includes a challenge: build a message ring of processes of size M and send a message around the ring N times, timing how long this takes. The author also suggests doing this in other languages and comparing the results. Having now done this, I can tell you that it is an interesting exercise.

    First, the Erlang results. Here's a sample run that creates 30,000 processes and sends a message around that ring 1,000 times:

    $ erl -noshell -s solution start 30000 1000
    Creating 30000 processes (32768 allowed)...
    Done.
    Timer started.
    Sending a message around the ring 1000 times...
    Done:  success
    Time in seconds:  29
    

    So we see about 30,000,000 message passes there in roughly 30 seconds. I should also note that Erlang creates those processes very, very fast. It's possible to raise the process limit shown there, but I'm more interested in comparing what these languages can do out of the box.

    Read more…

  • 3

    AUG
    2007

    The Ruby VM: Episode V

    You have told us before that one of the big reasons to move to a new Ruby VM was to provide new options for optimization. Can you talk a little about the optimizations you have added to the new Ruby VM thus far and what operations will likely be faster because of them?

    ko1:

    OK. At first, I write about basic of YARV instruction. YARV has two type instructions. First is primitive instruction. It's as written, primitive. Ruby code can be represented in these primitive instruction. Second is instructions for optimization. It's not needed to represent Ruby scripts, but they are added for optimization. Primitive instructions doesn't include _ in their name (like putobject), and optimize instructions do (like opt_plus). This policy helps you if you want to see VM instructions. Initially, you need to read primitive instructions.

    The most easy and effective optimization is Specialized Instructions. This optimization replace method call with another VM instruction, such as Fixnum#+ to opt_plus. Current Ruby's numeric calculation is slow because all operations are method call. For example, 1 + 2 means 1.+(2). But numeric operations are more lightweight than Ruby's method invocation. So method call is only overhead for numeric operation. Specialized Instructions allow the VM to skip method call overhead.

    But we can't know which expression is numeric operation or not at compile time. See this expression: a = c ? 1 : [:elem], a will be Fixnum or Array at runtime.

    So, we can't replace + expression with numeric operation instruction. Specialized Instruction, for example opt_plus which is replaced with + method invocation will do following code:

    def opt_plus(recv, val) # simple version
       if recv.class == Fixnum && val.class == Fixnum
         if Fixnum#+ is not redefined
           return calculate "recv + val" without method call
         end
       end
       # normal method invocation
       recv.+(val)
    end
    

    Check receiver and value are Fixnum or not, and check Fixnum#+ are not redefined. After these check, calculate them without method invocation. In fact, Float#+ are also checked. There are other specialized instructions.

    YARV eases to implement such instructions with VM generator. You shouldn't write bothersome code such as stack manipulation. If you write VM instruction such as opt_plus in simple VM DSL, VM generator will translate it to C code.

    Specialized Instruction is very simple, but effective for simple benchmark such as fib() or tak() and some calculate bound program.

    Read more…

  • 13

    JUL
    2007

    The Ruby VM: Episode IV

    We've talked about threads, so let's talk a little about character encodings. This is another big change planned for Ruby's future. Matz, you have stated that you plan to add m17n (multilingualization) support to Ruby. Can you talk a little about what that change actually means for Ruby users?

    Matz:

    Nothing much, except for some incompatibility in string manipulation, for example, "abc"[0] will give "a" instead of 97, and string indexing will be based on character instead of byte. I guess the biggest difference is that we can officially declare we support Unicode. ;-)

    Unlike Perl nor Python, Ruby's M17N is not Unicode based (Universal Character Set or USC). It's character set independent (CSI). It will handle Unicode, along with other encoding schemes such as ISO8859 or EUC-JP etc. without converting them into Unicode.

    Some misunderstand our motivation. We are no Unicode haters. Rather, I'd love to use Unicode if situation allows. We hate conversion between character sets. For historical reasons, there are many variety of character sets. For example, Shift_JIS character set has at least 5 variations, which differ each other in a few characters mapping. Unfortunately, we have no way to distinguish them. Thus conversion may cause information loss. If a language provide Unicode centric text manipulation, there's no way to avoid the problem, as long as we use that language.

    ko1:

    On my policy, I escape from this topic :)

    Read more…

  • 27

    APR
    2007

    The Ruby VM: Episode III

    Let's talk a little about threading, since that's a significant change in the new VM. First, can you please explain the old threading model used in Ruby 1.8 and also the new threading model now used in Ruby 1.9?

    Matz:

    Old threading model is the green thread, to provide universal threading on every platform that Ruby runs. I think it was reasonable decision 14 years ago, when I started developing Ruby. Time goes by situation has changed. pthread or similar threading libraries are now available on almost every platform. Even on old platforms, pth library (a thread library which implements pthread API using setjmp etc.) can provide green thread implementation.

    Koichi decided to use native thread for YARV. I honor his decision. Only regret I have is we couldn't have continuation support that used our green thread internal structure. Koichi once told me it's not impossible to implement continuation on YARV (with some restriction), so I expect to have it again in the future. Although it certainly has lower priority in 1.9 implementation.

    ko1:

    Matz explained old one, so I show you YARV's thread model.

    As you know, YARV support native thread. It means that you can run each Ruby thread on each native thread concurrently.

    It doesn't mean that every Ruby thread runs in parallel. YARV has global VM lock (global interpreter lock) which only one running Ruby thread has. This decision maybe makes us happy because we can run most of the extensions written in C without any modifications.

    Read more…

  • 16

    APR
    2007

    No Longer the Fastest Game in Town

    If your number one concern when working with CSV data in Ruby is raw speed, you might want to know that FasterCSV is no longer the fastest option.

    There are a couple of new contenders for Ruby CSV processing including a C extension called SimpleCSV and a pure Ruby library called LightCsv. I haven't been able to test SimpleCSV locally, because I can't get it to build on my box, but users do tell me it's faster. I have run some trivial benchmarks for LightCsv though and it too is pretty quick:

    $ rake benchmark
    (in /Users/james/Documents/faster_csv)
    time ruby -r csv -e '6.times { CSV.foreach("test/test_data.csv") { |row| } }'
    
    real    0m5.481s
    user    0m5.468s
    sys     0m0.010s
    time ruby -r lightcsv -e \
    '6.times { LightCsv.foreach("test/test_data.csv") { |row| } }'
    
    real    0m0.358s
    user    0m0.349s
    sys     0m0.008s
    time ruby -r lib/faster_csv -e \
    '6.times { FasterCSV.foreach("test/test_data.csv") { |row| } }'
    
    real    0m0.742s
    user    0m0.732s
    sys     0m0.009s
    

    It's important to note that LightCsv is indeed very "light." FasterCSV has grown up into a feature rich library that provides many different ways to look at your data. In contrast, LightCsv doesn't yet allow you to set column or row separators. Given that, it's only an option for vanilla CSV you just need to iterate over. If that's what you have though, and speed counts, it might just be the right choice.

    Read more…

  • 16

    MAR
    2007

    The Ruby VM: Episode II

    We started these talks because of the excitement around the alternate implementations, like JRuby and Rubinius. How do you feel about all of these new interpreters and how do you see them affecting the official development of Ruby?

    Matz:

    Alternate implementations mean maturity of Ruby language. I'm glad for the fact. But we have never had enough number of developers for core, so I think we need more cooperation between implementations. I had a good talk about future Ruby spec. with Charles Nutter recently. I expect occasion like this more often.

    ko1:

    I think having alternatives is very important. I want to know how to implement Ruby and apply these techniques to YARV.

    In fact, implementing from scratch is very fun. YARV (official Ruby implementation) has many problems resulted from historical reasons (a biggest problem is compatibility to extension libraries).

    Have you downloaded and installed any of the other interpreters?

    Read more…

  • 17

    FEB
    2007

    I believe in Ruby

    For the contest and all you Bull Durham fans…

    Well, I believe in blocks, iterators, closures, that everything should be an object, the power of reflection, garbage collection, exception handling, that multiple inheritance causes more problems than it solves. I believe interpreters should be totally free. I believe there ought to be a constitutional amendment outlawing pointers and verbose syntax. I believe in a strong standard library, green threads, that a language should trust the programmer rather than restrict his efforts and I believe in sheer fun of coding that truly is possible to achieve.

    In: Non-code | Tags: For Fun | 0 Comments
  • 16

    FEB
    2007

    The Ruby VM: Episode I

    Hello and thank you both for agreeing to answer my questions. To begin, would you please introduce yourselves and tell us about your role in Ruby's development?

    Matz:

    I am the designer and the first implementer of the Ruby language. My real name is Yukihiro Matsumoto, that sounds something like You-Key-Hero Matz-Motor in English. But it's too long to remember and pronounce, so just call me Matz.

    I have been developing Ruby since 1993. It is now quite complicated and has performance problem. I have had vague plan of rewriting the interpreter for long time, but I have never been motivated enough to throw out the current interpreter and start developing new one.

    Then Koichi came in with YARV that seemed to have much brighter future than my vaporware - it runs - so I asked him to take a role of the official implementer of the core. Although I enjoy both designing and implementation of the language, I don't think I am gifted for language implementation. So I thought that it might be the time to focus on designing when I saw YARV.

    ko1:

    Thank you for your interest in YARV and me. BTW, I'm thinking what "YARV" stand for. Because it is not Yet Another. Someone proposed that "YARV ain't RubyVM". If YARV means "YARV ain't RubyVM", what is YARV?

    I'm Koichi Sasada. Koichi is given name, and "ichi" means "one" in Japanese. So I use "ko1" as my nick. I'm an assistant at Department (...snip...) of Tokyo. My research interest is systems software, especially operating system, programming language, parallel systems, and so on. And I'm a member of Nihon Ruby no Kai (Ruby Association in Japan). I plan(ed) some Ruby events like RubyKaigi and am an editor of Rubyist Magazine. I also develop(ed) Nadoka, Rava, Rucheme, and some projects. Say, I'm a developer of YARV: Yet Another RubyVM.

    My role in Ruby's development? To steal VM hacking pleasure from Matz?

    Read more…