13
AUG2007
Erlang Message Passing
Like many Pragmatic Programmer fans, I've been having a look at Erlang recently by working my way through Programming Erlang. In the book, the author includes a challenge: build a message ring of processes of size M and send a message around the ring N times, timing how long this takes. The author also suggests doing this in other languages and comparing the results. Having now done this, I can tell you that it is an interesting exercise.
First, the Erlang results. Here's a sample run that creates 30,000 processes and sends a message around that ring 1,000 times:
$ erl -noshell -s solution start 30000 1000
Creating 30000 processes (32768 allowed)...
Done.
Timer started.
Sending a message around the ring 1000 times...
Done: success
Time in seconds: 29
So we see about 30,000,000 message passes there in roughly 30 seconds. I should also note that Erlang creates those processes very, very fast. It's possible to raise the process limit shown there, but I'm more interested in comparing what these languages can do out of the box.
Now Ruby doesn't have an equivalent to Erlang processes, so we need to decide what the proper replacement is. The first thing I tried was fork()ing some Unix processes:
$ ruby forked_mring.rb 100 10000
Creating 100 processes...
Timer started.
Sending a message around the ring 10000 times...
Done.
Done: success.
Time in seconds: 32
You should notice here the small number of processes I could create using the default limits imposed by my operating system. Again, it's possible to raise this limit but I don't think I'm going to get it up to 30,000 very easily. I did get these processes very quickly though, again.
So here we are passing 1,000,000 messages in about the same amount of time.
In an attempt to bypass the low process limit, I wrote another implementation with Ruby's threads. The results of that aren't too impressive though:
$ ruby threaded_mring.rb 100 1000
Using the standard Ruby thread library.
Creating 100 processes...
Timer started.
Sending a message around the ring 1000 times...
Done: success.
Time in seconds: 32
$ ruby threaded_mring.rb 1000 4
Using the standard Ruby thread library.
Creating 1000 processes...
Timer started.
Sending a message around the ring 4 times...
Done: success.
Time in seconds: 30
You should see from the second run that it is possible to create quite a few more threads, but I need to mention that creating that many took around 15 seconds. Sadly, both of these runs paint an ugly picture: introducing synchronization just kills performance. Using the fastthread library doesn't help as much as we would like:
$ ruby -rubygems threaded_mring.rb 100 1000
Using the fastthread library.
Creating 100 processes...
Timer started.
Sending a message around the ring 1000 times...
Done: success.
Time in seconds: 28
$ ruby -rubygems threaded_mring.rb 1000 5
Using the fastthread library.
Creating 1000 processes...
Timer started.
Sending a message around the ring 5 times...
Done: success.
Time in seconds: 29
So at best, we're passing 100,000 and 5,000 messages in our roughly 30 second timeframe, depending on how many processes we need.
Am I suggesting we all switch to Erlang? No. I've enjoyed seeing how the other side lives and I've learned a lot from getting into the functional mindset. Parts of Erlang are very impressive and concurrency is definitely one of them. It hasn't been enough to win me over from Ruby yet though. I couldn't ever see myself doing my day to day work without The Red Lady.
What I would love to see is some way to manage Erlang-like concurrency in Ruby. We could have some great fun building servers with that, I think.
I'll share the Erlang code here so the people that know the language better than me can provide corrections. First, here's the code the spawns processes and passes messages:
-module(mring).
-export([build/1, send_and_receive/2, round_and_round/3]).
build(RingSize) ->
ParentPid = self(),
spawn(fun() -> build(RingSize - 1, ParentPid) end).
build(0, StartPid) -> forward(StartPid);
build(RingSize, StartPid) ->
ChildPid = spawn(fun() -> build(RingSize - 1, StartPid) end),
forward(ChildPid).
forward(Pid) ->
receive
{message, Text, PassCount} ->
Pid ! {message, Text, PassCount + 1},
forward(Pid)
end.
send_and_receive(Ring, Text) ->
Ring ! {message, Text, 0},
receive Returned -> Returned end.
round_and_round(_, _, 0) -> success;
round_and_round(Ring, ProcessCount, Repeat) ->
Check = "Checking the ring...",
case send_and_receive(Ring, "Checking the ring...") of
{message, Check, ProcessCount} ->
round_and_round(Ring, ProcessCount, Repeat - 1);
Unexpected -> {failure, Unexpected}
end.
Next, we have a little helper I wrote to time things. I'm pretty confident there must be a better way to do this, but all my attempts to find it failed. Help me Erlang Jedi:
-module(stopwatch).
-export([time_this/1, time_and_print/1]).
time_this(Fun) ->
{StartMega, StartSec, StartMicro} = now(),
Fun(),
{EndMega, EndSec, EndMicro} = now(),
(EndMega * 1000000 + EndSec + EndMicro div 1000000) -
(StartMega * 1000000 + StartSec + StartMicro div 1000000).
time_and_print(Fun) ->
io:format("Timer started.~n"),
Time = time_this(Fun),
io:format("Time in seconds: ~p~n", [Time]).
Finally we have the application code that glues these modules together:
-module(solution).
-export([start/1]).
start([ProcessesArg, CyclesArg]) ->
Processes = list_to_integer(atom_to_list(ProcessesArg)),
Cycles = list_to_integer(atom_to_list(CyclesArg)),
io:format( "Creating ~p processes (~p allowed)...~n",
[Processes, erlang:system_info(process_limit)]),
Ring = mring:build(Processes),
io:format("Done.~n"),
stopwatch:time_and_print(
fun() ->
io:format("Sending a message around the ring ~p times...~n", [Cycles]),
Result = mring:round_and_round(Ring, Processes, Cycles),
io:format("Done: ~p~n", [Result])
end
),
init:stop().
You will see the Ruby solutions in this week's Ruby Quiz.
Comments (15)
-
Ricky Clarkson August 14th, 2007 Reply LinkWhile I'm interested in Erlang, I don't think it's fair to compare languages based on OS threads. Instead, in your favourite language, emulate Erlang's processes. An event queue would suffice.
I use a similar technique for simulating a computer network, and find it very fast (that said, I think Erlang's model would tend to simplify my networking code).
-
While I'm interested in Erlang, I don't think it's fair to compare languages based on OS threads.
Ruby doesn't have native threads.
My really hacky version which models the Erlang way:
http://snippets.dzone.com/posts/show/4421It performs much better, but this functionality should be part of the ruby interpreter.
-
While I'm interested in Erlang, I don't think it's fair to compare languages based on OS threads. Instead, in your favourite language, emulate Erlang's processes. An event queue would suffice.
An event queue would not be fair either.
Erlang processes are more advanced than simple queues; they will automatically spread themselves out among the available processor cores on a machine. This means a cleverly written program will automatically run faster if you get another processor core in your compy. By using an event queue instead, it would still run on a single core.
-
-
Ruby and Erlang are both awesome languages. Just out of curiosity, which one had fewer lines of code?
-
I abstracted the Erlang code a little more because I'm less comfortable there and have to move in baby steps. Taking that into account, there wasn't a huge difference.
-
-
Instead of your stopwatch, you can use timer:tc/3:
http://www.erlang.org/doc/man/timer.htmlBut if you do want to build your own timer it's usually easier to start with
erlang:statistics(runtime)anderlang:statistics(wall_clock):
http://www.erlang.org/doc/apps/kernel/index.html-
I did try to use the
statistics()function at first, but I couldn't seem to figure out how to get correct answers out of it. I'm sure it was my fault though. Thanks for the timer link, that's very helpful.-
statistics(wall_clock). % Do stuff {_, Delta} = statistics(wall_clock).
Delta now holds the number of ms since you last called
statistics(wall_clock).
-
-
-
There are a couple of papers that talk about how the Actors model is
implemented in Scala:
http://lamp.epfl.ch/~phaller/doc/haller07actorsunify.pdf and
http://lamp.epfl.ch/~phaller/doc/haller06jmlc.pdf. See also
http://lamp.epfl.ch/~phaller/actors.html for other links. Would be
interesting to compare on your machine. -
So, a couple things you might be interested in:
The Omnibus Concurrency Library for Ruby, which provides Actor-style
concurrency: http://rubyforge.org/projects/concurrent/Erlectricity, a Ruby to Erlang bridge: http://code.google.com/p/erlectricity/
Rebar, another Ruby to Erlang bridge:
http://rubyisawesome.com/2007/4/30/calling-erlang-from-ruby-teaser -
When might Erlang-to-Ruby bridges be needed or wanted in a Ruby or Rails app?
-
I doubt the need for an Erlang bridge comes up often in a Rails application, but generally speaking such a thing could be useful anytime massive concurrency is called for.
I know Campfire has a C backend, but that might have been a place where Erlang could have been used to good effect.
-
I doubt the need for an Erlang bridge comes up often in a Rails application, but generally speaking such a thing could be useful anytime massive concurrency is called for.
I'm reading the same book in almost the same timeframe, and that lead me to the same conclusions :)
-
-
-
How much does Ruby 1.9 change things? Could we do this same thing with a single process and single thread and instead use a bazillion fibers?
What capabilities do you get get with an Erlang thread? Do you get something like Thread Local Storage? If not then maybe a Ruby 1.9 fibers is a functional equivalent to an Erlang thead.
-
I don't think just Fiber is going to get us all the way to Erlang style parallelism. I feel it's are more a coroutine than an Erlang process.
However, matz was recently discussing the plan for Ruby 2.0 and it seems that Ruby is definitely going to head down this road in the future…
-