Concurrency

Posts tagged with "Concurrency."
  • 6

    SEP
    2014

    Taking Rust to Task

    Now that I've reached the point where I can get some Rust code running without asking questions in IRC every five minutes, I really wanted to play with some tasks. Tasks are the way Rust handles multiprocessing code. Under the hood they can map one-to-one with operating system threads or you can use a many-to-one mapping that I'm not ready to go into yet.

    Probably one of the most exciting aspect of tasks in Rust, in my opinion, is that unsafe use of shared memory is rejected outright as a compile error. That lead me to want to figure out how you communicate correctly. (Spoiler: the same was you do in Ruby: just pass messages.)

    Ready to dive in, I grossly simplified a recent challenge from work and coded it up in Rust. You can get the idea with a glance at main():

    use std::collections::HashMap;
    
    // ...
    
    fn string_vec(strs: &[&'static str]) -> Vec<String> {
        let mut v = Vec::new();
        for s in strs.iter() {
            v.push(s.to_string());
        }
        v
    }
    
    fn main() {
        let mut services = HashMap::new();
        services.insert("S1".to_string(), string_vec(["A", "B"]));
        services.insert("S2".to_string(), string_vec(["A", "C"]));
        services.insert("S3".to_string(), string_vec(["C", "D", "E", "F"]));
        services.insert("S4".to_string(), string_vec(["D", "B"]));
        services.insert("S5".to_string(), string_vec(["A", "Z"]));
    
        let work = Work(Search::new("A".to_string(), "B".to_string()));
    
        let mut task_manager = TaskManager::new(services);
        task_manager.run(work);
    }
    

    Read more…

    In: Rusting | Tags: Concurrency & Rust | 1 Comment
  • 22

    AUG
    2014

    Sleepy Programs

    When we think of real multiprocessing, our thoughts probably drift more towards languages like Erlang, Go, Clojure, or Rust. Such languages really focus on getting separate "processes" to communicate via messages. This makes it a lot easier to know when one process is waiting on another, because calls to receive messages typically block until one is available.

    But what about Ruby? Can we do intelligent process coordination in Ruby?

    Yes, we can. The tools for it are more awkward though. It's easy to run into tricky edge cases and hard to code your way out of them correctly.

    Let's play with an example to see how good we can make things. Here's what we will do:

    1. We will start one parent process that will fork() a single child process
    2. The child will push three messages onto a RabbitMQ queue and exit()
    3. The parent will listen for three messages to arrive, then exit()

    Here's a somewhat sloppy first attempt at solving this:

    #!/usr/bin/env ruby
    
    require "benchmark"
    
    require "bunny"
    
    QUEUE_NAME = "example"
    MESSAGES   = %w[first second third]
    
    def send_messages(*messages)
      connection = Bunny.new.tap(&:start)
      exchange   = connection.create_channel.default_exchange
    
      messages.each do |message|
        exchange.publish(message, routing_key: QUEUE_NAME)
      end
    
      connection.close
    end
    
    def listen_for_messages(received_messages)
      connection = Bunny.new.tap(&:start)
      queue      = connection.create_channel.queue(QUEUE_NAME, auto_delete: true)
    
      queue.subscribe do |delivery_info, metadata, payload|
        received_messages << payload
      end
    
      time_it("Received #{MESSAGES.size} messages") do
        yield
      end
    
      connection.close
    end
    
    def time_it(name)
      elapsed = Benchmark.realtime do
        yield
      end
      puts "%s: %.2fs" % [name, elapsed]
    end
    
    def wait_for_messages(received_messages)
      until received_messages == MESSAGES
        sleep 0.1  # don't peg the CPU while we wait
      end
    end
    
    def send_and_receive
      pid = fork do
        sleep 3  # make sure we're receiving before they are sent
        send_messages(*MESSAGES)
      end
      Process.detach(pid)
    
      received_messages = [ ]
      listen_for_messages(received_messages) do
        wait_for_messages(received_messages)
      end
    end
    
    send_and_receive
    

    Read more…

  • 13

    AUG
    2007

    Erlang Message Passing

    Like many Pragmatic Programmer fans, I've been having a look at Erlang recently by working my way through Programming Erlang. In the book, the author includes a challenge: build a message ring of processes of size M and send a message around the ring N times, timing how long this takes. The author also suggests doing this in other languages and comparing the results. Having now done this, I can tell you that it is an interesting exercise.

    First, the Erlang results. Here's a sample run that creates 30,000 processes and sends a message around that ring 1,000 times:

    $ erl -noshell -s solution start 30000 1000
    Creating 30000 processes (32768 allowed)...
    Done.
    Timer started.
    Sending a message around the ring 1000 times...
    Done:  success
    Time in seconds:  29
    

    So we see about 30,000,000 message passes there in roughly 30 seconds. I should also note that Erlang creates those processes very, very fast. It's possible to raise the process limit shown there, but I'm more interested in comparing what these languages can do out of the box.

    Read more…

  • 27

    APR
    2007

    The Ruby VM: Episode III

    Let's talk a little about threading, since that's a significant change in the new VM. First, can you please explain the old threading model used in Ruby 1.8 and also the new threading model now used in Ruby 1.9?

    Matz:

    Old threading model is the green thread, to provide universal threading on every platform that Ruby runs. I think it was reasonable decision 14 years ago, when I started developing Ruby. Time goes by situation has changed. pthread or similar threading libraries are now available on almost every platform. Even on old platforms, pth library (a thread library which implements pthread API using setjmp etc.) can provide green thread implementation.

    Koichi decided to use native thread for YARV. I honor his decision. Only regret I have is we couldn't have continuation support that used our green thread internal structure. Koichi once told me it's not impossible to implement continuation on YARV (with some restriction), so I expect to have it again in the future. Although it certainly has lower priority in 1.9 implementation.

    ko1:

    Matz explained old one, so I show you YARV's thread model.

    As you know, YARV support native thread. It means that you can run each Ruby thread on each native thread concurrently.

    It doesn't mean that every Ruby thread runs in parallel. YARV has global VM lock (global interpreter lock) which only one running Ruby thread has. This decision maybe makes us happy because we can run most of the extensions written in C without any modifications.

    Read more…