-
30
JAN
2015Random Access Terminal
I've recently been playing around with fancy terminal output in Ruby. I've learned quite a bit about this arcane magic. I've also realized that the documentation is pretty spotty. I want to see if I can improve that with a few blog posts, so let's dive right in.
Output
Program output typically happens in a linear order from top to bottom. For example, this code:
puts "onez" puts "twos" puts "threes"
generates this output:
onez twos threesBut what if you need to change some output? Could you replace the
zabove with ansif you needed to? Yes, but it can get a little involved.ANSI Escape Codes
In many cases, we just push some characters to
$stdout(the streamKernel#putsis writing to above) and your terminal program happily shows them to the user. However, your terminal is watching these characters for special sequences that it understands. Some of those sequences of characters can cause your terminal to take actions other than just writing some output to the screen. -
31
DEC
2014Game Programming Patterns
I meet a lot of programmers that tell me they got started because they wanted to build games. However, when I ask most of them which games they have built, the list rarely includes anything more than mostly unplayable toy projects.
I can only guess at the reasons for this oddity, but I suspect it might be due to the fact that games are fairly complex. Even if you want to rebuild a fairly simple classic like Space Invaders or Snake you need to know at least a little about event loops, keyboard handling, animation, and collision detection. If your day job involves a different kind of programming, like Web application development, odds are good that you don't get a lot of practice with these concepts.
That may not be your story, but it was definitely mine. This year I decided that it was finally time to learn how to build games. I used several sources to gain this knowledge and some helped more than others, but the biggest win by far was a book called Game Programming Patterns by Bob Nystrom.
-
31
OCT
2014How to Avoid Taking a Dart to the Knee
I've been playing with Dart quite a bit lately. I really enjoy the language, but there are always snags that trip up those coming from other backgrounds. Here are the top three issues that have bit me in Dart, in the hopes of saving others some pain:
The Truth and Nothing But the Truth… Literally!
One of the challenges of any language is figuring out what it considers to be truthy in conditional expressions. Each system has its twists, but I find Dart to be extra strict in this case.
Here's some code illustrating the rule:
bool isTruthy(Object condition) { return !!condition; } void main() { var tests = [true, false, null, 42, 0, "", [ ], new Object()]; for (var test in tests) { print("$test is ${isTruthy(test)}"); } }
That outputs:
$ dart truthiness.dart true is true false is false null is false 42 is false 0 is false is false [] is false Instance of 'Object' is falseAs you can see the literal
true(just that one object) is truthy in Dart and everything else is consideredfalse. I'm in the habit of playing pretty fast and loose with truthiness from all of my time working with Ruby, so this has surprised me a few times. -
24
OCT
2014The Three Tick Sort
Yesterday I showed a newer programmer some code like
scores.sort_by(&:reverse). This provoked a comment about how they where going to look upsort_by()later to figure out what magic is involved here. It made me sad to realize how many cool tricks they weren't going to see in that bit of documentation.Allow me to enumerate those tricks for you, but first let's flesh out an example. Consider this code:
scores = { fifteen: 2, five_card_run: 5, five_card_flush: 5, four_card_run: 4, four_card_flush: 4, his_nobs: 1, pair: 2, three_card_run: 3, } scores.sort_by(&:reverse).each do |name, score| puts "Score #{score} for #{name}." end # >> Score 1 for his_nobs. # >> Score 2 for fifteen. # >> Score 2 for pair. # >> Score 3 for three_card_run. # >> Score 4 for four_card_flush. # >> Score 4 for four_card_run. # >> Score 5 for five_card_flush. # >> Score 5 for five_card_run.
In this case, the magic method call (
scores.sort_by(&:reverse)) has reordered a list of Cribbage hands first by point value and then alphabetically ("ASCIIabetically" in truth). How this happens is a pretty interesting journey though. -
25
SEP
2014Regex Code Equivalency
#!/usr/bin/env ruby -w Name = "Gray, James" !!(Name =~ /\AGray/) # => true Name.start_with?("Gray") # => true !!(Name =~ /James\z/) # => true Name.end_with?("James") # => true !!(Name =~ /Dana/) # => false Name.include?("Dana") # => false !!(Name =~ /\A\z/) # => false Name.empty? # => false !!(Name =~ /\AGray, James\z/) # => true Name == "Gray, James" # => true !!(Name =~ /\A(?:Gray, James|Gray, Dana)\z/) # => true ["Gray, James", "Gray, Dana"].include?(Name) # => true Name =~ /\A\w+/ && $& # => "Gray" Name[/\A\w+/] # => "Gray" Name =~ /\A(\w+),\s*(\w+)\z/ && $2 # => "James" Name[/\A(\w+),\s*(\w+)\z/, 2] # => "James" Name =~ /\A(?<last>\w+),\s*(?<first>\w+)\z/ && $~[:first] # => "James" Name[/\A(?<last>\w+),\s*(?<first>\w+)\z/, :first] # => "James" Name.scan(/^.*\n?/) # => ["Gray, James"] Name.lines # => ["Gray, James"] Name.scan(/./m) # => ["G", "r", "a", "y", ",", " ", "J", "a", "m", "e", "s"] Name.chars # => ["G", "r", "a", "y", ",", " ", "J", "a", "m", "e", "s"] Name.gsub(/[aeiou]/, "") # => "Gry, Jms" Name.delete("aeiou") # => "Gry, Jms" Name.gsub(/[aeiou]/, "X") # => "GrXy, JXmXs" Name.tr("aeiou", "X") # => "GrXy, JXmXs" # For the destructive operations that follow you can drop the `dup()` and # switch `sub()` to `sub!()`, as long as you don't care about the return value. Name.sub(/(?=,)/, " II") # => "Gray II, James" Name.dup.insert(Name.index(","), " II") # => "Gray II, James" Name.sub(/\A/, "Name: ") # => "Name: Gray, James" Name.dup.prepend("Name: ") # => "Name: Gray, James" Name.sub(/\A.*\z/m, "Gray, Dana") # => "Gray, Dana" Name.dup.replace("Gray, Dana") # => "Gray, Dana" Name.sub(/\A.*\z/m, "") # => "" Name.dup.clear # => "" Spacey = "\tsome space\r\n" Spacey.sub(/\A\s+/, "") # => "some space\r\n" Spacey.lstrip # => "some space\r\n" Spacey.sub(/\s+\z/, "") # => "\tsome space" Spacey.rstrip # => "\tsome space" Spacey.sub(/\A\s*(.+?)\s*\z/m, '\1') # => "some space" Spacey.strip # => "some space" Spacey.sub(/(?:\r?\n|\r)\z/m, "") # => "\tsome space" Spacey.chomp # => "\tsome space" Spacey.sub(/(?:\r\n|.)\z/m, "") # => "\tsome space" Spacey.chop # => "\tsome space" Spacey.gsub(/ +/, " ") # => "\tsome space\r\n" Spacey.squeeze(" ") # => "\tsome space\r\n"
-
22
SEP
2014A Regex Can't Match Balanced Parentheses
Can we do math with regular expressions?
#!/usr/bin/env ruby -w def build_preparation_regex(number_regex, ops) %r{ (?<number> #{number_regex} ){0} (?<operator> [#{ops.map(&Regexp.method(:escape)).join}] ){0} (?<term_operator_term> \g<term> \s* \g<operator> \s* \g<term> ){0} (?<term> \g<number> | \( \s* \g<term_operator_term> \s* \) ){0} \g<term_operator_term>(?=\s*\z|[^)]) }x end NUMBER_REGEX = %r{ -? # an optional minus \d+ # an integer (?: \. \d+)? # an optional fractional bit }x PREPARE_MULT_AND_DIV_REGEX = build_preparation_regex(NUMBER_REGEX, %w[* /]) PREPARE_ADD_AND_SUB_REGEX = build_preparation_regex(NUMBER_REGEX, %w[* / + -]) CHECK_REGEX = %r{ \A # the start of the expression (?<term> # a term, which is: #{NUMBER_REGEX} # a number | # or \( \s* # a parenthesized group of \g<term> # a term \s* [*/+\-] \s* # an operator \g<term> # and another term \s* \) # the end of the parenthesized group ) \z # the end of the expression }x MATH_REGEX = %r{ \( \s* (?<left> #{NUMBER_REGEX} ) \s* (?<operator> [*/+\-] ) \s* (?<right> #{NUMBER_REGEX} ) \s* \) }x verbose = ARGV.delete("-v") problem = ARGV.first.strip or abort "USAGE: #{$PROGRAM_NAME} MATH_EXPRESSION" steps = [ ] [PREPARE_MULT_AND_DIV_REGEX, PREPARE_ADD_AND_SUB_REGEX].each do |preparation| loop do steps << problem.dup if verbose problem.sub!(preparation) { |term| "(#{term})" } or break end end problem =~ CHECK_REGEX or abort "Error: Invalid expression" solution = problem.dup loop do steps << solution.dup if verbose solution.sub!(MATH_REGEX) { $~[:left].to_f.public_send($~[:operator], $~[:right].to_f) } or break end puts steps.uniq[0..-2] if verbose puts solution.sub(/\.0+\z/, "")
-
20
SEP
2014Can You snake_case/CamelCase With One Regex?
In Rails, methods like
underscore()andcamelize()use several regexen to transform theStringunder the hood. Many people have asked if you can do it with a single regex though. These specs I borrowed from Rails seem to say yes:#!/usr/bin/env ruby -w class String def snake_case(acronyms = self.class.acronyms) gsub( %r{ (?: (?<before> \b | [A-Za-z\d] ) (?<acronym> #{acronyms.regex} ) (?<after> \b | [^a-z] ) ) | (?: (?<before> [A-Z]+ ) (?<after> [A-Z][^A-Z] ) ) | (?: (?<before> [^A-Z:] ) (?<after> [A-Z] ) ) | (?<nesting> :: ) }x ) { |m| if $~[:nesting] "/" else [$~[:before], $~[:acronym], $~[:after]] .compact .reject(&:empty?) .join("_") end }.downcase end def CamelCase(acronyms = self.class.acronyms) gsub( %r{ (?: (?: \A | _ | (?<nesting> / ) ) (?<acronym> #{acronyms.inverted_regex} ) (?= \b | [A-Z_] ) ) | (?: (?: \A | _ ) (?<letter> . ) ) | (?: (?<nesting> / ) (?<letter> . ) ) }mx ) { nested = $~[:nesting] && "::" capitalized = acronyms.capitalize($~[:acronym]) { $~[:letter].upcase } "#{nested}#{capitalized}" } end def camelCase self.CamelCase.sub(/\A[A-Z]/) { |first_char| first_char.downcase } end def self.acronyms @acronyms ||= AcronymManager.new end end class AcronymManager NEVER_MATCHES = /\zA/ def initialize @acronyms = { } @inverted = { } end attr_reader :acronyms, :inverted private :acronyms, :inverted def add(acronym) acronyms[acronym] = acronym.downcase @inverted = acronyms.invert end def regex return NEVER_MATCHES if acronyms.empty? /(?:#{acronyms.keys.map(&Regexp.method(:escape)).join('|')})/ end def inverted_regex return NEVER_MATCHES if acronyms.empty? /(?:#{inverted.keys.map(&Regexp.method(:escape)).join('|')})/ end def capitalize(acronym, &default) inverted.fetch(acronym, &default) end end if $PROGRAM_NAME == __FILE__ require "minitest/autorun" describe "Case changing" do # https://github.com/rails/rails/blob/ # 620f4a4fc962c863b91a51876ffdf58f33bedb9c/activesupport/test/ # inflector_test_cases.rb#L118-L123 let(:examples) { { "Product" => "product", "SpecialGuest" => "special_guest", "ApplicationController" => "application_controller", "Area51Controller" => "area51_controller", } } # https://github.com/rails/rails/blob/ # 620f4a4fc962c863b91a51876ffdf58f33bedb9c/activesupport/test/ # inflector_test_cases.rb#L139-L145 let(:one_way_snake_examples) { { "HTMLTidy" => "html_tidy", "HTMLTidyGenerator" => "html_tidy_generator", "FreeBSD" => "free_bsd", "HTML" => "html", "ForceXMLController" => "force_xml_controller" } } # https://github.com/rails/rails/blob/ # 620f4a4fc962c863b91a51876ffdf58f33bedb9c/activesupport/test/ # inflector_test.rb#L98 let(:one_way_camel_examples) { { "CamelCase" => "Camel_Case" } } # added by James let(:path_examples) { { "SomeLib::WithClass" => "some_lib/with_class" } } # https://github.com/rails/rails/blob/ # 620f4a4fc962c863b91a51876ffdf58f33bedb9c/activesupport/test/ # inflector_test.rb#L101-L145 let(:acronym_examples) { { "API" => "api", "APIController" => "api_controller", "Nokogiri::HTML" => "nokogiri/html", "HTTPAPI" => "http_api", "HTTP::Get" => "http/get", "SSLError" => "ssl_error", "RESTful" => "restful", "RESTfulController" => "restful_controller", "Nested::RESTful" => "nested/restful", "IHeartW3C" => "i_heart_w3c", "PhDRequired" => "phd_required", "IRoRU" => "i_ror_u", "RESTfulHTTPAPI" => "restful_http_api", # misdirection "Capistrano" => "capistrano", "CapiController" => "capi_controller", "HttpsApis" => "https_apis", "Html5" => "html5", "Restfully" => "restfully", "RoRails" => "ro_rails" } } it "can snake_case a String" do examples.each do |camel, snake| camel.snake_case.must_equal(snake) end end it "can handle some tricky one-way cases for snake_case" do one_way_snake_examples.each do |camel, snake| camel.snake_case.must_equal(snake) end end it "can CamelCase a String" do examples.each do |camel, snake| snake.CamelCase.must_equal(camel) end end it "can handle some tricky one-way cases for CamelCase" do one_way_camel_examples.each do |camel, snakey| snakey.CamelCase.must_equal(camel) end end it "can camelCase a String" do "camel_case".camelCase.must_equal("camelCase") end it "can convert nesting to paths and back" do path_examples.each do |camel, snake| camel.snake_case.must_equal(snake) snake.CamelCase.must_equal(camel) end end it "is aware of acronyms" do acronyms = AcronymManager.new acronyms.add("API") acronyms.add("HTML") acronyms.add("HTTP") acronyms.add("RESTful") acronyms.add("W3C") acronyms.add("PhD") acronyms.add("RoR") acronyms.add("SSL") acronym_examples.each do |camel, snake| camel.snake_case(acronyms).must_equal(snake) snake.CamelCase(acronyms).must_equal(camel) end end end end
-
19
SEP
2014"You can't parse [X]HTML with regex."
The only explanation I'll give for the following code it to provide this link to my favorite Stack Overflow answer.
#!/usr/bin/env ruby -w require "open-uri" URL = "http://stackoverflow.com/questions/1732348/" + "regex-match-open-tags-except-xhtml-self-contained-tags" PARSER = %r{ (?<doctype_declaration> <!DOCTYPE\b (?<doctype> [^>]* ) > ){0} (?<comment> <!-- .* --> ){0} (?<script_tag> < \s* (?<tag_name> script ) \s* (?<attributes> [^>]* > ) (?<script> .*? ) < \s* / \s* script \s* > ){0} (?<self_closed_tag> < \s* (?<tag_name> \w+ ) \s* (?<attributes> [^>]* / \s* > ) ){0} (?<unclosed_tag> < \s* (?<tag_name> link | meta | br | input | hr | img ) \b \s* (?<attributes> [^>]* > ) ){0} (?<open_tag> < \s* (?<tag_name> \w+ ) \s* (?<attributes> [^>]* > ) ){0} (?<close_tag> < \s* / \s* (?<tag_name> \w+ ) \s* > ){0} (?<attribute> (?<attribute_name> [-\w]+ ) (?: \s* = \s* (?<attribute_value> "[^"]*" | '[^']*' | [^>\s]+ ) )? \s* ){0} (?<attribute_list> \g<attribute> (?= [^>]* > \z ) # attributes keep a trailing > to disambiguate from text ){0} (?<text> (?! [^<]* /?\s*> \z ) # a guard to prevent this from parsing attributes [^<]+ ){0} \G (?: \g<doctype_declaration> | \g<comment> | \g<script_tag> | \g<self_closed_tag> | \g<unclosed_tag> | \g<open_tag> | \g<attribute_list> | \g<close_tag> | \g<text> ) \s* }mix def parse(html) stack = [{attributes: [ ], contents: [ ], name: :root}] loop do html.sub!(PARSER, "") or break if $~[:doctype_declaration] add_to_tree(stack.last, "DOCTYPE", $~[:doctype].strip) elsif $~[:script_tag] add_to_stack(stack, $~[:tag_name], $~[:attributes], $~[:script]) elsif $~[:self_closed_tag] || $~[:unclosed_tag] || $~[:open_tag] add_to_stack(stack, $~[:tag_name], $~[:attributes], "", $~[:open_tag]) elsif $~[:close_tag] stack.pop elsif $~[:text] stack.last[:contents] << $~[:text] end end stack.pop end def add_to_tree(branch, name, value) if branch.include?(name) branch[name] = [branch[name]] unless branch[name].is_a?(Array) branch[name] << value else branch[name] = value end end def add_to_stack(stack, tag_name, attributes_html, contents, open = false) tag = { attributes: parse_attributes(attributes_html), contents: [contents].reject(&:empty?), name: tag_name } add_to_tree(stack.last, tag_name, tag) stack.last[:contents] << tag stack << tag if open end def parse_attributes(attributes_html) attributes = { } loop do attributes_html.sub!(PARSER, "") or break add_to_tree( attributes, $~[:attribute_name], ($~[:attribute_value] || $~[:attribute_name]).sub(/\A(["'])(.*)\1\z/, '\2') ) end attributes end def convert_to_bbcode(node) if node.is_a?(Hash) name = node[:name].sub(/\Astrike\z/, "s") "[#{name}]#{node[:contents].map { |c| send(__method__, c) }.join}[/#{name}]" else node end end html = open(URL, &:read).strip ast = parse(html) puts ast["html"]["body"]["div"] .find { |div| div[:attributes]["class"] == "container" }["div"] .find { |div| div[:attributes]["id"] == "content" }["div"]["div"] .find { |div| div[:attributes]["id"] == "mainbar" }["div"] .find { |div| div[:attributes]["id"] == "answers" }["div"] .find { |div| div[:attributes]["id"] == "answer-1732454" }["table"]["tr"] .first["td"] .find { |div| div[:attributes]["class"] == "answercell" }["div"]["p"] .first[:contents] .map(&method(:convert_to_bbcode)) # to reach a wider audience .join
-
11
SEP
2014Experimenting With Ownership
Let's use a trivial exercise to see what we can learn about ownership, moving, borrowing, and more in Rust. Here's the idea:
- We'll allocate a list of numbers
- We'll add one to each number in the list
- We'll print the resulting list of numbers
This is a simple process requiring only a few lines of code:
fn main() { let mut numbers = vec![1u, 2, 3]; for n in numbers.mut_iter() { *n += 1; } println!("{}", numbers); }
The output is hopefully what we all expect to see:
$ ./one_function [2, 3, 4]In this code there is just one variable:
numbers. That variable owns a list of numbers on the heap and it's scope is limited to themain()function, which is just a way to say that the data exists for the length of that function call. Since all three steps happen in that one function call, ownership doesn't really affect us here.To better examine what ownership really means, let's add one small twist to our exercise:
- The increment of each number in the list must happen in a separate function
-
6
SEP
2014Taking Rust to Task
Now that I've reached the point where I can get some Rust code running without asking questions in IRC every five minutes, I really wanted to play with some tasks. Tasks are the way Rust handles multiprocessing code. Under the hood they can map one-to-one with operating system threads or you can use a many-to-one mapping that I'm not ready to go into yet.
Probably one of the most exciting aspect of tasks in Rust, in my opinion, is that unsafe use of shared memory is rejected outright as a compile error. That lead me to want to figure out how you communicate correctly. (Spoiler: the same was you do in Ruby: just pass messages.)
Ready to dive in, I grossly simplified a recent challenge from work and coded it up in Rust. You can get the idea with a glance at
main():use std::collections::HashMap; // ... fn string_vec(strs: &[&'static str]) -> Vec<String> { let mut v = Vec::new(); for s in strs.iter() { v.push(s.to_string()); } v } fn main() { let mut services = HashMap::new(); services.insert("S1".to_string(), string_vec(["A", "B"])); services.insert("S2".to_string(), string_vec(["A", "C"])); services.insert("S3".to_string(), string_vec(["C", "D", "E", "F"])); services.insert("S4".to_string(), string_vec(["D", "B"])); services.insert("S5".to_string(), string_vec(["A", "Z"])); let work = Work(Search::new("A".to_string(), "B".to_string())); let mut task_manager = TaskManager::new(services); task_manager.run(work); }