Feature matrix of Ruby HTTP clients

I posted Ruby HTTP clients feature matrix which compares various Ruby HTTP client implementations, as a product of Asakusa.rb meetups. I hope Ruby users find it informative.

Listed clients: net/http, open-uri, httparty, rest-client, right_http_connection, rufus-verbs, simplehttp, curb, patron, typhoeus, eventmachine, excon, httpclient, faraday, wrest, activeresource and rfuzz.

Sample and test scripts for this matrix is at github repo. Let me know if you find any mistakes (it must include some!) and my misunderstandings.

This matrix is a part of my presentation Oedo HTTP client picture scroll at Oedo RubyKaigi01(in Japanese) hosted by Asakusa.rb yesterday. You can see other comparisons including dependency, timeline and performance in the presentation. There are some Japanese commentary in it but almost major parts are in English and images.

[EDIT]

Disclaimer: I'm the author of httpclient gem, which is one of a clients in the list. :)

Here's Summary in the presentation:

- net/http has various derivatives and alternatives because of old-style API and simple structure.

- Few derivatives of net/http offers full-features of net/http. Too simple to extend?

- Implemented features are vary from product to product.

- APIs are vary, too. Making simple API tends to introduce disunity.

- There's no performance difference between products but it's worth checking Keep-Alive and gzip compression support.

- Eventmachine is considerably fast, if you can do non-blocking all your code.

- Take care if you use C-ext libcurl variants. Some introduce interpreter blocking and JRuby/Rubinius incompatibility (at this moment).

And if you need SSL support, clients you can use are not much.

[/EDIT]

Have fun!

JRuby-OSSL (JRuby-OpenSSL) 0.7.2 released

JRuby-OSSL 0.7.2 is out.

• http://rubygems.org/gems/jruby-openssl
• http://github.com/jruby/jruby-ossl/blob/master/History.txt

It contains fixes for important SSL resource management issues; selector/file descriptor leak and timeout did not work.

• JRUBY-5018 SSLSocket holds selectors, keys, preventing quick cleanup of resources when dereferenced
• JRUBY-5059 Timeouts don't work correctly for https sessions

SSL users are recommended to upgrade to 0.7.2.

Other fixes are;

• JRUBY-5024 HMAC.new with digest name raises an Exception; Now HMAC.new(msg, "SHA1") works as same as HMAC.new(msg, Digest::SHA1.new)
• JRUBY-5023 Certificate#signature_algorithm returns wrong name; Now it returns "md5WithRSAEncryption", "sha1WithRSAEncryption", "dsaWithSHA1" or "dsaWithSHA2" instead of "SHA1" or "SHA2".
• JRUBY-5096 Inconsistent verification behavior; Caling Certificate#verify twice returns true and false for valid certificate.
• JRUBY-4965 OpenSSL::Config not implemented; Now you can use OpenSSL::Config for parsing openssl.cnf
• JRUBY-5060 x509store.PEMInputOutput.writeX509Request causes NullPointerException; Avoid NPE from to_pem for empty X509 Objects
• JRUBY-5125 Cipher#name should return SN(short name) of OpenSSL (uppercase in general); Now Cipher#name for arcfour is 'RC4' not 'rc4' for Net::SSH compatibility.
• JRUBY-5126 Cipher#reset should not reset the internal state for stream cipher; Net::SSH depends on this behavior.

Net::SSH users are also recommended to upgrade.

This release supports following environments. (Not changed from 0.7.1)

• JRuby 1.5.3 and 1.4.0 (1.5.X should run fine, too)
• JRE 6 and 5
• With or without Unlimited Strength policy files for JRE
• Any OS which runs above JRE (I tested against Ubuntu 10.10 and Windows 7)

Please file a ticket at http://bugs.jruby.org with 'Component: OpenSSL' when you find a problem with this release.

JRuby-OSSL (JRuby-OpenSSL) 0.7.1 released

JRuby-OSSL 0.7.1 is out.

• http://rubygems.org/gems/jruby-openssl
• http://github.com/jruby/jruby-ossl/blob/master/History.txt

It's not a big release like 0.7. Mainly fixing 1 HTTPS client bug.

• JRUBY-4826 - net/https client possibly raises "rbuf_fill': End of file reached (EOFError)" for HTTP chunked read.

And misc changes.

• JRUBY-4900 - Set proper String to OpenSSL::OPENSSL_VERSION. Make sure it's not an OpenSSL artifact: "OpenSSL 0.9.8b 04 May 2006 (JRuby-OpenSSL fake)" -> "jruby-ossl 0.7.1"
• JRUBY-4975 - Moving BouncyCastle jars out to its own gem.

That's all. Much thanks to bug reporters of JRUBY-4826 and JRUBY-4822 (considered as the same issue) for helping us to fix this bug.

NOTE: Now BouncyCastle jars has moved out to its own gem "bouncy-castle-java" (http://rubygems.org/gems/bouncy-castle-java). You don't need to care about it because "jruby-openssl" gem depends on it from now on.

You can uninstall "bouncy-castle-java" gem if you certainly know that you don't need BouncyCastle jar for your JRuby-OSSL usage. For example, Random, Digest, HMAC and Cipher does not depend on BC jars from 0.7. And take care about "bouncy-castle-java" gem is distributed under the same terms as BouncyCastle's license.

One more thing at the end. We changed the artifact name to JRuby-OSSL for clearing that it is not using/distributing OpenSSL Toolkit itself. JRuby-OSSL just includes some derived work from software developed by the OpenSSL Project in the OpenSSL Toolkit. (http://www.openssl.org/) And we keep the name 'jruby-openssl' at some point like gem name for avoiding users confusion. It's a cosmetic change and there's no change other than that.

Enjoy!

JRubyKaigi2010

I'll give a 20min. session talk on JRubyKaigi2010. (smart URL, isn't it?)

"JRuby Hacking Guide" - Aims; extracting what is where in JRuby source code, showing how those interacts when, and hacking/troubleshooting it by examples in 20 min.

Date: 8/28(Sat.) 13:00-18:00 (on the second day in RubyKaigi2010)

More Details: http://rubykaigi.tdiary.net/20100728.html#p02

JRuby co-lead Charles O.Nutter and Thomas Enebo comes to Japan and have a Keynote. Can't wait to discuss with them including many Japanese developers who are interested in JRuby's future.

Are you planning to go RubyKaigi2010? Interested in JRuby's compilation details, optimization scheme and trouble shooting? See you there!

Queue#pop is unfair in Ruby 1.9

Do you think Queue is fair?

"Fairness" here is, when two or more threads wait with Queue#pop, the control is sequentially passed from the thread for which it waits longest when Queue#push is invoked.

In this sense, Ruby 1.8 is "fair". Following script always dumps "[:go, :first]" for Ruby 1.8.

# http://gist.github.com/493930#file_gistfile1.rb

create_waiter = lambda { |name, command|
 t = Thread.new {
   p [command.pop, name]
 }
 Thread.pass until t.status == 'sleep'
 t
}

50.times do
 command = Queue.new
 # create 3 waiters, and...
 create_waiter.call(:first, command)
 create_waiter.call(:second, command)
 create_waiter.call(:third, command)
 # and pushes a command.
 command.push(:go)
 # then try to consume the command by a subsequent thread
 Thread.new {
   p [command.pop, :LATER]
 }
end

In contrast to this, Ruby 1.9 sometimes dumps "[:go, :LATER]" because Queue#pop became unfair from Ruby 1.9.

To tell the truth, there might be no use in making Queue fair because Mutex#lock is unfair in Ruby. However, Queue#pop implementation in Ruby 1.9 is almost fair so if there's a good solution... Anyone?

Anyway, person who is using 1.8 Queue as a thread scheduler should care this change. My typical usage was as follows.

# http://gist.github.com/430066

require 'thread'

q = Queue.new
t1 = Thread.new {
 p [:t1, q.pop]
 q.push(:t1)
}

q.push(:main)
p [:main, q.pop]

EDITED ADD (8/1): You should wait for the created thread sleeping like the first script on this page.

By the way, Queue#pop in JRuby (both 1.8 and 1.9 modes) is unfair as well but slightly different from Ruby 1.9; only the main thread break in the scheduler. It's written by @mentalguy who is the author or fastthread for Ruby 1.8.

JRuby-OpenSSL 0.7 released

JRuby-OpenSSL 0.7 released! http://www.jruby.org/2010/04/27/jruby-openssl-07.html.

All changes which should be noted are in the above release note.

And more: It passes all testcases for ruby-ossl for CRuby 1.8.7 except few classes which are not yet implemented like Cipher::EC. And I confirmed it passes all testcases of Net::SSH, httpclient, WEBrick, OAuth and rubygems except known issues which are 'wontfix'. (For example, JRuby-OpenSSL does not support 32bit DSA key which is not allowed in BouncyCastle. Key lengths must be 512-1024 and multiple of 64.)

This release supports JRuby 1.4.0+ but 1.5 is better tested. Please run your testcases against JRuby + JRuby-OpenSSL/0.7 and file a ticket at http://bugs.jruby.org when you find a problem. I think we can fix it at the next release.

It's the first release after I started contributing JRuby-OpenSSL. Thanks to @nicksieger for coordinating my commits and bad English to the release. It must be an annoying work. Also thanks to @olabini for creating this >30kloc sources for JRuby world. I bet no one can really understand what you did, except a person who understands Ruby internal and OpenSSL Toolkit internal. Great work, indeed.

I did a little benchmark testing on some http client implementations on Ruby. I'm the author of httpclient (http-access2), one of a http client implementation for Ruby.

Environment:

• HTTP Server:
  
• Apache/2.2.9 (Ubuntu) mpm-worker
  
• Ubuntu 8.10 32bit
  

HTTP Client:
Ruby: ruby 1.8.7 (2008-12-04 revision 20478) [i686-linux]
vendor:
eventmachine/0.12.2(*)
rfuzz/0.9
net/http (svn ruby_1_8)
httpclient/2.1.3-RC
open-uri (svn ruby_1_8) (net/http wrapper)
httparty/0.2.2 (net/http wrapper)
(*) EM::Protocols::HttpClient2 is not listed below because it does
not work when requests > 20 on my env.

HTTP Server and HTTP clients runs on the same host.

Benchmark suite:
http://dev.ctor.org/http-access2/browser/trunk/bench/bm.rb
proxy = nil
url = http://127.0.0.1/
threads = 2
requests = 250

Result:
HTTP Server KeeyAlive: Off
user system total real
curb 0.050000 0.420000 0.470000 ( 0.807049)
RFuzz::HttpClient 0.090000 0.430000 0.520000 ( 0.803075)
Net::HTTP 0.280000 0.520000 0.800000 ( 1.147281)
HTTPClient 0.140000 0.800000 0.940000 ( 1.299928)
open-uri 0.420000 0.880000 1.300000 ( 1.675272)
HTTParty 0.490000 0.620000 1.110000 ( 1.483024)

HTTP Server KeeyAlive: On
user system total real
curb 0.100000 0.460000 0.560000 ( 51.170485)
RFuzz::HttpClient 0.030000 0.580000 0.610000 ( 1.001198)
Net::HTTP 0.150000 0.490000 0.640000 ( 0.842344)
HTTPClient 0.140000 0.460000 0.600000 ( 0.801035)
open-uri 0.740000 0.740000 1.480000 ( 1.870712)
HTTParty 0.710000 0.880000 1.590000 ( 2.555711)

Considerations:
1. EM::Protocols::HttpClient2 does not work on my env. Is the
client code right?
2. curb needs too much 'real' time when connecting to KeepAlive: On
site. Is this expected?
3. rfuzz and curb are faster than net/http and httpclient when
connecting to KeepAlive: Off site.
4. net/http and httpclient run almost as fast as rfuzz when
connecting to KeepAlive: On site. Bear in mind that you need to call
Net::HTTP#start and #finish by yourself for keep-alive handling. Just
calling Net::HTTP#get takes 2 times longer than others. Bothersome?
Use others.

At the last, does somebody know how I can read
http://apocryph.org/more_indepth_analysis_ruby_http_client_performance
? I get 'Error establishing a database connection'.