How Fast is Ruby 3 on Rails?

If you’ve been following me awhile, you know that I was hired by AppFolio opens a new window years ago to measure Ruby 3’s performance, especially on Rails. This has been a long trip. And that very first project is finally over: Ruby 3 exists and I can check its final, released Rails performance.

If you have been following along, the numbers in this post won’t surprise you. But it’s important to do the final measurement. If you haven’t been following, this will bring you up to date.

What Am I Measuring?

In concept, I’m measuring “how fast is Ruby 3 versus Ruby 2.6 or 2.7 for a typical real-world Rails workload?”

“Typical real-world workload” is a vague, hand-wavy concept. But there are a lot of specifics that can back it up. I used Rails Ruby Bench opens a new window , something I’ve written for this purpose and used for years now.

Rails Ruby Bench opens a new window runs a copy of Discourse opens a new window , a common and popular Rails app to host internet forums. It’s one of the biggest available “real” open-source Rails apps, making it a fine choice for “real world” benchmarking. RRB runs a set of simulated pseudorandom user requests against the running Rails app, and times how long they all take to finish. So it’s a throughput test. You can run it yourself if you like opens a new window , though it’s a bit complex and finicky. The dark side of using real-world software is hitting real-world complexity and bugs.

The Ruby versions I measured were my hacked Ruby 3.0 versus 2.6.6 and 2.7.1.

RRB has run against AWS m4.2xlarge instances for nearly its whole existence. That’s what I’m doing again. For now: m4.2xlarge instances, 10 processes, 6 threads/process, the same as I’ve been using for over 3 years opens a new window for this purpose. With this post, it has fulfilled its purpose: to measure the total speedup from 2.0 to 3.0 of a typical real-world Ruby on Rails application.

And I can finally stop supporting some hideous old hacks to make that possible.

These results are running 90 batches per Ruby version of 15,000 requests each. That’s enough to get pretty solid results, but not to detect the tiniest differences. If you need to detect the tiniest differences in speed, you should probably be analysing your own workload more carefully instead. Anything below about half a percent in speed is going to be dwarfed by differences for your own specific operations. It kinda breaks the abstraction of asking “how fast is Ruby 2.X?” — at some point you don’t care about the language generally, you care about the parts of it that you specifically use a lot.

Methodology and Extra Pitfalls

This post clearly follows the methodology of my 3.0.0-preview1 review opens a new window , of course. The main difference is that the Ruby branch was updated to the final release of 3.0.

Since I’m benchmarking with very old code for compatibility reasons, I hacked together a fake branch of Ruby 3 opens a new window that doesn’t remove some of the deprecated features and reports its version number as 2.8 instead of 3.0. The branch includes all the new Ruby 3 code except for a recently-added per-thread deadlock detection feature opens a new window that replaced a bit opens a new window I still needed for a deprecated feature. The per-thread deadlock detection feature isn’t be used in my benchmark, especially because it didn’t exist until after Ruby 2.7.

Here are those reverted features:

Using a hacked version of the latest Ruby with Rails 4.2 means there were several incompatible code chunks in ancient dependencies. Specifically:

• I had to change tzinfo-1.2.3/lib/tzinfo/ruby_core_support.rb in its one-liner for “open_file”. It was passing the options as a hash, not keywords. Ruby 3.0 keyword-passing changes broke this.
• Discourse uses discourse/lib/freedom_patches/ams_include_without_root.rb, a horrible old monkeypatch for ActionModel 0.8/0.9, and it causes an infinite recursion. I made discourse/app/serializers/topic_list_item_serializer.rb’s method last_poster_username return a static string, sidestepping the problem. Sigh.

And a new hack, apparently due to something about my hideously-hacked Ruby 3.0 branch that lies about its version number:

• I had to rebuild BigDecimal 1.3.5 to remove three instances of rb_check_safe_obj() from the C extension code. Since safe mode isn’t being used, it does nothing. But it also doesn’t link correctly to the (no-op, only a warning) code.

These are pretty trivial changes, so I shouldn’t be unfairly benefiting or penalizing Ruby 3.0 compared to 2.6 or 2.7. The discourse change (probably the biggest for performance) was also made where it would affect every Ruby version, not just 3.0.

Results

First off, I did not see the occasional crashes with high numbers of threads, that I saw in Ruby 3.0.0-preview1 opens a new window . I was worried they were because of my changes to the Ruby code, but apparently it was just that 3.0.0-preview1 wasn’t the most stable prerelease. The final released Ruby 3.0 seems to be fine.

Second, when I initially ran 30 batches of 15,000 HTTP requests per batch… The performance numbers were nearly identical. 169.1 reqs/sec, 171.4 reqs/sec and 170.8 reqs/sec. Particularly for version 2.7 versus 3.0, that’s well within the noise threshold, where the test is just saying “these look pretty much the same to me.” They were within 0.6 requests/second of each other with a standard deviation of 1.39. That’s basically identical.

So: Ruby 3.0.0-preview1 had been a few percentage points slower than 2.7. Tested to the same tolerances, Ruby 3.0.0’s release version was exactly the same speed as 2.7.

I’m a glutton for punishment, so I started another VM instance and ran another 60 batches of 15,000 requests for each Ruby version.

Let’s have a table for at least some visual variation, shall we?

This is once again well within the noise threshold. Ruby 3.0 has a tiny bit more variance than 2.7, but by so little that it could easily just be random. And none of these have much variance — nothing that would suggest the test is unstable or that it’s getting lots of outliers and averaging them out. The 2.7 and 3.0 results are basically identical.

Conclusion

Ruby 3.0.0-preview1 was a tiny bit slower than 2.7. The release 3.0 is exactly the same speed as 2.7. Preview1 would also occasionally crash for me in a way that might have been the hackiness of my testing… But apparently wasn’t. The release version of 3.0 had zero crashes during any of my testing. So I think preview1 was a bit unstable — it happens with prerelease versions sometimes.

Does that mean Ruby 3 failed in its promises? I don’t think so, personally opens a new window . But that’s not what this article is about.

This article is asking “how is the Rails performance of Ruby 3.0.0?” And the answer is “exactly the same as Ruby 2.7.1, which is about two percent better than 2.6.6.” You can get more historical perspective on that here opens a new window . The total comes to 1.75x the speed of (75% faster than) Ruby 2.0.0-p0.

If you’re into the low-level crunchy details of Ruby performance, FastRuby.io also has a whole category of performance posts opens a new window which you may enjoy.