Runbook pages.

Builds & investigations Deep dive

What concurrent Ruby looks like, twelve years after the book

A cache benchmark stayed serial as thread count rose. I traced the Ruby runtime, then tested where mutexes, Ractors, fibers, and snapshots help.

20 min read · updated

rubyconcurrencyractorfiberasyncperformanceplatform

Share

I ran a cache-read benchmark with one thread, then kept adding threads. Median latency barely moved, but p99 went from 9 ms at one thread to 3.4 seconds at thirty-two.

threads      p50 ms     p95 ms     p99 ms     wall s
1             6.184      7.765      9.021      3.150
4             6.638    345.249    456.259     17.218
8             5.521    774.835    817.882     27.179
10            5.818   1036.983   1099.172     35.390
16            5.635   1661.889   1708.506     55.324
32            5.367   3101.961   3412.738     94.348

I couldn't find a mutex on the gem's read path. The class we were caching was frozen, and the cache itself was thread-safe. More threads still meant much longer waits.

These timings show contention, but they do not identify its cause. Separating time spent waiting for the GVL from time spent in GC needs a profile from the same run.

That benchmark sent me back to Working with Ruby Threads, which came out in 2014. I wanted to check which parts still applied and how Ractors, fibers, and snapshot-based caches changed the options.

I'll start with the thread bugs the book covers, then work through newer Ruby tools and return to the cache benchmark.

What the book teaches you, with the actual oopses

A thread in Ruby is what you'd expect from any language. An execution path with its own stack, sharing memory with the other threads in the same process. You start one with Thread.new { ... }, you wait for it to finish with thread.join, you read its return value with thread.value. The job of a thread is to let your program do more than one thing at once. Handle two web requests in parallel. Fetch two URLs concurrently. Run a background task while the main thread does something else.

The reason people get hurt by Ruby threads in particular is the GVL, the Global VM Lock. Ordinary threads within one CRuby Ractor share this lock: only its holder can execute Ruby code. The other threads aren't dead, they're waiting their turn.

Threads give you concurrency. The scheduler switches between them, but ordinary threads in one CRuby Ractor cannot execute Ruby code at the same instant. Native extensions can release the GVL, and separate Ractors have their own locks. The distinction matters when a benchmark mixes Ruby work with I/O or native code.

The trick that makes multi-threaded Ruby servers work at all is that the GVL releases on blocking I/O. When a thread calls Net::HTTP.get or hits a slow database query or reads from a socket, the VM hands the lock to whoever is waiting and lets the original thread sleep until the OS wakes it back up. So a Puma worker with five threads can hold five slow database queries open in parallel, even though none of those threads is executing Ruby. The Ruby part is sequential. The I/O part is parallel.

JRuby and TruffleRuby allow Ruby threads to execute in parallel on multiple cores. That changes the options for CPU work. I/O-heavy applications still need measurements on the runtime and drivers they actually use.

The GVL prevents most kinds of native-data-structure corruption. Two threads can't both be halfway through resizing the same hash. What it does not prevent is what trips you up in real code, which is everywhere the VM gives up the lock between two operations you assumed would be atomic.

Naive memoization is a check-then-act race

This is the one I see junior Rails engineers fall into all the time, because the lock makes it look safe.

class Cache
  def get
    @value ||= expensive_compute
  end
end

@value ||= expensive_compute checks the current value before it computes and assigns a replacement. Two threads can both start the computation before either publishes a result, especially when that computation performs I/O. A side effect inside it can happen twice. The final assignment also replaces the other thread's result.

Eager initialization at boot avoids the race. Mutex#synchronize can protect the whole check and computation. A Concurrent::Map offers compute_if_absent when the cache needs an atomic lookup-and-insert operation.1

The counter race

The other one every Ruby concurrency tutorial starts with, because it's the smallest demonstration of a check-then-act race.

@counter = 0

threads = 5.times.map do
  Thread.new do
    temp = @counter
    Thread.pass          # expose the gap between reading and writing
    temp = temp + 1
    @counter = temp
  end
end
threads.each(&:join)

puts @counter
# Can be less than 5, depending on the interleaving.

The GVL doesn't save you. Between temp = @counter and @counter = temp, the scheduler can switch out, run another thread to completion, and switch back. Your temp is stale. Your final assignment overwrites whatever the other thread published.

@counter += 1 expresses a read, an addition, and an assignment. Its compact syntax does not provide a portable synchronization contract. Individual array or hash methods are a different question from a sequence such as "check whether the key exists, then insert it." I protect the whole sequence with a mutex, or use a primitive such as Concurrent::AtomicFixnum that already owns that contract.

Deadlock

The bug everyone has heard of and has trouble reproducing, because it needs two locks and an unlucky scheduler interleaving. The smallest example is two threads taking two locks in opposite orders.

m_a = Mutex.new
m_b = Mutex.new

a = Thread.new do
  m_a.synchronize do
    sleep 0.01           # give the other thread time to take m_b
    m_b.synchronize { puts "got both, A then B" }
  end
end

b = Thread.new do
  m_b.synchronize do
    sleep 0.01
    m_a.synchronize { puts "got both, B then A" }
  end
end

[a, b].each(&:join)

Thread A holds one lock and waits for B's lock. B does the reverse. Ruby can detect a deadlock when no thread can make progress and raise a fatal error.2 A larger service may keep other threads running while these two remain stuck, so I would not rely on global deadlock detection to catch every lock cycle.

The remedy is a lock hierarchy. Decide once which order locks are taken in. If m_a always precedes m_b, two threads can both ask for both and only one ever gets m_a first. The other waits. No cycle.

Livelock

Deadlock has a more humiliating sibling that happens less often but ruins your day harder when it does. To get to it you first need Mutex#try_lock, which is the non-blocking cousin of Mutex#lock. Where synchronize blocks until the lock is yours, try_lock looks at the lock, returns true and takes it if it's free, returns false immediately if someone else has it, and never waits. People reach for try_lock because it sounds like a clean defense against the deadlock pattern above: take one lock, try the other, give up and retry from the top if the second one is taken. The retry loop is what eats you.

Workers that take opposite first locks can repeatedly force each other to release and retry. They run code without completing the operation. A tight retry loop can also burn CPU while another thread holds the lock. Consistent lock ordering is the first fix I would use. A retry design needs backoff and a bound on how long it can keep trying.

ConditionVariable

This is the primitive you reach for when one thread needs to wait until another publishes something. It's the building block under most thread-safe queues.

require 'thread'

results = []
m = Mutex.new
cv = ConditionVariable.new

producer = Thread.new do
  10.times do |i|
    sleep 0.1
    m.synchronize do
      results << i
      cv.signal
    end
  end
end

consumer = Thread.new do
  10.times do
    m.synchronize do
      cv.wait(m) while results.empty?
      puts results.shift
    end
  end
end

[producer, consumer].each(&:join)

The two things you'll get wrong on first attempt: cv.wait(m) releases the mutex before it sleeps and reacquires when signalled, which is the whole point. And the while results.empty? has to be a while, not an if, because spurious wakeups happen and because by the time you reacquire the mutex another consumer might have shifted the result out.

Sidekiq, and why its architecture is the worked example

Sidekiq is the multi-threaded background job system that pretty much every production Ruby application reaches for, and the book devotes a chapter to walking through how it's built because it's the cleanest worked example in the language of the primitives we just covered. The shape it draws is still the shape you'd draw today if you sat down to design a job runner from scratch in 2026.

Sidekiq's current implementation gives each Processor a thread. That thread fetches work through its capsule's fetcher, runs the job, and repeats. The manager supervises the processors.3 A fetcher object does not imply one separate fetcher thread handing jobs to every worker.

I find that division useful because it gives lifecycle decisions a place in the code. Processor shutdown, job execution, and fetching have different responsibilities. The queue's persistence and fetch strategy still determine what happens to work when a process disappears. A thread-safe runner alone cannot settle that delivery question.

What the book doesn't cover

This is the section I went looking for and couldn't find written down anywhere in one place. So here goes.

Ractors, and what they still don't fix

The biggest new thing in MRI concurrency since 2014 is the Ractor, introduced in Ruby 3.0 in late 2020. A Ractor runs Ruby code in true parallel with other Ractors. Each one has its own lock. Two on a two-core machine can crunch numbers at the same time.

port = Ractor::Port.new

worker = Ractor.new(port) do |out|
  loop do
    msg = Ractor.receive
    out << msg.upcase
  end
end

worker.send("hello")
puts port.receive   # => "HELLO"

The constraint that wins every argument about Ractor adoption is the shared-state one. Ractors can't directly share most mutable state. A normal send copies an unshareable object. A move-send transfers ownership and prevents the sender from using it again.

state = { count: 0 }
worker.send(state)
# worker receives a copy; this ractor still owns the original

Ractor.make_shareable(state) provides another option when sharing an immutable object fits the design. A move-send transfers ownership instead of copying. Ordinary mutable connection pools, loggers, and caches cannot simply cross that boundary unchanged. The library compatibility work remains a substantial part of adopting Ractors.4

Ruby 4.0 introduced Ractor::Port and removed the older Ractor.yield and Ractor#take APIs.5 The runtime also reduced shared contention: frozen strings and symbols use a lock-free hash set, method-cache lookups avoid locking in most cases, and object-allocation counters are per-Ractor.6 Those changes improve the implementation without removing the application's shared-state constraints.

Ruby 4.0's release notes still call Ractor experimental.6 A Rails application also needs its dependencies to support the chosen execution model. The release announcement cannot answer that compatibility question for its Gemfile.

For a self-contained CPU-bound problem, such as a parser or simulation, the Ractor boundary can be manageable. Retrofitting that boundary around an application and its dependencies is a different amount of work.

Fiber.scheduler and async

Quieter shift, and I think the one actually changing how new Ruby code gets written: the Fiber scheduler interface that landed in Ruby 3.0 alongside Ractor. Fibers themselves have been in Ruby since 1.9, but until 3.0 they were cooperative-only inside your own code. The Fiber scheduler hook lets a third-party library register itself with the VM so that blocking I/O operations (a Net::HTTP call, a read on a socket, a sleep) automatically yield to other fibers instead of blocking the whole thread.

Samuel Williams's async7 is the library that wires this up. The code looks synchronous. The runtime is event-driven.

require 'async'
require 'net/http'

Async do
  urls = (1..10).map { |i| URI("https://example.com/#{i}") }
  urls.map do |url|
    Async { Net::HTTP.get(url) }
  end.map(&:wait)
end
# 10 concurrent HTTP requests, in one thread, no threadpool.
# Each Async block runs as a fiber. The blocking Net::HTTP.get
# yields to the scheduler. The scheduler resumes the next ready fiber.

Fibers let many waiting operations share one thread rather than requiring an OS thread for each one. They still consume memory, and their scheduler runs cooperatively. CPU work holds up the other fibers on that thread until it yields. Ruby 4.0 added the Fiber::Scheduler#fiber_interrupt hook for interrupting a fiber when its waiting I/O closes.8

Then there's Falcon9, also from Williams, the Rack-compatible server built on async. It combines processes for parallelism with fibers for concurrent I/O. That makes it worth evaluating for streaming and other workloads with many waiting connections. Connection capacity still depends on the application, drivers, and memory budget.

Ruby's scheduler hooks cover standard I/O, but a native extension can still block its calling thread if it waits without using those hooks. Releasing the GVL lets another Ruby thread run. It does not by itself let another fiber run on the blocked thread.

The pg gem supports Fiber.scheduler from version 1.3.0. It routes ordinary blocking I/O through the registered scheduler. Its documentation lists exceptions, including large-object methods and some authentication paths.10 Compatibility needs a driver version and a call path, not a guess based on whether the gem contains C.

I haven't run Falcon in production. I want to. The honest blocker is the same as Ractor's. The gem ecosystem assumes thread-based concurrency. If your background-job adapter, your APM agent, your feature-flag client, or your DB driver assumes thread-local state where the right answer is fiber-local state, you'll find out at the wrong moment.

concurrent-ruby and the small primitives

concurrent-ruby supplies concurrent collections, atomic references, futures, and executors. These let application code work with a defined concurrency contract without implementing every primitive from mutexes.

Concurrent::Map provides a hash-like API for concurrent access. Its backend depends on the Ruby runtime. The method matters too: fetch can compute a fallback without storing it, while compute_if_absent provides an atomic lookup and insertion.1 A safe container does not make an arbitrary sequence of calls atomic.

Concurrent::AtomicFixnum handles a counter update. Thread pools cap the number of worker threads, and a bounded work queue gives the caller an explicit saturation policy. I would choose those contracts before choosing a particular thread count.

Concurrent::AtomicReference holds one reference that readers can access while a writer replaces it. A reader gets the old reference or the new one. The writer must finish building and freezing the object before publishing it. Atomic publication does not make later mutation safe, and the implementation's locking cost depends on the runtime.11

The GVL debate, and what's actually moving

Jean Boussier's "So You Want To Remove The GVL?"12 uses free-threaded Python as a comparison for the work a runtime takes on without one global lock. Python's design includes container locks, biased reference counting, and deferred reference counting.13 Biased reference counting deliberately lets the owning thread avoid atomic updates for its local count. Removing the GVL would still require Ruby to replace the safety it currently provides, and to measure the cost of that replacement.

The cheaper wins, per Boussier and John Hawthorn, sit one layer below "rip out the lock entirely." The first is a smarter thread scheduler. Ruby's current scheduler is FIFO with no notion that an I/O-bound thread that just woke up from a socket read should get the lock back faster than a CPU-bound one that's been hogging it. The second is widening the windows where the GVL is released during C-extension work, so threads inside those gems don't serialize each other unnecessarily. Aaron Patterson shipped a small but useful piece of the first direction in Ruby 3.4: an environment variable exposing the thread-switching quantum, which is the maximum time slice one thread is allowed to hold the lock before the scheduler considers handing it over. The default is 100ms, and now workloads that have measured their contention can tune it14.

The other moving piece in the same neighbourhood is the JIT story. A JIT (just-in-time compiler) watches hot paths and emits faster machine code while the program runs. YJIT is the production JIT written in Rust. Ruby ships it, but the Ruby 3.3 command-line docs still mark it disabled by default. An application enables it with --yjit, RUBY_YJIT_ENABLE, or RubyVM::YJIT.enable.15 ZJIT is the next-generation method-based JIT that Ruby 4.0 introduced. Neither JIT changes the GVL. They shorten work that happens while a thread holds it.

The pattern I keep coming back to

Now back to the bench at the top of this post.

The method under test was a cache lookup. Roughly this shape:

def find_cached(scope, key)
  cache_key = build_key(scope, key)   # composes a small object
  CACHE.fetch(cache_key) do
    load_and_freeze(scope, key)       # cold path, almost never runs
  end
end

Requests call find_cached repeatedly. The expected hot path returns an already-loaded, frozen object. In the opening benchmark, p99 went from 9 ms with one thread to seconds as I added threads.

So I went down the obvious list.

CACHE was a Concurrent::Map. Its thread-safety contract ruled out unsynchronized access to the map itself, but that contract did not prove the lookup was free from contention.

Is the cached object hot? No. The value is a frozen class. Once it's in the map, nothing mutates it. No Mutex anywhere on the read side, no instance state being read under contention.

Is something inside load_and_freeze running on every call? Also no. I traced it. The block on fetch only fires when the key is missing, which on a warm bench is never.

By that point the only thing left in the method was build_key. Which is, roughly, this:

def build_key(scope, key)
  { scope: scope, key: key }.freeze
end

build_key constructs a new Hash on every call. The symbol keys already exist, and freeze returns the same object. The method's actual allocation count needs to be measured on the runtime in use.

A warm cache can still make garbage.

Eight reads per thread. An illustrative allocation budget.

Read path
T1
T2
T3
T4
read GC pause
96wrapper allocations
4 threads × 8 reads × 3 objects

The example GC budget fills 4 times. Each pause affects every thread.

Model assumptions: three wrapper objects per read, one pause per 24 objects. These are teaching inputs, not measured Ruby allocation counts or GC thresholds. Aligned cells don’t mean simultaneous execution.

When a thread does an I/O call like Net::HTTP.get, a slow Postgres query, or a socket read, the VM hands the GVL to another thread and lets the first one sleep. The blocked thread costs nothing while it waits. That's what makes a ten-thread Puma worker actually serve ten concurrent requests against a slow database.

Ruby object allocation adds work while a thread holds the GVL. It can also trigger garbage collection. GC has phases that stop Ruby execution, so extra allocations can affect other threads as well as the caller. The size of that effect depends on the runtime, heap, and workload.

I would separate those costs before attributing the whole p99 jump to GC. Allocation counts show what the method creates. GC timings and a GVL profile show where the threads waited. The latency table alone cannot distinguish those causes.

I wanted the read path to stop constructing a composite key each time. An immutable snapshot let me move that preparation into the refresh path. Readers load one reference and look up an existing key.

SNAPSHOT = Concurrent::AtomicReference.new(build_snapshot)

def lookup(key)
  SNAPSHOT.value.fetch(key)
end

def refresh!
  snapshot = build_snapshot   # fully build and freeze before publication
  SNAPSHOT.set(snapshot)
end

build_snapshot loads the source data and deep-freezes the lookup structure. Publication happens only after that work finishes. The example assumes one refresh worker. Concurrent refreshers need ordering too, or an older build can replace a newer snapshot.

Readers no longer allocate the composite key shown earlier. That is the change this code demonstrates. A before-and-after benchmark needs the same workload and runtime settings before it can support a latency claim.

You can apply this anywhere the read pattern is "look something up many times per request" and the write pattern is "occasionally swap in a new version of the whole dataset." It's the same pattern the per-request feature-flag dial in my earlier ConfigMap post reaches for one level up. The channel announces "something changed". A background thread does the cold work of rebuilding the snapshot.

A full snapshot becomes expensive when the dataset is large or changes constantly. Concurrent::Map can fit a cache that updates entries independently. The choice depends on update frequency, retained memory, and the consistency a reader needs. Neither data structure wins every benchmark.

What I'd still revisit

Ractor adoption is the big one. The plausible path isn't "rewrite the application around Ractors". It's "one or two narrow, CPU-bound subsystems get a Ractor boundary around them and stay there." If you've actually shipped a Ractor in production for a Rails monolith and it worked, I'd want to hear about it. Every story I've seen so far has been a yak-shave that ended in "we put it back behind a thread."

Where Falcon belongs is the other one. The case for fiber-based servers is overwhelming for streaming and WebSockets. The case for ordinary request-response Rails is more nuanced, and likely lands at "Falcon for the things that look like real-time, Puma for everything else, in the same fleet." My own next step is to put a small streaming endpoint behind Falcon and measure.

The GVL-removal question I think is mostly answered for now. The improvements Ruby actually needs (smarter scheduler, longer GVL-released windows during DB work, more parallelism in the GC) aren't blocked on the lock. They're separate work, and they're the work I'd bet on shipping in the next few releases.

If you've hit a different shape of this same problem, or you're running Ractors or Falcon in production and want to compare notes, I'd love to hear about it. [email protected]. The war stories are the part I read first.

Footnotes

  1. concurrent-ruby, Concurrent::Map. The API distinguishes fetch, fetch_or_store, and atomic compute_if_absent. ↩ ↩2

  2. Ruby, Thread.ignore_deadlock. Deadlock detection defaults to enabled. The documentation shows that a global deadlock can raise a fatal error. ↩

  3. Sidekiq, Processor. Each processor runs its own fetch-and-process loop and obtains work through the capsule's fetcher. ↩

  4. Jean Boussier, To Thread or Not to Thread, Shopify Engineering. The Ractor section is the cleanest articulation of the "global mutable state is the bottleneck" framing I've read, and it predates Ruby 4.0 but holds up. ↩

  5. Ruby 4.0.0 release notes, Compatibility issues / Ractor. The removed methods are Ractor.yield, Ractor#take, Ractor#close_incoming, and Ractor#close_outgoing, all replaced by Ractor::Port. ↩

  6. Ruby 4.0.0 release notes, Ractor Improvements and Implementation improvements / Ractor. The lock-free hash set for the frozen-string table and the per-Ractor allocation counter are the two changes I'd flag as the most operationally meaningful. ↩ ↩2

  7. socketry/async, readme. The project description is "composable asynchronous I/O framework for Ruby based on io-event," and the guides on scheduler, tasks, and thread safety are worth reading in order. ↩

  8. Ruby 4.0.0 release notes, Fiber::Scheduler section. Fiber::Scheduler#fiber_interrupt is the one I'd point at. The original 3.0 interface had no clean way to cancel a fiber waiting on a closed IO. ↩

  9. socketry/falcon, readme. The architecture description ("multi-process, multi-fiber rack-compatible HTTP server built on top of async") is the one-line summary that explains every operational property below it. ↩

  10. ruby-pg, Fiber IO scheduler support. Scheduler support starts at pg 1.3.0. The README documents the remaining exceptions. ↩

  11. concurrent-ruby, Concurrent::AtomicReference. Atomic reference operations synchronize publication. The stored object's own mutations need separate coordination. ↩

  12. Jean Boussier, So You Want To Remove The GVL?. The comparison with Python's per-object lock implementation is the part I keep going back to when someone says "just remove it like Python did." ↩

  13. Python, PEP 703, especially biased reference counting, deferred reference counting, and container thread-safety. Free-threaded reference counting does not make every increment and decrement an atomic operation. ↩

  14. Ruby bug tracker, Feature #20861: Make the thread time quantum configurable. Aaron Patterson's change to expose the 100ms quantum as an env var. Not a fix, a knob, but a useful one if you've already characterised your contention. ↩

  15. Ruby documentation, YJIT command-line options, and the Ruby 4.0.0 release notes, JIT / ZJIT. Ruby documents --yjit as disabled by default. A framework or deploy image can still enable it for the application. ↩

Share

Comments