Virtual threads: the limits of infinity

Java threads were expensive at scale. When your server needs to handle many requests concurrently, and each request needed a thread, the number of threads spawned could be performance-prohibitive. This led to all sorts of thread-conserving measures. The default server worker pool limit for Spring Boot was just 200, which meant you could only process 200 requests at a time. In desperation, people may switch to Reactive programming to eliminate the thread-per-request association, but the work is still scheduled off thread pools that do the real work. 

With Java 21, virtual threads are now an officially supported feature. This means threads are suddenly very cheap, blowing away those assumptions that led to thread pooling and Reactive programming. In fact, Java language architect Brian Goetz boldly predicted the death of Reactive programming (see link below). But oxymoronic as it may sounds, we should know the limits of infinite threads.

Making concurrent HTTP requests in Java

There may be occasions when you need to make multiple HTTP requests. The easy way is to simply make one request after another in sequence. If that's all you need, there is no need to read further. This will only complicate your life. Go away. Shoo.

The hard way to do this is to make multiple requests concurrently. Generally at scale (otherwise, why bother?). That's what I will play with in this post. Most of the time spent in a HTTP request is waiting for the response, so we want to get those requests all out at once, then wait for their responses. The typical pattern is to use async requests.

Test scenario:

• Make 1000 concurrent HTTP calls to a REST endpoint
• For each request, the server will wait 3 seconds before responding, simulating "work".

The results of my experimentation follows.