There may be occasions when you need to make multiple HTTP requests. The easy way is to simply make one request after another in sequence. If that's all you need, there is no need to read further. This will only complicate your life. Go away. Shoo.
The hard way to do this is to make multiple requests concurrently. Generally at scale (otherwise, why bother?). That's what I will play with in this post. Most of the time spent in a HTTP request is waiting for the response, so we want to get those requests all out at once, then wait for their responses. The typical pattern is to use async requests.
Test scenario:
- Make 1000 concurrent HTTP calls to a REST endpoint
- For each request, the server will wait 3 seconds before responding, simulating "work".
Method 1: wrap your call in CompletableFuture
You could just wrap your existing old school synchronous request in a CompletableFuture:
List<CompletableFuture<Response>> futures = new ArrayList<>();
for (int i = 0; i < numRequests; i++) {
String url = "...";
futures.add(CompletableFuture.supplyAsync(() ->
restTemplate.getForObject(url, SomeResponse.class), executor));
}
CompletableFuture.allOf(futures.toArray(new CompletableFuture[0])).join();
// read and process results
How long this takes depends on at least a couple of things:
- What kind of Executor are those futures running on?
- When does the request actually get sent?
This performance is consistent with processing 1000 requests 50 at a time, and taking 3 seconds to process each of them. What holds back this fake-async code as configured is that you still need a thread per request, so it's limited by the size of the thread pool. Moreover, the actual request does not get sent until the worker thread starts processing.
Method 2: use native HttpClient's async API
Starting with Java 11, we have a HttpClient class with built-in support for async requests, speaking CompletableFutures natively.
List<CompletableFuture<Response>> futures = new ArrayList<>();
for (int i = 0; i < numRequests; i++) {
var request = HttpRequest.newBuilder()
.uri(URI.create("..."))
.build();
httpClient.sendAsync(request, HttpResponse.BodyHandlers.ofString())
.thenApply(response -> /* parse JSON response */ null);
}
CompletableFuture.allOf(futures.toArray(new CompletableFuture[0])).join();
// read and process results1000 calls using HttpClient took 3 seconds
Method 3: use Spring's WebClient
Spring's WebClient is part of their Reactive framework and has a Reactive -- therefore async -- API. For consistency with other code here we'll convert it into a CompletableFuture:
List<CompletableFuture<Response>> futures = new ArrayList<>();
for (int i = 0; i < numRequests; i++) {
var future = webClient.get()
.uri("...")
.retrieve()
.bodyToMono(Response.class)
.toFuture();
futures.add(future);
}
CompletableFuture.allOf(futures.toArray(new CompletableFuture[0])).join();
// read and process results
Method 4: use OkHttp
Not all async implementations are equivalent. Consider my results with OkHttp. It seems to be a popular library. It has a built-in async API using enqueue, but the callback style makes the code somewhat clunky. The following does show some error handling, which I skipped with previous examples. Here's what I wrote to set up 1 future:
var request = new Request.Builder()
.url("...")
.get()
.build();
var future = new CompletableFuture<ReflectResponse>();
okHttpClient.newCall(request).enqueue(new Callback() {
public void onFailure(Call call, IOException e) {
future.completeExceptionally(e);
}
public void onResponse(Call call, Response response)
throws IOException {
if (!response.isSuccessful()) {
future.completeExceptionally(new IOException("Unexpected response code: " + response.code()));
} else {
if (response.body() == null) {
throw new IOException("response is missing body");
} else {
var result = objectMapper.readValue(response.body().string(), Response.class);
future.complete(result);
}
}
}
});
1000 calls using OkHttp took 601 seconds
Other limits
- processes limits the number of threads you can have in your JVM
- file descriptors limits the number of network connections in your JVM
What about virtual threads, a.k.a Project Loom?
var executor = Executors.newVirtualThreadPerTaskExecutor();
Conclusions
- I don't see a significant advantage either way. The usual complaint about reactive/async code is the difficulty in debugging because you lost context. But a virtual thread's stack trace is also similarly context-free, starting where it was created.
- Your choice might be a matter of convenience. The newer fluent APIs may be more pleasant to work with, or may be more convenient wrappers around lower level HTTP clients. Spring's RestClient, for example, supersedes RestTemplate. Higher level wrappers take care of some basic plumbing like error handling, retries, metrics and JSON serialization/deserialization.
- Your choice might be limited to what version of Java you're on. Virtual threads only enjoy official status in Java 21.
- But if you do have access to virtual threads, the conventional wisdom on thread conservation and pooling may need to go out the window.
- Not all async HTTP clients are created equal. You might not like the Reactive vocabulary, or (as with OkHttp) an async API does not guarantee full concurrency.
No comments:
Post a Comment