The 101st Request

In our .NET 10 lab, we started 300 requests through an HttpClient at the same moment. Each spent about one second on the server. The slowest answer arrived after 3.04 seconds.

From the server's side, nothing looked slow. Its duration metric showed about one second for each of the 300 requests, and the process used under 0.1 % of the machine's CPU. Two hundred requests spent the extra time waiting in a queue inside the client, before anything went across the network.

We hadn't written any queueing code. The queue comes from HTTP/2 and depends on a single handler property. gRPC and YARP set it to true when they build the handler for you. The handler type itself defaults to false. If you supply your own SocketsHttpHandler, for a certificate, a proxy or keep-alive pings, that default is now yours.

Where the queue lives

HTTP/2 carries many requests over one TCP connection, each on its own stream, and the server limits how many streams can be open at once. Kestrel's default is 100, which it announces to the client when the connection starts. .NET's SocketsHttpHandler respects that limit: when all 100 streams are busy, request 101 waits in the handler's queue until a stream comes free.

A second connection would let it through without waiting. That is what the property controls:

With one connection and 100 streams, each request after the hundredth waits for an earlier one to finish. Our requests all took about one second, so the queue cleared in batches, one second apart:

The server handled at most 100 requests at a time, and each finished in 1.01 seconds or less. The rest of each answer's time passed before the server started the request, and the client's own queue metric, below, recorded waits of the same size.

Who is exposed

Plain HttpClient uses HTTP/1.1 unless you ask for HTTP/2, and HTTP/1.1 gives each concurrent request its own connection. In our lab, 150 requests at once opened 150 TCP connections and got their answers in 1.03 seconds. The wait for a connection stayed under 10 milliseconds, paid for with 150 connections.

gRPC calls normally run on HTTP/2 (the .NET client can also use HTTP/3 when the server offers it). Microsoft's gRPC performance guide says what happens at the limit: "When the number of active calls reaches the connection stream limit, additional calls are queued in the client." By default, GrpcChannel handles this for you: "When a GrpcChannel is created its internal SocketsHttpHandler is automatically configured to create additional HTTP/2 connections." The gRPC client factory (AddGrpcClient) configures its handler the same way.

That protection lives in the handler gRPC creates. Supply your own and you get the type's default. Here is Microsoft's keep-alive example from the gRPC performance guide, minus one line. It needs the Grpc.Net.Client package and using Grpc.Net.Client;:

var channel = GrpcChannel.ForAddress("https://orders.internal", new GrpcChannelOptions
{
    HttpHandler = new SocketsHttpHandler
    {
        PooledConnectionIdleTimeout = Timeout.InfiniteTimeSpan,
        KeepAlivePingDelay = TimeSpan.FromSeconds(60),
        KeepAlivePingTimeout = TimeSpan.FromSeconds(30),
        // EnableMultipleHttp2Connections = true,   <- the line that decides
    }
});

The guide also warns that keep-alive pings need the server's support. Its example includes the fourth line, so check that it survived in your copy. The same goes for AddGrpcClient(...).ConfigurePrimaryHttpMessageHandler(() => new SocketsHttpHandler { ... }), which replaces the handler the gRPC factory configured.

YARP requests HTTP/2 by default and gets it from HTTPS destinations that support it. Its default client factory sets EnableMultipleHttp2Connections to true, and YARP's documentation asks custom factories to do the same.

Any HttpClient you have switched to HTTP/2 yourself is exposed too, whether through DefaultRequestVersion, RequestVersionOrHigher or a Version on each request.

Nine scenarios

We sent bursts of requests to a real Kestrel server on localhost. Each request awaited a one-second delay on the server, and we changed one thing at a time. The HTTP/2 client used the keep-alive handler above, without the fourth line.

With up to 100 requests at once, one HTTP/2 connection kept pace with the server: one second. At 150, the slowest answer took twice as long. A load test that stays under the server's stream limit will never catch this.

Errors the server never saw

A queue turns into errors when it meets a timeout. We repeated the 300-request burst with HttpClient.Timeout set to 1.5 seconds. 100 requests succeeded. 200 failed with TaskCanceledException, and 100 of those never reached the server.

The server's http.server.request.duration metric recorded 200 requests, none longer than 1.01 seconds. At the ASP.NET Core template's log levels, the server logged nothing about any of it. Kestrel does report the limit one level down: at Debug, the Microsoft.AspNetCore.Server.Kestrel.Http2 category logs Connection id "..." reached the maximum number of concurrent HTTP/2 streams allowed. In our lab, that line appeared only after we turned Debug on, and it appeared with multiple connections enabled too: it says a connection reached the limit, and the client's metric says whether anyone waited.

The client keeps a better record. .NET 8 and later publish http.client.request.time_in_queue, "the amount of time requests spent on a queue waiting for an available connection". In our runs it recorded the wait every time: up to 2.02 seconds in the 300-request burst, and up to 1.5 seconds for the requests that timed out while still in the queue.

The fix, and what it does not limit

Setting EnableMultipleHttp2Connections = true removed the wait. The client opened two connections for 150 requests and three for 300, and every answer arrived in 1.01 seconds.

The fix has no ceiling built in. SocketsHttpHandler applies MaxConnectionsPerServer only to HTTP/1.1 connections; with it set to 2, we still got three HTTP/2 connections. While calls are waiting and every connection is full, the handler keeps opening connections, one at a time. Keeping 1,000 calls in flight against a 100-stream limit takes ten. If the service behind it needs protection, cap concurrency in your own code.

Raising the server's limit also removed the queue. With MaxStreamsPerConnection = 300, all 300 requests ran at once on one connection. The gRPC guide advises against this: "Too many streams on a single HTTP/2 connection introduces new performance issues", including thread contention and packet loss that blocks every call on the connection. And it is an option only when you own the server; the client honours whatever limit the server announces.

Check your own services

1. Search for new SocketsHttpHandler, HttpHandler = and ConfigurePrimaryHttpMessageHandler(. For each handler that uses HTTP/2, set EnableMultipleHttp2Connections = true, or make a deliberate choice to keep one connection.
2. Search for DefaultRequestVersion, HttpVersion.Version20 and RequestVersionOrHigher. Those clients can end up on HTTP/2; the response's Version or the client metrics' network.protocol.version tag tells you which ones do.
3. If you use YARP, look for EnableMultipleHttp2Connections set to false in cluster configuration, and check any custom IForwarderHttpClientFactory.
4. Collect http.client.request.time_in_queue from the System.Net.Http meter, broken down by network.protocol.version. Seconds recorded for version 2 are seconds spent before your server sees the request.
5. Compare the client's http.client.request.duration with the server's http.server.request.duration for the same calls. A gap that grows with load means time is going somewhere between the two, and time_in_queue tells you whether it is this queue.
6. Once you have enabled multiple connections, check how many the client opens at peak load, and add your own concurrency limit if the downstream service needs one.

What we measured

• One HTTP/2 connection, 300 requests at once: the server handled 100 at a time. The slowest answer took 3.04 seconds, though no request spent more than 1.01 seconds on the server.
• With a 1.5-second client timeout: 200 errors, 100 of them for requests the server never received.
• With EnableMultipleHttp2Connections = true, the client opened three connections and every answer arrived within 1.01 seconds. MaxConnectionsPerServer did not cap them.
• http.client.request.time_in_queue captured the wait in every scenario.

HTTP/2 lets one connection do the work of many. With the handler's defaults, that one connection is also the limit.

How we ran this

We used .NET 10.0.14 (SDK 10.0.403) and the ASP.NET Core shared framework, with no NuGet packages. A real Kestrel server and a real SocketsHttpHandler ran in one process on localhost, using HTTP/2 over cleartext with prior knowledge. The server endpoint awaits a one-second Task.Delay. For each scenario we created a new handler and sent one warm-up request before the burst. A MeterListener read the runtime's metrics in process. On localhost without TLS a new connection cost us a few milliseconds at most; over a real network, each new HTTP/2 connection needs several round trips (TCP, TLS, HTTP/2 setup) before its first call. We did not run gRPC or YARP; what we say about them comes from Microsoft's documentation and their source code. We compiled the handler part of the gRPC sample with warnings as errors; the channel part needs Grpc.Net.Client, which we did not install.

We ran every scenario six times. The counts matched in every run. The timings quoted here come from one run and moved by at most 0.03 seconds between runs.

One thing surprised us along the way. HttpClient.DefaultRequestVersion applies only to methods such as GetAsync. A request built by hand and passed to SendAsync keeps its own version, HTTP/1.1, so our first attempt sent HTTP/1.1 to an HTTP/2-only endpoint.