Asked Three Times, Answered Never

In our .NET 10 lab, a simulated language model needed 40 seconds to write a 780-token answer. Our application asked once. The model received three requests and generated 471 tokens. None reached the application, which got a TimeoutRejectedException at 30 seconds.

We hadn't configured a timeout ourselves. The model client was an ordinary HttpClient from IHttpClientFactory, and the app was configured like the Aspire service defaults. If your team added AI to an app like this, ask one question: which resilience policy does our model client inherit?

We tried what seemed like the obvious fix: giving the model client its own handler with a 90-second attempt timeout. Nothing changed. We still saw three requests and a failure at 30 seconds.

Streaming the same answer gave us the full response, outside every timeout. But when we stalled the stream after 100 tokens, the application kept waiting. Our lab finally gave up at 120 seconds.

We ran every scenario twice, with the same outcomes each time. Here is what we measured and the configuration that fixed it.

Where the policy comes from

Our previous field note, "The Retry You Never Wrote", traced this policy to one line in the Service Defaults project that the Aspire template generates: http.AddStandardResilienceHandler(), inside ConfigureHttpClientDefaults.

Every HttpClient created through IHttpClientFactory gets the standard resilience handler. That includes the plain HttpClient supplied by the container: the one we resolved from DI in our lab had the handler too. Libraries that ask DI for an HttpClient inherit the same policy.

For a model call, these are the defaults that matter:

A chat completion is a POST, and a long answer can take far longer than 10 seconds: ours took 40. When an attempt times out, the handler treats the failure as transient and sends the POST again.

Some AI libraries leave HTTP policy to the HttpClient you supply. Semantic Kernel's OpenAI connector does this: when given an HttpClient, it sets its own retry count to zero and its network timeout to infinite. Only the HttpClient's policies then apply. If you register it with a model id and an API key and pass no client, it asks the container for one. If you pass it an OpenAIClient instead, that client's own settings apply. We did not run Semantic Kernel; this comes from its source code on the main branch, commit df1cb33 of November 5, 2026.

What the model saw

We sent the same request to our simulated model, which takes one second to produce the first token, then generates 20 tokens per second. Each attempt that reaches a provider is a request the provider processes. Our model counts the tokens it produces and stops generating when the client disconnects. Without streaming, it sends the response, headers included, only when the answer is complete.

Short answers get through. A long answer gets cut off at 10 seconds, about 180 tokens in, and the request is sent again. The third attempt exhausted the total time budget after 109 tokens. A slow first token fails the same way: our model sends its response headers with the first token, and each attempt times out before that token arrives.

The fixes that look right

We tried the changes a team reaches for first, one at a time.

Turning retries off stops the duplicate requests, but the call still fails at 10 seconds, just as it did in our previous note. Raising the total timeout makes things worse: each attempt still gets cut off at 10 seconds, so the extra time allows a fourth request before the call finally fails.

The third row surprised us. Giving a named client its own AddStandardResilienceHandler does not replace the defaults. It adds a second handler inside them. We found two ResilienceHandler instances in the client's handler chain, with the defaults on the outside. The outer handler still enforced its 10-second attempt timeout and retried the request. Our 90-second settings never got a chance to help.

The logs show which handler was responsible:

fail: Polly[0] Resilience event occurred. EventName: 'OnTimeout', Source: '-standard//Standard-AttemptTimeout'
warn: Polly[0] Resilience event occurred. EventName: 'OnRetry', Source: '-standard//Standard-Retry'

Our handler is named llm-standard, but both events came from -standard. The leading dash comes from the empty client name used to name the defaults' pipeline.

To fix this, remove the inherited handler before adding your own. Put this in Program.cs, where builder is the WebApplicationBuilder, with the using at the top of the file:

using Microsoft.Extensions.Http.Resilience;

#pragma warning disable EXTEXP0001 // RemoveAllResilienceHandlers is marked experimental
builder.Services.AddHttpClient("llm")
    .RemoveAllResilienceHandlers()
    .AddStandardResilienceHandler(o =>
    {
        o.Retry.DisableForUnsafeHttpMethods(); // never resend a generation
        o.AttemptTimeout.Timeout = TimeSpan.FromSeconds(90);
        o.TotalRequestTimeout.Timeout = TimeSpan.FromSeconds(180);
        o.CircuitBreaker.SamplingDuration = TimeSpan.FromSeconds(180); // >= 2x attempt timeout
    });
#pragma warning restore EXTEXP0001

With that change, we got one request, all 780 tokens, and HTTP 200 at 40 seconds. Set the attempt timeout to the longest you are willing to wait for response headers. With our model, that meant waiting for the whole generation; 90 is simply the value we used. The handler validates these settings: the total timeout must be at least the attempt timeout, and the circuit breaker's sampling duration must be at least twice the attempt timeout.

Calling RemoveAllResilienceHandlers() on its own also worked, but it left the client without a timeout. The defaults had already set HttpClient.Timeout to infinite, and removing the handler does not restore it.

This fix only helps if the model call uses the configured client, through IHttpClientFactory.CreateClient("llm") or a typed client. The plain HttpClient from DI is the factory's default client and keeps the defaults. If your AI library takes an HttpClient, pass it this one explicitly.

The timeouts stop at the headers

With HttpCompletionOption.ResponseHeadersRead, the same 40-second answer completed under the inherited defaults: one request, 780 tokens, finished at 40.0 seconds.

The handler's only attempt ended at 1.0 second, when the response headers arrived. Its timeouts apply until it returns the response. The application reads the body after that, outside the handler's timeouts. With HttpClient.Timeout set to infinite, that read has no timeout.

This also happens without streaming. We changed our model to send the headers immediately and the JSON body at the end. A plain SendAsync under the inherited defaults received all 780 tokens at 40.0 seconds, even though the handler's attempt ended at 0.1 seconds. HttpClient buffers the body after the handlers have returned. Whether a non-streaming call gets cut off at 10 seconds therefore depends on when your provider sends its headers. You can check the timing in the Received HTTP response headers after line in the HttpClient log.

That gives a long stream time to finish, but it also lets a stalled stream hang. We made our model stop after 100 tokens and leave the connection open. The application had received all 100 tokens by 6.0 seconds. It was still waiting at 120 seconds, when our lab's watchdog cancelled it. No timeout fired. The last log entry, at 1.0 second, said the request had ended with status 200.

A streaming client needs a separate deadline for each line it reads. With http, request and a CancellationToken ct already in scope:

using var response = await http.SendAsync(request, HttpCompletionOption.ResponseHeadersRead, ct);
response.EnsureSuccessStatusCode();
using var reader = new StreamReader(await response.Content.ReadAsStreamAsync(ct));
while (true)
{
    string? line;
    using (var idle = CancellationTokenSource.CreateLinkedTokenSource(ct))
    {
        idle.CancelAfter(TimeSpan.FromSeconds(15)); // the next line must arrive within 15 s
        line = await reader.ReadLineAsync(idle.Token);
    }
    if (line is null) break;
    // handle one server-sent event line
}

With a 15-second deadline per line, our application gave up on the stalled stream at 21.0 seconds, 15 seconds after the last token, with a TaskCanceledException. The deadline applies to each complete line: keep-alive lines reset it without any model progress, and a single line that trickles in for more than 15 seconds fails it. To also limit the total time for an answer, cancel ct after a deadline that covers both sending the request and reading the body.

When the model says slow down

The default policy also retries HTTP 429 responses. When our model returned 429 for every request without a Retry-After header, the handler sent four requests in 7.4 seconds before passing the last 429 back to the caller. When the 429 included Retry-After: 20, the handler waited the 20 seconds and tried once more, then failed at 30 seconds with TimeoutRejectedException. Each caller using this policy can send up to three extra requests to a provider that is already limiting it.

Check your own services

• Check how your model client is created. If your service defaults call AddStandardResilienceHandler() inside ConfigureHttpClientDefaults, as the Aspire template does, every factory-created client inherits it, including the HttpClient supplied by DI, unless its own configuration removes it.
• If your AI library accepts an HttpClient, check whether it disables its own retries and timeout. Semantic Kernel's OpenAI connector does, leaving the inherited policy in charge.
• Look for named clients that use AddStandardResilienceHandler in an app that also applies defaults. Unless you call RemoveAllResilienceHandlers() first, the named client's handler runs inside the defaults.
• Look through your logs for OnTimeout or OnRetry with Source: '-standard// alongside model calls. Those entries show the inherited policy at work.
• Check when your provider sends the response headers. The inherited timeouts cover only the wait for them: 10 seconds per attempt, 30 in total. Nothing covers the body that follows, so give the whole call a deadline, and a streaming call a deadline per line.

What we measured

• With the inherited defaults, a request for a 40-second answer reached the model three times and generated 471 tokens. None reached the caller, which got an exception at 30 seconds.
• Giving a named client its own longer timeouts on top of the defaults made no difference. Raising the total timeout led to four requests and a failure at 46.8 seconds.
• Calling RemoveAllResilienceHandlers() and then adding a handler sized for the model produced the full answer in one request.
• The handler's timeouts stop at the response headers, with or without streaming. A complete stream passed, but a stalled stream kept waiting until our lab cancelled it.

The standard handler makes sense for quick calls. Model calls need a timeout and retry policy chosen for the time they actually take.

How we ran this

We ran everything in one ASP.NET Core process on loopback, using .NET 10.0.14 and Microsoft.Extensions.Http.Resilience 10.11.0 (Polly 8.4.2). Packages were restored from the local cache with a lock file. A local endpoint simulated the language model: it waited one second, then produced 20 tokens per second, either as server-sent events or as one JSON response, sent when complete (with the headers sent first in one scenario). It stopped when the client disconnected. Each scenario had its own client configuration, with the defaults applied exactly as in the Aspire template, minus service discovery. A handler inside the resilience pipeline recorded every attempt. Logging used the ASP.NET Core template's levels.

We ran 13 scenarios twice, then two more twice after review: headers sent first, and the per-line deadline as printed above. The outcomes and request counts matched across runs. Retry jitter shifted timings by up to 1.8 seconds and changed the number of tokens generated before a cut. The numbers in this article come from the first run of each set.