Combining Polly Policies: Advanced Resilience

Real microservices rarely use a single Polly policy in isolation. Combining retry, circuit breaker, timeout, bulkhead, and fallback policies creates powerful resilience strategies that handle multiple failure modes. However, policy composition order is critical: wrapping policies in the wrong order can create race conditions, infinite retries, or timeout conflicts. This guide teaches composition patterns used in production systems.

Policy Wrapping Order Matters

When you wrap multiple policies, they nest like layers of an onion. The outermost policy executes first; the innermost policy executes last. Order determines execution flow and timing:

Execution: Outer → Middle → Inner → Target Operation

Consider a real scenario: you retry a network call, but each retry has its own timeout, and you fallback if all retries fail and the circuit is open. The correct nesting is:

var fallback = Policy<T>.Handle<Exception>().FallbackAsync(...);
var circuitBreaker = Policy<T>.Handle<Exception>().CircuitBreakerAsync(...);
var timeout = Policy.TimeoutAsync<T>(duration);
var retry = Policy<T>.Handle<Exception>().WaitAndRetryAsync(...);

// Wrap order: fallback → circuit breaker → timeout → retry
var combined = Policy.WrapAsync(fallback, circuitBreaker, timeout, retry);

Execution order for a request:

1. Fallback policy checks if the wrapped policy throws; if yes, returns fallback.
2. Circuit breaker checks state (Closed, Open, Half-Open).
3. Timeout enforces a per-retry deadline.
4. Retry attempts the operation (and repeats the timeout/circuit breaker checks).

The "Timeout per Retry" Pattern

Each retry should have its own timeout, not a global timeout for all retries. Use this composition:

using Polly;
using System;
using System.Net.Http;
using System.Threading.Tasks;

var retryPolicy = Policy
    .Handle<OperationCanceledException>()
    .Or<HttpRequestException>()
    .WaitAndRetryAsync(
        retryCount: 3,
        sleepDurationProvider: attempt => 
            TimeSpan.FromMilliseconds(Math.Pow(2, attempt) * 100),
        onRetryAsync: async (outcome, span, attemptNumber, context) =>
        {
            Console.WriteLine($"Retry {attemptNumber} after {span.TotalMilliseconds}ms");
            await Task.CompletedTask;
        }
    );

var timeoutPolicy = Policy
    .TimeoutAsync<HttpResponseMessage>(
        TimeSpan.FromSeconds(5),
        timeoutStrategy: TimeoutStrategy.Optimistic
    );

// Correct order: timeout wraps retry
var combined = Policy.WrapAsync(timeoutPolicy, retryPolicy);

using var client = new HttpClient();
var response = await combined.ExecuteAsync(async (cancellationToken) =>
    await client.GetAsync("https://api.example.com/data", cancellationToken)
);

Timing:

• Attempt 1: 5-second timeout, fails, waits 100ms.
• Attempt 2: 5-second timeout, fails, waits 200ms.
• Attempt 3: 5-second timeout, succeeds.

Total time: ~15-20 seconds maximum. Each attempt gets a fresh 5-second window.

Wrong order (don't do this):

// WRONG: retry wraps timeout
var wrong = Policy.WrapAsync(retryPolicy, timeoutPolicy);
// This gives only one global 5-second timeout for all 3 retries combined.

Bulkhead + Circuit Breaker + Timeout + Retry + Fallback

A production-grade resilience policy for a microservice call:

using Polly;
using System;
using System.Net.Http;
using System.Threading.Tasks;

// 1. Bulkhead isolation (limit concurrent calls to this service)
var bulkheadPolicy = Policy.BulkheadAsync<HttpResponseMessage>(
    maxParallelization: 16,
    maxQueuingActions: 10,
    onBulkheadRejectedAsync: async (context) =>
    {
        Console.WriteLine("Bulkhead rejected request; service at capacity.");
        await Task.CompletedTask;
    }
);

// 2. Retry for transient failures
var retryPolicy = Policy
    .Handle<HttpRequestException>()
    .Or<TimeoutException>()
    .WaitAndRetryAsync<HttpResponseMessage>(
        retryCount: 2,
        sleepDurationProvider: attempt => 
            TimeSpan.FromMilliseconds(100 * attempt),
        onRetryAsync: async (outcome, span, attemptNumber, context) =>
        {
            Console.WriteLine($"Retry {attemptNumber}");
            await Task.CompletedTask;
        }
    );

// 3. Timeout per retry
var timeoutPolicy = Policy
    .TimeoutAsync<HttpResponseMessage>(
        TimeSpan.FromSeconds(5),
        timeoutStrategy: TimeoutStrategy.Optimistic
    );

// 4. Circuit breaker for sustained failures
var circuitBreakerPolicy = Policy
    .Handle<HttpRequestException>()
    .Or<TimeoutException>()
    .CircuitBreakerAsync<HttpResponseMessage>(
        handledEventsAllowedBeforeBreaking: 5,
        durationOfBreak: TimeSpan.FromSeconds(30),
        onBreakAsync: async (outcome, duration) =>
        {
            Console.WriteLine($"Circuit opened for {duration.TotalSeconds}s");
            await Task.CompletedTask;
        }
    );

// 5. Fallback for final failures
var fallbackPolicy = Policy<HttpResponseMessage>
    .Handle<Exception>()
    .FallbackAsync<HttpResponseMessage>(
        fallbackAction: async (context) =>
        {
            Console.WriteLine("Fallback: returning cached response");
            await Task.CompletedTask;
            return new HttpResponseMessage(System.Net.HttpStatusCode.OK)
            {
                Content = new StringContent("[Cached Data from Last Known Good State]")
            };
        }
    );

// Wrap all policies: bulkhead → fallback → circuit breaker → timeout → retry
var resilientPolicy = Policy.WrapAsync(
    bulkheadPolicy,
    fallbackPolicy,
    circuitBreakerPolicy,
    timeoutPolicy,
    retryPolicy
);

using var client = new HttpClient();
try
{
    var response = await resilientPolicy.ExecuteAsync(async (cancellationToken) =>
    {
        return await client.GetAsync("https://api.example.com/data", cancellationToken);
    });
    Console.WriteLine($"Success: {response.StatusCode}");
}
catch (BulkheadRejectedException)
{
    Console.WriteLine("Request rejected due to bulkhead limit");
}

Execution flow:

1. Bulkhead grants a slot or queues/rejects.
2. Fallback wraps the next layer.
3. Circuit breaker checks state.
4. Timeout enforces 5 seconds per attempt.
5. Retry repeats up to 2 times.
6. Target: HTTP GET.
7. If all fail, fallback returns cached response.

Policy Composition with Policy.WrapAsync

The Policy.WrapAsync method accepts multiple policies in order:

var combined = Policy.WrapAsync(
    outerPolicy,
    middlePolicy,
    ...,
    innerPolicy
);

Tip: Document the wrapping order in a comment for future maintainers.

// Resilience policy wrapping order (outer to inner):
// 1. Fallback (catch all failures)
// 2. Circuit Breaker (protect against sustained failures)
// 3. Timeout (prevent hanging)
// 4. Retry (handle transient glitches)
var resilientPolicy = Policy.WrapAsync(
    fallbackPolicy,
    circuitBreakerPolicy,
    timeoutPolicy,
    retryPolicy
);

Testing Combined Policies

Unit tests verify behavior of combined policies:

using Polly;
using Xunit;
using System;
using System.Threading.Tasks;

public class CombinedPoliciesTests
{
    [Fact]
    public async Task RetryThenFallbackOnAllFailures()
    {
        var retryCount = 0;
        var retryPolicy = Policy
            .Handle<Exception>()
            .WaitAndRetryAsync(retryCount: 2, sleepDurationProvider: _ => TimeSpan.Zero);

        var fallbackPolicy = Policy<string>
            .Handle<Exception>()
            .FallbackAsync<string>("Fallback");

        var combined = Policy.WrapAsync(fallbackPolicy, retryPolicy);

        var result = await combined.ExecuteAsync(() =>
        {
            retryCount++;
            return Task.FromException<string>(new Exception("Always fails"));
        });

        // Retried twice (2 retries + 1 initial = 3 attempts), then fallback triggered
        Assert.Equal(3, retryCount);
        Assert.Equal("Fallback", result);
    }

    [Fact]
    public async Task TimeoutPerRetryApplied()
    {
        var attempts = 0;
        var retryPolicy = Policy
            .Handle<OperationCanceledException>()
            .WaitAndRetryAsync(retryCount: 2, sleepDurationProvider: _ => TimeSpan.Zero);

        var timeoutPolicy = Policy
            .TimeoutAsync(TimeSpan.FromMilliseconds(100));

        var combined = Policy.WrapAsync(timeoutPolicy, retryPolicy);

        // Simulate slow operation that times out on each retry
        await Assert.ThrowsAsync<OperationCanceledException>(
            () => combined.ExecuteAsync(async (cancellationToken) =>
            {
                attempts++;
                await Task.Delay(200, cancellationToken);
            })
        );

        // Each retry got its own timeout window
        Assert.Equal(1, attempts); // First attempt timed out; retries did not proceed
    }
}

Common Pitfalls and How to Avoid Them

Pitfall	Problem	Solution
Wrong wrap order	Circuit breaker closes before retry exhausted	Document order; use outermost-first wrapping
Single timeout for all retries	Effective timeout = total time, not per-retry	Timeout wraps retry (timeout outer)
Missing bulkhead	Resource exhaustion from one slow service	Add bulkhead as outermost layer
Retry on non-transient failures	Wastes time and masks real bugs	Only retry on HttpRequestException, TimeoutException
No fallback	User sees error instead of degraded response	Add fallback for user-facing operations

Key Takeaways

• Policy composition order is critical: Outermost executes first, innermost executes last.
• Recommended order: Bulkhead (outermost) → Fallback → Circuit Breaker → Timeout → Retry (innermost).
• Timeout per retry: Timeout should wrap retry so each attempt gets its own deadline.
• Test composition: Verify behavior with combined policies; single-policy tests are insufficient.
• Document your wrapping: Comments explaining the order aid future maintenance.

Frequently Asked Questions

What if my fallback itself fails?

The fallback exception propagates. Wrap the fallback factory in try-catch or nest another fallback. For critical paths, ensure fallback never throws (return safe defaults or empty collections).

Can I use the same policy instance for multiple calls?

Yes, policies are thread-safe and reusable. Create a single instance (static field or singleton) and reuse it across requests. This is more efficient than creating new policies per call.

Should all microservice calls use the same policy?

No. Different endpoints have different characteristics. A database query needs a faster timeout than an external API. Create separate policies per logical service boundary.

Can I nest policies other than WrapAsync?

WrapAsync is the standard composition. Advanced scenarios use policy registries (named policies) or dependency injection to manage multiple policies. Polly Resilience Pipeline (.NET 8+) provides a newer abstraction.

Further Reading

• Polly Policy.Wrap Documentation
• Microservices Resilience Patterns
• Resilience Pipeline .NET 8
• Release It! Chapter 7: Composing Resilience Patterns