In modern distributed systems, a single user action can trigger a cascade of requests across multiple microservices. When something goes wrong, tracing that single action through a sea of log entries can feel like finding a needle in a haystack. This is where correlation IDs become an indispensable tool for observability.

A correlation ID is a unique identifier that follows a request from start to finish, even as it hops between services. By including this ID in every log message related to that request, you can easily filter and piece together the entire transaction.

The most effective way to implement this in ASP.NET Core is with custom middleware. Let’s dive into how to build it.

Why Middleware is the Perfect Solution

The ASP.NET Core request pipeline is a series of components, or middleware, that process an HTTP request. By creating our own middleware and placing it early in the pipeline, we can ensure that:

  1. Every request is intercepted: No request gets processed without a correlation ID.
  2. Logic is centralized: We avoid cluttering our controllers or services with repetitive logging code.
  3. It’s set up once: Once the middleware is registered, the correlation ID is available for the entire scope of the request.

This approach is clean, efficient, and aligns perfectly with the ASP.NET Core design philosophy.

Building the Correlation ID Middleware

First, we’ll create the middleware component itself. This class will inspect incoming request headers for an existing correlation ID. If one isn’t found, it will generate a new one.

Create a new file named CorrelationIdMiddleware.cs:

public class CorrelationIdMiddleware
{
    private readonly RequestDelegate _next;
    private readonly ILogger<CorrelationIdMiddleware> _logger;
    private const string CorrelationIdHeaderName = "X-Correlation-ID";

    public CorrelationIdMiddleware(RequestDelegate next, ILogger<CorrelationIdMiddleware> logger)
    {
        _next = next;
        _logger = logger;
    }

    public async Task InvokeAsync(HttpContext context)
    {
        // Try to get the correlation ID from the request header.
        var correlationId = GetOrGenerateCorrelationId(context);

        // Add the correlation ID to the logging scope.
        // Any logs generated within this scope will have the CorrelationId property.
        using (_logger.BeginScope("{@CorrelationId}", correlationId))
        {
            // Add the correlation ID to the response headers.
            // This allows the client to see the ID for its request.
            context.Response.OnStarting(() =>
            {
                if (!context.Response.Headers.ContainsKey(CorrelationIdHeaderName))
                {
                    context.Response.Headers.Add(CorrelationIdHeaderName, correlationId);
                }
                return Task.CompletedTask;
            });

            // Pass control to the next middleware in the pipeline.
            await _next(context);
        }
    }

    private static string GetOrGenerateCorrelationId(HttpContext context)
    {
        if (context.Request.Headers.TryGetValue(CorrelationIdHeaderName, out var correlationIdValues) &&
            correlationIdValues.FirstOrDefault() is { Length: > 0 } correlationId)
        {
            return correlationId;
        }

        return Guid.NewGuid().ToString();
    }
}

Let’s break down the key parts:

  • Constructor: We inject the RequestDelegate to call the next middleware and ILogger to use its scoping feature.
  • GetOrGenerateCorrelationId: This helper method checks the X-Correlation-ID header. If the header exists and has a value, we use it. This is crucial for propagating the ID across service boundaries. If not, we generate a new Guid.
  • _logger.BeginScope(...): This is the most important line. It creates a logging scope. Any log created within the using block will automatically be enriched with the CorrelationId property. The @ symbol in the scope data tells Serilog to serialize the object, which is good practice.
  • context.Response.OnStarting(...): We attach the correlation ID to the response headers. This is helpful for clients and developers, as they can correlate a specific response with the logs on the server side.

Callout: Scoped Logging is a Game Changer

Using ILogger.BeginScope is far superior to passing the correlation ID as a parameter to every method. It leverages the ambient context of the request, making the ID available to any service resolved from the dependency injection container during that request without any extra code. This keeps your application logic clean and focused on its primary responsibilities.

Registering the Middleware

To make registration cleaner, we can create a simple extension method.

Create a new file named MiddlewareExtensions.cs:

public static class MiddlewareExtensions
{
    public static IApplicationBuilder UseCorrelationIdMiddleware(this IApplicationBuilder builder)
    {
        return builder.UseMiddleware<CorrelationIdMiddleware>();
    }
}

Now, register it in your Program.cs file. Make sure to place it early in the pipeline, before controllers or other middleware that might perform logging.

// Program.cs

var builder = WebApplication.CreateBuilder(args);

// ... Add services ...
builder.Services.AddControllers();

var app = builder.Build();

// Register the custom middleware
app.UseCorrelationIdMiddleware();

// ... Other middleware ...
app.UseHttpsRedirection();
app.UseAuthorization();
app.MapControllers();

app.Run();

Configuring a Logger to Use the Scope

The middleware adds the correlation ID to the logging scope, but your logger needs to be configured to read from that scope and include it in the final output. Here’s how you can do it with a popular structured logging provider, Serilog.

First, install the necessary NuGet packages:

dotnet add package Serilog.AspNetCore
dotnet add package Serilog.Sinks.Console

Next, configure Serilog in Program.cs. Replace the default logging with Serilog and configure its output template.

using Serilog;

var builder = WebApplication.CreateBuilder(args);

// Configure Serilog
builder.Host.UseSerilog((context, configuration) =>
    configuration.ReadFrom.Configuration(context.Configuration)
        .Enrich.FromLogContext() // This is the key part!
        .WriteTo.Console(
            outputTemplate: "[{Timestamp:HH:mm:ss} {Level:u3}] {Message:lj} {Properties:j}{NewLine}{Exception}"));

builder.Services.AddControllers();

var app = builder.Build();

app.UseCorrelationIdMiddleware();

// ... rest of the pipeline ...

app.Run();

The crucial parts are:

  • .Enrich.FromLogContext(): This tells Serilog to pull in all properties from the active logging scope (like our CorrelationId).
  • {Properties:j}: This token in the outputTemplate instructs Serilog to render all enriched properties (including our ID) as a JSON object in the log message.

Seeing It in Action

Now, let’s create a simple API controller to test our setup.

[ApiController]
[Route("[controller]")]
public class WeatherForecastController : ControllerBase
{
    private readonly ILogger<WeatherForecastController> _logger;

    public WeatherForecastController(ILogger<WeatherForecastController> logger)
    {
        _logger = logger;
    }

    [HttpGet]
    public IActionResult Get()
    {
        _logger.LogInformation("Fetching weather forecast.");
        
        // Some business logic here...

        _logger.LogInformation("Successfully retrieved weather forecast.");

        return Ok("Success");
    }
}

Run your application and make a request to the /weatherforecast endpoint using a tool like Postman or curl.

Request 1: Without providing a correlation ID

curl http://localhost:5000/weatherforecast -v

You’ll see a log entry in your console like this:

[11:30:05 INF] Fetching weather forecast. {"CorrelationId":"d4f5b5f5-c3e1-4b7c-8a9d-123456789abc"}
[11:30:05 INF] Successfully retrieved weather forecast. {"CorrelationId":"d4f5b5f5-c3e1-4b7c-8a9d-123456789abc"}

Notice how both log messages share the same, newly generated CorrelationId. The response will also contain the header X-Correlation-ID: d4f5b5f5-c3e1-4b7c-8a9d-123456789abc.

Request 2: Providing a correlation ID

Now, let’s pretend our request is coming from another service that has already assigned a correlation ID.

curl http://localhost:5000/weatherforecast -H "X-Correlation-ID: my-custom-trace-id-123" -v

The console output will now use the ID we provided:

[11:31:10 INF] Fetching weather forecast. {"CorrelationId":"my-custom-trace-id-123"}
[11:31:10 INF] Successfully retrieved weather forecast. {"CorrelationId":"my-custom-trace-id-123"}

Success! Our middleware correctly reads the existing ID and applies it, ensuring end-to-end traceability.

Conclusion

Implementing correlation IDs via custom middleware is a powerful pattern in ASP.NET Core. It provides a robust, centralized, and non-invasive way to enhance your application’s observability. With just one piece of middleware, you can enrich every log entry with a traceable identifier, dramatically simplifying debugging and analysis in complex, distributed environments.

References