Code-based Instrumentation of .NET using OpenTelemetry

Learn how to instrument .NET applications using OpenTelemetry SDK for comprehensive telemetry data generation and integration with Edge Delta.

8 minute read

Overview

Code-based instrumentation in .NET with OpenTelemetry involves manually writing code to generate telemetry data using the OpenTelemetry .NET SDK. The .NET implementation uses the existing System.Diagnostics API, repurposing ActivitySource and Activity classes to be OpenTelemetry-compliant. This means you can use familiar .NET constructs while generating OpenTelemetry-standard telemetry data. This approach provides fine-grained control over what telemetry is captured, allowing you to instrument custom business logic and application-specific operations.

Prerequisites

Ensure that you have:

.NET SDK: Version 6.0 or later (8.0+ recommended)
IDE: Visual Studio 2022, VS Code, or Rider
NuGet Package Manager: For installing dependencies
Edge Delta: An account and pipeline with an OTLP input node

Setup OpenTelemetry

1. Install Required Dependencies

Install the core OpenTelemetry packages via NuGet:

dotnet add package OpenTelemetry.Extensions.Hosting
dotnet add package OpenTelemetry.Instrumentation.AspNetCore
dotnet add package OpenTelemetry.Exporter.OpenTelemetryProtocol

These packages include:

OpenTelemetry.Extensions.Hosting: Integration with .NET dependency injection and hosting
OpenTelemetry.Instrumentation.AspNetCore: Automatic instrumentation for ASP.NET Core
OpenTelemetry.Exporter.OpenTelemetryProtocol: OTLP exporter for sending telemetry to Edge Delta

For console applications, also install:

dotnet add package OpenTelemetry

2. Configure OpenTelemetry in ASP.NET Core

In your Program.cs, configure OpenTelemetry using the dependency injection system:

using OpenTelemetry;
using OpenTelemetry.Exporter;
using OpenTelemetry.Resources;
using OpenTelemetry.Trace;
using OpenTelemetry.Metrics;
using OpenTelemetry.Logs;

var builder = WebApplication.CreateBuilder(args);

// Configure resource attributes
var serviceName = "my-dotnet-service";
var serviceVersion = "1.0.0";

// Add OpenTelemetry services
builder.Services.AddOpenTelemetry()
    .ConfigureResource(resource => resource
        .AddService(
            serviceName: serviceName,
            serviceVersion: serviceVersion)
        .AddAttributes(new Dictionary<string, object>
        {
            ["deployment.environment"] = "production",
            ["service.namespace"] = "my-namespace"
        }))
    .WithTracing(tracing => tracing
        .AddSource(serviceName)
        .AddAspNetCoreInstrumentation()
        .AddHttpClientInstrumentation()
        .AddOtlpExporter(options =>
        {
            options.Endpoint = new Uri("http://localhost:4318/v1/traces");
            options.Protocol = OtlpExportProtocol.HttpProtobuf;
        }))
    .WithMetrics(metrics => metrics
        .AddMeter(serviceName)
        .AddAspNetCoreInstrumentation()
        .AddHttpClientInstrumentation()
        .AddRuntimeInstrumentation()
        .AddOtlpExporter(options =>
        {
            options.Endpoint = new Uri("http://localhost:4318/v1/metrics");
            options.Protocol = OtlpExportProtocol.HttpProtobuf;
        }));

// Add logging with OpenTelemetry
builder.Logging.AddOpenTelemetry(logging => logging
    .AddOtlpExporter(options =>
    {
        options.Endpoint = new Uri("http://localhost:4318/v1/logs");
        options.Protocol = OtlpExportProtocol.HttpProtobuf;
    }));

var app = builder.Build();

// Your middleware and endpoints
app.MapGet("/", () => "Hello OpenTelemetry!");

app.Run();

3. Configure OTLP Endpoints

The endpoint configuration depends on your deployment environment:

Kubernetes Environment:

.AddOtlpExporter(options =>
{
    options.Endpoint = new Uri("http://ed-data-supply-svc:4318/v1/traces");
    options.Protocol = OtlpExportProtocol.HttpProtobuf;
})

Non-Kubernetes (same host as Edge Delta agent):

.AddOtlpExporter(options =>
{
    options.Endpoint = new Uri("http://localhost:4318/v1/traces");
    options.Protocol = OtlpExportProtocol.HttpProtobuf;
})

Using gRPC (port 4317) instead of HTTP:

.AddOtlpExporter(options =>
{
    options.Endpoint = new Uri("http://localhost:4317");
    options.Protocol = OtlpExportProtocol.Grpc;
})

Note: When using HTTP/protobuf, include the signal-specific path (/v1/traces, /v1/metrics, /v1/logs). For gRPC, omit the path.

See Ingest Data from an OTLP Source for detailed Edge Delta configuration.

4. Configure for Console Applications

For non-ASP.NET Core applications, configure providers directly:

using OpenTelemetry;
using OpenTelemetry.Resources;
using OpenTelemetry.Trace;
using OpenTelemetry.Metrics;

var serviceName = "my-console-app";
var serviceVersion = "1.0.0";

// Configure TracerProvider
var tracerProvider = Sdk.CreateTracerProviderBuilder()
    .AddSource(serviceName)
    .ConfigureResource(resource => resource
        .AddService(serviceName, serviceVersion))
    .AddOtlpExporter(options =>
    {
        options.Endpoint = new Uri("http://localhost:4318/v1/traces");
        options.Protocol = OtlpExportProtocol.HttpProtobuf;
    })
    .Build();

// Configure MeterProvider
var meterProvider = Sdk.CreateMeterProviderBuilder()
    .AddMeter(serviceName)
    .ConfigureResource(resource => resource
        .AddService(serviceName, serviceVersion))
    .AddOtlpExporter(options =>
    {
        options.Endpoint = new Uri("http://localhost:4318/v1/metrics");
        options.Protocol = OtlpExportProtocol.HttpProtobuf;
    })
    .Build();

// Your application code here

// Cleanup
tracerProvider?.Dispose();
meterProvider?.Dispose();

Manual Instrumentation

Creating an Instrumentation Class

Create a class to hold your ActivitySource and Meter instances:

using System.Diagnostics;
using System.Diagnostics.Metrics;

public class Instrumentation : IDisposable
{
    internal const string ActivitySourceName = "my-dotnet-service";
    internal const string ActivitySourceVersion = "1.0.0";

    public Instrumentation()
    {
        ActivitySource = new ActivitySource(ActivitySourceName, ActivitySourceVersion);
        Meter = new Meter(ActivitySourceName, ActivitySourceVersion);
    }

    public ActivitySource ActivitySource { get; }
    public Meter Meter { get; }

    public void Dispose()
    {
        ActivitySource.Dispose();
        Meter.Dispose();
    }
}

builder.Services.AddSingleton<Instrumentation>();

Creating Spans (Activities)

In .NET, spans are called “Activities”. Use ActivitySource to create them:

public class OrderService
{
    private readonly ActivitySource _activitySource;

    public OrderService(Instrumentation instrumentation)
    {
        _activitySource = instrumentation.ActivitySource;
    }

    public async Task ProcessOrder(string orderId)
    {
        // Create an activity (span)
        using var activity = _activitySource.StartActivity("ProcessOrder");

        // Add tags (attributes)
        activity?.SetTag("order.id", orderId);
        activity?.SetTag("order.type", "standard");

        try
        {
            await ValidateOrder(orderId);
            await ChargePayment(orderId);
            await FulfillOrder(orderId);

            activity?.SetStatus(ActivityStatusCode.Ok);
        }
        catch (Exception ex)
        {
            activity?.SetStatus(ActivityStatusCode.Error, ex.Message);
            activity?.RecordException(ex);
            throw;
        }
    }

    private async Task ValidateOrder(string orderId)
    {
        // Nested activity
        using var activity = _activitySource.StartActivity("ValidateOrder");
        activity?.SetTag("order.id", orderId);

        // Validation logic
        await Task.Delay(50);
    }
}

Adding Tags (Attributes) to Activities

Enrich activities with contextual information:

activity?.SetTag("http.method", "GET");
activity?.SetTag("http.url", "/api/users");
activity?.SetTag("http.status_code", 200);
activity?.SetTag("user.id", userId);
activity?.SetTag("transaction.amount", 99.99);

Adding Events to Activities

Record events at specific points in time:

using var activity = _activitySource.StartActivity("ProcessPayment");

activity?.AddEvent(new ActivityEvent("PaymentValidationStarted"));

// Process payment
await ValidatePaymentMethod();

activity?.AddEvent(new ActivityEvent("PaymentValidationCompleted",
    tags: new ActivityTagsCollection
    {
        { "validation.result", "success" },
        { "validation.duration_ms", 123 }
    }));

Recording Exceptions

Capture exceptions in activities:

try
{
    await SomeOperation();
}
catch (Exception ex)
{
    activity?.RecordException(ex);
    activity?.SetStatus(ActivityStatusCode.Error, ex.Message);
    throw;
}

Creating Activities with Links

Link activities from different traces:

var links = new List<ActivityLink>
{
    new ActivityLink(parentContext1),
    new ActivityLink(parentContext2)
};

using var activity = _activitySource.StartActivity(
    "CombinedOperation",
    ActivityKind.Internal,
    parentContext: default,
    links: links);

Recording Metrics

Creating Metric Instruments

Define metrics in your instrumentation class:

public class Instrumentation : IDisposable
{
    public Instrumentation()
    {
        ActivitySource = new ActivitySource(ActivitySourceName, ActivitySourceVersion);
        Meter = new Meter(ActivitySourceName, ActivitySourceVersion);

        // Counter
        RequestCounter = Meter.CreateCounter<long>(
            "http.requests",
            unit: "{request}",
            description: "Number of HTTP requests");

        // Histogram
        RequestDuration = Meter.CreateHistogram<double>(
            "http.request.duration",
            unit: "ms",
            description: "Duration of HTTP requests");

        // UpDownCounter
        ActiveConnections = Meter.CreateUpDownCounter<int>(
            "http.active_connections",
            unit: "{connection}",
            description: "Number of active connections");

        // ObservableGauge
        ProcessMemory = Meter.CreateObservableGauge(
            "process.memory.usage",
            () => GC.GetTotalMemory(false),
            unit: "By",
            description: "Process memory usage");
    }

    public ActivitySource ActivitySource { get; }
    public Meter Meter { get; }

    public Counter<long> RequestCounter { get; }
    public Histogram<double> RequestDuration { get; }
    public UpDownCounter<int> ActiveConnections { get; }
    public ObservableGauge<long> ProcessMemory { get; }

    public void Dispose()
    {
        ActivitySource.Dispose();
        Meter.Dispose();
    }
}

Using Metrics in Your Code

public class ApiMiddleware
{
    private readonly Instrumentation _instrumentation;

    public ApiMiddleware(RequestDelegate next, Instrumentation instrumentation)
    {
        _next = next;
        _instrumentation = instrumentation;
    }

    public async Task InvokeAsync(HttpContext context)
    {
        var stopwatch = Stopwatch.StartNew();

        // Increment active connections
        _instrumentation.ActiveConnections.Add(1,
            new KeyValuePair<string, object?>("endpoint", context.Request.Path));

        try
        {
            await _next(context);

            // Increment request counter
            _instrumentation.RequestCounter.Add(1,
                new KeyValuePair<string, object?>("method", context.Request.Method),
                new KeyValuePair<string, object?>("status", context.Response.StatusCode));
        }
        finally
        {
            // Record duration
            stopwatch.Stop();
            _instrumentation.RequestDuration.Record(stopwatch.Elapsed.TotalMilliseconds,
                new KeyValuePair<string, object?>("method", context.Request.Method),
                new KeyValuePair<string, object?>("endpoint", context.Request.Path));

            // Decrement active connections
            _instrumentation.ActiveConnections.Add(-1,
                new KeyValuePair<string, object?>("endpoint", context.Request.Path));
        }
    }
}

Capturing Logs

Integrate OpenTelemetry with Microsoft.Extensions.Logging:

public class OrderController : ControllerBase
{
    private readonly ILogger<OrderController> _logger;
    private readonly ActivitySource _activitySource;

    public OrderController(
        ILogger<OrderController> logger,
        Instrumentation instrumentation)
    {
        _logger = logger;
        _activitySource = instrumentation.ActivitySource;
    }

    [HttpPost("orders")]
    public async Task<IActionResult> CreateOrder([FromBody] OrderRequest request)
    {
        using var activity = _activitySource.StartActivity("CreateOrder");
        activity?.SetTag("order.type", request.Type);

        // Logs are automatically correlated with the active trace
        _logger.LogInformation("Creating order for customer {CustomerId}", request.CustomerId);

        try
        {
            var orderId = await ProcessOrder(request);

            _logger.LogInformation(
                "Order created successfully: {OrderId}",
                orderId);

            return Ok(new { orderId });
        }
        catch (Exception ex)
        {
            _logger.LogError(ex,
                "Failed to create order for customer {CustomerId}",
                request.CustomerId);

            activity?.SetStatus(ActivityStatusCode.Error);
            return StatusCode(500);
        }
    }
}

Complete ASP.NET Core Example

Here’s a complete example instrumenting an ASP.NET Core Web API:

using OpenTelemetry;
using OpenTelemetry.Exporter;
using OpenTelemetry.Resources;
using OpenTelemetry.Trace;
using OpenTelemetry.Metrics;
using OpenTelemetry.Logs;
using System.Diagnostics;
using System.Diagnostics.Metrics;

var builder = WebApplication.CreateBuilder(args);

// Add instrumentation singleton
builder.Services.AddSingleton<Instrumentation>();

// Configure OpenTelemetry
var serviceName = "dice-api";
var serviceVersion = "1.0.0";

builder.Services.AddOpenTelemetry()
    .ConfigureResource(resource => resource
        .AddService(serviceName, serviceVersion)
        .AddAttributes(new Dictionary<string, object>
        {
            ["deployment.environment"] = builder.Environment.EnvironmentName
        }))
    .WithTracing(tracing => tracing
        .AddSource(serviceName)
        .AddAspNetCoreInstrumentation(options =>
        {
            options.RecordException = true;
        })
        .AddHttpClientInstrumentation()
        .AddOtlpExporter(options =>
        {
            options.Endpoint = new Uri("http://localhost:4318/v1/traces");
            options.Protocol = OtlpExportProtocol.HttpProtobuf;
        }))
    .WithMetrics(metrics => metrics
        .AddMeter(serviceName)
        .AddAspNetCoreInstrumentation()
        .AddHttpClientInstrumentation()
        .AddRuntimeInstrumentation()
        .AddProcessInstrumentation()
        .AddOtlpExporter(options =>
        {
            options.Endpoint = new Uri("http://localhost:4318/v1/metrics");
            options.Protocol = OtlpExportProtocol.HttpProtobuf;
        }));

builder.Logging.AddOpenTelemetry(logging => logging
    .AddOtlpExporter(options =>
    {
        options.Endpoint = new Uri("http://localhost:4318/v1/logs");
        options.Protocol = OtlpExportProtocol.HttpProtobuf;
    }));

var app = builder.Build();

// Simple dice rolling endpoint
app.MapGet("/rolldice/{player?}", (string? player, Instrumentation instrumentation, ILogger<Program> logger) =>
{
    using var activity = instrumentation.ActivitySource.StartActivity("RollDice");

    var roll = Random.Shared.Next(1, 7);

    activity?.SetTag("player.name", player ?? "anonymous");
    activity?.SetTag("dice.value", roll);

    instrumentation.RequestCounter.Add(1,
        new KeyValuePair<string, object?>("endpoint", "/rolldice"),
        new KeyValuePair<string, object?>("player", player ?? "anonymous"));

    logger.LogInformation("Player {Player} rolled a {Roll}", player ?? "anonymous", roll);

    return Results.Ok(new { player = player ?? "anonymous", roll });
});

app.Run();

// Instrumentation class
public class Instrumentation : IDisposable
{
    internal const string ActivitySourceName = "dice-api";
    internal const string MeterName = "dice-api";

    public Instrumentation()
    {
        ActivitySource = new ActivitySource(ActivitySourceName);
        Meter = new Meter(MeterName);

        RequestCounter = Meter.CreateCounter<long>(
            "dice.rolls",
            unit: "{roll}",
            description: "Number of dice rolls");
    }

    public ActivitySource ActivitySource { get; }
    public Meter Meter { get; }
    public Counter<long> RequestCounter { get; }

    public void Dispose()
    {
        ActivitySource.Dispose();
        Meter.Dispose();
    }
}

Best Practices

Use Dependency Injection: Register ActivitySource and Meter as singletons for reuse across your application.
Dispose Resources: Always dispose of activities, providers, and instrumentation classes to prevent memory leaks.

Use Semantic Conventions: Follow OpenTelemetry semantic conventions for attribute names:

using OpenTelemetry.Trace;
activity?.SetTag(SemanticConventions.AttributeHttpMethod, "GET");
activity?.SetTag(SemanticConventions.AttributeHttpStatusCode, 200);

Record Exceptions: Always record exceptions in activities for better error tracking:

catch (Exception ex)
{
    activity?.RecordException(ex);
    activity?.SetStatus(ActivityStatusCode.Error);
}

Use Structured Logging: Leverage structured logging with ILogger for automatic log-to-trace correlation.
Configure Sampling: In production, configure sampling to control data volume (via environment variables or code).
Add Resource Attributes: Include deployment and version information for better filtering in Edge Delta.
Use Activity.Current: Access the current activity with Activity.Current when needed:
```
Activity.Current?.SetTag("custom.attribute", value);
```

Troubleshooting

Telemetry Not Appearing in Edge Delta

Verify endpoint configuration:

Console.WriteLine($"OTLP Endpoint: {options.Endpoint}");

Test connectivity:

curl -X POST http://localhost:4318/v1/traces \
  -H "Content-Type: application/x-protobuf"

Enable debug logging:

builder.Logging.SetMinimumLevel(LogLevel.Debug);

Activities Not Being Created

Ensure ActivitySource is registered:

.WithTracing(tracing => tracing.AddSource("your-activity-source-name"))

Check activity is being disposed:

using var activity = activitySource.StartActivity("operation");
// Activity automatically ends when disposed

Metrics Not Recording

Verify Meter is registered:

.WithMetrics(metrics => metrics.AddMeter("your-meter-name"))

Check metric export interval:

.WithMetrics(metrics => metrics
    .AddMeter("your-meter-name")
    .AddOtlpExporter((exporterOptions, readerOptions) =>
    {
        readerOptions.PeriodicExportingMetricReaderOptions.ExportIntervalMilliseconds = 10000;
    }))

Performance Issues

Enable sampling:

.WithTracing(tracing => tracing
    .SetSampler(new TraceIdRatioBasedSampler(0.1))) // Sample 10%

Reduce metric cardinality:

Limit the number of unique tag combinations
Use bounded tag values

Disable unnecessary instrumentations:

.WithTracing(tracing => tracing
    .AddAspNetCoreInstrumentation()
    // Don't add instrumentations you don't need
)

Next Steps

Learn about zero-code instrumentation for automatic telemetry
Set up Edge Delta processing for your telemetry data
Review sampling strategies to control telemetry volume in production
Configure resource attributes for richer metadata and filtering
Explore context propagation for distributed tracing across services

For more information on OpenTelemetry in .NET, visit the official OpenTelemetry .NET documentation.