Code-based Instrumentation of Java using OpenTelemetry

Instrument Java using Code-based OpenTelemetry to emit useful telemetry.

  5 minute read  

Overview

Code-based instrumentation in Java with OpenTelemetry involves manually writing code to generate telemetry data using the OpenTelemetry API - a set of libraries provided by OpenTelemetry and added to your project. The API provides functions and classes you can use to manually instrument your code to capture telemetry data. You write additional code using the OpenTelemetry API to specify what parts of your program should be monitored. This typically involves creating “spans” to track the execution timing of certain blocks of code or using “meters” to capture metrics like the number of requests processed.

Prerequisites

Ensure that you have:

  • Java 8 or later
  • OpenTelemetry API and SDK dependencies added to your project
  • An Edge Delta account and a fleet with an OTLP input node.

Setup OpenTelemetry

1. Import Required Classes

First, import the necessary OpenTelemetry classes:

import io.opentelemetry.api.trace.*;
import io.opentelemetry.api.metrics.*;
import io.opentelemetry.api.logs.*;
import io.opentelemetry.api.OpenTelemetry;
import io.opentelemetry.sdk.OpenTelemetrySdk;
  • api.trace.* for creating and managing spans (tracing).
  • api.metrics.* for defining and recording metrics.
  • api.logs.* for logging support.
  • api.OpenTelemetry the main interface for acquiring OpenTelemetry instances.
  • sdk.OpenTelemetrySdk provides the SDK implementation, required for exporting telemetry data.

2. Set Up Resource Attributes

Define resource attributes, such as service name, to associate telemetry data with your application:

Resource resource = Resource.create(Attributes.of(ResourceAttributes.SERVICE_NAME, "my-service"));

3. Configure Traces

Create a TracerProvider to enable distributed tracing:

SdkTracerProvider tracerProvider = SdkTracerProvider.builder()
    .setResource(resource)
    .addSpanProcessor(BatchSpanProcessor.builder(OtlpHttpSpanExporter.builder()
        .setEndpoint("http://localhost:4318/v1/traces")
        .build()).build())
    .build();

The SdkTracerProvider is a critical SDK component responsible for creating and managing traces. You can customize its behavior by configuring the associated resources, samplers, span processors, and span exporters.

The endpoint configuration depends on your environment. For example, in a Kubernetes Environment, you need a service open, for example ed-data-supply-svc, for the port that the exporters will send logs, metrics, and traces to. For non-kubernetes environments with exporters running in the same environment as the Edge Delta agent, such as on a Linux VM, the exporters will be able to communicate directly with the Edge Delta agent on localhost. For HTTP (4318), include the path /v1/traces, /v1/metrics, or /v1/logs. For gRPC (4317), do not include the path. See Ingest Data from an OTLP Source.

4. Configure Metrics

Set up a MeterProvider to track application metrics:

SdkMeterProvider meterProvider = SdkMeterProvider.builder()
    .setResource(Resource.getDefault().toBuilder()
        .put(ResourceAttributes.SERVICE_NAME, "my-service").build())
    .registerMetricReader(PeriodicMetricReader.builder(
        OtlpGrpcMetricExporter.builder()
            .setEndpoint("http://localhost:4318/v1/metrics") 
            .build()).build())
    .build();

As with the Traces configuration, the endpoint configuration depends on your environment.

5. Configure Logs

OpenTelemetry supports logging via OpenTelemetry Logging SDK (since v1.22+).

SdkLoggerProvider loggerProvider = SdkLoggerProvider.builder()
    .setResource(Resource.getDefault().toBuilder()
        .put(ResourceAttributes.SERVICE_NAME, "my-service").build())
    .addLogRecordProcessor(BatchLogRecordProcessor.builder(
        OtlpGrpcLogExporter.builder()
            .setEndpoint("http://localhost:4318/v1/logs")
            .build()).build())
    .build();

As with the Traces configuration, the endpoint configuration depends on your environment.

6. Debugging and Console Export (Optional):

To view traces, metrics, and logs locally, configure console exporters:

ConsoleSpanExporter spanExporter = ConsoleSpanExporter.create();
ConsoleMetricExporter metricExporter = ConsoleMetricExporter.create();

Instrumentation

1. Acquire a Tracer and Start a Span

Obtain a Tracer and use it to create spans for monitoring execution timing:

Tracer tracer = GlobalOpenTelemetry.getTracer("example.Tracer");

Span span = tracer.spanBuilder("process_request").startSpan();
try (Scope scope = span.makeCurrent()) {
    // Business logic being traced
} finally {
    span.end();
}

Add attributes and events for richer trace data:

span.setAttribute("http.method", "GET");
span.addEvent("User authenticated", Attributes.of(AttributeKey.stringKey("user.id"), "123"));

2. Measure Metrics

Use Meter to track application metrics:

Meter meter = GlobalOpenTelemetry.getMeter("example.Meter");
LongCounter requestCounter = meter.counterBuilder("http_requests").setDescription("Counts HTTP requests").build();
requestCounter.add(1);

Utilize different types of instruments such as Counter, Histogram, etc., to capture various types of metric data. To track fluctuating values, use gauges:

ObservableDoubleGauge gauge = meter.gaugeBuilder("cpu_usage")
    .setDescription("CPU Usage")
    .setUnit("%")
    .buildWithCallback(observableMeasurement -> {
        observableMeasurement.record(getCpuUsage());
    });

3. Capture Logs

Capture logs with trace context for better observability:

Logger logger = GlobalOpenTelemetry.getLogger("example.Logger");
Span currentSpan = Span.current();
logger.logRecordBuilder()
    .setBody("User login event")
    .setSeverity(Severity.INFO)
    .setAttribute("trace_id", currentSpan.getSpanContext().getTraceId())
    .emit();

Testing and Validation

  • Verify Logs: Use Edge Delta to check logs for trace and span IDs.
  • Check Metrics: Use Edge Delta.
  • Trace Requests: Use Edge Delta.

Advanced

Context Propagation

Ensure trace continuity across services by using ContextPropagators. In distributed systems, context propagation maintains trace continuity as requests traverse different services and components. OpenTelemetry provides APIs to manage and propagate context across service boundaries. The Context in OpenTelemetry serves as an immutable bundle of key-value pairs that travels along the execution path within and across application boundaries. It implicitly maintains and provides access to the current active Span, among other contextual data. The ContextPropagators API is central to context propagation. It enables the injection of context into outgoing requests and extraction from incoming requests, allowing trace contexts and baggage to move across process boundaries. OpenTelemetry auto-instrumentation often handles propagation automatically, but manual configuration may be necessary for custom instrumentation.

To configure context propagation, create a ContextPropagators instance using various TextMapPropagators that define the serialization of context over wire protocols like HTTP headers.

ContextPropagators propagators = ContextPropagators.create(
    TextMapPropagator.composite(W3CTraceContextPropagator.getInstance(), W3CBaggagePropagator.getInstance())
);

This setup, including W3CTraceContextPropagator and W3CBaggagePropagator, ensures trace context and baggage are serialized and deserialized according to W3C specs.

Inject and extract context in HTTP requests:

propagators.getTextMapPropagator().inject(Context.current(), request, (carrier, key, value) -> request.setHeader(key, value));

Context extractedContext = propagators.getTextMapPropagator().extract(Context.current(), exchange, TEXT_MAP_GETTER);

Advanced Customizations

In advanced scenarios, the default telemetry processing provided by OpenTelemetry might not fully meet all application requirements. You can extend the SDK functionality by implementing custom components, such as custom Samplers or SpanProcessors, to fine-tune telemetry data capture and processing.

Custom Sampler:

A Sampler in OpenTelemetry determines which traces are sampled and recorded. By implementing a custom Sampler, you can define complex logic to control trace sampling based on specific criteria.

public class CustomSampler implements Sampler {
    @Override
    public SamplingResult shouldSample(Context parentContext, String traceId, String name, SpanKind spanKind, Attributes attributes, List<LinkData> parentLinks) {
        return SpanKind.SERVER == spanKind ? SamplingResult.recordAndSample() : SamplingResult.drop();
    }
}

Custom Span Processor: SpanProcessors process the lifecycle of spans from when they start until they end. Creating a custom SpanProcessor allows additional span enrichment or processing at the end of a span’s lifecycle.

public class CustomSpanProcessor implements SpanProcessor {
    @Override
    public void onStart(Context parentContext, ReadWriteSpan span) {
        span.setAttribute("custom.attribute", "custom_value");
    }
}

Custom Metric Exporter: This example demonstrates how to create a custom MetricExporter in OpenTelemetry Java that processes and logs exported metric data:

public class CustomMetricExporter implements MetricExporter {
    @Override
    public CompletableResultCode export(Collection<MetricData> metrics) {
        for (MetricData metric : metrics) {
            System.out.println("Exporting metric: " + metric.getName() + " Value: " + metric.getData());
        }
        return CompletableResultCode.ofSuccess();
    }
}

Fine Tuning Performance

Use batch processing instead of immediate export.

Adjust sampling rates to control data volume.

Optimize context propagation for efficiency in distributed systems.

Semantic Conventions

Ensure trace and metric standardization:

Attributes attributes = Attributes.builder()
    .put(HttpAttributes.HTTP_REQUEST_METHOD, "GET")
    .put(ServerAttributes.SERVER_ADDRESS, "example.com")
    .build();