Code-based Instrumentation of Node.js using OpenTelemetry

Learn how to instrument Node.js applications using OpenTelemetry SDK for comprehensive telemetry data generation and integration with Edge Delta.

8 minute read

Overview

Code-based instrumentation in Node.js with OpenTelemetry involves manually writing code to generate telemetry data using the OpenTelemetry SDK. The SDK provides a comprehensive set of APIs and packages for capturing traces, metrics, and logs from your Node.js applications. This approach gives you fine-grained control over what telemetry is captured and how it’s enriched with custom attributes, allowing you to track application-specific business logic and performance characteristics.

Prerequisites

Ensure that you have:

Node.js: Version 14 or later (Active LTS versions recommended)
npm or yarn: Package manager for installing dependencies
TypeScript (optional): For type-safe instrumentation code
Edge Delta: An account and pipeline with an OTLP input node

Setup OpenTelemetry

1. Install Required Dependencies

Install the core OpenTelemetry packages for Node.js:

npm install --save @opentelemetry/api \
  @opentelemetry/sdk-node \
  @opentelemetry/sdk-trace-node \
  @opentelemetry/sdk-metrics \
  @opentelemetry/resources \
  @opentelemetry/semantic-conventions \
  @opentelemetry/exporter-trace-otlp-http \
  @opentelemetry/exporter-metrics-otlp-http

For automatic instrumentation of common frameworks:

npm install --save @opentelemetry/auto-instrumentations-node

These packages include:

@opentelemetry/api: Core OpenTelemetry API for creating spans and metrics
@opentelemetry/sdk-node: Node.js SDK for managing telemetry pipeline
@opentelemetry/exporter-trace-otlp-http: OTLP HTTP exporter for traces
@opentelemetry/exporter-metrics-otlp-http: OTLP HTTP exporter for metrics
@opentelemetry/auto-instrumentations-node: Automatic instrumentation for popular libraries

2. Create Instrumentation Configuration

Create an instrumentation.ts (or instrumentation.mjs for JavaScript) file to initialize the OpenTelemetry SDK. This file must be loaded before your application code.

For TypeScript (instrumentation.ts):

import { NodeSDK } from '@opentelemetry/sdk-node';
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-http';
import { OTLPMetricExporter } from '@opentelemetry/exporter-metrics-otlp-http';
import { PeriodicExportingMetricReader } from '@opentelemetry/sdk-metrics';
import { Resource } from '@opentelemetry/resources';
import {
  ATTR_SERVICE_NAME,
  ATTR_SERVICE_VERSION,
  ATTR_DEPLOYMENT_ENVIRONMENT,
} from '@opentelemetry/semantic-conventions';
import { getNodeAutoInstrumentations } from '@opentelemetry/auto-instrumentations-node';

// Configure resource with service information
const resource = new Resource({
  [ATTR_SERVICE_NAME]: 'my-nodejs-service',
  [ATTR_SERVICE_VERSION]: '1.0.0',
  [ATTR_DEPLOYMENT_ENVIRONMENT]: 'production',
});

// Configure OTLP trace exporter
const traceExporter = new OTLPTraceExporter({
  url: 'http://localhost:4318/v1/traces',
  headers: {},
});

// Configure OTLP metric exporter
const metricExporter = new OTLPMetricExporter({
  url: 'http://localhost:4318/v1/metrics',
  headers: {},
});

// Create SDK instance
const sdk = new NodeSDK({
  resource: resource,
  traceExporter: traceExporter,
  metricReader: new PeriodicExportingMetricReader({
    exporter: metricExporter,
    exportIntervalMillis: 60000, // Export every 60 seconds
  }),
  instrumentations: [
    getNodeAutoInstrumentations({
      // Customize auto-instrumentation
      '@opentelemetry/instrumentation-fs': {
        enabled: false, // Disable file system instrumentation
      },
    }),
  ],
});

// Start the SDK
sdk.start();

// Graceful shutdown
process.on('SIGTERM', () => {
  sdk
    .shutdown()
    .then(() => console.log('Tracing terminated'))
    .catch((error) => console.log('Error terminating tracing', error))
    .finally(() => process.exit(0));
});

console.log('OpenTelemetry instrumentation initialized');

For JavaScript (instrumentation.mjs):

import { NodeSDK } from '@opentelemetry/sdk-node';
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-http';
import { OTLPMetricExporter } from '@opentelemetry/exporter-metrics-otlp-http';
import { PeriodicExportingMetricReader } from '@opentelemetry/sdk-metrics';
import { Resource } from '@opentelemetry/resources';
import {
  ATTR_SERVICE_NAME,
  ATTR_SERVICE_VERSION,
} from '@opentelemetry/semantic-conventions';
import { getNodeAutoInstrumentations } from '@opentelemetry/auto-instrumentations-node';

const resource = new Resource({
  [ATTR_SERVICE_NAME]: 'my-nodejs-service',
  [ATTR_SERVICE_VERSION]: '1.0.0',
});

const traceExporter = new OTLPTraceExporter({
  url: 'http://localhost:4318/v1/traces',
});

const metricExporter = new OTLPMetricExporter({
  url: 'http://localhost:4318/v1/metrics',
});

const sdk = new NodeSDK({
  resource: resource,
  traceExporter: traceExporter,
  metricReader: new PeriodicExportingMetricReader({
    exporter: metricExporter,
  }),
  instrumentations: [getNodeAutoInstrumentations()],
});

sdk.start();

process.on('SIGTERM', () => {
  sdk.shutdown().finally(() => process.exit(0));
});

3. Configure OTLP Endpoints

The endpoint configuration depends on your deployment environment:

Kubernetes Environment:

const traceExporter = new OTLPTraceExporter({
  url: 'http://ed-data-supply-svc:4318/v1/traces',
});

const metricExporter = new OTLPMetricExporter({
  url: 'http://ed-data-supply-svc:4318/v1/metrics',
});

Non-Kubernetes (same host as Edge Delta agent):

const traceExporter = new OTLPTraceExporter({
  url: 'http://localhost:4318/v1/traces',
});

const metricExporter = new OTLPMetricExporter({
  url: 'http://localhost:4318/v1/metrics',
});

For gRPC (port 4317) instead of HTTP, install and use the gRPC exporters:

npm install @opentelemetry/exporter-trace-otlp-grpc \
  @opentelemetry/exporter-metrics-otlp-grpc

See Ingest Data from an OTLP Source for detailed Edge Delta configuration.

4. Run Your Application with Instrumentation

The instrumentation file must be loaded before your application code. Use Node.js’s --import flag (Node 18.19+) or --require flag (older versions).

TypeScript with tsx:

npx tsx --import ./instrumentation.ts app.ts

JavaScript (ESM):

node --import ./instrumentation.mjs app.js

JavaScript (CommonJS):

node --require ./instrumentation.js app.js

Alternative: Use package.json script:

{
  "scripts": {
    "start": "node --import ./instrumentation.mjs app.js",
    "dev": "npx tsx --import ./instrumentation.ts app.ts"
  }
}

Manual Instrumentation

Beyond automatic instrumentation, you can add custom spans, metrics, and logs to capture application-specific telemetry.

Creating Custom Spans

Use the tracing API to create custom spans for tracking specific operations:

import { trace, SpanStatusCode } from '@opentelemetry/api';

// Get a tracer
const tracer = trace.getTracer('my-service', '1.0.0');

async function processOrder(orderId: string) {
  // Create a span
  const span = tracer.startSpan('processOrder', {
    attributes: {
      'order.id': orderId,
    },
  });

  try {
    // Your business logic
    await validateOrder(orderId);
    await chargePayment(orderId);
    await fulfillOrder(orderId);

    span.setStatus({ code: SpanStatusCode.OK });
  } catch (error) {
    span.recordException(error);
    span.setStatus({
      code: SpanStatusCode.ERROR,
      message: error.message,
    });
    throw error;
  } finally {
    span.end();
  }
}

Using Active Spans for Context Propagation

For better context propagation, use startActiveSpan:

import { trace, context } from '@opentelemetry/api';

const tracer = trace.getTracer('my-service', '1.0.0');

async function handleRequest(req, res) {
  // Start an active span
  return tracer.startActiveSpan('handleRequest', async (span) => {
    span.setAttributes({
      'http.method': req.method,
      'http.url': req.url,
    });

    try {
      // Nested operations automatically become child spans
      const data = await fetchData();
      const result = await processData(data);

      span.setStatus({ code: SpanStatusCode.OK });
      res.json(result);
    } catch (error) {
      span.recordException(error);
      span.setStatus({ code: SpanStatusCode.ERROR });
      res.status(500).json({ error: error.message });
    } finally {
      span.end();
    }
  });
}

async function fetchData() {
  // This automatically becomes a child span
  return tracer.startActiveSpan('fetchData', async (span) => {
    try {
      const data = await database.query('SELECT * FROM users');
      span.setAttribute('result.count', data.length);
      return data;
    } finally {
      span.end();
    }
  });
}

Creating Nested Spans

Create parent-child span relationships for hierarchical tracing:

tracer.startActiveSpan('parentOperation', (parentSpan) => {
  // Parent span work
  parentSpan.addEvent('Starting child operations');

  tracer.startActiveSpan('childOperation1', (childSpan1) => {
    // First child work
    childSpan1.setAttribute('step', 1);
    childSpan1.end();
  });

  tracer.startActiveSpan('childOperation2', (childSpan2) => {
    // Second child work
    childSpan2.setAttribute('step', 2);
    childSpan2.end();
  });

  parentSpan.addEvent('Child operations completed');
  parentSpan.end();
});

Adding Span Attributes and Events

Enrich spans with contextual information:

const span = tracer.startSpan('databaseQuery');

// Add attributes
span.setAttributes({
  'db.system': 'postgresql',
  'db.name': 'users_db',
  'db.statement': 'SELECT * FROM users WHERE active = true',
  'db.user': 'app_user',
});

// Add events
span.addEvent('query_started');
span.addEvent('query_completed', {
  'result.rows': 42,
  'result.duration_ms': 23,
});

span.end();

Recording Metrics

Create and use metric instruments to track application performance:

import { metrics } from '@opentelemetry/api';

// Get a meter
const meter = metrics.getMeter('my-service', '1.0.0');

// Create a counter
const requestCounter = meter.createCounter('http.requests', {
  description: 'Count of HTTP requests',
  unit: '{request}',
});

// Create a histogram
const requestDuration = meter.createHistogram('http.request.duration', {
  description: 'Duration of HTTP requests',
  unit: 'ms',
});

// Create an up-down counter
const activeConnections = meter.createUpDownCounter('http.active_connections', {
  description: 'Number of active HTTP connections',
  unit: '{connection}',
});

// Use in your application
app.use((req, res, next) => {
  const startTime = Date.now();

  // Increment active connections
  activeConnections.add(1, {
    'http.method': req.method,
  });

  // Increment request counter
  requestCounter.add(1, {
    'http.method': req.method,
    'http.route': req.route?.path || 'unknown',
  });

  res.on('finish', () => {
    // Decrement active connections
    activeConnections.add(-1, {
      'http.method': req.method,
    });

    // Record duration
    const duration = Date.now() - startTime;
    requestDuration.record(duration, {
      'http.method': req.method,
      'http.status_code': res.statusCode,
      'http.route': req.route?.path || 'unknown',
    });
  });

  next();
});

Observable Metrics

Create asynchronous metrics that are collected periodically:

// Observable Gauge for current memory usage
meter.createObservableGauge('process.memory.usage', {
  description: 'Current memory usage',
  unit: 'By',
}, (observableResult) => {
  const memUsage = process.memoryUsage();
  observableResult.observe(memUsage.heapUsed, {
    'memory.type': 'heap',
  });
  observableResult.observe(memUsage.rss, {
    'memory.type': 'rss',
  });
});

// Observable Counter for total request count
let totalRequests = 0;

meter.createObservableCounter('requests.total', {
  description: 'Total number of requests',
  unit: '{request}',
}, (observableResult) => {
  observableResult.observe(totalRequests);
});

Instrumenting Express Applications

Here’s a complete example of instrumenting an Express.js application:

import express from 'express';
import { trace, SpanStatusCode, metrics } from '@opentelemetry/api';

const app = express();
const tracer = trace.getTracer('express-app', '1.0.0');
const meter = metrics.getMeter('express-app', '1.0.0');

// Create metrics
const requestCounter = meter.createCounter('http.requests');
const requestDuration = meter.createHistogram('http.request.duration');

// Middleware for custom instrumentation
app.use((req, res, next) => {
  const startTime = Date.now();

  tracer.startActiveSpan(`${req.method} ${req.path}`, (span) => {
    span.setAttributes({
      'http.method': req.method,
      'http.url': req.url,
      'http.target': req.path,
      'http.host': req.hostname,
    });

    res.on('finish', () => {
      span.setAttribute('http.status_code', res.statusCode);

      if (res.statusCode >= 400) {
        span.setStatus({ code: SpanStatusCode.ERROR });
      } else {
        span.setStatus({ code: SpanStatusCode.OK });
      }

      // Record metrics
      const duration = Date.now() - startTime;
      requestCounter.add(1, {
        'http.method': req.method,
        'http.status_code': res.statusCode,
      });
      requestDuration.record(duration, {
        'http.method': req.method,
        'http.route': req.route?.path || req.path,
      });

      span.end();
    });

    next();
  });
});

// Route handlers
app.get('/api/users', async (req, res) => {
  const span = trace.getActiveSpan();
  span?.addEvent('Fetching users from database');

  try {
    const users = await fetchUsers();
    span?.setAttribute('users.count', users.length);
    res.json(users);
  } catch (error) {
    span?.recordException(error);
    span?.setStatus({ code: SpanStatusCode.ERROR });
    res.status(500).json({ error: 'Failed to fetch users' });
  }
});

app.listen(3000, () => {
  console.log('Server running on port 3000');
});

Best Practices

Initialize Early: Load the instrumentation file before any other application code to ensure all modules are instrumented.

Use Semantic Conventions: Use the @opentelemetry/semantic-conventions package for standardized attribute names:

import { ATTR_HTTP_REQUEST_METHOD, ATTR_HTTP_RESPONSE_STATUS_CODE } from '@opentelemetry/semantic-conventions';

Handle Errors Properly: Always record exceptions in spans and set appropriate status codes:

catch (error) {
  span.recordException(error);
  span.setStatus({ code: SpanStatusCode.ERROR, message: error.message });
}

Use Active Spans: Prefer startActiveSpan over startSpan for automatic context propagation.
Add Meaningful Attributes: Include business-relevant attributes for better filtering and analysis in Edge Delta.

Graceful Shutdown: Always flush telemetry data before process termination:

process.on('SIGTERM', async () => {
  await sdk.shutdown();
  process.exit(0);
});

Configure Sampling: In production, use sampling to control data volume (configure via environment variables).
Batch Export: Use batch processors (default) rather than simple span processors for better performance.

Troubleshooting

Telemetry Not Appearing in Edge Delta

Check endpoint configuration:

console.log('Trace exporter URL:', traceExporter.url);

Verify connectivity:

curl -X POST http://localhost:4318/v1/traces \
  -H "Content-Type: application/json" \
  -d '{"resourceSpans":[]}'

Enable debug logging: Set the OTEL_LOG_LEVEL environment variable:

export OTEL_LOG_LEVEL=debug
node --import ./instrumentation.mjs app.js

Instrumentation Not Loading

Problem: Instrumentation file runs after application code

Solution: Use --import or --require flags, not manual imports
Verify: Check that SDK initialization happens before express/framework imports

Problem: TypeScript compilation issues

Solution: Use tsx for development: npx tsx --import ./instrumentation.ts app.ts
For production: Compile TypeScript first, then run compiled JavaScript

Missing Spans or Metrics

Check auto-instrumentation is enabled:

instrumentations: [getNodeAutoInstrumentations()],

Verify supported library versions:

Check the compatibility matrix

Ensure spans are ended:

span.end(); // Always call end() or use startActiveSpan

Performance Issues

High memory usage:

Reduce metric export interval
Enable sampling: export OTEL_TRACES_SAMPLER=parentbased_traceidratio
Disable unnecessary instrumentations

High CPU usage:

Set appropriate log level: export OTEL_LOG_LEVEL=warn
Review custom instrumentation for excessive span creation

Next Steps

Explore automatic instrumentation libraries for your frameworks
Learn about zero-code instrumentation for automatic telemetry
Set up Edge Delta processing for your telemetry data
Review sampling strategies to control telemetry volume in production
Configure resource detection for automatic metadata enrichment

For more information on OpenTelemetry in Node.js, visit the official OpenTelemetry JavaScript documentation.