OTEL Collector with Cloud Pipelines

Configure the OTEL Collector to send telemetry data to Edge Delta Cloud Pipelines using OTLP input nodes.

  6 minute read  

A cloud pipeline is able to collect data from third party agents. In this scenario you do not need to install an Edge Delta agent in your environment, but rather point your existing agent, such as an OTEL collector, to an Edge Delta cloud pipeline. On the cloud pipeline, you configure an OTLP input node.

OTEL Collector

The OTLP source node consumes data items directly from OTLP configured data sources. The node is configured with the port that the agent should listen on.

Configure OTLP

To configure the OTLP source node, you must obtain the port number from the OTLP configuration:

  • Instrumentation Libraries: When using the OpenTelemetry SDKs, the port used to emit OTLP logs is part of the exporter configuration. The endpoint (which includes the host and port) is set when setting up the OpenTelemetry exporter within your application code. See Instrument Code using OpenTelemetry.
  • OpenTelemetry Collector: The port number on which the collector should send outgoing OTLP data is specified in the exporter section.
  • Zero-Code Instrumentation Agents: Similar to the instrumentation libraries, auto-instrumentation agents are configured to send data to a specified endpoint. This configuration includes the port number to which OTLP logs will be sent. See Instrument Code using OpenTelemetry.
  • Sidecars: In a Kubernetes environment, a sidecar that runs an instance of the OpenTelemetry Collector is set up using a configuration file, in which you can find the port for the OTLP receiver and exporter.
  • Log Routers and Forwarders: Log routers and forwarders may have plugins or output configurations that support OTLP. Within these configurations,the endpoint is defined, including the port, where the logs should be sent in OTLP format.

Example Collector Configuration (Cloud Pipeline)

gRPC (Cloud Pipeline)

If you are sending OTEL telemetry from the collector to a cloud pipeline, you update the Collector configuration with exporters pointing to the cloud pipeline endpoints. You use secure TLS and port 443:

    exporters:
      otlp/ed-data-supply_trace:
        endpoint: '12345678-1a2b-3c4d-5e6f-7890ghijklmn-grpc-us-west2-cf.aws.edgedelta.com:443'
        tls:
          insecure: false
      otlp/ed-data-supply_metric:
        endpoint: '12345678-1a2b-3c4d-5e6f-7890ghijklmn-grpc-us-west2-cf.aws.edgedelta.com:443'
        tls:
          insecure: false
      otlp/ed-data-supply_log:
        endpoint: '12345678-1a2b-3c4d-5e6f-7890ghijklmn-grpc-us-west2-cf.aws.edgedelta.com:443'
        tls:
          insecure: false

Replace the endpoint with one provided in your Cloud pipeline settings. Include the port number but no route is required. Do not include grpcs://.

And you update the Collector’s Pipeline to use the new exporters:

    service:
      extensions:
      - health_check
      pipelines:
        logs:
          exporters:
          ...
          - otlp/ed-data-supply_log
          ...
        metrics:
          exporters:
          ...
          - otlp/ed-data-supply_metric
          ...
        traces:
          exporters:
          ...
          - otlp/ed-data-supply_trace
          ...

The Cloud pipeline contains an OTLP input node by default, which does not need to be adjusted for this gRPC configuration:

- name: otlp_input
  type: otlp_input
  port: 4317
  protocol: grpc

HTTP (Cloud Pipeline)

To send OTLP telemetry to an Edge Delta Cloud pipeline you configure otlphttp exporters and disable compression. You use secure TLS and port 443 for HTTPS:

    exporters:
      otlphttp/ed-data-supply_trace:
        endpoint: 'https://12345678-1a2b-3c4d-5e6f-7890ghijklmn-http-us-west2-cf.aws.edgedelta.com:443'
        compression: none
        tls:
          insecure: false
      otlphttp/ed-data-supply_metric:
        endpoint: 'https://12345678-1a2b-3c4d-5e6f-7890ghijklmn-http-us-west2-cf.aws.edgedelta.com:443'
        compression: none
        tls:
          insecure: false
      otlphttp/ed-data-supply_log:
        endpoint: 'https://12345678-1a2b-3c4d-5e6f-7890ghijklmn-http-us-west2-cf.aws.edgedelta.com:443'
        compression: none
        tls:
          insecure: false

Replace the endpoint with one provided in your Cloud pipeline settings. You include the port number at the end but no route is required. Unlike gRPC you include https://.

And you update the Collector’s Pipeline to use the new exporters:

    service:
      extensions:
      - health_check
      pipelines:
        logs:
          exporters:
          ...
          - otlphttp/ed-data-supply_log
          ...
        metrics:
          exporters:
          ...
          - otlphttp/ed-data-supply_metric
          ...
        traces:
          exporters:
          ...
          - otlphttp/ed-data-supply_trace
          ...

The Cloud pipeline contains an HTTP input node by default, which you need to delete. Replace it with an OTLP input node listening on port 80 for HTTP traffic:

- name: otlp_input_80
  type: otlp_input
  port: 80
  protocol: http
- name: otlp_input
  type: otlp_input
  port: 4317
  protocol: grpc

Note: You may also need to include an unused gRPC OTLP node to pass configuration validation.

Example Collector Configuration (Edge Delta Agent)

When sending OTLP telemetry from an OpenTelemetry Collector to an Edge Delta agent (not a Cloud Pipeline), the configuration differs from cloud pipeline setups. Recent versions of the OTLP collector have different default behaviors, particularly around compression settings. The following examples show working configurations for connecting to a Linux Edge Delta agent without TLS certificates.

gRPC (Edge Delta Agent)

For gRPC connections to an Edge Delta agent, configure the exporter with insecure TLS. The agent typically listens on port 4317:

exporters:
  debug:
    verbosity: detailed
  otlp:
    endpoint: 18.117.115.89:4317
    tls:
      insecure: true

Replace the IP address with your Edge Delta agent’s IP address or hostname. The debug exporter is optional but helpful for troubleshooting.

Update your service pipeline to use these exporters (keep your existing receivers and processors):

service:
  pipelines:
    logs:
      receivers: [...]  # Keep your existing receivers
      processors: [...]  # Keep your existing processors
      exporters:
      - otlp
      - debug
    metrics:
      receivers: [...]  # Keep your existing receivers
      processors: [...]  # Keep your existing processors
      exporters:
      - otlp
      - debug
    traces:
      receivers: [...]  # Keep your existing receivers
      processors: [...]  # Keep your existing processors
      exporters:
      - otlp
      - debug

HTTP with Protobuf Encoding (Edge Delta Agent)

For HTTP connections using protobuf encoding, you must explicitly disable compression and specify the encoding format. The agent typically listens on port 4318:

exporters:
  debug:
    verbosity: detailed
  otlphttp:
    endpoint: http://18.117.115.89:4318
    tls:
      insecure: true
    encoding: proto
    compression: none

Important: The compression: none setting is required. Recent OTLP collector versions may enable compression by default, which can cause connection issues with Edge Delta agents.

HTTP with JSON Encoding (Edge Delta Agent)

For HTTP connections using JSON encoding, similar settings apply. Note that there is a known regression in collector version 2.6.0 affecting JSON encoding:

exporters:
  debug:
    verbosity: detailed
  otlphttp:
    endpoint: http://18.117.115.89:4318
    tls:
      insecure: true
    encoding: json
    compression: none

Known Issue: OpenTelemetry Collector version 2.6.0 has a regression where JSON encoding may not work as expected. If you encounter issues with JSON encoding, consider using protobuf encoding (encoding: proto) or a different collector version.

Configuration Notes

When configuring the OTLP collector to send to Edge Delta agents:

  1. Compression: Always set compression: none for HTTP exporters. The default compression settings in newer collector versions may cause compatibility issues.

  2. TLS Configuration: For development or internal networks without TLS certificates, set tls.insecure: true. For production environments with proper certificates, configure TLS appropriately.

  3. Encoding Format: Choose between proto (protobuf) and json encoding based on your requirements. Protobuf is more efficient and recommended for production use.

  4. Port Numbers: Standard ports are 4317 for gRPC and 4318 for HTTP. Verify which ports your Edge Delta agent is configured to use.

  5. Debug Output: Include the debug exporter during initial setup to verify telemetry is flowing correctly. Remove it in production to reduce overhead.

Edge Delta Agent OTLP Input Configuration

Ensure your Edge Delta agent configuration includes the appropriate OTLP input nodes:

For gRPC (port 4317):

nodes:
- name: otlp_grpc_input
  type: otlp_input
  port: 4317
  protocol: grpc

For HTTP (port 4318):

nodes:
- name: otlp_http_input
  type: otlp_input
  port: 4318
  protocol: http
  read_timeout: 1m0s

See OTLP Input Node for complete configuration details.