Agent v2.7.0

November 7, 2025

New Features

• Elastic API Input: Introduced Filebeat endpoint to consume data from Vector, Logstash, and Filebeat, enabling seamless integration with Elastic-compatible data sources.
• EDXLookup Function: Added custom OTTL function for data enrichment by looking up values from lookup tables, supporting dynamic data enrichment with configurable column access.
• Edge Delta AI Event Output Node: Released AI Event Output node for public use, enabling AI-powered event processing with support for non-log data types and enhanced connector metadata.
• Apache Kudu Destination: Introduced secure Apache Kudu destination with Kerberos authentication and encryption support for enterprise data warehouse integration.
• BigQuery Destination: Added Google Cloud BigQuery destination node for sending log data directly to BigQuery via API, enabling native GCP data warehouse integration with nil object handling.
• Secrets and Keys Management: Added secrets and keys sections to pipeline configuration with reference validation, enabling secure credential management and encryption key handling.

Enhancements

• HTTP Input JSON Parsing: Added parse mode configuration with auto-detection, JSON object preservation, and line-delimited modes to HTTP Input source, resolving issues with embedded newlines in JSON payloads and ensuring complete JSON objects are preserved as single log entries.
• HTTP Pull Source Enhancements: Added body and body expression fields to HTTP Pull source with POST method support, and configurable request timeout for flexible HTTP data retrieval.
• OTLP JSON Encoding: Fixed JSON encoding support in OTLP source and destination nodes for HTTP protocol, resolving HTTP 400 errors when exporting logs.
• Splunk TCP Dynamic Indexing: Added dynamic index value support to Splunk TCP destination based on data item attributes, enabling flexible index routing similar to GCS bucket expression capability.
• TCP Input Concurrency: Enabled concurrent TCP connections with proper connection limits in TCP input, allowing multiple simultaneous client connections for improved throughput.
• Live Tail Payload Management: Added configurable truncation for large payloads with environment variables (ED_CAPTURER_ITEM_MAX_BODY_SIZE and ED_CAPTURER_ITEM_TRUNCATION_SIZE) for body size and truncation thresholds, preventing UI freezing with oversized data during live capture.
• Pipeline Node Disabling: Added disabled property support for source and destination nodes, allowing nodes to remain visible in UI while being operationally inactive without requiring removal from pipeline configuration.
• Edge Delta Gateway Destination: Extended Edge Delta Gateway destination availability to all environment types including non-Kubernetes deployments, enabling gateway connectivity via Kubernetes ingress or host ports.
• Splunk HEC Input Docker Support: Enhanced Splunk HEC input to handle Docker Splunk driver time field format and endpoint path variations, ensuring compatibility with Docker logging drivers.
• Splunk HEC Authentication: Added authentication support to Splunk HEC input node with hec_authentication configuration, enabling token validation and access control for incoming HEC requests.
• Splunk HEC Error Codes: Improved Splunk HEC input error code responses with proper feedback mechanisms, including code 1 for disabled tokens, providing better diagnostics for token validation issues.
• Splunk Destination Compression: Added gzip compression support to Splunk destination, enabling data compression before transmission to reduce bandwidth usage and improve transfer efficiency.
• TLS Configuration Separation: Separated TLS specifications into client and server roles with appropriate field restrictions, improving mutual TLS configuration clarity and preventing misconfiguration.
• Trace Type Recognition: Enhanced trace type recognition with expanded support for databases and messaging systems, and relaxed detection logic to prevent overwriting existing trace types set earlier in the pipeline.
• OTLP Input Compression: Added compression support for OTLP input to ensure compatibility with newer OpenTelemetry Collector versions that enable compression by default.

Bug Fixes

• S3/SQS Data Loss Prevention: Fixed critical bug in S3 input where SQS messages were deleted even when S3 object downloads failed. The agent now only deletes SQS messages after all downloads succeed for that message, ensuring failed downloads are automatically retried after the visibility timeout. Messages with partial download failures remain in the queue for retry.
• IRSA Authentication: Fixed authentication on EKS when role_arn is specified in S3/SQS configuration by using AWS SDK default credential chain with AssumeRole, enabling cross-region access and maintaining backward compatibility.
• OTLP Destination Endpoints: Fixed HTTP endpoint construction in OTLP destination, ensuring proper URL formatting for outbound telemetry.
• Output Buffer Encoding: Expanded gob encoding/decoding support to all item types beyond metrics, resolving serialization issues in HTTP output buffering.
• BigQuery Operations: Added nil object handling for BigQuery responses to prevent nil pointer errors when API returns nil without explicit errors.
• Kubernetes Trace IDs: Fixed trace and span ID generation to use hex-encoded strings, resolving failures when sending traces in OTEL format through gateway destination.
• Elastic API Input Configuration: Fixed Elastic API input to properly honor configuration variables including enable_health_check, active_request_limit, custom_api_version_response, and api_version that were previously not being applied in the tailer.
• Splunk HEC Input JSON Handling: Fixed Splunk HEC input to properly handle incoming requests with JSON payload events, resolving parsing issues for JSON-formatted event data.

Configuration Changes

• Sample Collection: Changed live capture sample collection to disabled by default to reduce unnecessary data processing and performance overhead in production environments. Live capture is designed for the pipeline design phase and can be re-enabled by setting the ED_DISABLE_SAMPLE_COLLECTOR=0 environment variable via helm (see helm values) if needed for ongoing pipeline development.
• Pipeline Secret: Coordinator now automatically generates secure pipeline.secret during initial run for encryption and decryption of secrets and configuration keys.

Deployment and Upgrade Notes

Coordinator Agent Upgrade (v2.7.0):

A new volume mount has been introduced for the coordinator agent to support persistent storage of the pipeline.secret.

• Upgrades: If upgrading an existing coordinator agent to this version, you must run helm upgrade to apply the updated deployment configuration.
• Requirements: Ensure that your Kubernetes cluster has a default StorageClass defined, as the new PersistentVolumeClaim depends on it.

Backward Compatibility Notes:

The following new configuration fields require agent version v2.7.0 or higher. Agents running older versions will not honor these fields:

• HTTP Pull Source - request_body and request_body_expression fields (added for POST request support)
• Splunk TCP Destination - index_expression field (added for dynamic index routing)
• Splunk HEC Input - hec_authentication field (added for token validation and access control)