Agent v2.0.0

June 16, 2025

New Features

• Node, gateway, and coordinator pipelines: Introduced a new agent architecture that separates responsibilities across three distinct components: node, gateway, and coordinator pipelines. The node pipeline remains lightweight and focused on local data collection and preprocessing at the node or host level. The gateway pipeline acts as a centralized point for data aggregation, transformation, and advanced processing tasks such as trace tail sampling. The coordinator pipeline manages orchestration and configuration distribution across node and gateway agents, ensuring consistent behavior and scalability across large deployments. This modular structure improves performance, simplifies management, and enables more advanced use cases in distributed environments.
• Windows Event Source: Added a new Windows Event source node for collecting logs from Windows events, enabling observability for Windows-based node pipelines.
• OTTL Conditional Functions: Introduced EDXIfElse and EDXCoalesce custom OTTL functions for inline conditional operations and null coalescing, enabling ternary-like operations and fallback value handling.
• Add Trace and Span ID to Log Items: Added trace ID and span ID as top-level fields to log item payloads, enabling improved traceability and correlation across pipelines.
• Tail Sampling: Introduced the tail sample processor to filter and manage traces and spans by applying various sampling policies

Improvements

• Aggregate Metric Processor Passthrough: Introduced a passthrough on no match flag for the aggregate metric processor, allowing unmatched items to be forwarded to subsequent nodes in sequence processors.
• OTLP GRPC Connection State Management: Implemented connection state management, cleanup, and reconnection logic for OTLP gRPC outputs to prevent orphaned connections and address issues with excessive pings after config reloads.
• Data Transformation Hook for Output Strategies: Updated the output node strategy interface to encapsulate data transformation (compression/formatting) before sending to third-party destinations.
• Set Cluster Name Only from Environment Variable: The agent now sets the ed.cluster.name field exclusively from the environment variable to avoid ambiguity between different sources.
• Output Node Self Log and ObsReport Gaps: Ensured all output nodes record error logs and upload errors using both self logs and obsreport metrics.
• Expand Delete Empty Values Validation: Made validation error messages for the Delete Empty Field processor more descriptive and specific to improve user feedback.
• GZIP Node and HTTP Source Improvements: Added decompression support for HTTP sources and increased max item size to enhance GZIP handling, particularly with AWS Firehose data.
• Remove Log from ED Output: Cleaned up unnecessary extra logging in the ED output node for metrics.
• Modify Dimensions of Trace Data in OTLP Input: Adjusted the scope of resources for trace data in OTLP input for accurate dimension labeling.
• Update Validation for JSON Unroll Node: Fixed path extraction logic in the JSON Unroll processor to handle OTTL properly.
• Implemented a number of changes to improve graceful agent shutdown.
• Self Telemetry Memory Usage by cgroups: Migrated self telemetry memory usage reporting from cAdvisor to cgroup-based calculation.
• Memory Working Set Fallback Scenario: Fixed fallback logic for memory working set determination in self telemetry to handle scenarios where cgroup data is unavailable.

Bug Fixes

• Fix Dedup Processor Timestamp: Resolved an issue where deduplicate processor output items had incorrect timestamps due to ordering in processing logic.
• Fix Sequence Node OTTL Data Type Filtering: Corrected the sequence node’s behavior when filtering by data type with OTTL filters to avoid unexpected drops.
• Fix JSON Unroll Path Extraction: Updated the JSON Unroll node to skip CEL expressions when extracting paths, resolving path extraction bugs.
• Do Not Build sysctl on Windows: Excluded sysctl build for Windows to fix Windows image build failures.
• Prevent Crash on Source Detector During Shutdown: Applied context cancellation to stop blocking source detector processes on config reload shutdown, preventing agent crashes.

Deprecations

• Deprecate Component Health Node: Deprecated the ed_component_health_input node in favor of relying on self telemetry for health metrics.