Agent v2.12.0

Splunk S2S v4 protocol support, Kubernetes event input enhancements, performance optimizations, and GCL destination improvements.

February 2026 - Agent v2.12.0

New Features

  • Splunk S2S v4 Protocol: Added support for v4 protocol in Splunk S2S sources and destinations, enabling compatibility with the latest Splunk S2S communication standard. This simplifies forwarder configuration by eliminating the need for legacy protocol settings like enableOldS2SProtocol and negotiateProtocolLevel.
  • Kubernetes Event Input Enhancements: Added support for watching resource lifecycle events (ADDED, MODIFIED, DELETED) across 20 built-in Kubernetes resource types and Custom Resource Definitions (CRDs) such as ArgoCD Applications and Prometheus Operator resources in the Kubernetes Event input. HPA events are enabled by default alongside CoreEvent. RBAC validation provides clear error messages with remediation guidance when permissions are missing.
  • Last Aggregation Type: Added a last aggregation type to the Aggregate Metric processor, enabling capture of the most recent metric value within an aggregation interval. This is useful for gauge-type metrics such as memory usage, queue depth, and connection counts.
  • Seeker Strategies: Introduced explicit start and end strategies for file input, providing clear and predictable file tailing behavior. When a file is encountered, the agent checks for a persistent cursor first, then applies the configured strategy (beginning or end), replacing the implicit behavior based on ED_FRESH_FILE_AGE_LIMIT.
  • Route Processor Error Mode: Added error_mode configuration to the Route (OTTL) and Route (legacy) processors, allowing control over how routing errors are handled. When set to strict, errors are logged and reported as metrics. The default silent mode silently ignores errors.
  • Live Tail Log Expiration: Added a configurable retention period for live tail logs in pipeline settings. When set, logs automatically expire after the configured duration. The default behavior (no expiration until new writes trigger cleanup) remains unchanged.
  • GCL Destination Metadata and Body Mapping: Added metadata mapping and body mapping sections to the Google Cloud Logging destination, enabling direct path-based mapping of OTEL item fields to destination-specific metadata and body content. This provides a structured alternative to using OTTL processors in the pipeline for field mapping.
  • Memory Buffer for Output Nodes: Added a configurable memory_buffer setting to output nodes, allowing control over the maximum number of events held in the in-memory buffer queue before blocking.

Improvements

  • Input Node Performance: Parallelized processing and added backpressure mechanisms for push-based input nodes including HTTP port, OTLP, Datadog Agent, Splunk HEC, Elastic, and Edge Delta port. These changes eliminate response writing bottlenecks, prevent excessive memory consumption and GC pressure under high load, and use bounded worker pools to avoid unbounded goroutine creation under high concurrency.
  • PII Mask Processor Optimization: Integrated a hybrid approach using Ragel for predefined PII patterns and a fast-match package with regex for custom patterns, improving performance for PII masking operations.
  • Live Capture Performance: Improved live capture performance by making data collection non-blocking, reducing CPU usage under high load.
  • Histogram Bucket Support for Rollup Agent: Improved histogram metric aggregation in the rollup agent with canonical bucket types and merge logic, enabling proper bucket alignment and quantile calculations (p50, p90, p95, p99) across agents.
  • S3 Source Configurable Polling: Added poll_interval and immediate_poll_threshold fields to the S3 source (and CrowdStrike FDR) node. The immediate poll threshold triggers re-polling without delay when message volume is high, improving throughput for high-volume SQS queues.
  • Kafka Source Multiple Topics: Enabled the Kafka source node to consume from multiple topics simultaneously using consumer groups. Replaced comma-separated topic fields with text array fields.
  • Parallel Worker Count and Channel Size Configuration: Moved parallel_worker_count and channel_size settings from environment variables to node configuration for push-based input nodes including HTTP, OTLP, Datadog Agent, Elastic, Splunk HEC, Edge Delta Gateway, TCP, and UDP.
  • Relaxed Config Unmarshaling: Introduced lenient YAML unmarshaling for agent configuration, discarding unknown fields during config parsing. This prevents agent crash loops when new configuration fields are introduced in newer versions.
  • Live Tail Settings: Retired lower and upper threshold size settings from live tail configuration.

Bug Fixes

  • Seeker EOF and Buffer Issues: Fixed a buffer padding bug where the seeker returned zero-padded data when fewer bytes were read than the buffer size, causing log line corruption and approximately 1% data loss. Also fixed EOF handling to retain the final chunk of data instead of discarding it.
  • File Rotation Fingerprinting: Introduced a cross-platform fingerprinting approach for file identity tracking during rotation, using initial file content as a fingerprint when proper inode support is unavailable. This resolves event loss during file rotation on Windows.
  • Fatal Crash in Capturer: Fixed a race condition in concurrent map iteration and write in the capturer by switching from shallow to deep copy for map operations.
  • Suspended Pipeline Configuration: Fixed an issue with suspended pipeline configuration handling.
  • GCL Destination Custom Label Handling: Fixed an issue where messages were dropped entirely when a custom label key did not exist in the attributes. Messages are now delivered without the missing label instead of being dropped.
  • eBPF Tracer Shutdown: Fixed eBPF tracer shutdown timeout that caused agent pod crash loops during configuration reloads. Replaced the stop flag with context-based cancellation to properly interrupt blocked epoll syscalls and ensure graceful shutdown within the allowed timeout.
  • Live Tail Settings Persistence: Fixed an issue where disabling live tail in pipeline settings was not persisted after page refresh.
  • Nil Map Panic in Processors: Fixed GetResource and GetAttributes methods to return empty maps instead of nil, preventing a panic when processors attempted to write to nil map attributes.
  • EDXLookup Config Parsing: Fixed config parsing to handle pcommon.Map type in addition to native Go maps, resolving failures when EDXLookup was used with match_mode configuration in the stateful alert processor.
  • File Tailer Telemetry with Glob Patterns: Fixed telemetry reporting for file tailers using glob patterns. Multiple tailers from the same glob pattern now report telemetry correctly instead of conflicting on a shared identifier.
  • Windows Event Message Encoding: Fixed an OTTL transform error where Windows Event input stored the body as bytes instead of string, causing ParseJSON and other string-based OTTL functions to fail with a type mismatch.
  • HTTP Input Buffer Size: Parameterized buffer size in the HTTP input node via configuration, replacing the hardcoded default.

Security

  • Go Runtime Upgrade: Upgraded Go to 1.25.6 to address CVE-2025-61726, CVE-2025-61728, and CVE-2025-61730.

Breaking Changes

  • Input Node Configuration Changes: The parallel_worker_count and channel_size settings moved from environment variables to node configuration for push-based input nodes including HTTP, OTLP, Datadog Agent, Elastic, Splunk HEC, Edge Delta Gateway, TCP, and UDP. Update your configuration to use node-level settings instead of environment variables.
  • Kafka Source Comma-Separated Fields Retired: The Kafka source node no longer accepts comma-separated values for topic and other multi-value fields. Use the new text array field format instead.
  • Live Tail Threshold Settings Retired: The lower and upper threshold size settings have been removed from live tail configuration. Existing configurations that reference these settings should be updated.