Agent v2.13.0
Fasthttp migration for HTTP inputs, ED Watcher for agent health monitoring, OTTL v0.146.0 with new functions, and performance improvements across input, output, and processor nodes.
4 minute read
March 10, 2026
Breaking Changes
HTTP Input Server Migration to Fasthttp
All HTTP-based source nodes (HTTP, Splunk HEC, OTLP, Datadog Agent, Elastic API) have been migrated from Go’s standard net/http to fasthttp for improved performance and lower memory usage. This introduces new default limits on request body size and header size that did not exist previously.
New configuration fields:
| Parameter | Default | Description |
|---|---|---|
drop_limit | 128 MB | Maximum item size. Items exceeding this limit are dropped with a metric recorded. |
truncate_limit | disabled | Maximum item size before truncation. Items between this and drop_limit are truncated. |
read_buffer_size | 16 KB | Per-connection read buffer size. Also controls the maximum header size. |
write_buffer_size | 4 KB | Per-connection write buffer size. |
max_connections | unlimited | Maximum concurrent connections. |
max_connections_per_ip | unlimited | Maximum concurrent connections per IP. |
idle_timeout | 1m | Idle connection timeout. |
write_timeout | 10s | Response write timeout. |
Who is affected: Environments sending large request bodies or headers that previously had no size limit may encounter HTTP 413 (Request Entity Too Large) or HTTP 431 (Request Header Fields Too Large) errors. Increase drop_limit or read_buffer_size in the input node configuration to resolve.
The reader_header_timeout parameter is deprecated in v2.13.0.
New Features
- ED Watcher: Added a new agent health monitoring component that detects out-of-memory events, crash loops, evictions, and scheduling failures. For Kubernetes deployments, ED Watcher runs as a sidecar container enabled via
--set watcherProps.enabled=truein Helm. For host deployments, it runs as a standalone process. ED Watcher is disabled by default and reports issues as time series metrics (ed.agent.oom,ed.agent.crashloop,ed.agent.evicted,ed.agent.failed_scheduling,ed.agent.process_not_found). - OTTL v0.146.0: Upgraded the OTTL engine to v0.146.0, adding 9 new converter and editor functions:
TrimPrefix,TrimSuffix,Bool,XXH3,XXH128,IsInCIDR,CommunityID,ParseSeverity, anddelete_index. - Metric Sampling: The sample processor now supports metric items in addition to traces and logs, enabling probabilistic sampling of metric data.
- Custom Kubernetes Event Selection: Added the ability to select specific custom Kubernetes events in the Kubernetes Event input, allowing inclusion or exclusion of custom priority events.
- GCS Custom Item Support: Added custom and deotelized item support for the GCS destination, enabling direct write of non-OTEL formatted data.
- Splunk HEC Weighted Destinations: Added weighted destination configuration for load balancing in the Splunk Load Balanced destination, aligning with standard load balancing configuration patterns.
- Connector Event Filtering: Added filtering for incoming connector events, allowing agents to process and filter connector event items.
- Demo Template Fractional Values: The telemetry generator input node now supports fractional values for more precise telemetry generation.
Improvements
- Splunk Input Parallelization: Added parallel worker support for Splunk HEC and Splunk TCP input nodes, enabling configurable parallel worker counts at the node level for higher throughput.
- OTLP and Gateway Backpressure: Improved backpressure handling for the OTLP and Gateway input nodes, reducing memory pressure under high load.
- S3 Source Concurrency: Added concurrent message processing and file downloads for the S3 source node, improving throughput when consuming from high-volume SQS queues.
- Regex Performance: Integrated the fastmatch package for regex matching across the mask processor, tail sampling,
EDXDeleteMatchingKeys, andEDXKeepMatchingKeys, improving regex evaluation performance. - EDXCompress and EDXDecompress Performance: Improved compression and decompression performance in the
EDXCompressandEDXDecompressOTTL extension functions. - EDXParseKeyValue Memory Optimization: Reduced memory usage in
EDXParseKeyValuethrough buffer pool implementation, significantly lowering memory consumption for key-value parsing workloads. - Lookup Processor Optimization: Optimized the lookup processor with inverted indexes for exact match lookups, delivering 2-3x performance improvement on lookup operations.
- Kubernetes Event Enrichment: Kubernetes resource events now include
service.nameenrichment for improved correlation. - Agent Heartbeat Versioning: Agent heartbeats now include the deploy configuration version timestamp for improved configuration tracking.
- GCL Destination Mapping: Added API and pipeline editor support for metadata mapping and body mapping in the Google Cloud Logging destination, enabling configuration through the UI.
- HTTP Input Truncation and Drop Options: Added configurable
truncate_limitanddrop_limitoptions for fasthttp-backed HTTP input nodes, allowing fine-grained control over oversized request handling.
Bug Fixes
- Aggregate Metric Validation: Fixed a validation error that prevented selecting the “Last” aggregation type in the pipeline editor, which was introduced as a feature in v2.12.0 but was missing from backend config validation.
- Kubernetes Event Severity and Naming: Fixed severity text values (normalized to lowercase) and aligned event type naming conventions for Kubernetes events and resource CRDs.
- OTLP gRPC Shutdown: Fixed a shutdown issue affecting OTLP with gRPC connections that could cause hanging during agent restart.
- Fasthttp TLS Configuration: Fixed TLS configuration handling for fasthttp-backed HTTP input nodes.
- Kubernetes Event Resource Forwarding: Fixed a concurrent map read/write race condition in Kubernetes event resource forwarding.
- OTTL Context Filter Race Condition: Fixed a time-of-check-time-of-use race condition in the OTTL context filter that caused inconsistent flush counts.
- Processor Node Validation: Fixed config validation for newly added processor nodes.
Security
- Go Runtime Upgrade: Upgraded Go to 1.26.1 to address CVE-2026-25679, CVE-2026-27137, CVE-2026-27138, and CVE-2026-27142.
- Go Runtime Upgrade: Upgraded Go to 1.25.7 to address CVE-2025-68121.