Ingest Kubernetes Metrics in Edge Delta

Prepare your Kubernetes cluster to ingest metrics using the Edge Delta agent.

2 minute read

Overview

You can use the Kubernetes Traffic source node and the Kubernetes Metrics source node in a Pipeline to ingest Kubernetes metrics, including metrics via eBPF. All Kubernetes metrics families are scraped by default. These metrics can be processed and sent to destinations that can consume metrics.

In addition they enable a number of functions within Edge Delta, including but not limited to the following:

The Kubernetes Dashboard requires KSM and Node Exporter installed in the same cluster as your fleet and you must have a Kubernetes Metrics source node connected to the Edge Delta Destination node.
The types of metrics available in the Metrics Explorer will depend on what Source nodes you’ve configured for Metrics ingestion.
The Kubernetes Traffic node is required to enable the service map, and that node must be connected to the Edge Delta Destination node.

For a complete list of Kubernetes metrics ingested by the Edge Delta agent, please see the metrics list.

Prepare the Cluster

To ingest Kubernetes metrics, you need to prepare your cluster:

To ingest Kube State Metrics, KSM must be deployed in the same cluster as the Edge Delta agent.
To ingest node-exporter metrics, node-exporter must be deployed in the same cluster as the Edge Delta agent.
All metric source nodes and processing nodes that produce metrics, such as logs to metrics nodes, must be connected to the Edge Delta Destination node.
As of v1.27.0 the Metrics Source node scrapes kube_state_metrics only. The kubelet, cadvisor, and node_exporter metrics are excluded by default. You can remove them from the exclude list if you want to include them.

The Edge Delta agent uses eBPF to collect network metrics. Therefore, the following Kubernetes environment configuration is required for Kubernetes network metrics and eBPF to work:

Linux kernel version 5.8 or later.
Linux kernel built with the CONFIG_DEBUG_INFO_BTF=y and CONFIG_DEBUG_INFO_BTF_MODULES=y flags.

To check for the flag:

docker run -it --rm --privileged --pid=host ubuntu nsenter -t 1 -m -u -n -i sh -c 'cat /proc/config.gz | gunzip | grep CONFIG_DEBUG_INFO_BTF'

The output should show CONFIG_DEBUG_INFO_BTF=y and CONFIG_DEBUG_INFO_BTF_MODULES=y.

In the case of minikube, the agent can run on minikube with the Docker driver. Docker must be at least v26.0.0 (Docker Desktop v4.29.0) and it is started as follows:

minikube start --driver docker

If you want to disable eBPF:

Delete the Kubernetes Traffic source node.
Disable the tracer:

Helm Rerun the Helm upgrade command with the --set tracerProps.enabled=false flag.

Kubectl Update and re-apply the kubernetes manifest with the following parameter change:

ED_ENABLE_TRAFFIC_TRACER = "0"