Edge Delta on Kubernetes

The setup process for deploying Edge Delta within a Kubernetes cluster natively leverages the Kubernetes APIs and other components to provide a seamless onboarding, and automated monitoring experience.

Many of the commonly used Kubernetes Services are supported by Edge Delta:

• Amazon EKS
• Azure AKS
• Google GKE
• OpenShift Enterprise Kubernetes
• Bare Metal Kubernetes

If you are new to Kubernetes, please refer to documentation for a few tools to create your first cluster:

• Docker
• Kubeadm
• AWS & Kops
• AWS EKS
• GCP GKE
• Azure AKS

Kubernetes Requirements

The Edge Delta fleet uses eBPF to collect network metrics. Therefore, the following Kubernetes environment configuration is required for Kubernetes network metrics and eBPF to work:

• Linux kernel version 5.8 or later.
• Linux kernel built with the CONFIG_DEBUG_INFO_BTF=y and CONFIG_DEBUG_INFO_BTF_MODULES=y flags.

To check for the flag:

docker run -it --rm --privileged --pid=host ubuntu nsenter -t 1 -m -u -n -i sh -c 'cat /proc/config.gz | gunzip | grep CONFIG_DEBUG_INFO_BTF'

The output should show CONFIG_DEBUG_INFO_BTF=y and CONFIG_DEBUG_INFO_BTF_MODULES=y.

In the case of minikube, the agent can run on minikube with the Docker driver. Docker must be at least v26.0.0 (Docker Desktop v4.29.0) and it is started as follows:

minikube start --driver docker

Kubernetes Source

Once the Edge Delta fleet is running within a Kubernetes cluster, telemetry data (logs and metrics) generated by other pods in the cluster are automatically monitored and analyzed by the Edge Delta Fleet in real time. The Edge Delta Fleet is aware of the Kubernetes components of the cluster (pods, namespaces, deployments, etc.) natively by utilizing the Edge Delta Kubernetes source.

Version 0.1.70 and older agents use the v2 configuration’s Kubernetes source. See Agent Configuration for more information about agent and configuration versions.

Kubernetes Deployment Overview

There are a number of different ways to deploy an Edge Delta fleet in a Kubernetes environment: kubectl apply commands, using Helm, or via Terraform.You can also automate deployments with using a tool such as ArgoCD.

Kubernetes Manifest

The recommended installation method for Edge Delta is using the Helm chart. However, a Kubernetes manifest is available. It is generated automatically for each helm chart release. The manifest contains the following sections:

• Namespace: The edgedelta namespace is reserved for all Edge Delta components, providing a separate environment to enhance organization and isolation from other services within the cluster.
• ServiceAccount: A service account named edgedelta is created specifically for authentication purposes. This account is vital for ensuring secure interactions of Edge Delta components with the Kubernetes API.
• ClusterRole and ClusterRoleBinding: These elements assign the necessary permissions to the edgedelta service account, enabling it to access and manage various Kubernetes resources critical for Edge Delta’s operations. Permissions include getting, listing, watching, creating, updating, patching, and deleting resources such as namespaces, pods, events, nodes, services, daemonsets, deployments, statefulsets, jobs, cronjobs, and leases.
• DaemonSet: Deploys the Edge Delta Processor Agent across all nodes in the cluster via a DaemonSet, ensuring extensive data collection and processing coverage. The agent is provided with significant privileges to access and gather necessary metrics and logs through system interactions. It leverages host-level permissions like hostPID and hostNetwork, with volume mounts for persistent storage and access to critical system paths.
• Services: Two distinct services are configured: ed-compactor-svc and ed-rollup-svc. These services establish communication paths for backend processing units, using ports 9199 and 9200, respectively, to maintain connectivity with the Compactor and Rollup agents.
• Deployment and StatefulSet:
  • edgedelta-rollup: This Deployment manages the Rollup component, facilitating metric data aggregation. It includes configurations for container resources, persistent storage, and a rolling update strategy.
  • edgedelta-compactor: A StatefulSet is used to deploy the Compactor, processing log data with configurations that mirror those of the Rollup component, including container setup, resource allocation, and data persistency.

Kubernetes Permissions

Based Access Control (RBAC) is configured for Edge Delta out of the box. In the default Edge Delta installation, the ClusterRole defines a set of permissions (verbs) for certain resources across the entire Kubernetes cluster or within specific namespaces. For Edge Delta, the permissions are critical for its operation as it needs to monitor and log activity across multiple resources. These verbs include get, list, watch, create, update, patch, and delete. These operations allow Edge Delta to observe the cluster state and interact with resource definitions. The permissions give access to certain resources:

• Namespaces, Pods, Events, Nodes: For capturing overall system state and pod-level logs and metrics.
• Services, DaemonSets, Deployments, StatefulSets: To manage and observe the health and status of applications and their components.
• Jobs, CronJobs, Leases: For understanding and monitoring background and scheduled tasks.

A service account ensures Edge Delta components can securely interact with the Kubernetes API to perform operations such as log collection, monitoring, and data processing without manual or insecure handling of credentials. The edgedelta service account, as defined in the manifest, is essential for managing the necessary access to system-level resources in a controlled manner. ClusterRoleBinding associates the ClusterRole with the edgedelta service account. This binding is what actually grants the service account the permissions defined in the ClusterRole.

The DaemonSet is responsible for deploying the Edge Delta Processor Agent across all nodes, ensuring seamless monitoring and log aggregation from all parts of the cluster. Given the nature of its work, the agent requires elevated privileges to access node-level data and system logs. Host-level permissions like hostPID and hostNetwork allow the agent to operate with broader system visibility and network access, which are crucial for collecting comprehensive data.

For some environments, additional securityContext configurations are required. For example see SELinux-enforced Kubernetes clusters.

Identity and Access Management (IAM) in Cluster

Within Kubernetes, IAM is primarily managed using service accounts in conjunction with RBAC (Role-Based Access Control). Each service account, like the edgedelta account in your deployment, is assigned specific roles and permissions, dictated by RBAC. Roles and role bindings ensure that each component of Edge Delta has access only to the required resources necessary for its operation. This integration of IAM ensures least privilege access, reducing the risk of unauthorized operations. IAM in Kubernetes is designed to separate roles and responsibilities, allowing developers, administrators, and operations teams to have access only to the tools and data they need. This approach limits the impact of potential security breaches or insider threats.

When running Kubernetes in cloud environments, integrating cloud IAM with Kubernetes RBAC becomes crucial. Cloud IAM roles and permissions can be extended to control access to cloud-native services and resources that Edge Delta might interact with, such as storage, networking, or specific APIs.

See AWS Authentication for Edge Delta and AWS Assumed Roles for Edge Delta.

Automated Kubernetes Monitoring

Once Edge Delta is running within your Kubernetes environment, it automatically begins analyzing and monitoring telemetry data locally from the active pods within the cluster. The incoming data is enriched with detailed metadata from the environment and analyzed.

Without any custom configuration, Edge Delta will begin analyzing data, generating patterns, findings, and statistics, as well as baselining the behavior to provide automated Anomaly Detection with Machine Learning.

Insights, Statistics, Patterns, and Findings generated by Edge Delta are streamed in real time to streaming destination nodes (or Streaming Destinations for v0.1.70 and older agents). Additionally, these insights are exposed by default in the Edge Delta UI, with flexible filtering, aggregation, and other commonly used investigation tools.

Related Content