Edge Delta Kubernetes Source

Monitor Kubernetes pods and namespaces.

5 minute read

Overview

You can use the Kubernetes source node to specify which Kubernetes pods and namespaces the agent should monitor.

outgoing_data_types: log

Example Configuration

nodes:
- name: my_k8s_input
  type: kubernetes_input
  include:
    - k8s.namespace.name=.*
  exclude:
    - k8s.namespace.name=kube-system
    - k8s.namespace.name=kube-public
    - k8s.namespace.name=kube-node-lease
    - k8s.pod.name=edgedelta
    - k8s.pod.name=prometheus
    - k8s.pod.name=promtail
    - k8s.pod.name=node-exporter
  resource_fields:
    pod_labels:
      - app.kubernetes.io/instance
      - app.kubernetes.io/name
      - helm.sh/chart
      - kubernetes.io/.*
    pod_annotations:
      - cluster-autoscaler.kubernetes.io/safe-to-evict
      - cluster-autoscaler.kubernetes.io/.*
    node_labels:
      - beta.kubernetes.io/arch
      - beta.kubernetes.io/instance-type
      - eks.amazonaws.com/.*
      - topology.kubernetes.io/zone
    namespace_labels:
      - kubernetes.io/metadata.name
      - name
  line_pattern: '^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}'
  preserve_original_timestamp: true
  enable_persisting_cursor: true

Controlling Ingestion Based on Object Tenancy

Each include/exclude line prefaced with a - is a complete object reference, and each object is treated as a complete set of selection criteria. For example, this configuration will exclude all telemetry data from namespace nginx and all pods that start with webserver anywhere in the cluster:

    exclude:
      - k8s.namespace.name=nginx
      - k8s.pod.name=webserver.*

However the following configuration will only exclude telemetry data from pods prefaced with webserver when they exist in the namespace nginx. webserver.* pods that exist in other namespaces will not be excluded in this example:

    exclude:
      - k8s.namespace.name=nginx,k8s.pod.name=webserver.*

Required Parameters

name

A descriptive name for the node. This is the name that will appear in Visual Pipelines and you can reference this node in the YAML using the name. It must be unique across all nodes. It is a YAML list element so it begins with a - and a space followed by the string. It is a required parameter for all nodes.

nodes:
  - name: <node name>
    type: <node type>

type: kubernetes_input

The type parameter specifies the type of node being configured. It is specified as a string from a closed list of node types. It is a required parameter.

nodes:
  - name: <node name>
    type: <node type>

Optional Parameters

auto_detect_line_pattern

YAML Only

The auto_detect_line_pattern parameter is a Boolean value that, when set to true, enables automatic detection of line patterns in logs. This automatically determine the structure of log lines, rather than relying on pre-defined patterns.

nodes:
  - name: my_file_input
    type: kubernetes_input
    auto_detect_line_pattern: true

The auto_detect_line_pattern parameter is optional and defaults to false if not specified.

boost_stacktrace_detection

YAML Only

The boost_stacktrace_detection parameter is used with auto_detect_line_pattern: true. It enables stack trace detection based on the Ragel FSM Based Lexical Recognition process. Found stack traces will be grouped together in the same log message. It is specified with a Boolean, the default is false and it is optional.

nodes:
  - name: my_file_input
    type: kubernetes_input
    auto_detect_line_pattern: true
    boost_stacktrace_detection: true

enable_persisting_cursor

The enable_persisting_cursor parameter is used to enable a persisting cursor in the event of an agent restart. It is specified as a Boolean, the default is false and it is optional.

nodes:
  - name: <node name>
    type: kubernetes_input
    enable_persisting_cursor: true

discovery

The discovery parameter over-rides Edge Delta’s default file discovery logic for looking for files on Kubernetes pods in mounted filesystems. This is useful for retaining Kubernetes metadata context. You specify two parameters:

file_path: A glob path that defines a source for logs files.
parsing_pattern: A Go regex pattern to parse captured literal file paths in order to observe the pod UID. This requires a capture group called pod_uid. You can optionally observe the container name, which requires the capture group container_name.

nodes:
  - name: <node name>
    type: kubernetes_input
    discovery:
      file_path: /var/logs/pods/*/kubernetes.io~emptyDir/**/*.log
      parsing_pattern: ^/var/logs/pods/(?P<pod_uid>[a-zA-Z0-9\-]+)/(?P<container_name>[a-zA-Z0-9\-_]+)/.*

exclude

The exclude parameter is used to excluded traffic from certain components from being ingested. It is specified in OTEL format with Golang regex or full text. If exclude is specified it drops logs that match its Golang regex rules. Comma separated values indicate AND rules, while separate rules indicate OR matching.

In this example logs that match rule-1 and rule-2 are excluded. Logs that match rule-3 are also excluded.

      exclude:
        - "rule-1,rule-2"
        - "rule-3"

Exclude is optional.

If include is specified as well as exclude, then logs matching in include are passed unless they are dropped by exclude.

nodes:
  - name: <node name>
    type: kubernetes_input
    include:
    - k8s.namespace.name=.*
    exclude:
    - k8s.namespace.name=kube-system

The following values can be excluded from a Kubernetes source:

k8s.pod.name=
k8s.namespace.name=
k8s.cronjob.name=
k8s.daemonset.name=
k8s.deployment.name=
k8s.job.name=
k8s.replicaset.name=
k8s.statefulset.name=
k8s.pod.labels.=
k8s.container.name=
container.image.name=

For example:

k8s.namespace.name=.*
k8s.namespace.name=kube-system
k8s.namespace.name=nginx,k8s.deployment.name=nginx-canary.*
k8s.pod.labels.app=nginx

You can use k8s.{kind}.name=.+ rather than k8s.controller.kind=xxx (which is no longer supported), where the values of {kind} match the Kubernetes kind, such as k8s.daemonset.name, k8s.deployment.name, k8s.job.name, etc.

include

The include parameter is used to specify the scope of traffic sources to be ingested. It is specified in OTEL format with Golang regex or full text. If exclude is also specified, then logs matching in include are passed unless they are dropped by exclude. Include is optional.

nodes:
  - name: <node name>
    type: kubernetes_input
    include:
    - k8s.namespace.name=.*

The following values can be included in the scope for a Kubernetes source:

k8s.pod.name=
k8s.namespace.name=
k8s.cronjob.name=
k8s.daemonset.name=
k8s.deployment.name=
k8s.job.name=
k8s.replicaset.name=
k8s.statefulset.name=
k8s.pod.labels.=
k8s.container.name=
container.image.name=

For example:

k8s.namespace.name=.*
k8s.namespace.name=kube-system
k8s.namespace.name=nginx,k8s.deployment.name=nginx-canary.*
k8s.pod.labels.app=nginx

You can use k8s.{kind}.name=.+ rather than k8s.controller.kind=xxx (which is no longer supported), where the values of {kind} match the Kubernetes kind, such as k8s.daemonset.name, k8s.deployment.name, k8s.job.name, etc.

line_pattern

YAML Only

The line_pattern parameter specifies a Golang regex pattern that will be used as a line break rather than using a new line. It is specified with a Golang regex pattern, including a ^ for the line start, and it is optional.

nodes:
  - name: <node name>
    type: <node type>
    line_pattern: ^\d{4}-\d{2}-\d{2}

preserve_original_timestamp

The preserve_original_timestamp parameter is used to configure the agent to preserve the timestamp that the incoming log already has. It is specified as a Boolean value and the default is false. It is optional.

nodes:
  - name: <node name>
    type: kubernetes_input
    preserve_original_timestamp: true

resource_fields

The resource_fields parameters define the labels and annotations to be collected as metadata fields and included. It is specified as a dictionary of regexes and is optional.

You can specify:

pod_labels
pod_annotations
node_labels
namespace_labels

nodes:
  - name: <node name>
    type: kubernetes_input
    resource_fields:
      pod_labels:
        - app.kubernetes.io/instance
        - app.kubernetes.io/name
        - helm.sh/chart
        - kubernetes.io/.*
      pod_annotations:
        - cluster-autoscaler.kubernetes.io/safe-to-evict
        - cluster-autoscaler.kubernetes.io/.*
      node_labels:
        - beta.kubernetes.io/arch
        - beta.kubernetes.io/instance-type
        - eks.amazonaws.com/.*
        - topology.kubernetes.io/zone
      namespace_labels:
        - kubernetes.io/metadata.name
        - name

sampling

YAML Only

The sampling parameter specifies the sampling rate for the input payloads. It is specified with a Boolean with false being 0.0 and true being 1.0, the default is false, and it is optional.

nodes:
  - name: my_file_input
    type: kubernetes_input
    sampling: true

Edge Delta Kubernetes Source

Overview

Example Configuration

Controlling Ingestion Based on Object Tenancy

Required Parameters

name

type: kubernetes_input

Optional Parameters

auto_detect_line_pattern

boost_stacktrace_detection

enable_persisting_cursor

discovery

exclude

include

line_pattern

preserve_original_timestamp

resource_fields

sampling

See Also