Edge Delta Metrics

Overview

Metrics, logs and traces are types of signals emitted by (properly instrumented) applications or systems to enable monitoring and troubleshooting. Metrics are numeric data or measurements of a system and the data is usually aggregated over time.

Metric Signals vs Log to Metric

There are two types of metrics in Edge Delta. Log to metrics are measurements that are derived by the Edge Delta agent by evaluating the content of logs. For example, the rate of 4xx errors reported in logs that are emitted by NGINX servers is determined by the agent by by examining /var/log/nginx/error.log, identifying 4xx error logs, counting the number of such errors for a specific duration, and emitting a metric data item.

Metric signals, on the other hand, are generated directly by the data source. For example, the rate of 4xx errors being reported by NGINX servers can be determined by consuming metric signals from an nginx-prometheus-exporter.

Metric Signals Examples

• Golden Signals: CPU, RAM, Network
• Kubernetes Metrics: kubeapi, kubelet, cAdvisor
• Kube State Metrics: kube_secret_type{namespace="vm",secret="vm-grafana",type="Opaque"} 1
• Application Metrics:
  • Metrics that are generated by applications directly, either alongside or instead of log data such as nginxplus_upstream_server_responses{code=”4xx”,server=”192.168.45.119”,upstream”my_servers”} 23451
  • Raw state metrics such as Google Nest metrics: nest_ambient_temperature{id="abcd1234",label="Living-Room",scale=”fahrenheit”} 72

Metric Signal Sources

• Linux/VM
  • eBPF (node, syscall)
  • System (various exporters like node_exporter, /proc)
• Docker
  • Linux + cAdvisor (container metrics)
• Kubernetes
  • Docker + cAdvisor (kubelet) + KSM (Kube State Metrics, kubeapi)
• Custom Metrics
  • Any/all of the other metric signal sources
  • Custom exporters from infrastructure and app providers (Cilium, NGINX, Mongo, etc)

Edge Delta Metrics

The Edge Delta agent can ingest metric signals with full dimensions, including Kubernetes Source metrics and it handles metrics routing using pipeline builder.

Requirements

See Ingest Kubernetes Metrics.