Edge Delta Data Item

Data items are handled by nodes and transported by links.

  3 minute read  

Paid
Visual Pipelines Beta

Overview

As of agent version 0.1.78, several parameters of the OTEL schema are applied by default for logs. This is relevant when configuring agent nodes that contain CEL and when referencing any field path.

OTEL Parameters

Field Name Description
Timestamp Time when the event occurred as measured by the source. It is formatted as uint64 nanoseconds since Unix epoch.
severity_text A description of the log level.
body The body of the log record.
resource The resource parameter uses a map to list the resources and tags that describe the source of the log.
_type The OTEL signal type.
attributes Event specific information such as the resource, custom field or other context.

Example Kubernetes Input

The following example is a log for a Kubernetes input.

{
  "timestamp": "1581452773000000789",
  "severity_text": "Error",
  "body": "{\"user\": \"userA\",\"operation\": \"delete\",\"outcome\": \"failed\"}",
  "resource": {
    "host.name": "host-1",
    "ed.tag": "test-config",
    "ed.org.id": "0481a213-....",
    "config_id": "12345678-abcd-...",
    "__src_name": "source_name",
    "src_type": "K8s",
    "__logical_source": "logical_source",
    "__short_src_name": "short_source_name",
    "__group_name": "group_name",
    "k8s.namespace.name": "edgedelta",
    "k8s.pod.name": "api-deployment-d79fab72249c",
    "k8s.container.name": "echo:latest",
    "k8s.controller.kind": "Deployment",
    "k8s.controller.logical.name": "api-deployment",
  },
  "_type": "log",
  "attributes": {
    "pod_id":"api-deployment-d79fab72249c-vtq9x", // user enrichment
    "instance_id":"i-1234567890abcdef0",  // user enrichment
    "instance_name":"test-name",  // user enrichment
    "user": "userA",
    "operation": "delete",
    "outcome": "failed",
    "k8s_labels": {
      "app":"my-api"
    },
    "k8s_annotations": {
      "service": "edgedelta",
    },
  },  
}

timestamp

The time at which the log was created. The timestamp is created by the source component.

body

The body of the log containing the raw data.

resource

host.name This is a semantic convention from OTEL for the host server name or IP address.

ed.tag The Edge Delta agent name specified when you create the agent configuration.

ed.org.id The ID for the organization the Edge Delta agent was created in.

__parameter Parameters starting with a double underscore are internal for use by the Edge Delta backend.

Source Components The resources section also references source components. It uses periods that do not indicate nesting. This means that when referencing resources in the UI form or in a YAML file you should use the following format:

item.resource["k8s.namespace.name"]

_type

The OTEL signal type.

attributes

Custom fields such as user enrichments and parsed fields are contained in the attributes section.

Example Docker Input

The following example illustrates the OTEL schema for a Docker input.

{
  "timestamp": "1581452773000000789",
  "severity_text": "Error",
  "body": "{\"user\": \"userA\",\"operation\": \"delete\",\"outcome\": \"failed\"}",
  "resource": {
    "host.name": "host-1",
    "ed.tag": "test-config",
    "ed.org.id": "0481a213-...."
    "config_id": "12345678-abcd-...",
    "__src_name": "source_name",
    "src_type": "K8s",
    "__logical_source": "logical_source",
    "__short_src_name": "short_source_name",
    "__group_name": "group_name",
    "container.name": "container_name",
    "container.image.name": "container_image_name",
  },
  "attributes": {
    "instance_id":"i-1234567890abcdef0",  // user enrichment
    "instance_name":"test-name",  // user enrichment
    "user": "userA",
    "operation": "delete",
    "outcome": "failed",
  },  
}

Example File Input

The following example illustrates the OTEL schema for a File input.

{
  "timestamp": "1581452773000000789",
  "severity_text": "Error",
  "body": "{\"user\": \"userA\",\"operation\": \"delete\",\"outcome\": \"failed\"}",
  "resource": {
    "host.name": "host-1",
    "ed.tag": "test-config",
    "ed.org.id": "0481a213-...."
    "config_id": "12345678-abcd-...",
    "__src_name": "source_name",
    "src_type": "K8s",
    "__logical_source": "logical_source",
    "__short_src_name": "short_source_name",
    "__group_name": "group_name",
    "ed.filepath": "filepath"
  },
  "attributes": {
    "instance_id":"i-1234567890abcdef0",  // user enrichment
    "instance_name":"test-name",  // user enrichment
    "user": "userA",
    "operation": "delete",
    "outcome": "failed",
  },  
}