Edge Delta Loki Output

Stream data to Loki.

Overview

The Loki output will stream analytics and insights to your Loki endpoint.

Example

      - name: loki-integration
        type: loki
        endpoint: "https://localhost:3000/loki/api/v1/push"
        api_key: "api_key"
        user: "user"
        custom_tags:
          "app": "test"
          "region": "us-west-2"
        message_template:
          "File Path": "{{.FileGlobPath}}"
          "K8s PodName": "{{.K8sPodName}}"
          "K8s Namespace": "{{.K8sNamespace}}"
          "K8s ControllerKind": "{{.K8sControllerKind}}"
          "K8s ContainerName": "{{.K8sContainerName}}"
          "K8s ContainerImage": "{{.K8sContainerImage}}"
          "K8s ControllerLogicalName": "{{.K8sControllerLogicalName}}"
          "ECSCluster": "{{.ECSCluster}}"
          "ECSContainerName": "{{.ECSContainerName}}"
          "ECSTaskVersion": "{{.ECSTaskVersion}}"
          "ECSTaskFamily": "{{.ECSTaskFamily}}"
          "DockerContainerName": "{{.DockerContainerName}}"
          "ConfigID": "{{.ConfigID}}"
          "Host": "{{.Host}}"
          "Source": "{{.Source}}"
          "SourceType": "{{.SourceType}}"
          "Tag": "{{.Tag}}"

Parameters

name

Required

Enter a descriptive name for the output or integration.

For outputs, this name will be used to map this destination to a workflow.

name: loki-integration

integration_name

Optional

This parameter refers to the organization-level integration created in the Integrations page.

If you need to add multiple instances of the same integration into the config, then you can add a custom name to each instance via the name parameter. In this situation, the name should be used to refer to the specific instance of the destination in the workflows.

integration_name: orgs-kafka

type

Required

Enter loki.

type: loki

endpoint

Required

Enter the Loki endpoint.

endpoint: "https://localhost:3000/loki/api/v1/push"

user

Optional

Enter the username for Loki.

user: "user"

api_key

Optional

Enter the Loki API key.

api_key: "api_key"

alert_as_log

Optional

Enter true to send alerts as a log.

Additionally, for this parameter to work, you must also enter log as a feature type. Enter false to send alerts as events.

alert_as_log: true

custom_tags

Optional

This parameter defines key-value pairs that are streamed with every request.

This parameter supports templating.

      custom_tags:
        "app": "test"
        "region": "us-west-2"
        "File Path": "{{.FileGlobPath}}"
        "K8s PodName": "{{.K8sPodName}}"
        "K8s Namespace": "{{.K8sNamespace}}"
        "K8s ControllerKind": "{{.K8sControllerKind}}"
        "K8s ContainerName": "{{.K8sContainerName}}"
        "K8s ContainerImage": "{{.K8sContainerImage}}"
        "K8s ControllerLogicalName": "{{.K8sControllerLogicalName}}"
        "ECSCluster": "{{.ECSCluster}}"
        "ECSContainerName": "{{.ECSContainerName}}"
        "ECSTaskVersion": "{{.ECSTaskVersion}}"
        "ECSTaskFamily": "{{.ECSTaskFamily}}"
        "DockerContainerName": "{{.DockerContainerName}}"
        "ConfigID": "{{.ConfigID}}"
        "Host": "{{.Host}}"
        "Source": "{{.Source}}"
        "SourceType": "{{.SourceType}}"
        "Tag": "{{.Tag}}"

message_template

Optional

This parameter customizes the message content.

This parameter supports templating.

See Template Fields.

      message_template:
        "File Path": "{{.FileGlobPath}}"
        "K8s PodName": "{{.K8sPodName}}"
        "K8s Namespace": "{{.K8sNamespace}}"
        "K8s ControllerKind": "{{.K8sControllerKind}}"
        "K8s ContainerName": "{{.K8sContainerName}}"
        "K8s ContainerImage": "{{.K8sContainerImage}}"
        "K8s ControllerLogicalName": "{{.K8sControllerLogicalName}}"
        "ECSCluster": "{{.ECSCluster}}"
        "ECSContainerName": "{{.ECSContainerName}}"
        "ECSTaskVersion": "{{.ECSTaskVersion}}"
        "ECSTaskFamily": "{{.ECSTaskFamily}}"
        "DockerContainerName": "{{.DockerContainerName}}"
        "ConfigID": "{{.ConfigID}}"
        "Host": "{{.Host}}"
        "Source": "{{.Source}}"
        "SourceType": "{{.SourceType}}"
        "Tag": "{{.Tag}}"

features

Optional

This parameter defines which data types to stream to the destination.

If you do not provide a value, then all will be set.

To learn more, see the following section on supported feature types.

features: log

buffer_ttl

Optional

Enter a length of time to retry failed streaming data.

After this length of time is reached, the failed streaming data will no longer be tried.

buffer_ttl: 2h

buffer_path

Optional

Enter a folder path to temporarily store failed streaming data.

The failed streaming data will be retried until the data reaches its destinations or until the Buffer TTL value is reached.

If you enter a path that does not exist, then the agent will create directories, as needed.

buffer_path: /var/log/edgedelta/pushbuffer/

buffer_max_bytesize

Optional

Enter the maximum size of failed streaming data that you want to retry.

If the failed streaming data is larger than this size, then the failed streaming data will not be retried.

buffer_max_bytesize: 100MB

Template Fields

As an optional step, you can customize the message payload and custom tags that are sent to a Loki destination.

  • Loki does not support the dash ( - ) character as a key value.
Field Description
Tag This field is the user-defined tag that describes the environment, such as prod_us_west_2_cluster.
Host This field is the hostname of the environment where the agent is running on.
ConfigID This field is the configuration ID of the corresponding agent.
Source This field is the source name, specifically the identifier of the source, such as docker container id or file name.
SourceType This field is the source type, such as Docker or system.
FileGlobPath This field is the file global path.
K8sPodName This field is the Kubernetes pod name.
K8sNamespace This field is the Kubernetes namespace.
K8sControllerKind This field is the Kubernetes controller kind.
K8sContainerName This field is the Kubernetes container name.
K8sContainerImage This field is the Kubernetes container image.
K8sControllerLogicalName This field is the Kubernetes controller logical name.
ECSCluster This field is the ECS cluster name.
ECSContainerName This field is the ECS container name.
ECSTaskVersion This field is the ECS task version.
ECSTaskFamily This field is the ECS task family.
DockerContainerName This field is the Docker container name.

Supported Features

See Streaming Features.

Feature Type Supported?
Log Yes
Metrics No
Alert as event No
Alert as log Yes
Health No
Dimensions as attribute No
Send as is No
Send as JSON No
Custom tags Yes
EDAC enrichment No
Message template Yes
outgoing_bytes.sum Yes
outgoing__raw_bytes.sum No
outgoing_lines.count Yes
output buffering to disk No