Edge Delta Loki Output

Stream data to Loki.

4 minute read

Version 2 is no longer supported. This section is for historical reference only.

Overview

The Loki output will stream analytics and insights to your Loki endpoint.

Example

      - name: loki-integration
        type: loki
        endpoint: "https://localhost:3000/loki/api/v1/push"
        api_key: "api_key"
        user: "user"
        custom_tags:
          "app": "test"
          "region": "us-west-2"
        message_template:
          "File Path": "{{.FileGlobPath}}"
          "K8s PodName": "{{.K8sPodName}}"
          "K8s Namespace": "{{.K8sNamespace}}"
          "K8s ControllerKind": "{{.K8sControllerKind}}"
          "K8s ContainerName": "{{.K8sContainerName}}"
          "K8s ContainerImage": "{{.K8sContainerImage}}"
          "K8s ControllerLogicalName": "{{.K8sControllerLogicalName}}"
          "ECSCluster": "{{.ECSCluster}}"
          "ECSContainerName": "{{.ECSContainerName}}"
          "ECSTaskVersion": "{{.ECSTaskVersion}}"
          "ECSTaskFamily": "{{.ECSTaskFamily}}"
          "DockerContainerName": "{{.DockerContainerName}}"
          "ConfigID": "{{.ConfigID}}"
          "Host": "{{.Host}}"
          "Source": "{{.Source}}"
          "SourceType": "{{.SourceType}}"
          "Tag": "{{.Tag}}"

Parameters

name

Required

Enter a descriptive name for the output or integration.

For outputs, this name will be used to map this destination to a workflow.

name: loki-integration

integration_name

Optional

This parameter refers to the organization-level integration created in the Integrations page.

If you need to add multiple instances of the same integration into the config, then you can add a custom name to each instance via the name parameter. In this situation, the name should be used to refer to the specific instance of the destination in the workflows.

integration_name: orgs-kafka

type

Required

Enter loki.

type: loki

endpoint

Required

Enter the Loki endpoint.

endpoint: "https://localhost:3000/loki/api/v1/push"

user

Optional

Enter the username for Loki.

user: "user"

api_key

Optional

Enter the Loki API key.

api_key: "api_key"

alert_as_log

Optional

Enter true to send alerts as a log.

Additionally, for this parameter to work, you must also enter log as a feature type. Enter false to send alerts as events.

alert_as_log: true

custom_tags

Optional

This parameter defines key-value pairs that are streamed with every request.

This parameter supports templating.

      custom_tags:
        "app": "test"
        "region": "us-west-2"
        "File Path": "{{.FileGlobPath}}"
        "K8s PodName": "{{.K8sPodName}}"
        "K8s Namespace": "{{.K8sNamespace}}"
        "K8s ControllerKind": "{{.K8sControllerKind}}"
        "K8s ContainerName": "{{.K8sContainerName}}"
        "K8s ContainerImage": "{{.K8sContainerImage}}"
        "K8s ControllerLogicalName": "{{.K8sControllerLogicalName}}"
        "ECSCluster": "{{.ECSCluster}}"
        "ECSContainerName": "{{.ECSContainerName}}"
        "ECSTaskVersion": "{{.ECSTaskVersion}}"
        "ECSTaskFamily": "{{.ECSTaskFamily}}"
        "DockerContainerName": "{{.DockerContainerName}}"
        "ConfigID": "{{.ConfigID}}"
        "Host": "{{.Host}}"
        "Source": "{{.Source}}"
        "SourceType": "{{.SourceType}}"
        "Tag": "{{.Tag}}"

message_template

Optional

This parameter customizes the message content.

This parameter supports templating.

See Template Fields.

      message_template:
        "File Path": "{{.FileGlobPath}}"
        "K8s PodName": "{{.K8sPodName}}"
        "K8s Namespace": "{{.K8sNamespace}}"
        "K8s ControllerKind": "{{.K8sControllerKind}}"
        "K8s ContainerName": "{{.K8sContainerName}}"
        "K8s ContainerImage": "{{.K8sContainerImage}}"
        "K8s ControllerLogicalName": "{{.K8sControllerLogicalName}}"
        "ECSCluster": "{{.ECSCluster}}"
        "ECSContainerName": "{{.ECSContainerName}}"
        "ECSTaskVersion": "{{.ECSTaskVersion}}"
        "ECSTaskFamily": "{{.ECSTaskFamily}}"
        "DockerContainerName": "{{.DockerContainerName}}"
        "ConfigID": "{{.ConfigID}}"
        "Host": "{{.Host}}"
        "Source": "{{.Source}}"
        "SourceType": "{{.SourceType}}"
        "Tag": "{{.Tag}}"

features

Optional

This parameter defines which data types to stream to the destination.

If you do not provide a value, then all will be set.

To learn more, see the following section on supported feature types.

features: log

buffer_ttl

Optional

Enter a length of time to retry failed streaming data.

After this length of time is reached, the failed streaming data will no longer be tried.

buffer_ttl: 2h

buffer_path

Optional

Enter a folder path to temporarily store failed streaming data.

The failed streaming data will be retried until the data reaches its destinations or until the Buffer TTL value is reached.

If you enter a path that does not exist, then the agent will create directories, as needed.

buffer_path: /var/log/edgedelta/pushbuffer/

buffer_max_bytesize

Optional

Enter the maximum size of failed streaming data that you want to retry.

If the failed streaming data is larger than this size, then the failed streaming data will not be retried.

buffer_max_bytesize: 100MB

Template Fields

As an optional step, you can customize the message payload and custom tags that are sent to a Loki destination.

Loki does not support the dash ( - ) character as a key value.

Field	Description
Tag	This field is the user-defined tag that describes the environment, such as prod_us_west_2_cluster.
Host	This field is the hostname of the environment where the agent is running on.
ConfigID	This field is the configuration ID of the corresponding agent.
Source	This field is the source name, specifically the identifier of the source, such as docker container id or file name.
SourceType	This field is the source type, such as Docker or system.
FileGlobPath	This field is the file global path.
K8sPodName	This field is the Kubernetes pod name.
K8sNamespace	This field is the Kubernetes namespace.
K8sControllerKind	This field is the Kubernetes controller kind.
K8sContainerName	This field is the Kubernetes container name.
K8sContainerImage	This field is the Kubernetes container image.
K8sControllerLogicalName	This field is the Kubernetes controller logical name.
ECSCluster	This field is the ECS cluster name.
ECSContainerName	This field is the ECS container name.
ECSTaskVersion	This field is the ECS task version.
ECSTaskFamily	This field is the ECS task family.
DockerContainerName	This field is the Docker container name.

Supported Features

See Streaming Features.

Feature Type	Supported?
Log	Yes
Metrics	No
Alert as event	No
Alert as log	Yes
Health	No
Dimensions as attribute	No
Send as is	No
Send as JSON	No
Custom tags	Yes
EDAC enrichment	No
Message template	Yes
outgoing_bytes.sum	Yes
outgoing__raw_bytes.sum	No
outgoing_lines.count	Yes
output buffering to disk	No