Edge Delta Fluentd Destination

Overview

The Fluentd node enables the agent to output logs in the Fluentd forward protocol.

• incoming_data_types: cluster_pattern_and_sample, log, metric, signal, custom

Example Configuration

nodes:
  - name: my_fluentd
    type: fluentd_output
    host: log-repo-host
    port: 23131
    pool_size: 10
    tag_prefix: "tail.ed."

Required Parameters

name

A descriptive name for the node. This is the name that will appear in Visual Pipelines and you can reference this node in the YAML using the name. It must be unique across all nodes. It is a YAML list element so it begins with a - and a space followed by the string. It is a required parameter for all nodes.

nodes:
  - name: <node name>
    type: <node type>

type: fluentd_output

The type parameter specifies the type of node being configured. It is specified as a string from a closed list of node types. It is a required parameter.

nodes:
  - name: <node name>
    type: <node type>

host

The host parameter along with the port defines the TCP server endpoint. It is specified as a string and is required for TCP traffic.

nodes:
  - name: <node name>
    type: fluentd_output
    host: log-repo-host
    port: 23131

port

The port parameter defines the TCP stream port number along with the host. It is specified as a string and is required for TCP traffic.

nodes:
  - name: <node name>
    type: fluentd_output
    host: log-repo-host
    port: 23131

Optional Parameters

buffer_max_bytesize

The buffer_max_bytesize parameter configures the maximum byte size for total unsuccessful items. If the limit is reached, the remaining items are discarded until the buffer space becomes available. It is specified as a datasize.Size, has a default of 0 indicating no size limit, and it is optional.

nodes:
  - name: <node name>
    type: fluentd_output
    host: log-repo-host
    port: 23131
    buffer_max_bytesize: 10MB

buffer_path

The buffer_path parameter configures the path to store unsuccessful items. Unsuccessful items are stored there to be retried back (exactly once delivery). It is specified as a string and it is optional.

nodes:
  - name: <node name>
    type: fluentd_output
    host: log-repo-host
    port: 23131
    buffer_path: <path to unsuccessful items folder>

buffer_ttl

The buffer_ttl parameter configures the time-to-Live for unsuccessful items, which indicates when to discard them. It is specified as a duration, has a default of 10m, and it is optional.

nodes:
  - name: <node name>
    type: fluentd_output
    host: log-repo-host
    port: 23131
    buffer_ttl: 20m

features

The features parameter defines which data types to stream to the destination. It is specified as a string of comma-separated list of item types. The default is metric,edac,cluster. It is optional.

nodes:
  - name: <node name>
    type: fluentd_output
    host: log-repo-host
    port: 23131
    features: <item type>,<item type> 

pool_size

The pool_size parameter defines the size of the connection pool for TCP type connections. It specifies how many connections can be maintained simultaneously for the given endpoint. It is specified as an integer and it is optional.

nodes:
  - name: <node name>
    type: fluentd_output
    host: log-repo-host
    port: 23131
    pool_size: <number of connections>

tag_prefix

The tag_prefix parameter is used for adding a prefix to the tag of incoming data. If the source config already defines a tag enrichment, then that configuration will be used. Otherwise, a tag will be generated in the following format: "{TagPrefix}{Agent’s Tag}". It is specified as a string and it is optional.

nodes:
  - name: <node name>
    type: fluentd_output
    host: log-repo-host
    port: 23131
    tag_prefix: <prefix>