Edge Delta FluentD Input

FluentD ports and protocols for the agent to listen for incoming traffic.

  3 minute read  

Version 2 is no longer supported. This section is for historical reference only.

See the latest version here.

Overview

The Fluentd input can consume data sent from a fluentd agent that has been configured to output using the fluent-forward protocol. It can be connected with secure or insecure connections.

fluentd.conf

<match **>
      https://docs.fluentd.org/v1.0/articles/in_forward
      @type forward
      heartbeat_type none
      keepalive true
      <security>
            shared_key "sharedKey" shared key is used between nodes.
      </security>
      <server>
            host IP or host to an endpoint which distributes the requests to the ED Agents.
            port 3421 Port which the ED Agents listens for fluentd input.
      </server>
</match>

Example

inputs:
 fluentds:
    - labels: my-fluentd-collector
      port: 9898
      read_timeout: 60s
      shared_key: sharedKey
      tls:
        ca_file: /var/run/secrets/secure-forward/ca-bundle.crt
        crt_file: /var/run/secrets/secure-forward/tls.crt
        key_file: /var/run/secrets/secure-forward/tls.key
        disable_verify: true

Required Parameters

labels

Enter a descriptive name for this input. When you create a workflow, you will use this label to enter your input into the workflow.

inputs:
 fluentds:
    - labels: <input name>
      port: <port number>
      read_timeout: <duration>

port

Enter the FluentD port. This parameter is required to support tcp stream.

inputs:
 fluentds:
    - labels: <input name>
      port: <port number>
      read_timeout: <duration>

read_timeout

Enter a maximum time to wait and listen for data. This parameter only applies if you select tcp as the protocol.

inputs:
 fluentds:
    - labels: <input name>
      port: <port number>
      read_timeout: <duration>

Optional Parameters

add_ingestion _time

Enter true to ingest a timestamp if the input format is in JSON.

inputs:
 fluentds:
    - labels: <input name>
      port: <port number>
      read_timeout: <duration>
      add_ingestion_time: true

enable_incoming_line_anomalies

Generate anomaly scores.

inputs:
 fluentds:
    - labels: <input name>
      port: <port number>
      read_timeout: <duration>
      enable_incoming_line_anomalies: true

filters

Enter an existing filter to add to this input. To learn how to create a filter, see Filters.

inputs:
 fluentds:
    - labels: <input name>
      port: <port number>
      read_timeout: <duration>
      filters:
        - info

read_size

Enter the number of lines to read from the incoming data. If you do not enter a value, then 1 will be used. This parameter only applies if you select tcp as the protocol.

inputs:
 fluentds:
    - labels: <input name>
      port: <port number>
      read_timeout: <duration>
      read_size: 10000

skip_ingestion _time_on_failure

Enter true to skip the ingestion of the timestamp when the input is broken or in an invalid format.

inputs:
 fluentds:
    - labels: <input name>
      port: <port number>
      read_timeout: <duration>
      skip_ingestion_time_on_failure: true

tls: ca_file

Enter the absolute path to scan the CA certificate file.

inputs:
 fluentds:
    - labels: <input name>
      port: <port number>
      read_timeout: <duration>
      tls:
        ca_file: /certs/ca.pem

tls: ca_path

Enter the absolute path to the certificate file.

inputs:
 fluentds:
    - labels: <input name>
      port: <port number>
      read_timeout: <duration>
      tls:
        crt_file: /certs/server-cert.pem

tls: client_auth_type

Enter a client authorization type. You can enter:

  • noclientcert
  • requestclientcert
  • requireanyclientcert
  • verifyclientcertifgiven
  • requireandverifyclientcert
inputs:
 fluentds:
    - labels: <input name>
      port: <port number>
      read_timeout: <duration>
      tls:
        client_auth_type: noclientcert

tls: crt_path

Enter the absolute path to the certificate file.

inputs:
 fluentds:
    - labels: <input name>
      port: <port number>
      read_timeout: <duration>
      tls:
        crt_file: /certs/server-cert.pem

tls: disable_verify

To disable a TLS verification of a certificate, enter disable_verify: true. To enable a TLS verification of the certificate, you can enter disable_verify: false or you can remove this line entirely.

inputs:
 fluentds:
    - labels: <input name>
      port: <port number>
      read_timeout: <duration>
      tls:
        disable_verify: true

tls: key_file

Enter the absolute path to the private key file.

inputs:
 fluentds:
    - labels: <input name>
      port: <port number>
      read_timeout: <duration>
      tls:
        key_file: /certs/server-key.pem

tls: key_password

Enter the password for the key file.

inputs:
 fluentds:
    - labels: <input name>
      port: <port number>
      read_timeout: <duration>
      tls:
        key_password: <password>

tls: max_version

Enter the maximum version of TLS to accept.

inputs:
 fluentds:
    - labels: <input name>
      port: <port number>
      read_timeout: <duration>
      tls:
        max_version: TLSv1_3

tls: min_version

Enter the minimum version of TLS to accept.

inputs:
 fluentds:
    - labels: <input name>
      port: <port number>
      read_timeout: <duration>
      tls:
        min_version: TLSv1_1