Edge Delta HTTP Destination Node

Overview

The HTTP destination node streams analytics and insights to streaming destinations. It sends raw bytes that are generated via marshaling items as JSON. You can create any header using key/value pairs for the HTTP destination node. It currently supports hardcoded strings only. For example, headers like X-Token: abc123456 can be configured.

• incoming_data_types: cluster_pattern_and_sample, log, metric, signal, custom

Example Configuration

nodes:
  - name: my_http_port
  type: http_output
  endpoint: http://localhost:4545/v0/collect

Required Parameters

name

A descriptive name for the node. This is the name that will appear in Visual Pipelines and you can reference this node in the YAML using the name. It must be unique across all nodes. It is a YAML list element so it begins with a - and a space followed by the string. It is a required parameter for all nodes.

nodes:
  - name: <node name>
    type: <node type>

type: http_output

The type parameter specifies the type of node being configured. It is specified as a string from a closed list of node types. It is a required parameter.

nodes:
  - name: <node name>
    type: <node type>

endpoint

The endpoint parameter defines the HTTP stream endpoint. It is specified as a string and is required for HTTP traffic.

nodes:
  - name: <node name>
    type: http_output
    endpoint: <http endpoint>

Optional Parameters

batch_bytes

The batch_bytes parameter specifies the maximum size of items that a request can have. It is specified in bytes. By default, there is no limit and the parameter is optional.

nodes:
  - name: <node name>
    type: http_output
    endpoint: <http endpoint>
    batch_bytes: 1048576

batch_size

The batch_size parameter specifies the maximum number of items that can be sent in request. By default, there is no limit and the parameter is optional.

nodes:
  - name: <node name>
    type: http_output
    endpoint: <http endpoint>
    batch_size: 1000

buffer_max_bytesize

The buffer_max_bytesize parameter configures the maximum byte size for total unsuccessful items. If the limit is reached, the remaining items are discarded until the buffer space becomes available. It is specified as a datasize.Size, has a default of 0 indicating no size limit, and it is optional.

nodes:
  - name: <node name>
    type: http_output
    endpoint: <http endpoint>
    buffer_max_bytesize: 2048

buffer_ttl

The buffer_ttl parameter configures the time-to-Live for unsuccessful items, which indicates when to discard them. It is specified as a duration, has a default of 10m, and it is optional.

nodes:
  - name: <node name>
    type: http_output
    endpoint: <http endpoint>
    buffer_ttl: 20m

buffer_path

The buffer_path parameter configures the path to store unsuccessful items. Unsuccessful items are stored there to be retried back (exactly once delivery). It is specified as a string and it is optional.

nodes:
  - name: <node name>
    type: http_output
    endpoint: <http endpoint>
    buffer_path: <path to unsuccessful items folder>

encoder

The encoder parameter specifies the encoding type used to stream data. It is specified as a string and is optional. You can specify msgpack or json. The default is raw data.

nodes:
  - name: <node name>
    type: http_output
    endpoint: <URI>
    encoder: json

features

The features parameter defines which data types to stream to the destination. It is specified as a string of comma-separated list of item types. The default is metric,edac,cluster. It is optional.

nodes:
  - name: <node name>
    type: http_output
    endpoint: <http endpoint>
    features: <item type>,<item type>

headers

The headers parameter defines headers to add to the HTTP destination data. It is specified as a list of dictionaries. It is optional.

nodes:
  - name: <node name>
    type: http_output
    endpoint: <http endpoint>
    headers:
    -  header: X-Token
       value: abc123456