Edge Delta Pipeline Configuration v3

Edge Delta Pipelines can be configured visually using Visual Pipelines.

Recap:

A Fleet is one or more sets of agents all configured by a single Pipeline definition. One Fleet can consists of sets of agents in different environments.

Overview

Visual Pipelines is a tool you can use to view and modify your Pipeline configurations.

Requirements

The Visual Pipelines tool requires Fleets with an agent version of 0.1.71 or higher. See here for more information.

View a Pipeline

  1. In the Edge Delta App, click Pipelines.
  2. Select the Fleet with the Pipeline you want to view.
  3. Click View/Edit Pipeline.

Nodes

The v3 configuration structure is a graph consisting of three types of nodes, the links between them, and the data handled by the pipeline:

View Mode

The Pipeline lists the outgoing data rate for each node. Using this view you can quickly see how the configuration contributes to the overall pipeline efficiency.

See Configuring a Pipeline to build your data pipeline.

YAML Only Parameters

Some configuration v3 parameters are advanced or in beta, requiring them to be configured in YAML using a text editor rather than Visual Pipelines.

Applying Changes

After making changes to a Pipeline configuration using the Edge Delta Web UI, there is no need to restart or reinstall the Edge Delta Fleet. The configuration settings will be automatically applied after a few minutes.


Configuring an Edge Delta Pipeline

Use Visual Pipelines or a text editor to configure a Pipeline.

Connecting Edge Delta Nodes

Connect nodes to build your pipeline.

Edge Delta Data Item

Data items are handled by nodes and transported by links.

CEL Custom Macros

Use CEL Custom Macros to reference log fields.

Designing Pipelines and Testing Edge Delta Nodes

Test node functions to design and build effective pipelines.

Edge Delta Input Nodes

Input Nodes in v3 configurations.

Edge Delta Processor Nodes

Processor nodes in v3 configurations.

Edge Delta Output Nodes

Output nodes in Visual Pipelines.

Advanced Edge Delta Configuration

Methods for fine-tuning your configuration.

Edge Delta Compound Nodes

Compound Nodes in the Edge Delta Visual Pipeline.

Edge Delta Diagnostics Nodes

Diagnostics Nodes in the Edge Delta Visual Pipeline.

Edge Delta Agent Settings

Global Pipeline configuration options.

Edge Delta Regex Testing

Build and test regex patterns for use in your pipeline nodes.