Edge Delta Pipeline Configuration v3

Recap:

A Fleet is one or more sets of agents all configured by a single Pipeline definition. One Fleet can consists of sets of agents in different environments.

Overview

Visual Pipelines is a tool you can use to view and modify your Pipeline configurations.

Requirements

The Visual Pipelines tool requires Fleets with an agent version of 0.1.71 or higher. See here for more information.

View a Pipeline

1. In the Edge Delta App, click Pipelines.
2. Select the Fleet with the Pipeline you want to view.
3. Click View/Edit Pipeline.

The v3 configuration structure is a graph consisting of three types of nodes, the links between them, and the data handled by the pipeline:

• Source, Processor and Destination Nodes
• Links
• Data Items

View Mode

The Pipeline lists the outgoing data rate for each node. Using this view you can quickly see how the configuration contributes to the overall pipeline efficiency.

The link weight indicates the relative volume of traffic on link between two nodes.

For example, the paymentservice link to the mask-credit-card node with no traffic is rendered smaller than the 129MB of traffic flowing on the link to the Easytrade Team Pack node.

See Configuring a Pipeline to build your data pipeline.

Nodes

Nodes are configuration objects. Each node performs a function and you design a pipeline by creating a flow of traffic through a set of nodes. Select a node to view details about it.

You can toggle between viewing traffic metrics as events or bytes and view a traffic graph for the node. Matched events indicates the input to successful output ratio in event numbers. Bear in mind that nodes that generate events might have a value of more than 100%. Nodes that fail to process traffic due to a configuration error will have a low matching rate.

The details pane also shows the configuration part of the node’s YAML.

Filter Pipeline

You can apply filters to the Sources, Processors and Destinations to view only the connections between the selected objects. Note this changes the view only, not the pipeline configuration.

For example, you can view the data flow along the pipeline between only the Kubernetes source and the Log destination:

YAML Only Parameters

Some configuration v3 parameters are advanced or in beta, requiring them to be configured in YAML using a text editor rather than Visual Pipelines.

Applying Changes

After making changes to a Pipeline configuration using the Edge Delta Web UI, there is no need to restart or reinstall the Edge Delta Fleet. The configuration settings will be automatically applied after a few minutes.

Related Content