Edge Delta Pipeline View

Pipelines view in the Edge Delta web application.

  4 minute read  

Overview

  1. In the Edge Delta App, click Pipelines.
  2. Select the pipeline you want to view.
  3. The pipeline opens in the Visual Builder tab.

The v3 configuration structure is a graph consisting of four types of nodes, the links between them, and the data handled by the pipeline:

Visual Builder

The Visual Builder opens with Edit Mode off. Turn it on to build a pipeline.

Filter Pipeline

You can filters by name the Sources, Processors and Destinations to view only the connections between the selected objects. Note this changes the view only, not the pipeline configuration.

For example, you can view the pipeline with only kubernetes_logs, k8s_traffic_metrics, k8s_trace_input, and k8s_metrics nodes and their connections to only the the edgedelta destination:

Traffic Volume

The Visual Builder lists the outgoing data rate for each node. Using this view you can quickly see how the configuration contributes to the overall pipeline efficiency.

The link weight indicates the relative volume of traffic on link between two nodes.

For example, the link from the kubernetes_logs node carrying 6GB of traffic is rendered thicker than the 896MB of traffic flowing on the link from the k8s_trace_input node.

Nodes

Nodes are configuration objects. Each node performs a function and you design a pipeline by creating a flow of traffic through a set of nodes. Select a node to view view its paths or double click to view details about it.

You can toggle between viewing traffic metrics as events or bytes and view a traffic graph for the node. Matched events indicates the input to successful output ratio in event numbers. Bear in mind that nodes that generate events might have a value of more than 100%. Nodes that fail to process traffic due to a configuration error will have a low matching rate.

The details pane also shows the configuration part of the node’s YAML.

Pipeline Options

Click the kebab icon to view settings for the pipeline.

  • Suspend: This option stops the Edge Delta agents associated with the pipeline from tailing data and collecting metrics or traces. However, they still emit small health events occasionally.
  • Duplicate: This option creates a new pipeline with the same configuration but it has a new Pipeline ID and name. This is not available for cloud pipelines.
  • Delete: This option deletes the pipeline from the back end. You still need to delete the agents on the edge. See Delete a Pipeline Configuration.

Cloud pipelines have a Settings option. See Edit Cloud Pipeline Resources.

Throughput

The Throughput tab on the Pipeline view shows the traffic statistics for that particular pipeline only. Similarly to the Pipelines overview page you can view Bytes in and out as well as Events in and out.

You can filter the graphs by service, node type and name or data type:

Packs

If any packs are used in your pipeline they are listed on the Packs tab. Here you can determine if they are up to date with the latest pack version.

Resource Usage

On the Resource Usage tab you can view graphs for the CPU (agent_cpu_millicores.value and ed.agent.cpu.milicores metrics) and memory usage (agent_mem_alloc.value and ed.agent.memory.allocation metrics) of the Edge Delta agents belonging to this pipeline.

You can filter the graphs by Host:

Agents

On the Agents tab the Edge Delta agents belonging to the pipeline are listed on the Host Name table.

Click Upgrade Agents to view the upgrade commands for the latest version of the agents, and in the environment you selected when you created the pipeline.

Agent Details

Click an agent to view its details for troubleshooting an Edge Delta installation. It opens on the Health tab for the agent:

Here you can examine the health metrics for the agent internal components.

On the Logs tab you can view internal logs generated by that agent:

The metrics tab shows resource usage for the agent, similar to viewing pipeline resource usage filtered by Host Name.

On the Profiling tab you can view the agent’s performance profile. Select a dimension to view the flame graph. See Performance Profiling Edge Delta Agents for more information.

Logs

Back on the Pipeline view page, the Logs tab shows logs generated by Edge Delta agents. You can filter them by Severity:

Legacy Pipelines

Starting at agent version 0.1.71, Edge Delta agents are configured with a pipeline that can be represented in a YAML file with a configuration v3 structure. This structure enables configuration using Visual Builder. You can also using a text editor to make changes directly to the YAML file. In addition, v0.1.71 agents support v2 configurations.

You can’t edit version 2 configurations using Visual Builder. Contact Support to migrate your configurations to the v3 format.

Identifying Your Version

The version is specified in the configuration YAML:

  1. Click Pipelines
  2. Select the pipeline.
  3. Enable Edit Mode.
  4. Expand the kebab menu and select Edit YAML:

The configuration version is at the start:

version: v3

In this instance it is version 3.