Pipeline Status

Pipeline Status in the Edge Delta web application.

  2 minute read  

Overview

You can use this document to learn about the Pipeline Status page in the Edge Delta App.

Specifically, this page displays data for incoming and outgoing data.

Concepts

Review the following terms types that populate the the Pipeline Status page

Data Optimization

This data type represents the ratio between the amount of incoming data versus the amount of outgoing data.

Objects Analyzed

This data type represents the number of pieces of data analyzed by Edge Delta.

In other words, this data type is represents log lines count or (JSON objects count).

This data type does not relate to the events data displayed in the Insights page.

Bytes Analyzed

This data type represents incoming data.

This value is the sum of all the raw data obtained by the agent from your source environments.

This value does not include extra / enriched metadata.

Bytes Streamed

This data type represents outgoing data.

This value is the sum of all the bytes that is sent to streaming destinations, which includes raw data messages and extra / enriched metadata.

Active Agents

This data type represents the number of active agent instances that run under the Edge Delta agent.

The table will include agents that have had a heartbeat in the previous 5 minutes. As a result, this data may fluctuate if an agent has not yet sent a heartbeat within the timeframe.

In the Pipeline Status page, in the Active Agents table, the term Agent Tab represents the tag of the agent configuration.

Agent Health

View Agent Health

  1. In the Edge Delta App, on the left-side navigation, click Data Pipeline, and then click Pipeline Status.
  2. Navigate to the table of Active Agents, then locate the desired agent, then under Actions, click the vertical ellipses, and then click View Health.
  3. The Instance Count column displays the number of unique components.
    • By default, streamer contains at least 3 components (Elastic, Influx, and S3).
  4. Review the health check types.

Health Data

streamer

This type represents the streaming destination, such as Sumo Logic, Elastic, etc.

tailer

This type represents a source, such as files, Kubernetes, etc.

http_client

This type represents anything that communicates to the HTTP endpoint, which includes streaming destinations and Edge Delta.

backend

This type reports the responses when trying to send data, such as 200, 400, 500, etc.

archive_reporter

This type represents the archive destination, such as S3.