Log Patterns
2 minute read
Pattern Detection
The Edge Delta agent uses a proprietary algorithm to automatically detect repeated patterns in log messages. This allows it to optimize data by reporting patterns and their frequency rather than streaming the full log messages. Variant values within a pattern are expressed as a wildcard (*
).

Cluster Processors
Log patterns are detected by processors that are configured on an Edge Delta agent. The cluster processor will track and report the most frequently occurring patterns for the default or specified interval, and it will send the specified number of full log samples for each pattern. You configure pattern detection by adding a cluster processor to a workflow.

Sentiment Analysis
Every pattern detected by the Edge Delta agent is further analyzed to check for negative sentiment. Negative sentiment is determined by checking for the presence of specific keywords in the pattern (e.g. error
, exception
, fail
, etc.). Some keywords such as debug
are considered neutralizing because they automatically offset negative keyword matches in the pattern.
The negative and neutralizing keywords used in sentiment analysis can be configured in the Pipeline Settings for an account. They are applied to all agents within the account.

Pattern Visualization
Log patterns detected by the Edge Delta agent can be viewed in the Edge Delta web app as well as 3rd party observability tools.
Edge Delta Web App
Log patterns can be viewed in multiple places in the Edge Delta Web app:
The Patterns screen shows a breakdown of Negative Patterns (all patterns with negative sentiment).

All Patterns shows patterns with any sentiment.

All patterns also shows a tabular view of the most frequently occurring patterns and their associated statistics.

The Kubernetes Overview shows patterns for a particular cluster, namespace, controller, or container image.

3rd Party Tools
Patterns can also be viewed in any streaming destination that accepts log data.


Anomaly Detection in Log Patterns
Once pattern data is sent to the Edge Delta backend, it can be further analyzed for anomalies. See our article on anomaly detection for more details.