Edge Delta Cluster Processor

Find patterns in logs.

See the latest version here.


This processor type finds patterns in logs, and then groups (or clusters) these patterns based on similarities. This processor populates the Patterns page.

Most users, especially new users, will have default processors already configured for their account; however, if your account does not have any existing monitors, then the Patterns page will be empty.


    name: clustering
    num_of_clusters: 100
    samples_per_cluster: 20
    reporting_frequency: 30s
    retention: 10m
    cpu_friendly: true
    throttle_limit_per_sec: 200
      - info

Required Parameters


Enter a descriptive label for this processor.

When you create a workflow, you will use this label to enter your processor into the workflow.

name: clustering


This parameter sets the maximum number of clusters to generate for an input.

num_of_clusters: 100


This parameter sets the frequency to send clustering results to a streaming destination. These results include patterns and samples.

reporting_frequency: 1m

Optional Parameters


This parameter sets CPU rate limiting. Specifically, the agent will review the soft_cpu_limit parameter from Agent Settings and drop some percentage of events to keep agent’s CPU usage at less than the limit. By default, this parameter is disabled.

This parameter only applies to users who have more 1,000 logs per second.

Analyzing patterns in high-volume log environments may cause strains on your computing resources. As a result, by default, this processor type only processes 200 logs per source (such as a container or file) per second.

You can change this setting with the cpu_friendly and throttle_limit_per_sec parameters.

cpu_friendly: true


Enter an existing filter to add to this input. To learn how to create a filter, see Filters.

  - extract_severity


Enter true to include pattern information (pattern, pattern count, sentiment score) as tags in the cluster sample. The default value is false.

include_pattern_info_in_samples: true


This parameter is a golang duration string that represents a cluster’s retention rate. Clusters that do not have any new logs within the retention period will dropped and will no longer be reported until logs appear again. For example, if you set this parameter at 10m, then clusters without new logs for the last 10 minutes will be dropped. The default retention rate is 1 hour (1h).

retention: 30m


This parameter sets the number of sample events to report when providing cluster details.

samples_per_cluster: 20


This parameter sets a limit on the number of logs that can be clustered per second from a single source. If the cpu_friendly parameter is enabled, then this parameter will be ignored.

throttle_limit_per_sec: 200

Additional Information

To learn how Edge Delta calculates and clusters invariant components, see Clustered Invariants.