Edge Delta Ratio Node

Overview

The ratio processor identifies two patterns and generates a metric to indicate their ratio. For example you can monitor the success to failure ratio.

Example Configuration

The following example calculates a ratio using following formula: failure / (failure+success)

nodes:
  - name: request_error_ratio 
      type: ratio
      success_pattern: "request succeeded"
      failure_pattern: "request failed"
      interval: 1m
      retention: 3h
      metric_name: "request_ratio"
      enable_reporting_in_prometheus: true
      group_by:
        - item["resource"]["ed.filepath"]

Required Parameters

name

A descriptive name for the node. This is the name that will appear in Visual Pipelines and you can reference this node in the YAML using the name. It must be unique across all nodes. It is a YAML list element so it begins with a - and a space followed by the string. It is a required parameter for all nodes.

nodes:
  - name: <node name>
    type: <node type>

type: ratio

The type parameter specifies the type of node being configured. It is specified as a string from a closed list of node types. It is a required parameter.

nodes:
  - name: <node name>
    type: <node type>

failure_pattern

The failure_pattern parameter specifies the Golang regex pattern to match log items in the body field for the failure value that will be used to calculate the ratio. It is a required parameter.

nodes:
  - name: <node name> 
      type: ratio
      success_pattern: "request succeeded"
      failure_pattern: "request failed"

success_pattern

The success_pattern parameter specifies the Golang regex pattern to match log items in the body field for the success value that will be used to calculate the ratio. It is a required parameter.

nodes:
  - name: <node name> 
      type: ratio
      success_pattern: "request succeeded"
      failure_pattern: "request failed"

Optional Parameters

anomaly_coefficient

The anomaly_coefficient parameter specifies the coefficient to amplify calculated anomaly scores between [0, 100] range. Default value is 10 and it is optional.

nodes:
  - name: <node name> 
      type: ratio
      success_pattern: "request succeeded"
      failure_pattern: "request failed"
      anomaly_coefficient: 11

anomaly_confidence_period

The anomaly_confidence_period parameter specifies the initial duration during which anomaly scores are not reported. This prevents inaccurate reports from new deployments. The default value is 30m and it is optional.

nodes:
  - name: <node name> 
      type: ratio
      success_pattern: "request succeeded"
      failure_pattern: "request failed"
      anomaly_confidence_period: 40m

anomaly_tolerance

The anomaly_tolerance parameter is used for handling edge cases for anomaly scores where standard deviation is too small. The default value is 0.01 and it is optional.

nodes:
  - name: <node name> 
      type: ratio
      success_pattern: "request succeeded"
      failure_pattern: "request failed"
      anomaly_tolerance: 0.02

group_by

The group_by parameter defines how to aggregate log items based on their properties. Each entry should be an expression (CEL or Go template). When group_by is not specified, metrics are grouped by their source (ie. item[“resource”]["__group_name"] or item[“resource”]["__logical_source"] if aggregator in place). It is optional.

nodes:
  - name: <node name> 
      type: ratio
      success_pattern: "request succeeded"
      failure_pattern: "request failed"
      group_by:
        - item["resource"]["ed.filepath"]

interval

The interval parameter specifies the duration after which the node flushes metric items. It can also be defined at dimension group level. It is specified as a duration, the default is 1m and it is optional.

nodes:
  - name: <node name> 
      type: ratio
      success_pattern: "request succeeded"
      failure_pattern: "request failed"
      interval: 2m

metric_name

The metric_name parameter specifies the name to use for the metric. If not specified, the processor name will be used. It is specified as a string and is optional.

nodes:
  - name: <node name> 
      type: ratio
      success_pattern: "request succeeded"
      failure_pattern: "request failed"
      metric_name: "request_ratio"

only_report_nonzeros

The only_report_nonzeros parameter excludes metrics with a value of zero. It is specified with a Boolean and the default value is false.

nodes:
  - name: <node name> 
      type: ratio
      success_pattern: "request succeeded"
      failure_pattern: "request failed"
      only_report_nonzeros: true

retention

The retention parameter defines a look back period to generate an anomaly score. Anything older than the specified period will not be included in the score. It can also be defined at dimension group level. It is specified as a duration and is optional. The default is 3h.

nodes:
  - name: <node name> 
      type: ratio
      success_pattern: "request succeeded"
      failure_pattern: "request failed"
      retention: 3h

skip_empty_intervals

The skip_empty_intervals parameter skips intervals so the anomaly scores are calculated based on history of only non-zero intervals. It is specified with a Boolean, the default value is false and it is optional.

nodes:
  - name: <node name> 
      type: ratio
      success_pattern: "request succeeded"
      failure_pattern: "request failed"
      skip_empty_intervals: true