Edge Delta Log to Metric Node
5 minute read
Overview
The Log to Metric Node evaluates the body
field for matching patterns and generates metrics.

Example Configuration
Different types of metrics are supported:
Occurrence Count
A simple count of occurrences of logs that match the pattern, for example a count of logs: pattern: "HIGH|high"
.
Numeric Capture
This type matches numeric values in a capture group, for example pattern: " (\\d+)$"
and generates standard metrics: count, min, max, avg, anomaly1 and anomaly2.
Dimension Counter
If named captures in the regex pattern are dimensions, and dimension groups are given, then dimension occurrence stats are generated.
nodes:
- name: http
type: log_to_metric
pattern: "] \"(?P<method>\\w+)"
enabled_stats: ["count", "anomalymin"]
dimension_groups:
- dimensions: ["method"]
This node will generate an occurrence count
and anomalymin
for each HTTP method:
http_method_get.count
http_method_get.anomalymin
http_method_post.count
http_method_post.anomalymin
etc.
Dimension Numeric Capture
If both dimension and numeric captures are defined in the regex pattern and also in one of the dimension groups, then numeric stats per dimension and per numeric value are generated.
nodes:
- name: "http_request"
type: log_to_metric
pattern: "] \"(?P<method>\\w+) took (?P<latency>\\d+) ms"
interval: 1m
retention: 1h
skip_empty_intervals: true
dimension_groups:
- dimensions: ["method"]
numeric_dimension: "latency"
In this example numeric stats for each HTTP method will be generated such as:
http_request_method_get_latency.avg
http_request_method_get_latency.min
http_request_method_get_latency.max
http_request_method_get_latency.p95
http_request_method_get_latency.p99
http_request_method_get_latency.sum
http_request_method_post_latency.avg
http_request_method_post_latency.min
http_request_method_post_latency.max
http_request_method_post_latency.p95
http_request_method_post_latency.p99
http_request_method_post_latency.sum
Dimensions as Attributes
Setting dimensions_as_attributes
to true configures dimensions to be delivered as metric attributes rather than appended to the metric name. This is illustrated by the data objects:
nodes:
- name: "http_single"
type: log_to_metric
pattern: "] \"(?P<method>\\w+) (?P<uri>\\S*) (?P<httpversion>\\S*)\" (?P<code>\\d+)"
dimensions_as_attributes: true
dimension_groups:
- dimensions: ["method"]
- dimensions: ["httpversion"]
- dimensions: ["code"]
Required Parameters
name
A descriptive name for the node. This is the name that will appear in Visual Pipelines and you can reference this node in the yaml using the name. It must be unique across all nodes. It is a yaml list element so it begins with a -
and a space followed by the string. It is a required parameter for all nodes.
nodes:
- name: <node name>
type: <node type>
type: log_to_metric
The type
parameter specifies the type of node being configured. It is specified as a string from a closed list of node types. It is a required parameter.
nodes:
- name: <node name>
type: <node type>
pattern
The pattern
parameter is used to match log items in the body
field. It is specified as a regex expression and it can include a capture group. If one or more dimension groups are defined, there should be at least one capture group definition. A pattern
is required.
nodes:
- name: <node name>
type: log_to_metric
pattern: <regex pattern>
Optional Parameters
dimension_groups
The dimension_groups
parameter is used to group attributes for metrics. There can be one or more dimension groups. It is specified with child dimensions
elements. It is optional. The dimensions
parameter specifies the names from capture groups that will be used in the metric name or attribute. It can take a number of options that apply only to that dimension:
custom_suffix
A suffix to append to the metric name.numeric_dimension
The metric value won’t be accepted as 1.0 but rather the value captured from the given dimensionvalue_adjustment_rule
A rule to transform the metric using a variable calledvalue
, for example with an equation such as"value * 5.0"
. The adjustment applies only to the parent dimension.
Parameters from the parent level specification can also be used as dimensions
child parameters, meaning they only apply to the particular parent dimension
anomaly_coefficient
anomaly_confidence_period
anomaly_tolerance
dimensions_as_attributes
enable_reporting_in_prometheus
enabled_stats
histogram_limit
interval
only_report_nonzeros
retention
skip_empty_intervals
nodes:
- name: <node name>
type: log_to_metric
pattern: <regex pattern>
dimension_groups:
- dimensions: ["service"]
numeric_dimension: "duration"
custom_suffix: "by_duration"
value_adjustment_rule: "value * 5.0"
enabled_stats
The enabled_stats
parameter specifies the statistics that should be reported. It is specified as a list of strings and is optional.
Valid options are:
count
- the number of instances matched.sum
- the sum of matched values.avg
- the average (mean) matching value.min
- the smallest matching value.max
- the largest matching value.p25
- count of values in the 25th percentile.p75
- count of values in the 75th percentile.p95
- count of values in the 95th percentile.p99
- count of values in the 99th percentile.stddev
- the standard deviation.anomaly1
- the proprietary Edge Delta anomaly score 1.anomaly2
- the proprietary Edge Delta anomaly score 2.anomalymin
- the min ofanomaly1
andanomaly2
. This is useful to reduce the alert noise.
The
count
,anomaly1
andanomaly2
metrics are generated for occurrence captures. Whereascount
,min
,max
,avg
,anomaly1
andanomaly2
metrics are generated for numeric captures.
nodes:
- name: <node name>
type: log_to_metric
pattern: <regex pattern>
enabled_stats: <statistic type>, <statistic type>
histogram_limit
The histogram_limit
parameter configures the maximum number of histograms allowed per dimension group (or node if no dimension groups are specified). It is specified as an integer, the default is 1000 and it is optional.
nodes:
- name: <node name>
type: log_to_metric
pattern: <regex pattern>
histogram_limit: <integer>
interval
The interval
parameter specifies the reporting interval for the statistics that the node will generate. It will collect values for the duration of the interval before calculating metrics such as the average. It is specified as a duration and the default is 1 minute. It is optional.
nodes:
- name: <node name>
type: log_to_metric
pattern: <regex pattern>
interval: 2m
metric_name
The metric_name
parameter specifies a custom name for the generated metric. It is specified as a string and the default, if not specified, is to use the node name. It is optional.
nodes:
- name: <node name>
type: log_to_metric
pattern: <regex pattern>
metric_name: <custom name>
retention
The retention
parameter specifies how far back to look to generate anomaly scores. A short retention period will be more sensitive to spikes in metric values. It is specified as a duration and the default is 3 hours. It is optional.
nodes:
- name: <node name>
type: log_to_metric
pattern: <regex pattern>
retention: <duration>