Edge Delta Filter Processor

The Edge Delta filter processor filters certain data items from the pipeline.

  3 minute read  

Overview

You can drop data items based on a specific condition in each data item. For example, you can drop activity performed by a test user.

See Pipeline Quickstart: Filter Data Items for a walkthrough.

Configuration

In this example, all logs with user email jane.smith@exampleemail.com are dropped.

- name: Multi Processor
  type: sequence
  processors:
  - type: ottl_filter
    metadata: '{"id":"C583fhTKWFgbhdWCZOXEz","type":"filter","name":"Filter out test
      user Jane"}'
    condition: attributes["user"]["email"] == "jane.smith@exampleemail.com"
    data_types:
    - log
    filter_mode: exclude

Options

Select a telemetry type

You can specify, log, metric, trace or all. It is specified using the interface, which generates a YAML list item for you under the data_types parameter. This defines the data item types against which the processor must operate. If data_types is not specified, the default value is all. It is optional.

It is defined in YAML as follows:

- name: multiprocessor
  type: sequence
  processors:
  - type: <processor type>
    data_types:
    - log

Action

This option determines what the processor does with matching vs non-matching data items. Include means matching data items are passed and all others are dropped. Exclude means matching data items are dropped and all others are passed. The default is exclude. It is populated for you in YAML using the filter_mode parameter.

It is defined in YAML as follows:

- name: Multi Processor
  type: sequence
  processors:
  - type: ottl_filter
    filter_mode: exclude

In this example, the action is set to Include so only logs containing the jane.smith@exampleemail.com attribute are dropped.

condition

The condition parameter contains a conditional phrase of an OTTL statement. It restricts operation of the processor to only data items where the condition is met. Those data items that do not match the condition are passed without processing. You configure it in the interface and an OTTL condition is generated. It is optional. You can select one of the following operators:

Operator Name Description Example
== Equal to Returns true if both values are exactly the same attributes["status"] == "OK"
!= Not equal to Returns true if the values are not the same attributes["level"] != "debug"
> Greater than Returns true if the left value is greater than the right attributes["duration_ms"] > 1000
>= Greater than or equal Returns true if the left value is greater than or equal to the right attributes["score"] >= 90
< Less than Returns true if the left value is less than the right attributes["load"] < 0.75
<= Less than or equal Returns true if the left value is less than or equal to the right attributes["retries"] <= 3
matches Regex match Returns true if the string matches a regular expression isMatch(attributes["name"], ".*\\.name$"

It is defined in YAML as follows:

- name: _multiprocessor
  type: sequence
  processors:
  - type: <processor type>
    condition: attributes["request"]["path"] == "/json/view"

s

See Also