Edge Delta Enrichment Node

This node is deprecated, replaced by the the Log Transform, Output Transform or Mask nodes.

Overview

An enrichment node modifies data as it is being streamed through the node. The value used to enrich the logs is calculated only at the first log. This calculated value is then applied to all subsequent logs per source. You can configure enrichment to, for example add attributes or metadata to a new or existing field to make troubleshooting easier; or transform a field to match a particular data format.

Enrichment does not re-calculate the enrichment value for each log - it uses the value determined by the first log. Transform node transformations are calculated for each log processed.

Example Configuration 1

In this specific configuration, a field named environment with the value "kind" is added to each incoming log message as an attribute. The override_if_exists parameter is set to true, which means that if there is an existing environment field, its value will be updated to "kind". In addition an attribute called enriched-hostname is added and it uses the value that it finds in the host.name field within the resources parameter that was added by the input node.

See here for details on field path references.

nodes:
- name: enrich_test
  type: enrich
  field_mappings:
  - field_name: environment
    value: '"kind"'
    override_if_exists: true
  - field_name: enriched-hostname
    value: item["resource"]["host.name"]

Suppose the following log is sent through a pipeline with this configuration.

{
    "timestamp": "2023-04-23T12:34:56.789Z",
    "logLevel": "ERROR",
    "serviceName": "AuthService",
    "nodeId": "node3",
    "message": "Login failed",
    "clientIP": "192.168.1.10",
    "username": "user123",
    "event": "login_attempt",
    "outcome": "failure"
}

The enrichments are added as attributes.

Required Parameters

name

A descriptive name for the node. This is the name that will appear in Visual Pipelines and you can reference this node in the yaml using the name. It must be unique across all nodes. It is a yaml list element so it begins with a - and a space followed by the string. It is a required parameter for all nodes.

nodes:
  - name: <node name>
    type: <node type>

type: enrich

The type parameter specifies the type of node being configured. It is specified as a string from a closed list of node types. It is a required parameter.

nodes:
  - name: <node name>
    type: <node type>

field_mappings

The field_mappings parameter define the fields that will be enriched by the Enrichment node. All field_mappings have - field_name and a value child parameters. A field_mappings parameter is required for an enrichment node.

The field_name parameter defines the log field that will be enriched either by updating the value if the field already exists in the log, or by adding the field if it doesn’t already exist. It is specified as a string. The field_name parameter is required for an enrichment node.

The value parameter specifies the logic for creating a derived value for the field_name. You can use a CEL custom macro as the value to reference the field.

The override_if_exists parameter configures whether to over-write the values derived from enrichment if they exist already. It is a child parameter of the field mappings and is specific to a particular field_name. It is specified as a Boolean and the default is false. It is optional.

nodes:
  - name: <node name>
    type: enrich
    field_mappings:
    - field_name: <name of field to enrich>
      value: <source of data to enrich the field>  
    - field_name: <name of field to enrich>
      value: <source of data to enrich the field>
      override_existing_value: true | false