Edge Delta Split With Delimiter Node

Overview

The Split with delimiter node splits the body field using a given delimiter. It creates individual logs for each delimited event discovered in the input log.

To ingest logs that include line breaks, such as multi-line JSON, consider using the line_pattern parameter on Kubernetes source or File source nodes.

• incoming_data_types: log
• outgoing_data_types: log

Example Configuration

This configuration splits logs where it detects a semi-colon.

nodes:
- name: split_items_test
  type: split_with_delimiter
  delimiter: ;

Input log

Suppose this log in processed by the node. It is a single log line consisting of three events that are separated with a semi-colon.

2023-04-14 09:15:00 node12 INFO: User login successful;2023-04-14 09:16:25 node12 WARN: Low disk space on server;2023-04-14 09:17:45 node12 ERROR: Database connection failed

The output is split into three individual logs.

{
  "_type": "log",
  "body": "2023-04-14 09:15:00 node12 INFO: User login successful",
  "resource": {...},
  "timestamp": 1726710206606
},
{
  "_type": "log",
  "body": "2023-04-14 09:16:25 node12 WARN: Low disk space on server",
  "resource": {...},
  "timestamp": 1726710206606
},
{
  "_type": "log",
  "body": "2023-04-14 09:17:45 node12 ERROR: Database connection failed",
  "resource": {...},
  "timestamp": 1726710206606
}

Each of these logs can now be individually processed for further analysis, such as alerting for warnings or errors, monitoring user activity, or tracking system health.

Required Parameters

name

A descriptive name for the node. This is the name that will appear in Visual Pipelines and you can reference this node in the YAML using the name. It must be unique across all nodes. It is a YAML list element so it begins with a - and a space followed by the string. It is a required parameter for all nodes.

nodes:
  - name: <node name>
    type: <node type>

type: split_with_delimiter

The type parameter specifies the type of node being configured. It is specified as a string from a closed list of node types. It is a required parameter.

nodes:
  - name: <node name>
    type: <node type>

delimiter

The delimiter parameter defines the string that will be used for splitting the raw log. It is specified as a string and is required.

nodes:
  - name: split_items
    type: split_with_delimiter
    delimiter: ","