Use Multiprocessors in Edge Delta

Overview

Multiprocessor nodes streamline the way you shape and manage data through your observability pipeline. Traditionally, you create and link individual processors, often resulting in complex and error-prone graph topologies. Multiprocessors solve this by allowing you to define a sequence of processors in a single, simplified step.

Each processor in the sequence is automatically connected to the next, enabling a clear, sequential flow of data processing. This not only speeds up pipeline creation but also significantly reduces operational complexity.

Note: There is no need to decode the body before operating on it, despite many processors using OTTL under the hood. Similarly, patterns are automatically escaped for you.

Create a Multiprocessor

When you add a source or destination, a multiprocessor is added automatically.

Alternatively, in pipeline Edit Mode, click Add Processor and select Multi Processor.

Configure a Processor

A multiprocessor node is configured using Live Capture data. To start, it recommends a particular processor based on the detected input traffic. In this instance, the logs are JSON formatted so it recommends a Parse JSON processor:

Click the + icon to add the recommended processor or click + Add a processor to add a different processor.

In this example the Parse JSON processor is added to the multiprocessor.

If you select a log in the input pane, the resulting log is shown in the output. Note the green diff block showing the parsed attributes, as well as the 24% increase in data item size:

Autocomplete

As you specify fields, the fields detected in the live capture sample are offered by autocomplete:

Similarly, as you specify values, autocomplete offers samples based on the live capture sample.

Configure a Sequence of Processors

As you configure additional processors, they are stacked vertically in the centre pane. Data items are usually processed by each in turn from the top down. In this example, three processors operate in turn with the final processor filtering out certain data items.

Take note of the affected markers: this time some are red indicating those data items have been dropped. Also take note of the size decrease: despite increasing the individual size of each data item, the overall throughput volume is less on balance due to the filtering.

Disable Processors

You can toggle specific processors in the sequence on or off. You might do this as you experiment with the sequence of processors, or when troubleshooting.

In this instance, the Extract Metric process is being used after parsing JSON but without string conversion or filtering.