Edge Delta Enrichment Node
Apply one enrichment to all logs in a stream.
2 minute read
Transformation nodes manipulate, enhance, and format log data in the pipeline before it is used for monitoring, analytics, or storage. Together, these nodes ensure that the data is optimized for actionable insights and a streamlined monitoring workflow.
Consider a basic log data item:
{
"_type": "log"
"attributes": {
"newfield": "I added this new field value"
}
"body": "{"timestamp":"2023-04-23T12:34:56.789Z","logLevel":"ERROR","serviceName":"AuthService","nodeId":"node2","message":"Login failed","clientIP":"192.168.1.10","username":"user123","event":"login_attempt","outcome":"failure"}"
"resource": {
"config_id": "87654321-1321-69874-9456-s5123456h7"
"ed.tag": "ed_parallel"
"host.name": "ED_TEST"
"ip": "10.0.0.1"
"src_type": ""
}
"timestamp": 1703677287365
}
Each transformation node is designed for enriching different sections of the data item:
Node | Attribute | Resource | Body | Timestamp | Any Field | Restrictions |
---|---|---|---|---|---|---|
Output Transform | Y | Y | Y | Y | Y | Outputs a Custom type data item that cant be ingested by the Edge Delta Archive node. The whole payload is flattened and sent as the event, with all other fields empty. |
Log Transform | Y | N | N | Y | N | Can only ingest logs, and it outputs only logs. |
Resource Transform | N | Y | N | N | N | Can only ingest logs, and it outputs only logs. |
Mask | N | N | Y | N | N | Can only ingest logs, and it outputs only logs. New value can only be a static string. |
The body field is protected from dynamic enrichment until the end of the pipeline (Output Transform) to prevent schema changes from disabling pipeline functionality.
Apply one enrichment to all logs in a stream.
Transform logs using upsert and delete operations.
Mask values in logs using the Edge Delta Mask Node.
Transform and output logs.
Transform resource fields.