Edge Delta's OTTL Extensions

Transform or route data using Edge Delta's OTTL extensions, including custom functions like EDXCoalesce and EDXCompress.

28 minute read

Edge Delta Custom OTTL

Edge Delta has extended the OTTL with custom functionality. The extensions are organized into two categories: Converter Functions and Editor Functions.

Converter Functions

Converter functions transform, extract, or evaluate data and return values that can be assigned to fields.

EDXCoalesce

Minimum Agent Version: v2.0.0

EDXCoalesce fills a gap in standard OTTL which lacks null coalescing capabilities. While OTTL can check for null values with IsMatch, it doesn’t provide an elegant way to select the first non-null value from multiple options. This Edge Delta extension works like the null coalescing operator (??) in modern languages, enabling graceful handling of missing or empty fields with a fallback value.

Syntax

EDXCoalesce(value, fallback)

value: The primary value or field reference to evaluate.
fallback: The fallback value or field reference to return if the primary value is null, empty, or zero.

For more than two values, nest multiple EDXCoalesce calls:

EDXCoalesce(value1, EDXCoalesce(value2, EDXCoalesce(value3, defaultValue)))

Input

{
  "_type": "log",
  "timestamp": 1735789600000,
  "body": "User activity log",
  "resource": {...},
  "attributes": {
    "user_id": null,
    "session_id": "",
    "tracking_id": "track-12345",
    "primary_email": null,
    "secondary_email": "",
    "fallback_email": "default@example.com",
    "error_message": "",
    "warning_message": null,
    "info_message": "Operation completed"
  }
}

Example

set(attributes["effective_id"], EDXCoalesce(attributes["user_id"], EDXCoalesce(attributes["session_id"], EDXCoalesce(attributes["tracking_id"], "anonymous"))))
set(attributes["contact_email"], EDXCoalesce(attributes["primary_email"], EDXCoalesce(attributes["secondary_email"], attributes["fallback_email"])))
set(attributes["log_message"], EDXCoalesce(attributes["error_message"], EDXCoalesce(attributes["warning_message"], EDXCoalesce(attributes["info_message"], "No message"))))

Output

{
  "_type": "log",
  "timestamp": 1735789630000,
  "body": "User activity log",
  "resource": {...},
  "attributes": {
    "user_id": null,
    "session_id": "",
    "tracking_id": "track-12345",
    "primary_email": null,
    "secondary_email": "",
    "fallback_email": "default@example.com",
    "error_message": "",
    "warning_message": null,
    "info_message": "Operation completed",
    "effective_id": "track-12345",
    "contact_email": "default@example.com",
    "log_message": "Operation completed"
  }
}

The EDXCoalesce function has selected the first non-null/non-empty value for each field: “track-12345” for the effective ID (skipping null user_id and empty session_id), “default@example.com” for the contact email (after checking primary and secondary), and “Operation completed” for the log message (after checking error and warning messages).

EDXCompress

Minimum Agent Version: v1.31.0

EDXCompress fills a gap in standard OTTL which lacks native compression capabilities. This Edge Delta extension enables you to compress strings or byte arrays using various algorithms (gzip, zlib, deflate, snappy, zstd), which is essential for reducing data size before storage or transmission in telemetry pipelines.

Syntax

EDXCompress(data, algorithm, asBytes...)

data: The string or byte array to compress (can be a field reference or literal).
algorithm: The compression algorithm to use. Supported values: "gzip", "zlib", "deflate", "snappy", "zstd", "lz4".
asBytes (optional): Boolean flag to control output format. When true (default), returns a byte array. When false, returns a base64-encoded string. This parameter is variadic and can be omitted.

Input

{
  "_type": "log",
  "timestamp": 1735789200000,
  "body": "This is a very long log message that contains repeated information. This is a very long log message that contains repeated information. This is a very long log message that contains repeated information.",
  "resource": {...},
  "attributes": {
    "original_size": 198
  },
  "cache": {
    "body": "This is a very long log message that contains repeated information. This is a very long log message that contains repeated information. This is a very long log message that contains repeated information."
  }
}

Example

set(body, EDXCompress(cache["body"], "gzip"))
set(attributes["compression"], "gzip")
set(attributes["compressed_size"], Len(body))

Output

{
  "_type": "log",
  "timestamp": 1735789230000,
  "body": "[compressed byte array]",
  "resource": {...},
  "attributes": {
    "original_size": 198,
    "compression": "gzip",
    "compressed_size": 52
  },
  "cache": {
    "body": "This is a very long log message that contains repeated information. This is a very long log message that contains repeated information. This is a very long log message that contains repeated information."
  }
}

The body field now contains the compressed version of the original text as a byte array, significantly reducing the data size from 198 bytes to approximately 52 bytes when using gzip compression.

EDXDataType

Minimum Agent Version: v1.32.0

EDXDataType fills a gap in standard OTTL which lacks type introspection capabilities. While OTTL provides type conversion functions like Int and String, it doesn’t offer a way to determine the current type of a field. This Edge Delta extension enables you to inspect data types at runtime, which is essential for debugging, conditional processing, and data validation.

Syntax

EDXDataType(input)

input: The field or value whose data type you want to determine.

Input

{
  "_type": "log",
  "timestamp": 1735789400000,
  "body": "Test log message",
  "resource": {...},
  "attributes": {
    "string_field": "hello",
    "number_field": 42,
    "float_field": 3.14,
    "boolean_field": true,
    "null_field": null,
    "array_field": ["item1", "item2"],
    "map_field": {
      "key1": "value1",
      "key2": "value2"
    }
  }
}

Example

set(attributes["types"]["string_type"], EDXDataType(attributes["string_field"]))
set(attributes["types"]["number_type"], EDXDataType(attributes["number_field"]))
set(attributes["types"]["float_type"], EDXDataType(attributes["float_field"]))
set(attributes["types"]["boolean_type"], EDXDataType(attributes["boolean_field"]))
set(attributes["types"]["null_type"], EDXDataType(attributes["null_field"]))
set(attributes["types"]["array_type"], EDXDataType(attributes["array_field"]))
set(attributes["types"]["map_type"], EDXDataType(attributes["map_field"]))
set(attributes["types"]["body_type"], EDXDataType(body))

Output

{
  "_type": "log",
  "timestamp": 1735789430000,
  "body": "Test log message",
  "resource": {...},
  "attributes": {
    "string_field": "hello",
    "number_field": 42,
    "float_field": 3.14,
    "boolean_field": true,
    "null_field": null,
    "array_field": ["item1", "item2"],
    "map_field": {
      "key1": "value1",
      "key2": "value2"
    },
    "types": {
      "string_type": "string",
      "number_type": "int",
      "float_type": "float",
      "boolean_type": "bool",
      "null_type": "nil",
      "array_type": "array",
      "map_type": "map",
      "body_type": "bytes"
    }
  }
}

The EDXDataType function has identified the data type of each field, returning type names such as “string”, “int”, “float”, “bool”, “nil”, “array”, “map”, and “bytes” (for the body field which is a byte array).

EDXDecompress

Minimum Agent Version: v1.31.0

EDXDecompress complements EDXCompress and fills a gap in standard OTTL which lacks native decompression capabilities. This Edge Delta extension enables you to decompress byte arrays that have been compressed with various algorithms, allowing you to work with compressed data sources in your telemetry pipeline.

Syntax

EDXDecompress(data, algorithm)

data: The compressed byte array or string to decompress (can be a field reference or literal).
algorithm: The decompression algorithm to use. Supported values: "gzip", "zlib", "deflate", "snappy", "zstd", "lz4".

Input

{
  "_type": "log",
  "timestamp": 1735789300000,
  "body": "[compressed byte array]",
  "resource": {...},
  "attributes": {
    "compression": "gzip",
    "compressed_size": 52
  },
  "cache": {
    "body": "[compressed byte array]"
  }
}

Example

set(attributes["decompressed_body"], EDXDecompress(cache["body"], "gzip"))
set(attributes["decompressed_size"], Len(attributes["decompressed_body"]))

Output

{
  "_type": "log",
  "timestamp": 1735789330000,
  "body": "[compressed byte array]",
  "resource": {...},
  "attributes": {
    "compression": "gzip",
    "compressed_size": 52,
    "decompressed_body": "This is a very long log message that contains repeated information. This is a very long log message that contains repeated information. This is a very long log message that contains repeated information.",
    "decompressed_size": 198
  },
  "cache": {
    "body": "[compressed byte array]"
  }
}

The compressed data from cache["body"] has been decompressed and stored as a string in attributes["decompressed_body"], restoring the original 198-byte message from the 52-byte compressed format.

EDXDecode

Minimum Agent Version: v2.5.0

The EDXDecode function decodes encoded strings using various encoding formats. It supports URL encoding (percent encoding) and hexadecimal encoding.

Syntax

EDXDecode(value, encoding_type)

Parameters

value: The encoded string to decode
encoding_type: The type of encoding to decode from. Supported values:
- "url": URL/percent encoding (e.g., %20 becomes a space)
- "hex": Hexadecimal encoding (e.g., 48656c6c6f becomes “Hello”)

Input

{
  "_type": "log",
  "timestamp": 1757000000000,
  "body": "url=https%3A%2F%2Fapi.example.com%2Fpath%3Fquery%3Dvalue&message=Hello%20World%21",
  "resource": {...},
  "attributes": {
    "hex_data": "4564676544656c7461"
  }
}

Example

// Parse the body to extract key-value pairs
set(attributes["kv_map"], ParseKeyValue(body))

// Decode URL encoded values
set(attributes["decoded_url"], EDXDecode(attributes["kv_map"]["url"], "url"))
set(attributes["decoded_message"], EDXDecode(attributes["kv_map"]["message"], "url"))

// Decode hex encoded value
set(attributes["decoded_hex"], EDXDecode(attributes["hex_data"], "hex"))

Output

{
  "_type": "log",
  "timestamp": 1757000000000,
  "body": "url=https%3A%2F%2Fapi.example.com%2Fpath%3Fquery%3Dvalue&message=Hello%20World%21",
  "resource": {...},
  "attributes": {
    "hex_data": "4564676544656c7461",
    "kv_map": {
      "url": "https%3A%2F%2Fapi.example.com%2Fpath%3Fquery%3Dvalue",
      "message": "Hello%20World%21"
    },
    "decoded_url": "https://api.example.com/path?query=value",
    "decoded_message": "Hello World!",
    "decoded_hex": "EdgeDelta"
  }
}

The EDXDecode function has decoded the URL-encoded strings (converting %3A to :, %2F to /, %20 to space, etc.) and the hexadecimal string to its ASCII representation.

EDXDecrypt

Minimum Agent Version: v2.5.0

EDXDecrypt restores original values from encrypted strings produced by EDXEncrypt. The function automatically extracts the key ID from the encrypted format and uses it along with the provided keyclass to locate the correct decryption key. To maintain JSON safety, the decrypted value is returned as a double-escaped string.

For comprehensive documentation including key configuration, deployment instructions, and best practices, see EDXEncrypt and EDXDecrypt Extensions.

Requirements

The ED_CRYPTO_PATH environment variable must be set, pointing to a directory containing keys.json
The keys.json file must be an array of key objects with matching keyId and keyclass

Syntax

EDXDecrypt(encryptedValue, keyclass)

encryptedValue: The encrypted string in format #keyId|iv|ciphertext# to decrypt
keyclass: An integer (1-99) that must match the key’s configured class for authorization

Input

{
  "_type": "log",
  "timestamp": 1756998000000,
  "body": "Processing encrypted user data",
  "resource": {...},
  "attributes": {
    "enc_ssn": "#prod-key|ixpb3NRQfrsV+bPR|qKCj8Vr1SjRBE6A6lIRVtOyHPrlemQMDQpsM#",
    "enc_credit_card": "#prod-key|k8T6MO7B/G2Oi/zJ|isk+OEucriLYVnKnr1PbIiwPxV533zv2DJzyT6bdfk0=#",
    "authorized": true
  }
}

This input shows encrypted data that was previously processed by EDXEncrypt. The encrypted values follow the format #keyId|iv|ciphertext# where prod-key is the key identifier, followed by the initialization vector and ciphertext in base64 encoding. The authorized attribute serves as a control flag for conditional decryption.

Example

set(attributes["decrypted_ssn"], EDXDecrypt(attributes["enc_ssn"], 1)) where attributes["authorized"] == true
set(attributes["decrypted_ssn_clean"], ReplaceAllStrings(attributes["decrypted_ssn"], "\\", "")) where attributes["decrypted_ssn"] != nil

The first statement decrypts the SSN only when the authorized attribute is true, using keyclass 1 for authorization. The second statement removes the double-escaping from the decrypted value using ReplaceAllStrings to provide a clean string for downstream use.

Output

{
  "_type": "log",
  "timestamp": 1756998030000,
  "body": "Processing encrypted user data",
  "resource": {...},
  "attributes": {
    "enc_ssn": "#prod-key|ixpb3NRQfrsV+bPR|qKCj8Vr1SjRBE6A6lIRVtOyHPrlemQMDQpsM#",
    "enc_credit_card": "#prod-key|k8T6MO7B/G2Oi/zJ|isk+OEucriLYVnKnr1PbIiwPxV533zv2DJzyT6bdfk0=#",
    "authorized": true,
    "decrypted_ssn": "\"214-16-0214\"",
    "decrypted_ssn_clean": "214-16-0214"
  }
}

The decrypted_ssn shows the double-escaped format with quotes, while decrypted_ssn_clean shows the cleaned value after normalization. If decryption fails (wrong keyclass, missing key, or invalid format), the function returns the original encrypted string unchanged.

EDXEncode

Minimum Agent Version: v1.25.0

EDXEncode fills a critical gap in standard OTTL by providing byte array encoding capabilities. While OTTL has a Decode function for converting byte arrays to strings, it lacks a corresponding encode function. This Edge Delta extension enables you to convert strings back to byte arrays, which is essential when working with the body field that must remain as a byte array.

When a log is ingested by the Edge Delta agent, a log data item is created and the log contents are placed in the body field. Importantly, this field is a byte array. Therefore, to perform any OTTL transformation that requires the body as a string input, it needs to be decoded from byte array to a string type as part of the transformation function. See Working with the body.

After copying, decoding and transforming the body into a field other than body, you can use the EDXEncode custom function to save a new body field in the appropriate byte array type.

Syntax

EDXEncode(target, encoding, asBytes...)

target: The string value that should be encoded (can be a field reference or literal string).
encoding: The encoding format from the IANA encoding index, such as "utf-8" or "base64".
asBytes (optional): Boolean flag to control output format. When true (default), returns a byte array. When false, returns a string. This parameter is variadic and can be omitted.

Suppose this log is ingested:

session_id=abc123 user_id=admin event_type=login event_status=success debug_info=verbose connection_id=conn456 temp_file=report_tmp.log temp_data=sensitiveData

The following operations are performed to extract, decode, parse, and transform the data:

set(attributes["decoded_body"], Decode(body, "utf-8"))
set(attributes["kv_map"], ParseKeyValue(attributes["decoded_body"]))
delete_matching_keys(attributes["kv_map"], "(temp_.*|debug_.*)")
set(attributes["new_body"], ToKeyValueString(attributes["kv_map"]))

See Decode, ParseKeyValue, delete_matching_keys, and ToKeyValueString.

These operations result in the following log:

Input:

{
  "_type": "log",
  "timestamp": 1735802408445,
  "body": "session_id=abc123 user_id=admin event_type=login event_status=success debug_info=verbose connection_id=conn456 temp_file=report_tmp.log temp_data=sensitiveData",
  "resource": {...},
  "attributes": {
    "decoded_body": "session_id=abc123 user_id=admin event_type=login event_status=success debug_info=verbose connection_id=conn456 temp_file=report_tmp.log temp_data=sensitiveData",
    "kv_map": {
      "connection_id": "conn456",
      "event_status": "success",
      "event_type": "login",
      "session_id": "abc123",
      "user_id": "admin"
    },
    "new_body": "event_status=success connection_id=conn456 session_id=abc123 user_id=admin event_type=login"
  }
}

Example:

set(body, EDXEncode(attributes["new_body"], "utf-8", true))

Output:

{
  "_type": "log",
  "timestamp": 1735802426292,
  "body": "connection_id=conn456 session_id=abc123 user_id=admin event_type=login event_status=success",
  "resource": {...},
  "attributes": {
    "decoded_body": "session_id=abc123 user_id=admin event_type=login event_status=success debug_info=verbose connection_id=conn456 temp_file=report_tmp.log temp_data=sensitiveData",
    "kv_map": {
      "connection_id": "conn456",
      "event_status": "success",
      "event_type": "login",
      "session_id": "abc123",
      "user_id": "admin"
    },
    "new_body": "connection_id=conn456 session_id=abc123 user_id=admin event_type=login event_status=success"
  }
}

The new_body attribute has been upserted into the body field and encoded as a byte array.

EDXEncrypt

Minimum Agent Version: v2.5.0

EDXEncrypt protects sensitive string values using field-level encryption. The function selects an encryption key based on the provided key ID and class, then returns either an encrypted value or a mask value on failure. The encrypted format preserves the key ID for future decryption while preventing plaintext exposure even when encryption cannot proceed.

For comprehensive documentation including key configuration, deployment instructions, and best practices, see EDXEncrypt and EDXDecrypt Extensions.

Requirements

The ED_CRYPTO_PATH environment variable must be set, pointing to a directory containing keys.json
The keys.json file must be an array of key objects with the referenced keyId and matching keyclass

Syntax

EDXEncrypt(value, keyclass, keyId, maskValue)

value: The string to encrypt (can be a field reference like body["user"]["ssn"] or a literal string)
keyclass: An integer (1-99) that must match the key’s configured class
keyId: The unique identifier of the encryption key to use, as defined in your keys.json file
maskValue: A fallback string returned when encryption fails (e.g., "<SSN-MASKED>")

Input

{
  "_type": "log",
  "timestamp": 1756998000000,
  "body": {
    "user": {
      "ssn": "214-16-0214",
      "credit_card": "4111-0214-1498-2782",
      "email": "user14@example.com"
    }
  },
  "resource": {...},
  "attributes": {
    "user_id": "usr_12345"
  }
}

This input shows a log with parsed JSON data in the body field. When using log_parsing_mode: full with a Kubernetes input, Edge Delta automatically parses JSON logs into structured fields within the body. This allows you to directly reference nested fields like body["user"]["ssn"] in your OTTL statements.

Example

set(attributes["enc_ssn"], EDXEncrypt(body["user"]["ssn"], 1, "prod-key", "<SSN-MASKED>")) where body["user"]["ssn"] != nil
set(attributes["enc_credit_card"], EDXEncrypt(body["user"]["credit_card"], 1, "prod-key", "<CC-MASKED>")) where body["user"]["credit_card"] != nil

These statements encrypt sensitive fields from the parsed JSON body. Each statement reads a value from the nested body structure, encrypts it using the prod-key with keyclass 1, and stores the result in a new attribute. If encryption fails for any reason, the mask value is stored instead.

Output

{
  "_type": "log",
  "timestamp": 1756998030000,
  "body": {
    "user": {
      "ssn": "214-16-0214",
      "credit_card": "4111-0214-1498-2782",
      "email": "user14@example.com"
    }
  },
  "resource": {...},
  "attributes": {
    "user_id": "usr_12345",
    "enc_ssn": "#prod-key|ixpb3NRQfrsV+bPR|qKCj8Vr1SjRBE6A6lIRVtOyHPrlemQMDQpsM#",
    "enc_credit_card": "#prod-key|k8T6MO7B/G2Oi/zJ|isk+OEucriLYVnKnr1PbIiwPxV533zv2DJzyT6bdfk0=#"
  }
}

The sensitive fields are now encrypted in the format #keyId|iv|ciphertext#. The original body fields remain unchanged. If encryption fails (missing key, wrong keyclass, expired key, or invalid configuration), the function returns the provided mask value instead.

For detailed key configuration, Kubernetes deployment, troubleshooting, and end-to-end examples, see the comprehensive EDXEncrypt and EDXDecrypt guide.

EDXEnv

Minimum Agent Version: v2.5.0

EDXEnv fills a gap in standard OTTL which lacks environment variable access. While OTTL can work with resource attributes and other telemetry data, it cannot access system environment variables at runtime. This Edge Delta extension enables you to inject configuration values, deployment-specific information, or secrets from environment variables into your telemetry data, making pipelines more dynamic and configurable.

Syntax

EDXEnv(env_variable_name, default_value)

env_variable_name: The name of the environment variable to retrieve.
default_value: The default value to return if the environment variable is not set or is empty (optional).

Input

{
  "_type": "log",
  "timestamp": 1735789700000,
  "body": "Application started",
  "resource": {...},
  "attributes": {
    "app_name": "telemetry-processor",
    "start_time": "2024-01-10T10:00:00Z"
  }
}

Assuming these environment variables are set:

ENVIRONMENT=production
REGION=us-west-2
CLUSTER_NAME=telemetry-cluster-01

Example

set(attributes["environment"], EDXEnv("ENVIRONMENT", "development"))
set(attributes["region"], EDXEnv("REGION", "us-east-1"))
set(attributes["cluster"], EDXEnv("CLUSTER_NAME", "default-cluster"))
set(attributes["debug_mode"], EDXEnv("DEBUG_MODE", "false"))
set(attributes["api_version"], EDXEnv("API_VERSION", "v1"))

Output

{
  "_type": "log",
  "timestamp": 1735789730000,
  "body": "Application started",
  "resource": {...},
  "attributes": {
    "app_name": "telemetry-processor",
    "start_time": "2024-01-10T10:00:00Z",
    "environment": "production",
    "region": "us-west-2",
    "cluster": "telemetry-cluster-01",
    "debug_mode": "false",
    "api_version": "v1"
  }
}

The EDXEnv function has retrieved the values from environment variables where they exist (ENVIRONMENT, REGION, CLUSTER_NAME) and used the provided default values for variables that don’t exist (DEBUG_MODE defaults to “false”, API_VERSION defaults to “v1”).

EDXExtractPatterns

Minimum Agent Version: v1.22.0

EDXExtractPatterns enhances the standard OTTL ExtractPatterns converter function. While the default OTTL function requires a hardcoded regex pattern, this Edge Delta extension enables you to use a field reference for the pattern parameter, allowing dynamic pattern matching based on runtime data.

Syntax

EDXExtractPatterns(input, "pattern")

input: The source string, such as the body of a log, from which patterns are to be extracted.
pattern: A Golang regex pattern or field reference containing a regex pattern, used to match and extract specific parts of the input string. Named patterns can be used to provide field names.

Input:

{
  "_type": "log",
  "timestamp": 1735810699536,
  "body": "session_id=abc123 event_type=login event_status=success debug_info=verbose temp_file=report_tmp.log temp_data=sensitiveData",
  "resource": {...},
  "attributes": {
    "decoded_body": "session_id=abc123 event_type=login event_status=success debug_info=verbose temp_file=report_tmp.log temp_data=sensitiveData",
    "pattern": "session_id=(?P<session_id>\\w+)"
  }
}

Notice how the pattern in the pattern attribute has been escaped. See Regex in OTTL.

Example:

set(attributes["id"], EDXExtractPatterns(attributes["decoded_body"], attributes["pattern"]))

Output:

{
  "_type": "log",
  "timestamp": 1735810732304,
  "body": "session_id=abc123 event_type=login event_status=success debug_info=verbose temp_file=report_tmp.log temp_data=sensitiveData",
  "resource": {...},
  "attributes": {
    "decoded_body": "session_id=abc123 event_type=login event_status=success debug_info=verbose temp_file=report_tmp.log temp_data=sensitiveData",
    "id": {
      "session_id": "abc123"
    },
    "pattern": "session_id=(?P<session_id>\\w+)"
  }
}

The pattern in the pattern attribute was used to extract the session_id’s value from the decoded body.

EDXHmac

Minimum Agent Version: v1.22.0

EDXHmac computes Hash-based Message Authentication Code (HMAC) for data integrity and authentication verification. HMAC combines a cryptographic hash function with a secret key to produce a message authentication code. This is commonly used for API request signing, webhook validation, JWT token generation, and ensuring data hasn’t been tampered with during transmission.

Syntax

EDXHmac(value, key, algorithm)

value: The string value to compute the HMAC for (can be a field reference or literal string).
key: The secret key used for HMAC computation (can be a field reference or literal string).
algorithm: The hash algorithm to use. Supported values: "md5", "sha1", "sha224", "sha256", "sha384", "sha512".

Input

{
  "_type": "log",
  "timestamp": 1735789800000,
  "body": "API request payload",
  "resource": {...},
  "attributes": {
    "request_body": "user_id=12345&action=purchase&amount=99.99",
    "webhook_payload": "{\"event\":\"user.signup\",\"user_id\":\"abc123\"}",
    "api_secret": "my-secret-key-2024",
    "jwt_claims": "{\"sub\":\"user123\",\"exp\":1735890000}"
  }
}

Example

set(attributes["request_signature"], EDXHmac(attributes["request_body"], attributes["api_secret"], "sha256"))
set(attributes["webhook_signature"], EDXHmac(attributes["webhook_payload"], "webhook-secret", "sha256"))
set(attributes["jwt_signature"], EDXHmac(attributes["jwt_claims"], EDXEnv("JWT_SECRET"), "sha512"))

Output

{
  "_type": "log",
  "timestamp": 1735789830000,
  "body": "API request payload",
  "resource": {...},
  "attributes": {
    "request_body": "user_id=12345&action=purchase&amount=99.99",
    "webhook_payload": "{\"event\":\"user.signup\",\"user_id\":\"abc123\"}",
    "api_secret": "my-secret-key-2024",
    "jwt_claims": "{\"sub\":\"user123\",\"exp\":1735890000}",
    "request_signature": "8f3e4d5c2a1b9f7e6d5c4b3a2e1d0c9b8a7f6e5d4c3b2a1f0e9d8c7b6a5f4e3d2",
    "webhook_signature": "7a2b3c4d5e6f7g8h9i0j1k2l3m4n5o6p7q8r9s0t1u2v3w4x5y6z7a8b9c0d1e2f3",
    "jwt_signature": "a1b2c3d4e5f6g7h8i9j0k1l2m3n4o5p6q7r8s9t0u1v2w3x4y5z6a7b8c9d0e1f2g3h4i5j6k7l8m9n0o1p2q3r4s5t6u7v8w9x0y1z2"
  }
}

The EDXHmac function has computed HMAC signatures for each input: a request signature using SHA-256 with the API secret, a webhook signature using SHA-256 with a hardcoded secret, and a JWT signature using SHA-512 with a secret from an environment variable.

EDXIfElse

Minimum Agent Version: v2.0.0

EDXIfElse fills a gap in standard OTTL which lacks inline conditional operators. While OTTL supports conditional logic with where clauses for conditional execution of entire statements, it doesn’t provide a ternary operator for conditional value assignment within a single statement. This Edge Delta extension enables you to choose between two values based on a condition, similar to the ternary operator (? :) in many programming languages.

Syntax

EDXIfElse(condition, value_if_true, value_if_false)

condition: A boolean expression or field that evaluates to true or false.
value_if_true: The value to return if the condition is true.
value_if_false: The value to return if the condition is false.

Input

{
  "_type": "log",
  "timestamp": 1735789500000,
  "body": "Transaction processed",
  "resource": {...},
  "attributes": {
    "transaction_amount": 150,
    "user_type": "premium",
    "is_weekend": true,
    "error_count": 0
  }
}

Example

set(attributes["transaction_size"], EDXIfElse(attributes["transaction_amount"] > 100, "large", "small"))
set(attributes["discount_eligible"], EDXIfElse(attributes["user_type"] == "premium", true, false))
set(attributes["processing_mode"], EDXIfElse(attributes["is_weekend"], "batch", "realtime"))
set(attributes["status"], EDXIfElse(attributes["error_count"] > 0, "failed", "success"))

Output

{
  "_type": "log",
  "timestamp": 1735789530000,
  "body": "Transaction processed",
  "resource": {...},
  "attributes": {
    "transaction_amount": 150,
    "user_type": "premium",
    "is_weekend": true,
    "error_count": 0,
    "transaction_size": "large",
    "discount_eligible": true,
    "processing_mode": "batch",
    "status": "success"
  }
}

The EDXIfElse function has evaluated each condition and assigned the appropriate values: “large” for transaction size (amount > 100), true for discount eligibility (premium user), “batch” for processing mode (weekend), and “success” for status (no errors).

EDXLookup

Minimum Agent Version: v1.22.0

EDXLookup performs safe map lookups with default value fallback. While OTTL provides direct map access using bracket notation (e.g., attributes["key"]), accessing a non-existent key results in a null value that can cause errors in subsequent operations. This Edge Delta extension provides a safe way to look up values in maps with a default value that’s returned when the key doesn’t exist, similar to the get() method with defaults in many programming languages.

Syntax

EDXLookup(map, key, default_value)

map: The map to perform the lookup on (typically attributes, resource, or a nested map field).
key: The key to look up in the map (can be a field reference or literal string).
default_value: The value to return if the key doesn’t exist in the map.

Input

{
  "_type": "log",
  "timestamp": 1735789900000,
  "body": "User event logged",
  "resource": {...},
  "attributes": {
    "status_codes": {
      "200": "OK",
      "404": "Not Found",
      "500": "Internal Server Error"
    },
    "current_status": "200",
    "missing_status": "999",
    "severity_map": {
      "error": "high",
      "warn": "medium",
      "info": "low"
    },
    "log_level": "debug"
  }
}

Example

set(attributes["status_message"], EDXLookup(attributes["status_codes"], attributes["current_status"], "Unknown Status"))
set(attributes["unknown_status_message"], EDXLookup(attributes["status_codes"], attributes["missing_status"], "Unknown Status"))
set(attributes["severity"], EDXLookup(attributes["severity_map"], attributes["log_level"], "low"))
set(attributes["priority"], EDXLookup(attributes["severity_map"], "critical", "unknown"))

Output

{
  "_type": "log",
  "timestamp": 1735789930000,
  "body": "User event logged",
  "resource": {...},
  "attributes": {
    "status_codes": {
      "200": "OK",
      "404": "Not Found",
      "500": "Internal Server Error"
    },
    "current_status": "200",
    "missing_status": "999",
    "severity_map": {
      "error": "high",
      "warn": "medium",
      "info": "low"
    },
    "log_level": "debug",
    "status_message": "OK",
    "unknown_status_message": "Unknown Status",
    "severity": "low",
    "priority": "unknown"
  }
}

The EDXLookup function has safely performed map lookups: found “OK” for status code 200, returned “Unknown Status” for non-existent status code 999, returned “low” default for missing “debug” key in severity map, and returned “unknown” default for missing “critical” key.

EDXParseKeyValue

Minimum Agent Version: v1.22.0

EDXParseKeyValue significantly enhances the standard OTTL ParseKeyValue function. While the default OTTL function overwrites duplicate keys (keeping only the last value), this Edge Delta extension provides multiple merge strategies for handling duplicate keys: keeping first, keeping last, appending as arrays, concatenating as strings, or creating indexed keys. This is essential when parsing logs that legitimately contain repeated keys.

Consider a function such as ParseKeyValue that parses key value pairs into attributes. In circumstances where a log contains multiple key-value pairs with the same key, only the last instance is used. For example:

{
  "_type": "log",
  "timestamp": 1735784191233,
  "body": "service=auth user=65532 action=login status=success location=us-east-1 service=billing action=invoice status=failure location=eu-west-2",
  "resource": {...},
  "attributes": {
    "decoded_body": "service=auth user=65532 action=login status=success location=us-east-1 service=billing action=invoice status=failure location=eu-west-2"
  }
}

The function: set(attributes["kv_map"], ParseKeyValue(attributes["decoded_body"])), would parse the string as follows:

{
  "_type": "log",
  "timestamp": 1735784250481,
  "body": "service=auth user=65532 action=login status=success location=us-east-1 service=billing action=invoice status=failure location=eu-west-2",
  "resource": {...},
  "attributes": {
    "decoded_body": "service=auth user=65532 action=login status=success location=us-east-1 service=billing action=invoice status=failure location=eu-west-2",
    "kv_map": {
      "action": "invoice",
      "location": "eu-west-2",
      "service": "billing",
      "status": "failure",
      "user": "65532"
    }
  }
}

The second values for duplicate fields, such as status, have over-written the first values.

Edge Delta’s EDXParseKeyValue editor function exposes a number of options for dealing with this situation:

Syntax: EDXParseKeyValue(<target>, "<delimiter>", "<pair delimiter>", <string conversion>, "<merge strategy>")

target: the location of the field to be parsed into attributes
delimiter: The delimiter between key and value.
pair delimiter: The delimiter between key-value pairs.
string conversion: A Boolean value indicating whether to convert strings into float or int if a number is detected.
merge strategy: Select a merge strategy: first|last|append|concat|indexed

Note: identical key-value pairs are treated as a single key-value.

Keep First

set(attributes["kv_map"], EDXParseKeyValue(attributes["decoded_body"], "=", " ", false, "first"))

In this example the first value is kept and the rest are dropped.

{
  "_type": "log",
  "timestamp": 1735784401372,
  "body": "service=auth user=65532 action=login status=success location=us-east-1 service=billing action=invoice status=failure location=eu-west-2",
  "resource": {...},
  "attributes": {
    "decoded_body": "service=auth user=65532 action=login status=success location=us-east-1 service=billing action=invoice status=failure location=eu-west-2",
    "kv_map": {
      "action": "login",
      "location": "us-east-1",
      "service": "auth",
      "status": "success",
      "user": "65532"
    }
  }
}

Keep Last

set(attributes["kv_map"], EDXParseKeyValue(attributes["decoded_body"], "=", " ", false, "last"))

In this example the last value is kept and the rest are dropped.

{
  "_type": "log",
  "timestamp": 1735784427676,
  "body": "service=auth user=65532 action=login status=success location=us-east-1 service=billing action=invoice status=failure location=eu-west-2",
  "resource": {...},
  "attributes": {
    "decoded_body": "service=auth user=65532 action=login status=success location=us-east-1 service=billing action=invoice status=failure location=eu-west-2",
    "kv_map": {
      "action": "invoice",
      "location": "eu-west-2",
      "service": "billing",
      "status": "failure",
      "user": "65532"
    }
  }
}

Append Values

set(attributes["kv_map"], EDXParseKeyValue(attributes["decoded_body"], "=", " ", false, "append"))

In this example the values are all included as an array.

{
  "_type": "log",
  "timestamp": 1735784499306,
  "body": "service=auth user=65532 action=login status=success location=us-east-1 service=billing action=invoice status=failure location=eu-west-2",
  "resource": {...},
  "attributes": {
    "decoded_body": "service=auth user=65532 action=login status=success location=us-east-1 service=billing action=invoice status=failure location=eu-west-2",
    "kv_map": {
      "action": [
        "login",
        "invoice"
      ],
      "location": [
        "us-east-1",
        "eu-west-2"
      ],
      "service": [
        "auth",
        "billing"
      ],
      "status": [
        "success",
        "failure"
      ],
      "user": "65532"
    }
  }
}

Concatenate Values

set(attributes["kv_map"], EDXParseKeyValue(attributes["decoded_body"], "=", " ", false, "concat"))

In this example the values are all included as a single comma separated value.

{
  "_type": "log",
  "timestamp": 1735784548994,
  "body": "service=auth user=65532 action=login status=success location=us-east-1 service=billing action=invoice status=failure location=eu-west-2",
  "resource": {...},
  "attributes": {
    "decoded_body": "service=auth user=65532 action=login status=success location=us-east-1 service=billing action=invoice status=failure location=eu-west-2",
    "kv_map": {
      "action": "login, invoice",
      "location": "us-east-1, eu-west-2",
      "service": "auth, billing",
      "status": "success, failure",
      "user": "65532"
    }
  }
}

Index Values

set(attributes["kv_map"], EDXParseKeyValue(attributes["decoded_body"], "=", " ", false, "indexed"))

In this example the values are all included by creating multiple indexed keys.

{
  "_type": "log",
  "timestamp": 1735784569792,
  "body": "service=auth user=65532 action=login status=success location=us-east-1 service=billing action=invoice status=failure location=eu-west-2",
  "resource": {...},
  "attributes": {
    "decoded_body": "service=auth user=65532 action=login status=success location=us-east-1 service=billing action=invoice status=failure location=eu-west-2",
    "kv_map": {
      "action_0": "login",
      "action_1": "invoice",
      "location_0": "us-east-1",
      "location_1": "eu-west-2",
      "service_0": "auth",
      "service_1": "billing",
      "status_0": "success",
      "status_1": "failure",
      "user": "65532"
    }
  }
}

Convert strings to int or float

set(attributes["kv_map"], EDXParseKeyValue(attributes["decoded_body"], "=", " ", true, "indexed"))

In this example one string, user, is detected as being an integer and converted into the appropriate data type.

{
  "_type": "log",
  "timestamp": 1735784594207,
  "body": "service=auth user=65532 action=login status=success location=us-east-1 service=billing action=invoice status=failure location=eu-west-2",
  "resource": {...},
  "attributes": {
    "decoded_body": "service=auth user=65532 action=login status=success location=us-east-1 service=billing action=invoice status=failure location=eu-west-2",
    "kv_map": {
      "action_0": "login",
      "action_1": "invoice",
      "location_0": "us-east-1",
      "location_1": "eu-west-2",
      "service_0": "auth",
      "service_1": "billing",
      "status_0": "success",
      "status_1": "failure",
      "user": 65532
    }
  }
}

EDXRedis

Minimum Agent Version: v2.5.0

EDXRedis fills a critical gap in standard OTTL which lacks native Redis integration capabilities. This Edge Delta extension provides direct Redis connectivity within OTTL pipelines, enabling real-time data enrichment, caching, deduplication, and storage operations. It supports all Redis commands across Standalone, Cluster, and Sentinel deployments with automatic connection pooling, TLS/mTLS support, and batch operations for optimal performance.

Syntax

EDXRedis(connection_config, commands)

connection_config: Map containing Redis connection details (URL, authentication, TLS settings).
commands: Array of command objects with command, key, args, and required outField parameters.

Input

{
  "_type": "log",
  "timestamp": 1758168205000,
  "body": "User login attempt",
  "resource": {...},
  "attributes": {
    "event_id": "evt_001",
    "user_id": "user_123",
    "action": "login"
  }
}

Example

// Enrich with user data from Redis
set(cache["cmd"], [{"command": "get", "key": "user:123", "outField": "user_data"}])
set(cache["result"], EDXRedis({"url": "redis://localhost:6379"}, cache["cmd"]))
set(attributes["user_name"], cache["result"]["user_data"]) where cache["result"]["user_data"] != nil

// Increment login counter
set(cache["incr_cmd"], [{"command": "incr", "key": "logins:today", "outField": "count"}])
set(cache["incr_result"], EDXRedis({"url": "redis://localhost:6379"}, cache["incr_cmd"]))
set(attributes["login_count"], cache["incr_result"]["count"])

Output

{
  "_type": "log",
  "timestamp": 1758168235000,
  "body": "User login attempt",
  "resource": {...},
  "attributes": {
    "event_id": "evt_001",
    "user_id": "user_123",
    "action": "login",
    "user_name": "John Doe",
    "login_count": 42
  }
}

The EDXRedis function has enriched the log with user data retrieved from Redis and incremented a login counter. The function supports all Redis commands including strings (GET/SET), hashes (HSET/HGET), lists (LPUSH/LPOP), sets (SADD/SREM), sorted sets (ZADD/ZRANGE), and key management (EXPIRE/TTL).

Due to the extensive capabilities of this function including batch operations, authentication methods, TLS/mTLS configuration, and deployment options, please refer to:

EDXRedis Complete Documentation - Full reference with all examples and use cases
EDXRedis Troubleshooting Guide - Common issues and solutions

EDXUnescapeJSON

Minimum Agent Version: v1.31.0

EDXUnescapeJSON fills a gap in standard OTTL’s JSON handling capabilities. While OTTL provides ParseJSON for parsing JSON strings, it cannot handle multiply-escaped JSON strings that are common in nested logging scenarios. This Edge Delta extension recursively unescapes JSON strings, making it possible to properly parse deeply nested or repeatedly escaped JSON data.

Syntax

EDXUnescapeJSON(input)

input: The escaped JSON string to unescape.

Input

{
  "_type": "log",
  "timestamp": 1735789800000,
  "body": "Processing escaped JSON data",
  "resource": {...},
  "attributes": {
    "escaped_json": "{\\\"level\\\":\\\"info\\\",\\\"message\\\":\\\"User logged in\\\",\\\"metadata\\\":\\\"{\\\\\\\"user_id\\\\\\\":\\\\\\\"12345\\\\\\\",\\\\\\\"session\\\\\\\":\\\\\\\"abc-xyz\\\\\\\"}\\\"}"
  },
  "cache": {
    "body": "{\\\"level\\\":\\\"info\\\",\\\"message\\\":\\\"User logged in\\\",\\\"metadata\\\":\\\"{\\\\\\\"user_id\\\\\\\":\\\\\\\"12345\\\\\\\",\\\\\\\"session\\\\\\\":\\\\\\\"abc-xyz\\\\\\\"}\\\"}"
  }
}

Example

set(attributes["unescaped_json"], EDXUnescapeJSON(cache["body"]))
set(attributes["parsed_data"], ParseJSON(attributes["unescaped_json"]))

Output

{
  "_type": "log",
  "timestamp": 1735789830000,
  "body": "Processing escaped JSON data",
  "resource": {...},
  "attributes": {
    "escaped_json": "{\\\"level\\\":\\\"info\\\",\\\"message\\\":\\\"User logged in\\\",\\\"metadata\\\":\\\"{\\\\\\\"user_id\\\\\\\":\\\\\\\"12345\\\\\\\",\\\\\\\"session\\\\\\\":\\\\\\\"abc-xyz\\\\\\\"}\\\"}",
    "unescaped_json": "{\"level\":\"info\",\"message\":\"User logged in\",\"metadata\":\"{\\\"user_id\\\":\\\"12345\\\",\\\"session\\\":\\\"abc-xyz\\\"}\"}",
    "parsed_data": {
      "level": "info",
      "message": "User logged in",
      "metadata": "{\"user_id\":\"12345\",\"session\":\"abc-xyz\"}"
    }
  },
  "cache": {
    "body": "{\\\"level\\\":\\\"info\\\",\\\"message\\\":\\\"User logged in\\\",\\\"metadata\\\":\\\"{\\\\\\\"user_id\\\\\\\":\\\\\\\"12345\\\\\\\",\\\\\\\"session\\\\\\\":\\\\\\\"abc-xyz\\\\\\\"}\\\"}"
  }
}

The EDXUnescapeJSON function has recursively unescaped the JSON string, making it possible to properly parse the nested JSON structure.

Editor Functions

Editor functions modify existing data structures in place, such as deleting or keeping specific keys in maps.

edx_delete_keys

Minimum Agent Version: v1.23.0

edx_delete_keys enhances the standard OTTL delete_key function. While the default OTTL function can only delete one key at a time, this Edge Delta extension enables batch deletion of multiple specific keys in a single operation. This is particularly useful when you need to remove several unrelated keys that don’t follow a common pattern, eliminating the need for multiple individual delete operations.

Syntax

edx_delete_keys(input, ["key1", "key2", "key3"])

input: A map (e.g., resource or attributes) from which keys are to be deleted.
keys: An array of key names to be deleted.

Input

{
  "_type": "log",
  "timestamp": 1735787654284,
  "body": "Firewall_action=block rule_id=R102 rule_name=Block_All_Outgoing user_id=admin user_group=network_admins protocol=TCP severity=high src_ip=10.0.0.1 dst_ip=192.168.1.100",
  "resource": {...},
  "attributes": {
    "decoded_body": "Firewall_action=block rule_id=R102 rule_name=Block_All_Outgoing user_id=admin user_group=network_admins protocol=TCP severity=high src_ip=10.0.0.1 dst_ip=192.168.1.100",
    "kv_map": {
      "Firewall_action": "block",
      "dst_ip": "192.168.1.100",
      "protocol": "TCP",
      "rule_id": "R102",
      "rule_name": "Block_All_Outgoing",
      "severity": "high",
      "src_ip": "10.0.0.1",
      "user_group": "network_admins",
      "user_id": "admin"
    }
  }
}

Example

edx_delete_keys(attributes["kv_map"], ["rule_id", "rule_name", "user_id", "user_group"])

Output

{
  "_type": "log",
  "timestamp": 1735787684654,
  "body": "Firewall_action=block rule_id=R102 rule_name=Block_All_Outgoing user_id=admin user_group=network_admins protocol=TCP severity=high src_ip=10.0.0.1 dst_ip=192.168.1.100",
  "resource": {...},
  "attributes": {
    "decoded_body": "Firewall_action=block rule_id=R102 rule_name=Block_All_Outgoing user_id=admin user_group=network_admins protocol=TCP severity=high src_ip=10.0.0.1 dst_ip=192.168.1.100",
    "kv_map": {
      "Firewall_action": "block",
      "dst_ip": "192.168.1.100",
      "protocol": "TCP",
      "severity": "high",
      "src_ip": "10.0.0.1"
    }
  }
}

The keys rule_id, rule_name, user_id, and user_group have been removed as a batch in a single operation.

edx_delete_matching_keys

Minimum Agent Version: v1.23.0

edx_delete_matching_keys enhances the standard OTTL delete_matching_keys function. While the default OTTL function accepts only a single regex pattern, this Edge Delta extension enables you to provide multiple regex patterns in a single operation. This allows concurrent deletion of keys matching different patterns without requiring multiple separate statements.

Syntax

edx_delete_matching_keys(input, ["key1", "key2"])

input: A map (e.g., resource or attributes) from which keys matching specified patterns are to be deleted.
keys: An array of regex-like patterns for matching and deleting keys.

Input

{
  "_type": "log",
  "timestamp": 1735788401264,
  "body": "session_id=abc123 user_id=admin event_type=login event_status=success debug_info=verbose connection_id=conn456 temp_file=report_tmp.log temp_data=sensitiveData",
  "resource": {...},
  "attributes": {
    "decoded_body": "session_id=abc123 user_id=admin event_type=login event_status=success debug_info=verbose connection_id=conn456 temp_file=report_tmp.log temp_data=sensitiveData",
    "kv_map": {
      "connection_id": "conn456",
      "debug_info": "verbose",
      "event_status": "success",
      "event_type": "login",
      "session_id": "abc123",
      "temp_data": "sensitiveData",
      "temp_file": "report_tmp.log",
      "user_id": "admin"
    }
  }
}

Example

edx_delete_matching_keys(attributes["kv_map"], ["(.*_id$)", "(temp_.*|debug_.*)"])

Output

{
  "_type": "log",
  "timestamp": 1735788441183,
  "body": "session_id=abc123 user_id=admin event_type=login event_status=success debug_info=verbose connection_id=conn456 temp_file=report_tmp.log temp_data=sensitiveData",
  "resource": {...},
  "attributes": {
    "decoded_body": "session_id=abc123 user_id=admin event_type=login event_status=success debug_info=verbose connection_id=conn456 temp_file=report_tmp.log temp_data=sensitiveData",
    "kv_map": {
      "event_status": "success",
      "event_type": "login"
    }
  }
}

Keys matching the two patterns (.*_id$) (connection_id, session_id, user_id) and (temp_.*|debug_.*) (debug_info, temp_data, temp_file) were removed.

edx_delete_empty_values

Minimum Agent Version: v1.28.0

edx_delete_empty_values fills a gap in standard OTTL which lacks a dedicated function for cleaning up empty or meaningless values. While OTTL can delete specific keys with delete_key, it doesn’t provide a way to automatically remove all keys with empty, null, or custom-defined “empty” values (like “unknown” or “N/A”). This Edge Delta extension enables comprehensive data cleanup by removing fields that contain no meaningful information based on customizable criteria.

Syntax

edx_delete_empty_values(input, excluded_keys, empty_patterns, strategies)

input: A map (e.g., attributes, resource) from which empty values should be deleted.
excluded_keys: An array of key names to exclude from deletion (optional, use [] to exclude none).
empty_patterns: An array of string patterns to consider as empty values (e.g., ["", "unknown", "N/A"]).
strategies (optional): An array of strategies for additional deletion behavior. Supported values:
- "deleteNull": Delete keys with null/empty values
- "deleteEmptyList": Delete keys with empty arrays/slices
- "deleteEmptyMap": Delete keys with empty maps/objects
- "deleteZero": Delete keys with zero integer or float values

Input

{
  "_type": "log",
  "timestamp": 1735788500000,
  "body": "user_id=12345 name=John email= status=unknown location= department=null notes=",
  "resource": {...},
  "attributes": {
    "decoded_body": "user_id=12345 name=John email= status=unknown location= department=null notes=",
    "kv_map": {
      "user_id": "12345",
      "name": "John",
      "email": "",
      "status": "unknown",
      "location": "",
      "department": null,
      "notes": ""
    }
  }
}

Example

edx_delete_empty_values(attributes["kv_map"], [], ["", "unknown"], ["deleteNull"])

Output

{
  "_type": "log",
  "timestamp": 1735788530000,
  "body": "user_id=12345 name=John email= status=unknown location= department=null notes=",
  "resource": {...},
  "attributes": {
    "decoded_body": "user_id=12345 name=John email= status=unknown location= department=null notes=",
    "kv_map": {
      "user_id": "12345",
      "name": "John"
    }
  }
}

The keys email, status, location, department, and notes have been removed because their values were either empty strings, matched the pattern “unknown”, or were null (with the deleteNull option enabled).

edx_keep_keys

Minimum Agent Version: v1.23.0

edx_keep_keys enhances the standard OTTL keep_keys function. While the default OTTL function requires individual key specifications, this Edge Delta extension accepts an array of keys to retain in a single batch operation. This is more efficient when you need to keep multiple specific keys and remove everything else, essentially providing an allowlist approach to field management.

Syntax

edx_keep_keys(input, ["key1", "key2"])

input: A map (e.g., resource or attributes) from which keys are selectively retained.
keys: An array of key names to be retained.

Input:

{
  "_type": "log",
  "timestamp": 1735796321870,
  "body": "session_id=abc123 user_id=admin event_type=login event_status=success debug_info=verbose connection_id=conn456 temp_file=report_tmp.log temp_data=sensitiveData",
  "resource": {...},
  "attributes": {
    "decoded_body": "session_id=abc123 user_id=admin event_type=login event_status=success debug_info=verbose connection_id=conn456 temp_file=report_tmp.log temp_data=sensitiveData",
    "kv_map": {
      "connection_id": "conn456",
      "debug_info": "verbose",
      "event_status": "success",
      "event_type": "login",
      "session_id": "abc123",
      "temp_data": "sensitiveData",
      "temp_file": "report_tmp.log",
      "user_id": "admin"
    }
  }
}

Example:

edx_keep_keys(attributes["kv_map"], ["event_status", "event_type"])

Output:

{
  "_type": "log",
  "timestamp": 1735796349262,
  "body": "session_id=abc123 user_id=admin event_type=login event_status=success debug_info=verbose connection_id=conn456 temp_file=report_tmp.log temp_data=sensitiveData",
  "resource": {...},
  "attributes": {
    "decoded_body": "session_id=abc123 user_id=admin event_type=login event_status=success debug_info=verbose connection_id=conn456 temp_file=report_tmp.log temp_data=sensitiveData",
    "kv_map": {
      "event_status": "success",
      "event_type": "login"
    }
  }
}

Only the keys event_status and event_type have been retained in the kv_map map.

edx_keep_matching_keys

Minimum Agent Version: v1.23.0

edx_keep_matching_keys enhances the standard OTTL keep_matching_keys function. While the default OTTL function accepts only a single regex pattern, this Edge Delta extension enables you to provide multiple regex patterns simultaneously. This allows you to retain keys matching any of several patterns in one operation, making complex filtering scenarios more efficient.

Syntax

edx_keep_matching_keys(input, ["key1", "key2"])

input: A map (e.g., resource or attributes) from which keys are selectively retained.
keys: An array of regex-like patterns for matching and retaining keys.

Input:

{
  "_type": "log",
  "timestamp": 1735796604731,
  "body": "session_id=abc123 user_id=admin event_type=login event_status=success debug_info=verbose connection_id=conn456 temp_file=report_tmp.log temp_data=sensitiveData",
  "resource": {...},
  "attributes": {
    "decoded_body": "session_id=abc123 user_id=admin event_type=login event_status=success debug_info=verbose connection_id=conn456 temp_file=report_tmp.log temp_data=sensitiveData",
    "kv_map": {
      "connection_id": "conn456",
      "debug_info": "verbose",
      "event_status": "success",
      "event_type": "login",
      "session_id": "abc123",
      "temp_data": "sensitiveData",
      "temp_file": "report_tmp.log",
      "user_id": "admin"
    }
  }
}

Example:

edx_keep_matching_keys(attributes["kv_map"], ["(.*_id$)", "(temp_.*|debug_.*)"])

Output:

{
  "_type": "log",
  "timestamp": 1735796570758,
  "body": "session_id=abc123 user_id=admin event_type=login event_status=success debug_info=verbose connection_id=conn456 temp_file=report_tmp.log temp_data=sensitiveData",
  "resource": {...},
  "attributes": {
    "decoded_body": "session_id=abc123 user_id=admin event_type=login event_status=success debug_info=verbose connection_id=conn456 temp_file=report_tmp.log temp_data=sensitiveData",
    "kv_map": {
      "connection_id": "conn456",
      "debug_info": "verbose",
      "session_id": "abc123",
      "temp_data": "sensitiveData",
      "temp_file": "report_tmp.log",
      "user_id": "admin"
    }
  }
}

The keys event_status and event_type have been removed as they do not match one of the patterns.

edx_map_keys

Minimum Agent Version: v1.26.0

edx_map_keys fills a gap in standard OTTL which lacks bulk field renaming capabilities. While OTTL can set and delete individual fields to achieve renaming, it doesn’t provide a way to transform multiple field names according to a schema mapping in a single operation. This Edge Delta extension enables schema normalization by renaming multiple keys at once, which is essential for adapting data to different standards or destination requirements.

Syntax

edx_map_keys(input, current_keys, new_keys, strategy)

input: A map (e.g., attributes, resource) whose keys should be mapped to new names.
current_keys: An array of current key names to be mapped from.
new_keys: An array of new key names to map to (must have the same length as current_keys).
strategy: The mapping strategy to use (e.g., “update” to rename keys).

Input

{
  "_type": "log",
  "timestamp": 1735789000000,
  "body": "src_ip=10.0.0.1 dst_ip=192.168.1.100 proto=TCP action=ALLOW bytes=1024",
  "resource": {...},
  "attributes": {
    "curr_schema": ["src_ip", "dst_ip", "proto", "action", "bytes"],
    "new_schema": ["source_address", "destination_address", "protocol", "firewall_action", "byte_count"],
    "decoded_body": "src_ip=10.0.0.1 dst_ip=192.168.1.100 proto=TCP action=ALLOW bytes=1024",
    "kv_map": {
      "src_ip": "10.0.0.1",
      "dst_ip": "192.168.1.100",
      "proto": "TCP",
      "action": "ALLOW",
      "bytes": "1024"
    }
  }
}

Example

edx_map_keys(attributes["kv_map"], attributes["curr_schema"], attributes["new_schema"], "update")

Output

{
  "_type": "log",
  "timestamp": 1735789030000,
  "body": "src_ip=10.0.0.1 dst_ip=192.168.1.100 proto=TCP action=ALLOW bytes=1024",
  "resource": {...},
  "attributes": {
    "curr_schema": ["src_ip", "dst_ip", "proto", "action", "bytes"],
    "new_schema": ["source_address", "destination_address", "protocol", "firewall_action", "byte_count"],
    "decoded_body": "src_ip=10.0.0.1 dst_ip=192.168.1.100 proto=TCP action=ALLOW bytes=1024",
    "kv_map": {
      "source_address": "10.0.0.1",
      "destination_address": "192.168.1.100",
      "protocol": "TCP",
      "firewall_action": "ALLOW",
      "byte_count": "1024"
    }
  }
}

The keys in kv_map have been renamed according to the mapping defined in curr_schema and new_schema. Each key from the current schema has been replaced with its corresponding key from the new schema, enabling seamless schema transformation for data normalization or integration with different systems.

Edge Delta's OTTL Extensions

Edge Delta Custom OTTL

Converter Functions

EDXCoalesce

EDXCompress

EDXDataType

EDXDecompress

EDXDecode

EDXDecrypt

EDXEncode

EDXEncrypt

EDXEnv

EDXExtractPatterns

EDXHmac

EDXIfElse

EDXLookup

EDXParseKeyValue

EDXRedis

EDXUnescapeJSON

Editor Functions

edx_delete_keys

edx_delete_matching_keys

edx_delete_empty_values

edx_keep_keys

edx_keep_matching_keys

edx_map_keys

Edge Delta AI Assistant

Quick Topics

Recent Questions

Hi! I'm your Edge Delta AI Assistant

Current Context