EDXEncode

Learn about the EDXEncode Edge Delta OTTL extension function.

  2 minute read  

Minimum Agent Version: v1.25.0

EDXEncode fills a critical gap in standard OTTL by providing byte array encoding capabilities. While OTTL has a Decode function for converting byte arrays to strings, it lacks a corresponding encode function. This Edge Delta extension enables you to convert strings back to byte arrays, which is essential when working with the body field that must remain as a byte array.

When a log is ingested by the Edge Delta agent, a log data item is created and the log contents are placed in the body field. Importantly, this field is a byte array. Therefore, to perform any OTTL transformation that requires the body as a string input, it needs to be decoded from byte array to a string type as part of the transformation function. See Working with the body.

After copying, decoding and transforming the body into a field other than body, you can use the EDXEncode custom function to save a new body field in the appropriate byte array type.

Syntax

EDXEncode(target, encoding, asBytes...)
  • target: The string value that should be encoded (can be a field reference or literal string).
  • encoding: The encoding format from the IANA encoding index, such as "utf-8" or "base64".
  • asBytes (optional): Boolean flag to control output format. When true (default), returns a byte array. When false, returns a string. This parameter is variadic and can be omitted.

Suppose this log is ingested:

session_id=abc123 user_id=admin event_type=login event_status=success debug_info=verbose connection_id=conn456 temp_file=report_tmp.log temp_data=sensitiveData

The following operations are performed to extract, decode, parse, and transform the data:

set(attributes["decoded_body"], Decode(body, "utf-8"))
set(attributes["kv_map"], ParseKeyValue(attributes["decoded_body"]))
delete_matching_keys(attributes["kv_map"], "(temp_.*|debug_.*)")
set(attributes["new_body"], ToKeyValueString(attributes["kv_map"]))

See Decode, ParseKeyValue, delete_matching_keys, and ToKeyValueString.

These operations result in the following log:

Input:

{
  "_type": "log",
  "timestamp": 1735802408445,
  "body": "session_id=abc123 user_id=admin event_type=login event_status=success debug_info=verbose connection_id=conn456 temp_file=report_tmp.log temp_data=sensitiveData",
  "resource": {...},
  "attributes": {
    "decoded_body": "session_id=abc123 user_id=admin event_type=login event_status=success debug_info=verbose connection_id=conn456 temp_file=report_tmp.log temp_data=sensitiveData",
    "kv_map": {
      "connection_id": "conn456",
      "event_status": "success",
      "event_type": "login",
      "session_id": "abc123",
      "user_id": "admin"
    },
    "new_body": "event_status=success connection_id=conn456 session_id=abc123 user_id=admin event_type=login"
  }
}

Example:

set(body, EDXEncode(attributes["new_body"], "utf-8", true))

Output:

{
  "_type": "log",
  "timestamp": 1735802426292,
  "body": "connection_id=conn456 session_id=abc123 user_id=admin event_type=login event_status=success",
  "resource": {...},
  "attributes": {
    "decoded_body": "session_id=abc123 user_id=admin event_type=login event_status=success debug_info=verbose connection_id=conn456 temp_file=report_tmp.log temp_data=sensitiveData",
    "kv_map": {
      "connection_id": "conn456",
      "event_status": "success",
      "event_type": "login",
      "session_id": "abc123",
      "user_id": "admin"
    },
    "new_body": "connection_id=conn456 session_id=abc123 user_id=admin event_type=login event_status=success"
  }
}

The new_body attribute has been upserted into the body field and encoded as a byte array.