OTTL Language Guide

OTTL Statements

Open Telemetry Transformation Language (OTTL) is a scripting language used within the OpenTelemetry ecosystem, primarily for transforming and manipulating telemetry data. It enables users to customize how telemetry data is handled, providing a way to filter, transform, and export the data. In Edge Delta, you can use an OTTL Transformation node to transform data items with OTTL, or you can use a Route node configured with OTTL to route data items with Boolean OTTL functions.

See the OTTL documentation for more information.

An OTTL statement follows a scripting structure:

<function_call>(<target_specification>,<argument_parameters>)

A function call can be an editor or a convertor.

The target specification defines the parameter that will be modified using bracket notation.

The argument parameters are required by the function to perform its operation, such as values to append, patterns to match, or replacement texts.

Note: To perform string functions on the body it needs to be converted from a byte array. See working with the body.

Editor Functions

Editors transform telemetry data. They are meant to modify the underlying telemetry data by applying various functions. Editors like append, delete_key and keep_keys can manipulate data maps and slices within the telemetry data. For example, append can add values to a target, converting it into an array if necessary.

append(attributes["tags"], "prod")

In this example, append is the function being called, attributes["tags"] specifies the target data structure, and "prod" is the argument to be appended to the tags attribute.

See editor functions.

Edge Delta has created bespoke OTTL extension functions.

Converter Functions

Converters provide utility functions for transforming telemetry data without altering the original data, for example, Decode, Concat, and ConvertCase.

See converter functions for a list of converters.

Converters are used within arguments to transform input values before they are applied to the target action or field. In the following example they are used to change the case within a set editor. This example sets severity_text using an attribute value, but it uses a converter to first change the value to uppercase.

Input

{
  "_type": "log",
  "body": "time=hostname=SIMPLE|product=Firewall",
  "resource": {
    "host.name": "simple-host",
    "service.name": "basic-service"
  },
  "attributes": {
    "event": {
      "level": "error"
    }
  },
  "timestamp": 1730511053177
}

Statement

set(severity_text, ConvertCase(attributes["event"]["level"], "upper"))

Output

{
  "_type": "log",
  "attributes": {
    "event": {
      "level": "error"
    }
  },
  "body": "time=hostname=SIMPLE|product=Firewall",
  "resource": {
    "host.name": "simple-host",
    "service.name": "basic-service"
  },
  "severity_text": "ERROR",
  "timestamp": 1730511053177
}

The attributes["event"]["level"] value is changed to uppercase and used as the value for severity_text.

Comments

You can include comments in OTTL statements using // characters. This is useful to document the node’s function.

// convert the event level attribute to uppercase and save it as the root severity_text.
set(severity_text, ConvertCase(attributes["event"]["level"], "upper"))

Conditional Logic and Comparators

Overview

You can use conditional logic to execute actions only when specific conditions are met. Conditions are used in two main contexts:

1. Within statements using the where clause for conditional execution
2. In route processors to determine which path data should follow

Important: All conditions must be written on a single line in YAML. Multi-line conditions are not supported.

Comparison Operators

These operators compare values and return true or false:

Logical Operators

Critical: Use lowercase and, or, not - uppercase operators (AND, OR, NOT) will cause errors!

Using Conditions with Where Clauses

The where clause allows conditional execution within OTTL statements:

// Simple condition
set(attributes["alert"], "Critical") where attributes["status"] == "error"

// With type checking
set(attributes["duration"], Duration(attributes["duration_str"])) where IsString(attributes["duration_str"])

// Complex conditions with AND
set(attributes["tier"], "High Priority") where attributes["importance"] == "high" and attributes["status"] == "active"

// Multiple conditions with OR
set(attributes["severity"], "urgent") where attributes["level"] == "ERROR" or attributes["level"] == "FATAL"

// Using NOT with functions
set(attributes["needs_review"], true) where not IsString(attributes["user_id"]) or attributes["user_id"] == ""

Field Existence Checks

Check whether fields exist or have values:

// Check if field exists
set(attributes["has_user"], true) where attributes["user_id"] != nil

// Check if field doesn't exist
set(attributes["anonymous"], true) where attributes["user_id"] == nil

// Check for empty string
set(attributes["needs_message"], true) where attributes["message"] == ""

// Combined checks
set(attributes["valid_user"], true) where attributes["user_id"] != nil and attributes["user_id"] != ""

Common Patterns and Examples

Checking Multiple Values

// WRONG - This doesn't work
set(attributes["excluded"], true) where attributes["log_type"] != "TRAFFIC" OR "THREAT"

// CORRECT - Must repeat the full comparison
set(attributes["excluded"], true) where attributes["log_type"] != "TRAFFIC" and attributes["log_type"] != "THREAT"

// Alternative using regex for multiple values
set(attributes["is_security"], true) where regex_match(attributes["log_type"], "^(TRAFFIC|THREAT)$")

Complex Conditions with Parentheses

// Group conditions with parentheses for clarity
set(attributes["critical"], true) where (attributes["level"] == "ERROR" or attributes["level"] == "FATAL") and attributes["env"] == "production"

// Nested conditions
set(attributes["needs_escalation"], true) where attributes["priority"] == "high" and (attributes["retries"] > 5 or attributes["duration"] > 10000)

Common Mistakes to Avoid

// WRONG - Uppercase operators
set(attributes["flag"], true) where attributes["status"] == "error" AND attributes["level"] == "critical"

// CORRECT - Lowercase operators
set(attributes["flag"], true) where attributes["status"] == "error" and attributes["level"] == "critical"

// WRONG - OTTL requires 'and' instead of '&&'
set(attributes["retry_alert"], "Too many retries") where IsInt(attributes["retry_count"]) && attributes["retry_count"] > 5

// CORRECT - Consistent OTTL syntax
set(attributes["retry_alert"], "Too many retries") where IsInt(attributes["retry_count"]) and attributes["retry_count"] > 5

Working with the body

When a log is ingested by the Edge Delta agent, a log data item is created and the log contents are placed in the body field. Importantly, this field is a byte array. Therefore, to perform any OTTL transformation that requires the body as a string input, it needs to be decoded from byte array to a string type as part of the transformation function.

Note: You can’t convert the body into a string type as it will appear as null in the Edge Delta interface.

To convert a field into byte array, for example to update the body, you can use the EDXEncode custom function.

Consider this log:

[
	{
		"_type": "log",
		"body": "Starting HTTP server on 10.120.15.221:8080, serving content from /usr/share/app",
		"resource": {...},
		"timestamp": 1733285404282
	}
]

To extract any IP address from the body and save it as an attribute, a set editor function is used with an embedded ExtractPatterns converter function. Embedded within the Converter function is the Decode function to convert the body type.

set(attributes, ExtractPatterns(Decode(body, "utf-8"), "(?P<ip_address>\\b\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\b)"))

Output:

[
	{
		"_type": "log",
		"attributes": {
			"ip_address": "10.120.15.221"
		},
		"body": "Starting HTTP server on 10.120.15.221:8080, serving content from /usr/share/app",
		"resource": {...},
		"timestamp": 1733285404282
	}
]

The body was converted to a string and the regex pattern identified the IP address, which was saved as an attribute using the capture name as the field.

Working with a Cache

A cache field enables you to store data temporarily while performing subsequent transformations. This field is not visible in node testing as it is meant to be temporary. In particular it is useful to create a cache version of the body that is a string type rather than a byte array. Subsequent operations within the same transformation node can operate on the cache body rather than using an embedded decode function. This may be required if the operation syntax contains a target that should point to the body, but the target field expects a key rather than a string. It is also useful for storing data types temporarily that are not allowed in the OTEL schema.

For example set(attribute, replace_pattern(target, regex, replacement)) can’t mask text from the body while setting it in an attribute because target must be a path expression. So you need an initial step of saving the decoded body in a cache, and running the replace and set function against the cached value.

Consider this log:

[
	{
		"_type": "log",
		"body": "Starting HTTP server on 10.120.15.221:8080, serving content from /usr/share/app",
		"resource": {...},
		"timestamp": 1733286563321
	}
]

set(cache["body"], Decode(body, "utf-8"))
set(attributes, ExtractPatterns(cache["body"], "(?P<ip_address>\\b\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\b)"))

[
	{
		"_type": "log",
		"attributes": {
			"ip_address": "10.120.15.221"
		},
		"body": "Starting HTTP server on 10.120.15.221:8080, serving content from /usr/share/app",
		"resource": {...},
		"timestamp": 1733286563321
	}
]

The first statement sets the cache with a decoded body. The second statement extracts the IP address from the cached version of the body without needing to decode it. Similarly, any subsequent operations that need to use the body can use the cached decoded version.

Note: The cache does not appear as part of the log metadata and it is purged when processing is complete on the data item.

See Also