keep_keys

Learn about the keep_keys OTTL editor function.

  2 minute read  

This function is used to retain specified keys within a target field or object in a log entry. It provides a precise method for filtering log data by keeping only the entries that match the specified keys. Unlike keep_matching_keys, this function requires an explicit list of keys to retain. It is best when you know the exact keys you want to keep ahead of time. The keep_matching_keys function uses a pattern or regular expression to determine which keys to retain, making it more flexible and powerful for dynamic or large datasets where the exact keys might not be known.

Syntax: keep_keys(target, keys)

  • Target: The target is the field or object within the log entry you wish to filter. It typically points to a JSON object or associative array with multiple key-value pairs.
  • Keys: The keys parameter is an array of specific keys you want to retain in the target. Only these keys will be kept, and all others will be removed.

Input

{
	"_type": "log",
	"attributes": {
		"decoded_body": "...",
		"parsed_body": {
			"awsRegion": "us-west-2",
			"eventCategory": "Management",
			"eventID": "mnop5678-abcd-1234-efgh-5678ijklqrst",
			"eventName": "ListStacks",
			"eventSource": "ec2.amazonaws.com",
			"eventTime": "2024-12-05T05:10:57.227003Z",
			"eventType": "AwsApiCall",
			"eventVersion": "1.08",
			"managementEvent": "true",
			"readOnly": "true",
			"recipientAccountId": 123456789012,
			"requestID": "abcd1234-efgh-5678-ijkl-9012mnopqrst",
			"resources.0": {
				"ARN": "arn:aws:iam::123456789012:role/aws-controltower-ForwardSnsNotificationRole",
				"accountId": 123456789012,
				"type": "AWS::IAM::Role"
			},
			"responseElements.assumedRoleUser.arn": "arn:aws:iam::123456789012:role/ABCDEFGHIJKLM123456789/AWSConfig-BucketConfigCheck",
			"responseElements.assumedRoleUser.assumedRoleId": "A1B2C3D4E5F6G7H8I9J0:AWSConfig-BucketConfigCheck",
			"responseElements.credentials.accessKeyId": "A1B2C3D4E5F6G7H8I9J0",
			"responseElements.credentials.expiration": "2024-12-05T05:10:57.227053Z",
			"responseElements.credentials.sessionToken": "123456789876",
			"sharedEventID": "01234567-89ab-cdef-edcb-a9876543210f",
			"sourceIPAddress": "211.46.216.146",
			"userAgent": "ec2.amazonaws.com",
			"userIdentity.invokedBy": "lambda.amazonaws.com",
			"userIdentity.type": "AssumedRole"
		}
	},
	"body": "...",
	"resource": {...},
	"timestamp": 1733377758772
}

Statement

keep_keys(attributes["parsed_body"], keys=["eventCategory", "eventName"])

Output

{
	"_type": "log",
	"attributes": {
		"decoded_body": "...",
		"parsed_body": {
			"eventCategory": "Management",
			"eventName": "ListStacks"
		}
	},
	"body": "...",
	"resource": {...},
	"timestamp": 1733377774908
}

The log entry is updated to retain only the specified eventCategory and eventName within the attributes["parsed_body"] object.