Deploying the Edge Delta Lambda Extension

Deploy the Edge Delta Lambda Extension to monitor AWS Lambda functions and send logs to an Edge Delta Cloud pipeline.

  4 minute read  

Overview

Edge Delta’s AWS Lambda Extension is deployed as a layer within your Lambda Function. See Serverless AWS Monitoring. It is published in the AWS Serverless Application Repository and there are layers for ARM64 or AMD64. The Lambda extension is run on AWS as a part of the lambda execution and lambda function logs are sent to the extension. The extension in turn sends logs to an Edge Delta Cloud pipeline.

To configure Lambda monitoring using the Lambda Extension:

  1. Create a Pipeline Configuration
  2. Create an Edge Delta Cloud pipeline.
  3. Deploy the Lambda extension.
  4. Specify the Cloud pipelines endpoint.
  5. Configure other environment variables as needed.

Create an Edge Delta Cloud Pipeline

Create a new Edge Delta Cloud pipeline.

  1. Click Pipelines.
  2. Click New Pipeline.
  3. Select Cloud
  4. Specify a name to identify the pipeline.
  5. Select Compute Units based on your estimated traffic volume. This is the maximum bandwidth the agent can handle before signalling an error. The number of compute units used per hour counts towards your plan usage.
  6. Click Deploy Cloud Pipeline.

Copy the HTTPS endpoint from the Cloud pipelines settings.

Configure the pipeline

Apply the Lambda Pack to your Cloud pipeline. The Pack handles JSON parsing, message extraction, and attribute cleanup automatically.

Add the following pipeline settings to handle large Lambda log payloads. These values should match the maximum Lambda/CloudWatch log size to prevent out-of-memory issues:

settings:
  multiline_max_byte_size: 11MB
  max_incomplete_line_buffer_size: 10MB

Deploying the Lambda Extension

Get the ARN from the AWS Serverless Application Repository

  1. Open AWS Serverless Application Repository and click Available Applications.
  2. Search for EdgeDelta and select the extension (either ARM64 or AMD64).
  3. Confirm the layer template such as the application name and click Deploy.

The extension can be deployed multiple times if necessary by providing different names. Deploying with an existing application name upgrades the existing deployment if an upgrade is available.

  1. On AWS Lambda, expand Additional Resources and click Layers.
  2. Click the Layer you just created (edgedelta-lambda-extension-arm64 or edgedelta-lambda-extension-amd64)
  3. Copy the ARN of the Lambda Layer.

Add the Layer to your Lambda Function

Add the layer to your Lambda Function using the CLI or using the Console.

CLI: Replace <Function Name> and <ARN>

aws lambda update-function-configuration --function-name <Function Name> \
 --layers “<ARN>"

Console

  1. Navigate to the Lambda Function you want to monitor.
  2. In the Layers section, click Add a Layer.
  3. Click Specify an ARN, paste the ARN you copied earlier and click Verify.
  4. Click Add.

Specify the Endpoint Environment Variable

To configure Environment Variables for the Lambda function:

  1. Open the Functions page of the Lambda console.
  2. Choose a function.
  3. Choose Configuration, then choose Environment variables.
  4. Under Environment variables, choose Edit.
  5. Choose Add environment variable.
  6. Enter a key and value.

Push Destination Variables

Specify the Cloud pipeline endpoint URL you copied earlier.

  • PUSHER_MODE: (Optional) Use kinesis only if sending to AWS Kinesis Firehose. For Edge Delta hosted environments, leave this unset (defaults to http).
  • ED_ENDPOINT: Specify the Edge Delta Cloud pipeline endpoint (e.g., https://xxx-http-us-west2-cf.aws.edgedelta.com).

Configuration Variables

Configure any other environment variables you need:

  • ED_FORWARD_LAMBDA_TAGS: If set to true, logs are enriched with lambda tags (in the field “lambda_tags”). Lambda function role needs to have the lambda:ListTags permission.
  • ED_LAMBDA_LOG_TYPES: The variable defines which types of logs you want to get from the Lambda Function. Options are function,platform,extension. Default is function,platform.
  • ED_PUSH_TIMEOUT_SEC: Push timeout is the duration between sending batches of logs (in seconds). The default is 5.
  • ED_RETRY_INTERVAL_MS: The initial interval to wait until the next retry (in milliseconds). It is increased exponentially until the Edge Delta process is shut down. The default is 100.
  • ED_BUFFER_SIZE_MB: Maximum buffer size to keep logs (in MB). The default is 20. Some logs may be dropped if the buffer exceeds this limit.
  • ED_FLUSH_AT_NEXT_INVOKE: If set to true, logs are flushed at the start of the next invocation of the function. Normally, logs are flushed after the function execution finishes, which can add to the total running time. For lambda functions that are executed periodically, ED_FLUSH_AT_NEXT_INVOKE can be turned on to reduce costs.
  • ED_PRINT_EXTENSION_LOGS: If set to true, the extension’s own logs are sent to CloudWatch.

Lambda can buffer logs and deliver them to the subscriber. You can configure this behavior in the subscription request by specifying the following optional fields:

  • ED_LAMBDA_MAX_ITEMS: The maximum number of events to buffer in memory. Default: 1000. Minimum: 1000. Maximum: 10000. This is also the size of the channel that the Edge Delta http server writes into and which inputs consume.
  • ED_LAMBDA_MAX_BYTES: The maximum size (in bytes) of the logs to buffer in memory. Default: 262144. Minimum: 262144. Maximum: 1048576.
  • ED_LAMBDA_TIMEOUT_MS: The maximum time (in milliseconds) to buffer a batch. Default: 1000. Minimum: 25. Maximum: 30000.