Lambda Pack

Edge Delta Pipeline Pack for AWS Lambda

Overview

The Lambda pack ensures ingestion and appropriate processing of AWS Lambda data. This pipeline recieves the logs from the Edge Delta Lambda layer. The Edge Delta Lambda layer sends logs to the endpoint where each log recieved is parsed and extracted.There is prerequisite to add the (Edge Delta Lambda Extension)[https://docs.edgedelta.com/aws-lambda-extension/].

Pack Description

1. body_attributes

The Parse JSON node parses the JSON attributes from the body of the message.

- name: body_attributes
  type: parse_json_attributes
  process_field: item.body

2. extract_logEvents

The Extract JSON Field node extracts values from JSON log content in a particular field and uses it as the log’s body field. This node is useful for focusing log analysis on specific pieces of data or for simplifying complex log documents into a more manageable form that retains only the essential information required for further processing and analysis.

- name: extract_message
  type: extract_json_field
  field_path: message
  keep_log_if_failed: true

Example Input

Consider the following example of a Lambda log.

Note: Sensitive information has been replaced with dummy data.

{
  "_type": "log",
  "attributes": {
    "ed.env.id": "c3bed94c-17b9-41c4-9cc4-f7f21adc3bc4"
  },
  "body": "{\"cloud\":{\"resource_id\":\"arn:aws:lambda:us-east-2:00000000000:function:example-log\",\"account_id\":\"00000000000\",\"region\":\"us-east-2\"},\"faas\":{\"name\":\"example-log\",\"version\":\"$LATEST\",\"request_id\":\"e8ba78a6-1028-40ba-ad62-21a74249cd59\"},\"timestamp\":\"2024-01-00T00:00:00.000Z\",\"log_type\":\"function\",\"message\":\"example-log\"}",
  "resource": {
    "ed.conf.id": "d3175b27-a510-444c-9055-006e5fc452fc",
    "ed.domain": "pipeline",
    "ed.org.id": "4fcd9ade-2aad-4a53-9214-18b7dc68190c",
    "ed.source.name": "http_input",
    "ed.source.type": "http_input",
    "ed.tag": "awslambda",
    "host.ip": "1.1.1.1",
    "host.name": "example-host",
    "http.method": "POST",
    "http.route": "/",
    "http.scheme": "http",
    "server.port": 80,
    "service.name": "http-80",
    "src_type": "http"
  },
  "timestamp": 1700000000000
}

Example Output

Consider the following log emitted from the Lambda pack.

Note: This example is not derived from the example above but the input log had the same structure.

It consists of a simple body:

{  
  "message": "example-log"
}

As well as a detailed Attributes field:

{
  "cloud": {
    "account_id": "00000000000",
    "region": "us-east-2",
    "resource_id": "arn:aws:lambda:us-east-2:00000000000:function:example-log"
  },
  "ed.env.id": "d5d3b7dc-1e86-4cf7-bbc1-b505bc307d97",
  "faas": {
    "name": "example-log",
    "request_id": "2e3b45fa-6f43-4981-a472-68dfa3010301",
    "version": "$LATEST"
  },
  "log_type": "function",
  "message": "example-log",
  "timestamp": "2024-01-01T00:00:00.000Z"
}

Sample Input

{"_type": "log","attributes": {"ed.env.id": "4f76f760-b479-4deb-a3f8-5221f79de98c"},"body": "{"cloud":{"resource_id":"arn:aws:lambda:us-east-2:000000000000:function:example","account_id":"000000000000","region":"us-east-2"},"faas":{"name":"example-log","version":"$LATEST","request_id":"a7a83425-69bc-422a-9977-695c9c375dcd"},"timestamp":"2024-01-01T00:00:00.000Z","log_type":"function","message":"example-log"}","resource": {"ed.conf.id": "206146b7-3ebf-47bb-9f39-c8dbb61657a8","ed.domain": "pipeline","ed.org.id": "314bbfe1-7ce3-45ac-91f9-99a60fdc61d6","ed.source.name": "http_input","ed.source.type": "http_input","ed.tag": "awslambda","host.ip": "1.1.1.1","host.name": "example-host","http.method": "POST","http.route": "/","http.scheme": "http","server.port": 80,"service.name": "http-80","src_type": "http"},"timestamp": 1730000000000}