CloudWatch Event Logs (AWS)
  • Dark
    Light

CloudWatch Event Logs (AWS)

  • Dark
    Light

Overview

This input type allows you to specify a set of AWS CloudWatch Log Events for Edge Delta to monitor.

With this input, you can monitor multiple regions and log streams.


Review Sample Configuration

Review the following sample configuration:

cloudwatches:
    - labels: "cloudwatch-input-with-rate-limiters"
      region: ".*"
      log_group: "^/ed-log"
      log_stream: ".*"
      interval: 5m    
      rate_limiters:
        - api_name: "DescribeLogGroups"
          every: 1s
          limit: 4
        - api_name: "DescribeLogStreams"
          every: 1s
          limit: 4
        - api_name: "GetLogEvents"
          every: 1s
          limit: 22
    - labels: "us-west-2_ed-log-group_admin-api"
      region: "^us.*$"
      log_group: /ed-log-group
      log_stream: "^log.*$"
      lookback: 1h
      interval: 1m
      prepend_timestamp: true
      result_limit: 5000
    - labels: "us_ed-log-group_admin"
      region: "^us.*$"
      log_group: "/ed-log-group"
      log_stream: "^admin.*$"
      interval: 5m
    - labels: "ed-log-group"
      region: ".*"
      log_group: "/ed-log-group"
      log_stream: ".*"
      interval: 5m
    - labels: "ed-log-with-regex-group-name"
      region: ".*"
      log_group: "^/ed-log"
      log_stream: ".*"
      interval: 5m
    - labels: "cloudwatch-input-assumes-role"
      region: "us-west-2"
      log_group: "/ed-log-group"
      log_stream: ".*"
      interval: 5m
      role_arn: "arn:aws:iam::<ACCOUNT_ID>:role/<ROLE_NAME>"
      external_id: "053cf606-8e80-47bf-b849-8cd1cc826cfc"
    - labels: "cloudwatch-input-host-filtered"
      host: "myhost"
      region: ".*"
      log_group: "^/ed-log"
      log_stream: ".*"
      interval: 5m
    - labels: "cloudwatch-input-with-aws-creds"
      region: ".*"
      log_group: "^/ed-log"
      log_stream: ".*"
      interval: 5m
      aws_key_id: '{{ Env "AWS_KEY_ID" }}'
      aws_sec_key: '{{ Env "AWS_SECRET_KEY" }}'

Review Parameters

Review the following parameters that you can configure in the Edge Delta App.


labels

Required

Enter a descriptive name for this input.

When you create a workflow, you will use this label to enter your input into the workflow.

Review the following example:

- labels: "us-west-2_ed-log-group_admin-api"

region

Optional

You can define a region pattern via regex expression.

For example, for all regions in United States, enter:

  • region: "^us.*$"
Note

By default, an AWS account is not enabled with all regions. As a result, you can monitor AWS CloudWatch Log Events for all regions without defining a region in the config file; the Edge Delta Agent will obtain and monitor logs from all enabled regions in your account. To accomplish this, you must add the ec2:DescribeRegions permission to your account.

To learn more, review the AWS document about DescribeRegions.

Review the following example:

region: "us-west-2"

log_group

Required

Enter the Log Group name via a regex expression that is associated with the CloudWatch Logs agent.

Review the following example:

log_group: /ed-log-group

log_stream

Optional

Enter a log stream pattern via a regex expression.

For example, for streams that start with log, enter:

  • log_stream: ^log.*$""

Review the following example:

log_stream: "^log.*$"

lookback

Optional

Enter an internal rate to monitor past data.

By default, this parameter is set to 1 hour.

Review the following example:

lookback: 1h

interval

Optional

Enter an internal rate to look for new, incoming log events.

By default, this parameter is set to 1 minute.

Review the following example:

interval: 1m

rate_limiters

Optional

Specify the API rate limit to conform with the quotas set by Amazon Web Services. 

Specify the CloudWatch resource using api_name. Specify the time frame with the every parameter and the number of transactions with the limit parameter. 

For example, the current quota from CloudWatch for DescribeLogGroups is 5 per second so an agent limit of 4 transactions every 1 second will fall under the quota. 


Note
If you have multiple CloudWatch inputs configured you need to split your quota between the inputs. For example, if you have two inputs using the DescribeLogGroups resource they should each be limited to half the total quota: 2 transactions per second each to fall under the 5 transactions per second total quota. 


      rate_limiters:
        - api_name: "DescribeLogGroups"
          every: 1s
          limit: 4
        - api_name: "DescribeLogStreams"
          every: 1s
          limit: 4
        - api_name: "GetLogEvents"
          every: 1s
          limit: 22

prepend_timestamp

Optional

Enter true or false to add an event timestamp as a prefix to the event message with a tab ("\t") delimiter.

Review the following example:

prepend_timestamp: true

result_limit

Optional

Enter a limit for the the maximum number of log events that should be returned.

By default, this parameter will return up to 1 MB of log events, which can be up to 10,000 log events.

Review the following example:

result_limit: 5000

filters

Optional

Enter an existing filter to add to this input.

To learn more, see Filters.

Review the following example:

filters: 
- info 

Was this article helpful?

Changing your password will log you out immediately. Use the new password to log back in.
First name must have atleast 2 characters. Numbers and special characters are not allowed.
Last name must have atleast 1 characters. Numbers and special characters are not allowed.
Enter a valid email
Enter a valid password
Your profile has been successfully updated.