Agent Settings
  • Dark
    Light

Agent Settings

  • Dark
    Light

Overview

You can use this document to learn about the configuration parameters available in a configuration file, specifically for Agent Settings.


Review Sample Configuration for Agent Settings

Review the following sample configuration for Agent Settings:

agent_settings:
  tag: prod_payments
  log:
    level: info
  persisting_cursor_settings:
    path: /var/lib/edgedelta/cursor_provider
    file_name: cursor_provider.json
    flush_interval: 5s
  soft_cpu_limit: 0.5
  anomaly_tolerance: 0.1
  anomaly_confidence_period: 1m
  skip_empty_intervals: false
  only_report_nonzeros: false
  anomaly_capture_size: 1000
  anomaly_capture_bytesize: "10 KB"
  anomaly_capture_duration: 1m
  anomaly_coefficient: 10.0
  grace_period: 30s

Review Parameters for Agent Settings

Review the following parameters that you can configure in the Edge Delta App.


anomaly_capture_size

Optional

This parameter represents the number of log lines (buffer size) to capture during an anomaly capture.

The default value is 125.

Review the following example:

anomaly_capture_size: 1000

anomaly_capture_bytesize

Optional

This parameter represents the maximum buffer size (in bytes) to capture during an anomaly capture.

The default value is 0b (disabled).

Review the following example:

anomaly_capture_bytesize: "10 KB"

anomaly_capture_duration

Optional

This parameter represents the maximum time span that the logs of an anomaly capture can belong to, such as logs from the last 10 minutes.

The default value is 0s (disabled).

Review the following example:

anomaly_capture_duration: 1m

anomaly_coefficient

Optional

This parameter represents the anomaly coefficient used to multiply the final score to a range of 0 - 100.

The higher the coefficient, the higher the anomaly score will be.

For some rules types, this parameter can be set at the rule level.

The default value is 10.

Review the following example:

anomaly_coefficient: 10.0

anomaly_confidence_period

Optional

This parameter represents a grace period between when a configuration is updated, which will restart the agent, and when data will flow into the app.

Anomaly scores will all be zero while baselines are established.

For some rules types, this parameter can be set at the rule level.

The default value is 30M.

Review the following example:

anomaly_confidence_period: 1m

anomaly_tolerance

Optional

When it is non-zero, anomaly scores handle edge cases better when standard deviation is too small.

For some rules types, this parameter can be set at the rule level.

The default value is 0.01.

Review the following example:

anomaly_tolerance: 0.2


attributes

Optional

This parameter defines a user-defined, key-value pair that is used to label and distinguish different running agents.

These key-value pairs are attached to the data that is collected and generated by the agent and sent to streaming destinations.

You can set keywords for app, environment, and region.

Review the following example:

attributes:
    environment: prod
    app: smp
    region: us-west

capture_flush_mode

Optional

This parameter sets the behavior for flushing captured contextual log buffers.

This parameter supports the following modes:

  • local_per_source flushes the captured buffer of a source when a local alert is triggered from the same source.
  • local_all flushes all captured buffers when a local alert is triggered, not necessarily from the same source. As a result, when an alert is triggered from an agent, all captured buffers from all active sources will be flushed.
  • tag_per_source flushes the captured buffer of a source when an alert is triggered from the same source and tag, including any agent within the current tag.
  • tag_all flushes all captured buffers for all agents within the same tag when any agent triggers an alert.
  • custom_local_per_group flushes specified custom groups that should flush together if one of the groups triggers an alert.

The default value is local_per_source.

Review the following example:

capture_flush_mode: custom_local_per_group

log

Optional

This parameter contains additional subfields to configure an agent's log settings.

Specifically, you can configure the following parameters:

Level

  • Use this parameter to organize the entires in your log file.
  • Review the following example:
agent_settings:
  tag: prod
  log:
    level: debug

Secure Logging

  • Use this parameter to hide sensitive data from the specified agent logs, such as API keys, secrets, and authentication information.
Note

When an agent runs inside a container, such as Kubernetes, the agent logs to a standard output.

When an agent runs as a Linux, Windows, or macOS service, the agent logs to a file named edgedelta.log next to the installed service location.

Review the following example:

log:
  level: debug
  secure_logging: true

multiline_max_bytesize

Optional

This parameter configures the buffer byte size for multiline accumulation.

The default value is 10 KB.

Review the following example:

multiline_max_bytesize: "10 KB"

multiline_max_size

Optional

This parameter defines the buffer length size for multiline accumulation.

If there is an overflow of lines, then those extra lines are dumped as a single line.

The default value is 250.

Review the following example:

multiline_max_size: 250

only_report_nonzeros

Optional

This parameter configures if non-zero stats should be reported or not.

For some rules types, this parameter can be set at the rule level.

The default value is false.

Review the following example:

only_report_nonzeros: true

rule_metrics_prom_stats_enabled

Optional

The rule_metrics_prom_stats_enabled parameter is used to configure all regex processors to expose rule metrics to the metrics endpoint for Prometheus to scrape. Specific processors can be excluded from exposing their rule metrics to Prometheus by setting disable_reporting_in_prometheus: to true in the regex processor configuration. The rule_metrics_prom_stats_enabled parameter is configured with a boolean true or false and it is optional.

agent_settings
  rule_metrics_prom_stats_enabled: <true|false>

skip_empty_intervals

Optional

This parameter configures if empty intervals should be skipped so that anomaly scores are calculated based on non-zero intervals.

For some rules types, this parameter can be set at the rule level.
The default value is false.

Review the following example:

skip_empty_intervals: true

soft_cpu_limit

Optional

This parameter allows you to use more CPU than what is specified in the allocation.

This parameter is only honored by the clustering processor at the moment. 0.5 means 50% of a core.

This parameter complements the cpu_friendly parameter for Processors.

To enable, in the clustering rule, set cpu_friendly=true.

The default value is 0.0.

Review the following example:

soft_cpu_limit: 0.5

tag

Optional

This parameter is a user-defined tag used to describe the environment, such as prod_us_west_2_cluster.

While the default value is Edge, we recommend that you set a value.

Review the following example:

tag: prod

max_file_per_glob_path

Optional

Enter the maximum number of files to tail, per glob path.

The default value is 100.

Review the following example:

max_file_per_glob_path: 100

forget_file_after

Optional

Enter a length of time to drop files that have not been modified.

The default value is 1h.

Review the following example:

forget_file_after: 1h

total_seek_capacity

Optional

Enter the maximum size that tailers can seek concurrently.

The default value is 5MB.

Review the following example:

total_seek_capacity: "5 MB"

max_seek_size

Optional

Enter the maximum size that a tailer can seek, per second.

Review the following example:

max_seek_size: "4 MB"

source_discovery_interval

Optional

Enter how often the source discovery is invoked.

The default value is 5s.

Review the following example:

source_discovery_interval: 5s

file_tailer_buffer_size

Optional

Enter the maximum number of logs that a file tailer can store in its memory until the logs are ingested by the agent’s internal router.

If the router is busy and cannot ingest the logs and the tailer’s buffer is reached, then the seeking will be blocked.

The default value is 1000.

Review the the following example:

file_tailer_buffer_size: 1000

router_per_source_buffer_size

Optional

Enter the maximum number of logs that an agent’s internal router can store in its memory, per source.

The default value is 1000.

Review the following parameter:

router_per_source_buffer_size: 1000

archive_flush_interval

Optional

Enter a time frame to flush and send logs to a configured archiving destination.

The default value is 5m.

Review the following example:

archive_flush_interval: 5m

archive_max_byte_limit

Optional

Enter the maximum number of bytes that can be buffered (in memory) before a flush is triggered to an archive destination.

The default value is 26MB.

Review the following example:

archive_max_byte_limit: "16MB"

Review Sample Configuration for Attributes

Review the following sample configuration for Attributes :

attributes:
    environment: prod
    app: smp
    region: us-west

Review Parameters for Attributes

Review the following parameters that you can configure in the Edge Delta App.


app

Optional

Enter a descriptive label that will be used to enrich data generated by the agent.

Review the following example:

attributes:
  app: smp

environment

Optional

Enter a descriptive label that will be used to enrich data generated by the agent.

Review the following example:

attributes:  environment: prod

region

Optional

Enter a descriptive label that will be used to enrich data generated by the agent.

Review the following example:

attributes:  region: us-west

agent_stats_enabled

Optional

Enter true or false to display agent-related information (such as CPU and memory usage) in the Metrics page. 

Note 
If you want to forward this data to an external destination, then we recommend that you use the agent_stats input.
To learn more, see Agent Stats.

Review the following example: 

agent_stats_enabled: true



Create and Manage Agent Settings

To create and manage agent settings, you must populate a YAML file.

To access the YAML file for a new configuration:

  1. In the Edge Delta App, on the left-side navigation, click Data Pipeline, and then click Agent Settings.
  2. Click Create Configuration.
  3. Click YAML.
  4. Enter the desired parameters, and then click Save.

To access the YAML file for an existing configuration:

  1. In the Edge Delta App, on the left-side navigation, click Data Pipeline, and then click Agent Settings.
  2. Locate the desired configuration, then under Actions, click the vertical ellipses, and then click Edit.
  3. Review the YAML file, make your changes, and then click Save.

Was this article helpful?

Changing your password will log you out immediately. Use the new password to log back in.
First name must have atleast 2 characters. Numbers and special characters are not allowed.
Last name must have atleast 1 characters. Numbers and special characters are not allowed.
Enter a valid email
Enter a valid password
Your profile has been successfully updated.