Agent v0.1.20

April 12, 2022

While this agent release contains many enhancements and bug fixes, review the following noteworthy updates:

Updated Log Enrichment In the Edge Delta App, log enrichment features have been updated to now support enrichment from Kubernetes annotations.

Specifically, you can use the from_k8s parameter to enrich streaming data with K8s attributes.

You can enter a pod, namespace, or node attributes.

        from_k8s:
          pod_identifier_pattern: /var/logs/anyDir/MyApp/users/(?:(.+)/)/.*
          field_mappings:
            * field_name: instance_id
              pod_attribute: pod
              transformers:
                # replace all "source" matches with "target"
                * source: "-"
                  target: "_"
                  type: "replace"
                # remove all "test" words
                * source: "test*"
                  target: ""
                  type: "regex"
            * field_name: namespace
              pod_attribute: namespace
            # fields from labels should have pod_attribute start with "labels."
            * field_name: service
              pod_attribute: labels.service

To learn more, review the Enrich Input Data section in the Inputs document.

Updated Numeric Capture (Regexes) Processors In the Edge Delta App, the Numeric Capture (Regexes) processor has been updated to support multiplication and division for numeric value captures.

Specifically, you can use the newly created value_adjustment_rules parameter to create a rule per capture group.

The rule must follow the “(*|/)” format where:

An asterisk ( * ) represents multiplication A slash ( / ) represents division

    * name: "flog"
      pattern: " (?P\\d+) (?P\\d+)$"
      value_adjustment_rules:
        responsesize:
          operator: "/"
          operand: 1000.0

To learn more, review theNumeric Capture (Regexes) Processor section in the Processors document.

Updated Edge Delta Agent In an effort to improve security during agent installation, makeself –sha256 option has been enabled on agent deployments.

Specifically, makeself performs md5 and crc checks for content integrity.

Starting with version 0.1.20 of the agent, this upgraded security measure update will be included in all agent deployments.

Updated Sumo Logic Output In the Edge Delta App, the Sumo Logic streaming output has been updated.

Specifically, you can use the newly created send_as_json parameter to send data in a JSON format, which allows the fields to be auto-parsed and extracted in Sumo.

    * name: sumo-us-2
      type: sumologic
      endpoint: '{{ Env "EMPTY" "https://endpoint4.collection.us2.sumologic.com/receiver/v1/http/XYZ" }}'
      send_as_json: true

Updated Enriched Data for AWS In the Edge Delta App, you can use the dynamic enrichment feature to obtain data from an AWS EC2 instance.

Specifically, in the value parameter, you must enter aws-instance.

      enrichments:
        dynamic:
          field_mappings:
            # if the field value starts with "aws-instance" then instance metadata is get from aws ec2 instance.
            # for more info ref: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html
            * field_name: "instance_id"
              value: '{{".aws-instance.instance-id"}}'
            * field_name: "instance_type"
              value: '{{".aws-instance.instance-type"}}'

To learn more, review the Enrich Input Data section of the Inputs document.

For additional information, please review the Retrieve instance metadata document from Amazon.

Updated Enriched Data for JSON In the Edge Delta App, the from_logs enrichment feature has been with the json_path parameter.

You can use the json_path parameter to enrich data with fields extracted from JSON logs.

        # from_logs is used to enrich data with fields extracted from logs
        from_logs:
          field_mappings:
            * field_name: component
              # extracting using json_path is also supported
              json_path: fields.[1].component

To learn more, review the Enrich Input Data section of the Inputs document.

Updated Workflows In the Edge Delta App, you can use the newly created enabled_hosts parameter to limit the workflow to specific hosts.

With this parameter, the workflow will only run for specified agent hosts.

  enabled_hosts_workflow:
    description: "runs only specified hosts"
    input_labels:
      * system
      * docker
      * agent
      * infa-processes
    filters:
      * info
    destinations:
      * '{{ Env "TEST_SUMO" }}'
    enabled_hosts:
      * my.host.us1
      * my.host.us2

To learn more, see Workflows.

New Filter / Process Type In the Edge Delta App, you can use the newly created OTLP filter / processor to process OTLP (Open Telemetry) logs.

  * name: opentelemetry_trace_filter
    type: buffered-otlp-trace-processor
    trace_deadline: 1m
    should_filter_traces: true
    failure_path: "attributes.result_code"
    failure_value_pattern: "(4|5)xx"
    latency_threshold: 2500.0
    success_sample_rate: 0.1

To learn more, see Filters.