Files
  • Dark
    Light

Files

  • Dark
    Light

Overview

This input type allows you to specify a set of files for Edge Delta to monitor.


Review Sample Configuration

Review the following sample configuration:

files:
    - labels: "billing,errorcheck"
      path: "/billing/logfolder1/*.log"
    - labels: "billing,errorcheck"
      path: "/etc/systemd/system/billingservice/*.log"
      exclude:
        - "/etc/systemd/system/billingservice/test.log"
        - "/etc/systemd/system/billingservice/dev.log"
      auto_detect_line_pattern: true
      boost_stacktrace_detection: true
      enable_persisting_cursor: true
      filters:
        - info
        - not_trace
        - mask_card
        - mask_password
    - labels: "docker,my_container"
      path: "/var/lib/docker/my_container/*.log"
      docker_mode: true
    - labels: "app,service_a"
      path: "/var/log/service_a.log"
      line_pattern: "^MMM dd, yyyy hh:mm:ss"
      late_arrival_handling:
        rule_metrics:
          ignore_after: 15m
        patterns:
          ignore_after: 4h
          report_with_original_timestamp: true
      filters:
      - source-detection-docker
    - labels: k8s_log
      path: /var/logs/anyDir/MyApp/users/MyPodID/transaction.log
      filters:
      - enrichment-full
    - labels: "billing,errorcheck"
      path: "/billing/logfolder1/*.log"
      separate_source: true  
    - labels: "billing,errorcheck"
      path: "/billing/logfolder1/*.log"
      add_ingestion_time: true
      skip_ingestion_time_on_failure: true 

Review Parameters

Review the following parameters that you can configure in the Edge Delta App.


labels

Required

Enter a descriptive label for this input.

When you create a workflow, you will use this label to enter your input into the workflow.

billing,errorcheck is the default label.

Review the following example:

labels: "billing,errorcheck"

path

Required

Enter the full path to the file (or files) that you want Edge Delta to monitor.

  • Wildcards are supported.
  • If you want the agent to process lines for a specific line separation rule (not for New Line("\n")), then you need to define a "line_pattern" regex rule.

Review the following example:

path: "/etc/systemd/system/billingservice/*.log"

exclude

Optional

Enter a path (in regex) to exclude matched patterns.

Review the following example:

 - labels: "billing,errorcheck"
      path: "/etc/systemd/system/billingservice/*.log"
      exclude:
        - "/etc/systemd/system/billingservice/test.log"
        - "/etc/systemd/system/billingservice/dev.log"

auto_detect_line_pattern

Optional

Enter true (or false ) to enable auto line detection for log messages with multiple lines.

Review the following example:

auto_detect_line_pattern: true

boost_stracktrace_detection

Optional

Enter true (or false ) to fallback (troubleshoot) missing content that resulted from the Auto Line Detection parameter.

Review the following example:

boost_stacktrace_detection: true

separate_source

Optional

Enter true (or false ) to have the agent treat each file that matches the glob expression as a separate source.

By default, this option is disabled; all files that match the glob path will be treated as a single source.

Review the following example:

separate_source: true

filters

Optional

Enter an existing filter to add to this input.

To learn how to create a filter, see Filters.

Review the following example:

filters:
  - info
  - not_trace
  - mask_card
  - mask_password

add_ingestion _time

Optional

Enter true (or false ) to ingest a timestamp if the input format is in JSON.

Review the following example:

add_ingestion_time: true

skip_ingestion _time_on_failure

Optional

Enter true or false to skip the ingestion of the timestamp when the input is broken or in an invalid format.

Review the following example:

skip_ingestion_time_on_failure: true`

late_arrival _handling

Optional

You can use this parameter to configure how to accommodate delayed data.

Specifically, you can use the ignore_after parameter to configure when to ignore metrics or pattern logs.

For example, for ignore_after, if you enter 15m, then the input will ignore logs whose timestamp is older than 15 minutes.

Review the following example:

late_arrival_handling:
  rule_metrics:
    ignore_after: 15m
  patterns:
    ignore_after: 4h
    report_with_original_timestamp: true

docker_mode

Optional

Enter true (or false ) to collect Docker container standard output logs on a file with the JSON file logging driver.

To learn more, review this document from Docker.

Review the following example:

docker_mode: true

Was this article helpful?

Changing your password will log you out immediately. Use the new password to log back in.
First name must have atleast 2 characters. Numbers and special characters are not allowed.
Last name must have atleast 1 characters. Numbers and special characters are not allowed.
Enter a valid email
Enter a valid password
Your profile has been successfully updated.