Edge Delta File Input

Specify a file from which to consume logs.

2 minute read

Overview

The File Input node captures log input from specific files. These messages are sent as logs into the pipeline as they arrive in the file. This node is useful were there’s a need to read and process logs in real time but the system logs are written to flat files on disk, it is also useful for testing and troubleshooting. For a detailed walkthrough, see the Ingest Logs from a File page.

Example Configuration

In this example the file_input node monitors one particular file. You can use wildcards to tail multiple files, and, if you use wildcards, you can exclude specific files.

nodes:
  - name: my_file_input
    type: file_input
    path: "/mnt/inputfile/logs/inputfile.log"

Bear in mind that you need to enable access for the agent to this location by configuring the path in the agent manifest as a volume and a volumemount. See here for details on adding a volume and a volumemount.

Required Parameters

name

A descriptive name for the node. This is the name that will appear in Visual Pipelines and you can reference this node in the yaml using the name. It must be unique across all nodes. It is a yaml list element so it begins with a - and a space followed by the string. It is a required parameter for all nodes.

nodes:
  - name: <node name>
    type: <node type>

type: file_input

The type parameter specifies the type of node being configured. It is specified as a string from a closed list of node types. It is a required parameter.

nodes:
  - name: <node name>
    type: <node type>

path

The path parameter specifies the file paths to tail for log messages. It is a string. A path is required. Wildcards are supported:

/etc/systemd/system/billingservice/*.log includes all .log files in the billingservice folder.
/etc/systemd/system/billingservice/**/*.log includes all .log files in the billingservice folder and any sub-folders.

nodes:
  - name: <node name>
    type: file_input
    path: "/var/logs/app/*.log"

Optional Parameters

exclude

The exclude parameter specifies the file paths to exclude from tailing for log messages. These are files that might otherwise be tailed as they are located within the path parameter. It is a string and it supports wildcards. The exclude parameter is optional.

nodes:
  - name: my_file_input
    type: file_input
    path: "/var/logs/app/*.log"
    exclude: "/var/logs/app/test*.log"