Edge Delta File Input
2 minute read
Overview
The File Input node captures log input from specific files. These messages are sent as logs into the pipeline as they arrive in the file. This node is useful were there’s a need to read and process logs in real time but the system logs are written to flat files on disk, it is also useful for testing and troubleshooting. For a detailed walkthrough, see the Ingest Logs from a File page.
Example Configuration
In this example the file_input node monitors one particular file. You can use wildcards to tail multiple files, and, if you use wildcards, you can exclude specific files.
nodes:
- name: my_file_input
type: file_input
path: "/mnt/inputfile/logs/inputfile.log"
Bear in mind that you need to enable access for the agent to this location by configuring the path in the agent manifest as a volume and a volumemount. See here for details on adding a volume and a volumemount.
Required Parameters
name
A descriptive name for the node. This is the name that will appear in Visual Pipelines and you can reference this node in the yaml using the name. It must be unique across all nodes. It is a yaml list element so it begins with a -
and a space followed by the string. It is a required parameter for all nodes.
nodes:
- name: <node name>
type: <node type>
type: file_input
The type
parameter specifies the type of node being configured. It is specified as a string from a closed list of node types. It is a required parameter.
nodes:
- name: <node name>
type: <node type>
path
The path
parameter specifies the file paths to tail for log messages. It is a string. A path
is required. Wildcards are supported:
/etc/systemd/system/billingservice/*.log
includes all.log
files in thebillingservice
folder./etc/systemd/system/billingservice/**/*.log
includes all.log
files in thebillingservice
folder and any sub-folders.
nodes:
- name: <node name>
type: file_input
path: "/var/logs/app/*.log"
Optional Parameters
exclude
The exclude
parameter specifies the file paths to exclude from tailing for log messages. These are files that might otherwise be tailed as they are located within the path
parameter. It is a string and it supports wildcards. The exclude
parameter is optional.
nodes:
- name: my_file_input
type: file_input
path: "/var/logs/app/*.log"
exclude: "/var/logs/app/test*.log"