Edge Delta Syslog Source

Overview

The Syslog source node enables Edge Delta to receive and process syslog messages sent from various sources including network devices, servers, applications, and security appliances. It supports both RFC 3164 (traditional BSD syslog format) and RFC 5424 (structured syslog format with enhanced metadata) over UDP or TCP transport protocols, allowing you to centralize log collection from your entire infrastructure.

• outgoing_data_types: log

The Syslog source node is ideal for:

• Network Device Monitoring: Collect logs from routers, switches, firewalls, and load balancers
• Application Logging: Receive logs from applications configured to send syslog
• Security Event Collection: Gather security events from IDS/IPS systems and firewalls
• System Log Aggregation: Centralize Linux/Unix system logs
• Legacy System Integration: Collect logs from older systems that only support syslog

Field Mapping

The Syslog source node extracts and maps the following fields from incoming messages:

Severity Mapping

Syslog priorities are automatically mapped to severity levels:

Example Configurations

Basic RFC 5424 UDP Configuration

This configuration receives RFC 5424 formatted syslog messages over UDP on port 5514. Using a non-standard port helps avoid permission requirements for privileged ports below 1024.

nodes:
- name: syslog_rfc5424
  type: syslog_input
  port: 5514
  protocol: rfc5424
  transport_protocol: udp

RFC 3164 TCP Configuration

This configuration receives traditional RFC 3164 syslog messages over TCP for reliable delivery. TCP ensures messages are not lost during transmission.

nodes:
- name: syslog_traditional
  type: syslog_input
  port: 6514
  protocol: rfc3164
  transport_protocol: tcp
  listen: "0.0.0.0"

Advanced Configuration with Encoding

This configuration demonstrates advanced options including custom encoding and listening address specification for multi-homed systems.

nodes:
- name: syslog_advanced
  type: syslog_input
  port: 514
  protocol: rfc5424
  transport_protocol: udp
  listen: "0.0.0.0"
  encoding: utf-8
  location: America/New_York
  parse_json: true

Example Output

When syslog messages are received, they are parsed and converted into structured log items. Below are examples showing the raw syslog message sent and the resulting processed data item.

Info Level Message

Raw syslog message sent:

<14>1 2025-09-04T15:00:00Z webserver nginx - - - User accessed /api/v1/status

Processed data item in pipeline:

{
  "_type": "log",
  "timestamp": 1756998000000,
  "body": "<14>1 2025-09-04T15:00:00Z webserver nginx - - - User accessed /api/v1/status",
  "resource": {
    "host.ip": "172.19.0.2",
    "host.name": "",
    "service.name": ""
  },
  "attributes": {
    "appname": "nginx",
    "facility": 1,
    "hostname": "webserver",
    "message": "User accessed /api/v1/status",
    "priority": 14,
    "version": 1
  },
  "observed_timestamp": 1756955155244,
  "severity_text": "info"
}

Critical Alert Message

Raw syslog message sent:

<10>1 2025-09-04T12:00:00Z database postgres - - - CRITICAL: Database connection lost

Processed data item in pipeline:

{
  "_type": "log",
  "timestamp": 1756955306000,
  "body": "<10>1 2025-09-04T12:00:00Z database postgres - - - CRITICAL: Database connection lost",
  "resource": {
    "host.ip": "172.19.0.2",
    "host.name": "",
    "service.name": ""
  },
  "attributes": {
    "appname": "postgres",
    "facility": 1,
    "hostname": "database",
    "message": "CRITICAL: Database connection lost",
    "priority": 10,
    "version": 1
  },
  "observed_timestamp": 1756955306171,
  "severity_text": "crit"
}

Required Parameters

name

A descriptive name for the node. This is the name that will appear in pipeline builder and you can reference this node in the YAML using the name. It must be unique across all nodes. It is a YAML list element so it begins with a - and a space followed by the string. It is a required parameter for all nodes.

nodes:
  - name: <node name>
    type: <node type>

type: syslog_input

The type parameter specifies the type of node being configured. It is specified as a string from a closed list of node types. It is a required parameter.

nodes:
  - name: <node name>
    type: <node type>

protocol

The protocol parameter specifies which syslog format standard to use for parsing incoming messages.

RFC 3164 (rfc3164) - Traditional BSD syslog:

• Older format widely supported by legacy systems
• Simple structure with limited metadata
• Format: <priority>timestamp hostname tag: message
• Best for compatibility with older devices and systems

RFC 5424 (rfc5424) - Structured syslog:

• Modern format with enhanced structure
• Supports structured data elements for key-value pairs
• ISO 8601 timestamps with timezone support
• Includes version, message ID, and structured data fields
• Recommended for new deployments

It is specified as a string and is a required parameter.

nodes:
  - name: <node name>
    type: syslog_input
    protocol: rfc5424
    transport_protocol: <transport>
    port: <port>

transport_protocol

The transport_protocol parameter specifies the network protocol for receiving syslog messages.

UDP Transport (udp):

• Connectionless, fire-and-forget protocol
• Lower overhead and resource usage
• No delivery confirmation
• Common for traditional syslog implementations
• Best for high-volume, non-critical logs

TCP Transport (tcp):

• Connection-oriented with delivery acknowledgment
• Reliable message delivery
• Higher resource usage
• Supports larger messages
• Recommended for critical logs requiring guaranteed delivery

It is specified as a string and is a required parameter.

nodes:
  - name: <node name>
    type: syslog_input
    protocol: <protocol>
    transport_protocol: udp
    port: <port>

port

The port parameter specifies the network port to listen on for incoming syslog messages. Common ports include:

• 514: Standard syslog port (requires elevated privileges)
• 5514, 6514: Common alternative ports

It is specified as an integer and is a required parameter.

nodes:
  - name: <node name>
    type: syslog_input
    protocol: <protocol>
    transport_protocol: <transport>
    port: 5514

Optional Parameters

listen

The listen parameter specifies the IP address to bind to for listening. Use “0.0.0.0” to listen on all interfaces. Default is “0.0.0.0”.

It is specified as a string and is optional.

nodes:
  - name: <node name>
    type: syslog_input
    protocol: <protocol>
    transport_protocol: <transport>
    port: <port>
    listen: "0.0.0.0"

encoding

The encoding parameter specifies the character encoding for syslog messages. Default is “utf-8”.

Common values:

• utf-8: Unicode (default)
• ascii: ASCII encoding
• latin-1: Latin-1/ISO-8859-1

It is specified as a string and is optional.

nodes:
  - name: <node name>
    type: syslog_input
    protocol: <protocol>
    transport_protocol: <transport>
    port: <port>
    encoding: utf-8

location

The location parameter specifies the timezone location for parsing timestamps that don’t include timezone information. Uses IANA timezone names.

Examples: “America/New_York”, “Europe/London”, “Asia/Tokyo”

It is specified as a string and is optional.

nodes:
  - name: <node name>
    type: syslog_input
    protocol: <protocol>
    transport_protocol: <transport>
    port: <port>
    location: America/New_York

parse_json

The parse_json parameter enables automatic parsing of JSON-formatted message content. When enabled, if the syslog message contains JSON, it will be parsed and added as attributes.

It is specified as a boolean and is optional. Default is false.

nodes:
  - name: <node name>
    type: syslog_input
    protocol: <protocol>
    transport_protocol: <transport>
    port: <port>
    parse_json: true

source_metadata

The source_metadata parameter is used to define which detected resources and attributes to add to each data item as it is ingested by the Edge Delta agent. In the GUI you can select:

• Required Only: This option includes the minimum required resources and attributes for Edge Delta to operate.
• Default: This option includes the required resources and attributes plus those selected by Edge Delta
• High: This option includes the required resources and attributes along with a larger selection of common optional fields.
• Custom: With this option selected, you can choose which attributes and resources to include. The required fields are selected by default and can’t be unchecked.

Based on your selection in the GUI, the source_metadata YAML is populated as two dictionaries (resource_attributes and attributes) with Boolean values.

See Choose Data Item Metadata for more information on selecting metadata.

Configuring Syslog Sources

Rsyslog Configuration

Configure rsyslog to forward logs to Edge Delta:

# /etc/rsyslog.conf or /etc/rsyslog.d/edge-delta.conf

# Forward all logs to Edge Delta via UDP
*.* @<edge-delta-host>:5514

# Forward only critical logs via TCP for guaranteed delivery
*.crit @@<edge-delta-host>:6514

# Forward specific facilities
auth,authpriv.* @<edge-delta-host>:5514
kern.* @<edge-delta-host>:5514
mail.err @@<edge-delta-host>:6514

# Forward with custom template (RFC 5424)
$ActionForwardDefaultTemplate RSYSLOG_SyslogProtocol23Format
*.* @<edge-delta-host>:5514

This rsyslog configuration demonstrates different forwarding patterns for various use cases. The single @ symbol indicates UDP transport while double @@ specifies TCP. The selector syntax (facility.priority) controls which messages are forwarded—for example, *.crit sends all critical-level messages, while auth,authpriv.* sends all authentication-related logs regardless of severity. The last section enables RFC 5424 formatting using rsyslog’s built-in template for modern syslog protocol compliance.

After configuration, restart rsyslog:

sudo systemctl restart rsyslog

Syslog-ng Configuration

Configure syslog-ng to forward to Edge Delta:

# /etc/syslog-ng/syslog-ng.conf

destination d_edge_delta {
    network("<edge-delta-host>" port(5514) transport("udp"));
};

# For TCP with RFC 5424
destination d_edge_delta_tcp {
    syslog("<edge-delta-host>" 
           port(6514) 
           transport("tcp")
           flags(syslog-protocol));
};

log {
    source(s_src);
    destination(d_edge_delta);
};

This syslog-ng configuration creates two destination definitions for Edge Delta. The first uses the network() driver for simple UDP forwarding, while the second uses the syslog() driver with the syslog-protocol flag to enable RFC 5424 formatting over TCP. The log statement connects your existing log source (s_src) to the Edge Delta destination. You can modify the log statement to include filters or use the TCP destination instead by changing d_edge_delta to d_edge_delta_tcp.

Network Device Configuration

Cisco IOS

Configure Cisco IOS devices to forward syslog messages to Edge Delta. The logging trap command sets the minimum severity level (informational captures levels 0-6), while logging origin-id hostname ensures each message includes the device hostname for identification.

logging host <edge-delta-host> transport udp port 5514
logging trap informational
logging origin-id hostname

Juniper Junos

Configure Juniper devices to send syslog messages to Edge Delta. The first command specifies the destination host and port, the second sets the minimum severity level (any info captures all facilities at info level and above), and facility-override remaps all messages to the local0 facility for consistent processing.

set system syslog host <edge-delta-host> port 5514
set system syslog host <edge-delta-host> any info
set system syslog host <edge-delta-host> facility-override local0

Linux IPTables

Enable iptables logging and forward the resulting kernel messages to Edge Delta. The LOG target writes packet information to the kernel log with a custom prefix for easy identification. The --log-level 4 corresponds to the warning severity. After enabling iptables logging, configure rsyslog to forward these kernel warnings using the configuration shown in the rsyslog section above.

# Forward iptables logs
iptables -A INPUT -j LOG --log-prefix "iptables: " --log-level 4
# Configure rsyslog to forward kern.warning to Edge Delta

Application Integration

Python Application

Python’s built-in logging framework provides native syslog integration through the SysLogHandler class, allowing applications to send structured logs directly to Edge Delta without requiring system-level syslog configuration. This approach gives developers fine-grained control over which application events are forwarded and how they’re formatted.

import logging
import logging.handlers

# Configure syslog handler
handler = logging.handlers.SysLogHandler(
    address=('<edge-delta-host>', 5514),
    facility=logging.handlers.SysLogHandler.LOG_USER,
    socktype=socket.SOCK_DGRAM  # UDP
)

# Set format for RFC 5424
formatter = logging.Formatter('%(message)s')
handler.setFormatter(formatter)

# Add to logger
logger = logging.getLogger()
logger.addHandler(handler)
logger.setLevel(logging.INFO)

# Use in application
logger.info("Application started successfully")
logger.error("Database connection failed")

This configuration creates a syslog handler that sends messages over UDP to port 5514 using the USER facility. The formatter is kept minimal to allow the message content to be properly formatted according to syslog standards. Once configured, all log statements at INFO level and above will be automatically forwarded to Edge Delta, where they’ll be parsed and enriched with the appropriate syslog fields including timestamp, hostname, and severity based on the log level used.

Java Application (Log4j2)

Log4j2 provides enterprise-grade logging capabilities for Java applications with built-in syslog support. The Syslog appender can be configured declaratively through XML configuration files, enabling centralized log management without modifying application code. This approach is particularly valuable in microservices architectures where consistent logging across services is critical.

<!-- log4j2.xml -->
<Configuration>
  <Appenders>
    <Syslog name="EdgeDelta" 
            format="RFC5424" 
            host="<edge-delta-host>" 
            port="5514" 
            protocol="UDP"
            appName="MyApp"
            facility="USER"
            includeMDC="true">
      <PatternLayout pattern="%m"/>
    </Syslog>
  </Appenders>
  <Loggers>
    <Root level="info">
      <AppenderRef ref="EdgeDelta"/>
    </Root>
  </Loggers>
</Configuration>

This Log4j2 configuration creates a Syslog appender that formats messages according to RFC 5424 standards and sends them via UDP to Edge Delta. The includeMDC="true" setting ensures that any Mapped Diagnostic Context (MDC) data, such as request IDs or user identifiers, is included as structured data in the syslog message. The PatternLayout is kept simple to prevent double-formatting, allowing Log4j2’s syslog appender to properly structure the message with all required syslog fields.

Docker Container Logging

Docker’s logging driver architecture allows containers to send their stdout and stderr streams directly to syslog destinations without requiring any changes to the containerized application. This container-level approach ensures that all output from the application, including startup messages and uncaught exceptions, is captured and forwarded to Edge Delta.

# Configure Docker to send container logs via syslog
docker run --log-driver=syslog \
  --log-opt syslog-address=udp://<edge-delta-host>:5514 \
  --log-opt syslog-format=rfc5424 \
  --log-opt tag="{{.Name}}/{{.ID}}" \
  myapp:latest

The Docker syslog driver configuration uses RFC 5424 format for structured logging and includes container metadata through the tag template. The {{.Name}}/{{.ID}} tag format provides both the container name and a unique identifier in each log message, making it easy to correlate logs across container restarts or scaling events. This approach works seamlessly with container orchestrators like Docker Swarm or standalone Docker hosts, though Kubernetes users typically rely on the Edge Delta DaemonSet for log collection instead.

Testing Syslog Configuration

After configuring your systems, you can test the connection using these methods:

Using Logger Command

# Send test message from Linux/Unix systems
logger -n <edge-delta-host> -P 5514 -p user.info "Test message from $(hostname)"

# Test with different priorities
logger -n <edge-delta-host> -P 5514 -p user.error "Test error message"
logger -n <edge-delta-host> -P 5514 -p user.crit "Test critical message"

Using Netcat for Manual Testing

# Send RFC 5424 formatted test message via UDP
echo '<14>1 2025-09-04T12:00:00Z test-host app - - - Test connection' | nc -u -w1 <edge-delta-host> 5514

# Test TCP connection
echo '<14>1 2025-09-04T12:00:00Z test-host app - - - TCP test' | nc -w1 <edge-delta-host> 6514

Verify Receipt

Check Edge Delta Live Capture or logs to confirm messages are being received and processed correctly.

Best Practices

Port Selection

When configuring the Syslog source node, choose ports above 1024 to avoid requiring root privileges for the Edge Delta agent. The standard syslog port 514 requires elevated permissions, which may not be available in containerized environments. Common alternatives include port 5514 for UDP traffic and port 6514 for TCP connections, both of which can be bound by non-privileged processes while maintaining compatibility with most syslog clients.

Protocol Selection

For new deployments, RFC 5424 is the recommended syslog format due to its enhanced structure, support for structured data elements, and precise ISO 8601 timestamp formatting with timezone information. This modern format provides better metadata handling and allows for key-value pairs within the message structure. Only use RFC 3164 when dealing with legacy systems that cannot be upgraded, as this older format lacks the structured data capabilities and has limited timestamp precision that can lead to parsing ambiguities.

Transport Selection

The choice between UDP and TCP transport protocols depends on your specific requirements for reliability versus performance. UDP is ideal for high-volume, non-critical log streams where some message loss is acceptable in exchange for lower overhead and better throughput. This connectionless protocol minimizes resource usage and handles traffic spikes more gracefully. TCP should be used for critical logs that require guaranteed delivery, such as security events or compliance data, though it comes with higher resource consumption and connection management overhead. Consider your network’s bandwidth limitations and reliability characteristics when making this decision.

Security Considerations

Traditional syslog transmits data in plain text without encryption, making it vulnerable to interception and tampering. When security is a concern, consider implementing TLS encryption for TCP connections where supported by your syslog sources. Implement strict firewall rules to restrict which IP addresses can send syslog messages to your Edge Delta agents, reducing the risk of log injection attacks. Regular monitoring for suspicious patterns or potential message spoofing attempts is essential, particularly when receiving logs from internet-facing sources. For highly sensitive environments, consider using dedicated network segments or VPNs for syslog traffic.

Performance Optimization

To optimize performance in high-volume environments, start with UDP transport to minimize processing overhead while accepting that some messages may be lost during traffic spikes. Adjust receive buffer sizes based on your message volume and burst patterns to prevent drops during peak periods. For environments generating substantial log volumes, distribute the load across multiple Edge Delta agents using DNS round-robin or load balancer configurations. Continuously monitor CPU and memory usage of your Edge Delta agents, adjusting resource allocations and scaling horizontally when utilization consistently exceeds 70%. Regular analysis of message patterns can help identify opportunities to filter unnecessary logs at the source, reducing overall processing requirements.

Troubleshooting

Messages Not Received

• Verify network connectivity between source and Edge Delta
• Check firewall rules allow traffic on configured port
• Ensure Edge Delta agent has permissions to bind to the port
• Verify protocol and transport settings match sender configuration

Parsing Errors

• Confirm correct protocol (rfc3164 vs rfc5424) is configured
• Check message format matches expected RFC standard
• Verify encoding settings match source encoding
• Review Edge Delta logs for parsing error details

Performance Issues

• Consider switching from TCP to UDP for high-volume sources
• Increase Edge Delta agent resources
• Distribute load across multiple agents
• Filter unnecessary messages at the source

Timestamp Issues

• Set correct location parameter for sources without timezone
• Ensure source systems have synchronized time (NTP)
• Verify timestamp format matches RFC specification