Splunk HTTP Event Collector Source

Overview

This source node is used to ingest log data from Splunk’s HTTP Event Collector. The Splunk HEC source listens on the following endpoints:

• JSON Events: <host>:<port>/services/collector/event - For structured JSON event data
• Raw Data: <host>:<port>/services/collector/raw - For raw text data

AI Team: Configure this source using the Splunk HEC connector for streamlined setup in AI Team.

Example Configuration

nodes:
  - name: my_hec_input
    type: splunk_hec_input
    port: 8088
    read_timeout: 30s

Splunk Output Configuration

To configure Splunk to send data to the Edge Delta HEC input, add the following to your Splunk outputs.conf:

[httpout]
httpEventCollectorToken = 1234abcd-67ef-89gh-1234-567890ijkl
uri = http://localhost:3421/services/collector/event
batchSize = 32768
batchTimeout = 10

Replace:

• httpEventCollectorToken with your desired token (omit this line if Edge Delta doesn’t enforce authentication)
• localhost:3421 with your Edge Delta HEC input host and port
• Use /services/collector/event for JSON events or /services/collector/raw for raw data

Required Parameters

name

A descriptive name for the node. This is the name that will appear in pipeline builder and you can reference this node in the YAML using the name. It must be unique across all nodes. It is a YAML list element so it begins with a - and a space followed by the string. It is a required parameter for all nodes.

nodes:
  - name: <node name>
    type: <node type>

type: splunk_hec_input

The type parameter specifies the type of node being configured. It is specified as a string from a closed list of node types. It is a required parameter.

nodes:
  - name: <node name>
    type: <node type>

read_timeout

The read_timeout parameter is used to specify how long to wait for incoming data. Default value is 0 which means no time out. It is specified as a duration and it is required.

nodes:
  - name: <node name>
    type: splunk_hec_input
    read_timeout: 10s

Optional Parameters

listen

The listen parameter is used to specify the address to listen to for incoming traffic. It is specified as a string and it is optional.

nodes:
  - name: <node name>
    type: splunk_hec_input
    read_timeout: 10s
    listen: <host>

port

Enter the port number that the splunk_hec_input type node should listen on. It is specified as an integer and is a required parameter.

nodes:
  - name: <node name>
    type: splunk_hec_input
    read_timeout: 10s
    port: <port number>

source_metadata

This option is used to define which detected resources and attributes to add to each data item as it is ingested by Edge Delta. You can select:

• Required Only: This option includes the minimum required resources and attributes for Edge Delta to operate.
• Default: This option includes the required resources and attributes plus those selected by Edge Delta
• High: This option includes the required resources and attributes along with a larger selection of common optional fields.
• Custom: With this option selected, you can choose which attributes and resources to include. The required fields are selected by default and can’t be unchecked.

Based on your selection in the GUI, the source_metadata YAML is populated as two dictionaries (resource_attributes and attributes) with Boolean values.

See Choose Data Item Metadata for more information on selecting metadata.

tls

Configure TLS settings for secure connections to this source. TLS is optional and typically used when the source node needs to accept encrypted connections or perform mutual TLS authentication with clients.

YAML Configuration Example:

nodes:
  - name: <node name>
    type: <source type>
    tls:
      <tls options>

CA Certificate File

Specifies the absolute path to a single CA (Certificate Authority) certificate file used to verify client certificates during mutual TLS authentication. This file contains the trusted CA certificate that signed the client certificates. Use this when you have a single CA certificate. (YAML parameter: ca_file)

When to use: Required when client_auth_type is set to verifyclientcertifgiven or requireandverifyclientcert. Choose either ca_file or ca_path, not both.

YAML Configuration Example:

nodes:
  - name: <node name>
    type: <source type>
    tls:
      ca_file: /certs/ca.pem
      client_auth_type: requireandverifyclientcert

CA Certificate Path

Specifies a directory path containing one or more CA certificate files for client certificate verification. Use this when you need to trust multiple CAs or when managing CA certificates across multiple files. All certificate files in the directory will be loaded. (YAML parameter: ca_path)

When to use: Alternative to ca_file when you have multiple CA certificates. Required when client_auth_type is set to verifyclientcertifgiven or requireandverifyclientcert. Choose either ca_file or ca_path, not both.

YAML Configuration Example:

nodes:
  - name: <node name>
    type: <source type>
    tls:
      ca_path: /certs/ca-certificates/
      client_auth_type: requireandverifyclientcert

Certificate File

Path to the server certificate file (public key) that will be presented to connecting clients during the TLS handshake. This certificate identifies the server and must match the private key. The certificate should be in PEM format and can include the full certificate chain. (YAML parameter: crt_file)

When to use: Required for TLS/HTTPS connections. Must be used together with key_file. Obtain this from your certificate authority or generate a self-signed certificate for testing.

YAML Configuration Example:

nodes:
  - name: <node name>
    type: <source type>
    tls:
      crt_file: /certs/server-cert.pem
      key_file: /certs/server-key.pem

Private Key File

Path to the private key file corresponding to the server certificate. This key must match the public key in the certificate file and is used to decrypt communications encrypted with the public key. Keep this file secure and restrict access permissions. (YAML parameter: key_file)

When to use: Required for TLS/HTTPS connections. Must be used together with crt_file. If the key file is encrypted, also specify key_password.

YAML Configuration Example:

nodes:
  - name: <node name>
    type: <source type>
    tls:
      crt_file: /certs/server-cert.pem
      key_file: /certs/server-key.pem
      key_password: <password>  # Only if key is encrypted

Private Key Password

Password (passphrase) used to decrypt an encrypted private key file. Only needed if your private key file is password-protected. If your key file is unencrypted, omit this parameter. (YAML parameter: key_password)

When to use: Optional. Only required if key_file is encrypted/password-protected. For enhanced security, use encrypted keys in production environments.

YAML Configuration Example:

nodes:
  - name: <node name>
    type: <source type>
    tls:
      crt_file: /certs/server-cert.pem
      key_file: /certs/encrypted-server-key.pem
      key_password: mySecurePassword123

Client Authentication Type

Controls how client certificates are requested and validated during the TLS handshake. This setting is specific to server-side TLS and determines the mutual TLS (mTLS) behavior. Use this to enforce certificate-based authentication from connecting clients for enhanced security. (YAML parameter: client_auth_type)

Valid options:

• noclientcert - No client certificate is requested; any sent certificates are ignored. Use for standard TLS encryption without client authentication.
• requestclientcert - Client certificate is requested but not required. Useful for optional client authentication where you want to accept both authenticated and unauthenticated clients.
• requireanyclientcert - Client must send a certificate, but it is not validated against a CA. Use when you need proof a client has a certificate but don’t need to verify its authenticity.
• verifyclientcertifgiven - Certificate is requested but not required; if sent, it must be valid and signed by a trusted CA. Balances security with flexibility for mixed client environments.
• requireandverifyclientcert - Client must send a valid certificate signed by a trusted CA (full mutual TLS). Use for maximum security when all clients can be provisioned with certificates.

Default: noclientcert

When to use: Set to verifyclientcertifgiven or requireandverifyclientcert for secure environments where you need to verify client identity. When using certificate verification options, you must also configure ca_file or ca_path to specify the trusted CA certificates.

YAML Configuration Example:

nodes:
  - name: <node name>
    type: <source type>
    tls:
      client_auth_type: requireandverifyclientcert
      ca_file: /certs/ca.pem  # Required for certificate validation

Minimum TLS Version

Minimum TLS protocol version that will be accepted from connecting clients. This enforces a baseline security level by rejecting connections from clients using older, less secure TLS versions. (YAML parameter: min_version)

Available versions:

• TLSv1_0 - Deprecated, not recommended (security vulnerabilities)
• TLSv1_1 - Deprecated, not recommended (security vulnerabilities)
• TLSv1_2 - Recommended minimum for production (default)
• TLSv1_3 - Most secure, use when all clients support it

Default: TLSv1_2

When to use: Set to TLSv1_2 or higher for production deployments. Only use TLSv1_0 or TLSv1_1 if you must support legacy clients that cannot upgrade, and be aware of the security risks. TLS 1.0 and 1.1 are officially deprecated and should be avoided.

YAML Configuration Example:

nodes:
  - name: <node name>
    type: <source type>
    tls:
      min_version: TLSv1_2

Maximum TLS Version

Maximum TLS protocol version that will be accepted from connecting clients. This is typically used to restrict newer TLS versions if compatibility issues arise with specific client implementations. (YAML parameter: max_version)

Available versions:

• TLSv1_0
• TLSv1_1
• TLSv1_2
• TLSv1_3

When to use: Usually left unset to allow the most secure version available. Only set this if you encounter specific compatibility issues with TLS 1.3 clients, or for testing purposes. In most cases, you should allow the latest TLS version.

YAML Configuration Example:

nodes:
  - name: <node name>
    type: <source type>
    tls:
      max_version: TLSv1_3

attributes_from_header

Minimum Agent Version: v2.5.0

The attributes_from_header parameter allows you to extract specific HTTP headers from incoming Splunk HEC requests and store them as attributes on the received data items. This is useful for capturing metadata such as authentication tokens, client identifiers, or custom headers. It is specified as an array of header names and is an optional advanced parameter.

When a Splunk HEC request includes an Authorization: Splunk <token> header and Authorization is included in this list, the token value will be stored in resource.splunk_token on the data item, which can then be used by the Splunk output node.

nodes:
  - name: <node name>
    type: splunk_hec_input
    read_timeout: 10s
    attributes_from_header:
      - Authorization
      - X-Splunk-Request-Channel
      - X-Custom-Header

Example Use Case: This feature enables token passthrough scenarios where the Splunk HEC input captures the original Splunk token from incoming requests, and the Splunk output node can use that same token when forwarding data to Splunk, maintaining authentication context throughout the pipeline.

hec_authentication

Minimum Agent Version: v2.7.0

The hec_authentication parameter enables token validation and access control for incoming HEC requests. When configured, the Splunk HEC input will validate tokens against the configured authentication settings, providing better security and diagnostics for token validation issues. It is specified as an object and is optional.

Backward Compatibility: Agents running versions older than v2.7.0 will not honor the hec_authentication field. Ensure all agents are upgraded to v2.7.0 or higher before using this parameter.

nodes:
  - name: <node name>
    type: splunk_hec_input
    read_timeout: 10s
    port: 8088
    hec_authentication:
      enabled: true
      tokens:
        - token: your-hec-token-1
          enabled: true
        - token: your-hec-token-2
          enabled: false  # Disabled token will return error code 1

Common Use Cases:

• Enable token validation for secure HEC endpoints
• Manage multiple tokens with individual enable/disable controls
• Provide detailed error responses for authentication failures (error codes 1-4)
• Implement access control for incoming HEC data

Authentication Error Codes: When authentication is enabled, the HEC input will return specific error codes for authentication failures:

• Code 1: Token is disabled
• Code 2: Token is required but not provided
• Code 3: Invalid authorization header format
• Code 4: Token is not recognized

For a complete reference of all HEC error codes, see the HEC Error Codes section below.

max_version

The max_version parameter is a child of the tls parameter. It specifies the maximum version of TLS to accept. It is specified as a string and is optional.

You can select one of the following options:

• TLSv1_0
• TLSv1_1
• TLSv1_2
• TLSv1_3

nodes:
  - name: <node name>
    type: splunk_hec_input
    read_timeout: 10s
    tls:
      max_version: <TLS version>

min_version

The min_version parameter is a child of the tls parameter. It specifies the minimum version of TLS to accept. It is specified as a string and is optional. The default is TLSv1_2.

You can select one of the following options:

• TLSv1_0
• TLSv1_1
• TLSv1_2
• TLSv1_3

nodes:
  - name: <node name>
    type: splunk_hec_input
    read_timeout: 10s
    tls:
      min_version: <TLS version>

HEC Error Codes

The Splunk HEC source node implements the Splunk HEC error code system, providing detailed feedback for client requests. When a request fails or encounters an issue, the response includes a numeric error code and descriptive text to help diagnose problems.

Error Code Reference

The table above includes recommended solutions for each error code. For more detailed troubleshooting guidance, see the Splunk Integration Troubleshooting Guide.

Error Response Format

When an error occurs, the response includes a JSON object with detailed information:

{
  "text": "Error description",
  "code": 4,
  "invalid-event-number": 0,
  "invalid-token": "abc123"
}

Response Fields:

• text: Human-readable error description
• code: Numeric error code (see table above)
• invalid-event-number: (Optional) Index of the problematic event in batch requests
• invalid-data-format: (Optional) Details about format issues
• invalid-index: (Optional) Name of the invalid index
• invalid-authorization: (Optional) Authorization header details
• invalid-token: (Optional) The invalid token value

Common Error Scenarios

Authentication Errors (Codes 1-4)

If you receive authentication-related errors:

• Code 2: Ensure your request includes the Authorization: Splunk <token> header
• Code 4: Verify the token matches what’s configured in your pipeline
• Code 1: Check if the token has been disabled in the configuration

Data Format Errors (Codes 5-6, 12-13)

For data format issues:

• Code 5: Verify request body is not empty
• Code 6: Ensure JSON is properly formatted for /services/collector/event endpoint
• Code 12-13: Include a valid event field in your JSON payload

Service Availability Errors (Codes 9, 18-20, 23-27)

Server capacity and health errors:

• Code 9, 26: Server is overloaded, implement retry logic with exponential backoff
• Code 27: ACK channel full, reduce request rate or increase buffer capacity
• Code 23: Server is shutting down, redirect traffic to another instance

Example Error Responses

Missing Token:

{
  "text": "Token is required",
  "code": 2
}

Invalid Data Format:

{
  "text": "Invalid data format",
  "code": 6,
  "invalid-event-number": 3,
  "invalid-data-format": "malformed JSON"
}

Token Disabled:

{
  "text": "Token disabled",
  "code": 1,
  "invalid-token": "CF179AE4-3C99-45F5-A7CC-3284AA91CF67"
}

Testing the Endpoint

You can test your Edge Delta HEC endpoint using curl:

# For JSON events
curl "http://localhost:3421/services/collector/event" \
  -H "Authorization: Splunk CF179AE4-3C99-45F5-A7CC-3284AA91CF67" \
  -d '{"event": "Hello, world!", "sourcetype": "manual"}'

# For raw data
curl "http://localhost:3421/services/collector/raw" \
  -H "Authorization: Splunk CF179AE4-3C99-45F5-A7CC-3284AA91CF67" \
  -d 'Raw log message'