Edge Delta Webhook Destination

Configure the Edge Delta Webhook Destination to send alerts and metrics using customizable JSON payloads with Go template variables.

Overview

The Webhook destination node sends alerts and metrics to a webhook endpoint using customizable payloads with Go template variables.

Key Features:

  • Supports both alert/signal and metric data
  • Customizable JSON payloads using Go templates
  • Suppression windows for alerts (not applied to metrics)
  • Integration with any webhook-compatible service

See Send Data to a Webhook for detailed instructions.

Template Variables Reference

The webhook node uses Go templates to create dynamic payloads. Variables are accessed through the .item object, which contains different fields depending on whether it’s an alert/signal or a metric.

Alert/Signal Variables

When processing alerts, the following variables are available:

Signal Fields (.item.signal)

Variable Description Example Usage
title Alert title {{ .item.signal.title }}
description Alert description {{ .item.signal.description }}
signal_id Unique signal identifier {{ .item.signal.signal_id }}
name Metric name that triggered the alert {{ .item.signal.name }}
value Current metric value {{ .item.signal.value }}
threshold_type Type of threshold (e.g., “above”, “below”) {{ .item.signal.threshold_type }}
threshold_value Configured threshold value {{ .item.signal.threshold_value }}
capture_flush_mode Capture mode for the alert {{ .item.signal.capture_flush_mode }}
capture_size Size of captured data {{ .item.signal.capture_size }}

Resource Fields (.item.resource)

Variable Description Example Usage
ed.tag EdgeDelta agent tag {{ index .item.resource "ed.tag" }}
ed.source_name Source name {{ index .item.resource "ed.source_name" }}
ed.source_type Source type (e.g., “file”, “k8s”) {{ index .item.resource "ed.source_type" }}
host.name Host name {{ index .item.resource "host.name" }}
custom attributes Any custom resource attributes {{ index .item.resource "custom_key" }}

Additional Fields

  • Attributes (.item.attributes): Custom tags and labels as key-value pairs
  • Timestamp (.item.timestamp): ISO 8601 formatted timestamp

Metric Variables

When processing metrics, the following variables are available:

Variable Description Example Usage
.item.name Metric name {{ .item.name }}
.item.value Metric value {{ .item.value }}
.item.stat_type Statistic type (gauge, counter, etc.) {{ .item.stat_type }}
.item.resource Same resource fields as alerts {{ index .item.resource "host.name" }}
.item.attributes Metric-specific attributes {{ index .item.attributes "environment" }}
.item.timestamp Metric timestamp {{ .item.timestamp }}

Template Functions

EdgeDelta supports all Sprig template functions plus standard Go template functions:

Function Description Example
index Access map values with special characters {{ index .item.resource "ed.tag" }}
js JavaScript escape strings {{ js .item.signal.description }}
default Provide default values `{{ .item.signal.value
toJson Convert to JSON `{{ .item.attributes
if Conditional logic {{ if .item.signal }}...{{ end }}
range Iterate over collections {{ range $k, $v := .item.attributes }}...{{ end }}

Example: Microsoft Teams Adaptive Card

In this example, a rich payload is sent with dynamic content pulled from the alert fields.

- name: richtemplate
  type: webhook_output
  endpoint: https://myco.webhook.office.com/webhookb2/88264c11-0bb8-44fc-/IncomingWebhook/1a2b3c4d5e6f/abcdef123456
  suppression_window: 1m0s
  payload: "      {\n       \"type\":\"message\",\n       \"attachments\":[\n          {\n
    \           \"contentType\":\"application/vnd.microsoft.card.adaptive\",\n            \"contentUrl\":null,\n
    \           \"content\":{\n              \"$schema\":\"http://adaptivecards.io/schemas/adaptive-card.json\",\n
    \             \"type\":\"AdaptiveCard\",\n              \"version\":\"1.2\",\n
    \             \"body\":[\n                {\n                  \"type\": \"TextBlock\",\n
    \                 \"text\": \"**{{ index .item.signal \"title\" }}**\"\n                },\n
    \               {\n                  \"type\": \"TextBlock\",\n                  \"text\":
    \"**Tag**: {{ index .item.resource \"ed.tag\" }}\\n\\r**Host**: {{ index .item.resource
    \"host.name\" }}\\n\\r**Description**: {{ js .item.signal.description }}\\n\\r**Signal
    ID**: {{ index .item.signal \"signal_id\" }}\\n\\r**Metric name**: {{ index .item.signal
    \"name\" }}\\n\\r**Value**: {{ index .item.signal \"value\" }}\"\n                }\n
    \             ]\n            }\n          }\n        ]\n      }    "

Required Parameters

name

A descriptive name for the node. This is the name that will appear in pipeline builder and you can reference this node in the YAML using the name. It must be unique across all nodes. It is a YAML list element so it begins with a - and a space followed by the string. It is a required parameter for all nodes.

nodes:
  - name: <node name>
    type: <node type>

type: webhook_output

The type parameter specifies the type of node being configured. It is specified as a string from a closed list of node types. It is a required parameter.

nodes:
  - name: <node name>
    type: <node type>

endpoint

The endpoint parameter defines the webhook URL. It is specified as a string and is required. You may need to define egress permissions for your chosen endpoint.

nodes:
  - name: <node name>
    type: webhook_output
    endpoint: "<end point address>"
    payload: |
      <message template>      

payload

The payload parameter defines the message template. It is specified in JSON and is required.

nodes:
  - name: <node name>
    type: webhook_output
    endpoint: "<end point address>"
    payload: |
      <message JSON>      

Integration Examples

ServiceNow Incident Creation

Create incidents in ServiceNow when alerts are triggered:

- name: servicenow_incidents
  type: webhook_output
  endpoint: https://instance.service-now.com/api/now/table/incident
  headers:
    Content-Type: "application/json"
    Accept: "application/json"
    Authorization: "Basic ${SERVICENOW_CREDENTIALS}"
  suppression_window: 30m
  payload: |
    {
      "short_description": "{{ .item.signal.title }}",
      "description": "Alert: {{ .item.signal.description }}\nHost: {{ index .item.resource "host.name" }}\nSource: {{ index .item.resource "ed.source_name" }}\nCurrent Value: {{ .item.signal.value }}\nThreshold: {{ .item.signal.threshold_value }}",
      "urgency": "2",
      "impact": "2",
      "category": "Infrastructure",
      "assignment_group": "Infrastructure Team",
      "cmdb_ci": "{{ index .item.resource "host.name" }}",
      "correlation_id": "{{ .item.signal.signal_id }}",
      "work_notes": "Automated alert from EdgeDelta at {{ .item.timestamp }}"
    }    

Slack Block Kit Alert

Send rich formatted alerts to Slack using Block Kit:

- name: slack_alerts
  type: webhook_output
  endpoint: https://hooks.slack.com/services/${SLACK_WEBHOOK_PATH}
  headers:
    Content-Type: "application/json"
  suppression_window: 10m
  payload: |
    {
      "text": "EdgeDelta Alert: {{ .item.signal.title }}",
      "blocks": [
        {
          "type": "header",
          "text": {
            "type": "plain_text",
            "text": "{{ .item.signal.title }}",
            "emoji": true
          }
        },
        {
          "type": "section",
          "fields": [
            {
              "type": "mrkdwn",
              "text": "*Host:*\n{{ index .item.resource "host.name" }}"
            },
            {
              "type": "mrkdwn",
              "text": "*Source:*\n{{ index .item.resource "ed.source_name" }}"
            },
            {
              "type": "mrkdwn",
              "text": "*Current Value:*\n{{ .item.signal.value }}"
            },
            {
              "type": "mrkdwn",
              "text": "*Threshold:*\n{{ .item.signal.threshold_value }}"
            }
          ]
        },
        {
          "type": "section",
          "text": {
            "type": "mrkdwn",
            "text": "*Description:*\n{{ .item.signal.description }}"
          }
        },
        {
          "type": "context",
          "elements": [
            {
              "type": "mrkdwn",
              "text": "Signal ID: `{{ .item.signal.signal_id }}` | {{ .item.timestamp }}"
            }
          ]
        }
      ]
    }    

PagerDuty Event Integration

Trigger PagerDuty incidents from EdgeDelta alerts:

- name: pagerduty_alerts
  type: webhook_output
  endpoint: https://events.pagerduty.com/v2/enqueue
  headers:
    Content-Type: "application/json"
  suppression_window: 1h
  payload: |
    {
      "routing_key": "${PAGERDUTY_ROUTING_KEY}",
      "event_action": "trigger",
      "dedup_key": "{{ .item.signal.signal_id }}",
      "payload": {
        "summary": "{{ .item.signal.title }}",
        "severity": "error",
        "source": "{{ index .item.resource "host.name" }}",
        "component": "{{ index .item.resource "ed.source_name" }}",
        "group": "{{ index .item.resource "ed.source_type" }}",
        "class": "metric_threshold",
        "custom_details": {
          "description": "{{ .item.signal.description }}",
          "current_value": {{ .item.signal.value }},
          "threshold": {{ .item.signal.threshold_value }},
          "threshold_type": "{{ .item.signal.threshold_type }}",
          "timestamp": "{{ .item.timestamp }}"
        }
      },
      "links": [
        {
          "href": "https://app.edgedelta.com",
          "text": "View in EdgeDelta"
        }
      ]
    }    

Custom Metrics Dashboard

Send metrics to a custom dashboard API:

- name: metrics_dashboard
  type: webhook_output
  endpoint: https://api.custom-dashboard.com/v1/metrics
  headers:
    Content-Type: "application/json"
    X-API-Key: "${DASHBOARD_API_KEY}"
  payload: |
    {
      "metric_name": "{{ .item.name }}",
      "value": {{ .item.value }},
      "timestamp": "{{ .item.timestamp }}",
      "tags": {
        "host": "{{ index .item.resource "host.name" }}",
        "source": "{{ index .item.resource "ed.source_name" }}",
        "source_type": "{{ index .item.resource "ed.source_type" }}",
        "environment": "{{ index .item.attributes "environment" | default "production" }}"
      },
      "metadata": {
        "stat_type": "{{ .item.stat_type }}",
        "agent_tag": "{{ index .item.resource "ed.tag" }}"
      }
    }    

Opsgenie Alert Creation

Integrate with Opsgenie for incident management:

- name: opsgenie_alerts
  type: webhook_output
  endpoint: https://api.opsgenie.com/v2/alerts
  headers:
    Content-Type: "application/json"
    Authorization: "GenieKey ${OPSGENIE_API_KEY}"
  suppression_window: 20m
  payload: |
    {
      "message": "{{ .item.signal.title }}",
      "alias": "{{ .item.signal.signal_id }}",
      "description": "{{ .item.signal.description }}\n\nHost: {{ index .item.resource "host.name" }}\nSource: {{ index .item.resource "ed.source_name" }}\nValue: {{ .item.signal.value }}\nThreshold: {{ .item.signal.threshold_value }}",
      "tags": [
        "EdgeDelta",
        "{{ index .item.resource "ed.source_type" }}",
        "{{ index .item.resource "host.name" }}"
      ],
      "details": {
        "signal_id": "{{ .item.signal.signal_id }}",
        "metric_name": "{{ .item.signal.name }}",
        "current_value": "{{ .item.signal.value }}",
        "threshold": "{{ .item.signal.threshold_value }}",
        "threshold_type": "{{ .item.signal.threshold_type }}",
        "timestamp": "{{ .item.timestamp }}"
      },
      "priority": "P2"
    }    

Best Practices

Handling Special Characters

When field names contain special characters (dots, hyphens, etc.), use the index function:

# Correct - using index for fields with dots
{{ index .item.resource "ed.tag" }}

# Incorrect - will fail
{{ .item.resource.ed.tag }}

Escaping JSON Values

Use appropriate escaping for string values in JSON:

payload: |
  {
    "description": "{{ js .item.signal.description }}",  # JavaScript escape for JSON strings
    "value": {{ .item.signal.value }},                    # Numbers don't need quotes
    "is_alert": {{ if .item.signal }}true{{ else }}false{{ end }}  # Booleans without quotes
  }  

Default Values

Provide defaults for optional fields to prevent template errors:

"environment": "{{ index .item.attributes "environment" | default "production" }}",
"priority": "{{ index .item.attributes "priority" | default "P3" }}"

Suppression Windows

  • Alerts: Use suppression windows to prevent notification flooding
  • Metrics: Suppression is not applied to metric webhooks
  • Guideline: Start with longer windows (20-30m) and adjust based on alert frequency
suppression_window: 20m  # Prevents duplicate alerts for 20 minutes

Using Environment Variables

Store sensitive data like API keys in environment variables:

headers:
  Authorization: "Bearer ${API_TOKEN}"  # Resolved from environment

Secure Testing Practices

Local Testing Methods

  1. Mock Server Approach: Use local mock servers to test webhook payloads

    # Using Python's built-in HTTP server to log requests
    python3 -m http.server 8080
    
    # Or use a dedicated mock server like WireMock
    docker run -p 8080:8080 wiremock/wiremock
    
  2. Docker-Based Testing: Create isolated test environments

    # docker-compose.yml for webhook testing
    services:
      webhook-receiver:
        image: mendhak/http-https-echo:latest
        ports:
          - "8080:8080"
    
  3. EdgeDelta Dry Run: Use EdgeDelta’s configuration validation

    edgedelta validate -c your-config.yaml
    

Testing with Mock Data

Always use synthetic test data, never production data:

# test-webhook.yaml - Safe test configuration
- name: test_webhook
  type: webhook_output
  endpoint: http://localhost:8080/webhook
  payload: |
    {
      "test": true,
      "alert": "SYNTHETIC_TEST_ALERT",
      "host": "test-host-001",
      "value": 99.9
    }    

Secure Tunneling (When External Access Required)

If you must expose a local endpoint for integration testing:

  1. Self-Hosted Solutions (Most Secure):

    • Cloudflare Tunnel: Enterprise-grade security with zero-trust access
    • Zrok (GitHub): Open-source, built on OpenZiti zero-trust network
    • Expose (GitHub): Open-source PHP-based tunnel, self-hostable
  2. Temporary Tunnels (Use with Caution):

    # Using Cloudflare Tunnel (recommended)
    cloudflared tunnel --url http://localhost:8080
    
    # Using Zrok (self-hosted option)
    zrok share public http://localhost:8080
    
    # Always authenticate and limit access
    

Security Checklist

  • ✅ Never use production credentials in test configurations
  • ✅ Always use mock/synthetic data for testing
  • ✅ Test locally first before any external exposure
  • ✅ Use environment variables for all sensitive data
  • ✅ Implement webhook signature verification when available
  • ✅ Log and monitor all webhook attempts
  • ✅ Use short-lived test endpoints that auto-expire
  • ✅ When using tunneling tools, prefer self-hosted solutions for sensitive environments

Troubleshooting

Common Issues

  1. Template parsing errors: Check for unescaped quotes in JSON strings
  2. Missing field errors: Use index function for fields with special characters
  3. Invalid JSON: Ensure proper escaping and no trailing commas
  4. Authentication failures: Verify environment variables are set correctly
  5. No webhook triggered: Check suppression window settings

Debug Tips

  • Enable trace logging to see rendered payloads
  • Test templates with sample data first
  • Use default values to handle missing fields gracefully
  • Validate webhook endpoints are accessible from EdgeDelta agents

Optional Parameters

headers

The headers parameter defines a set of key-value pairs for the request headers. For example Content-Type: "application/json".It is specified as a yaml.MapSlice and is optional.

nodes:
  - name: <node name>
    type: webhook_output
    endpoint: "<end point address>"
    headers:
      <key>: "<value>"
    payload: |
      <message JSON>