Edge Delta Webhook Destination

Configure the Edge Delta Webhook Destination to send alerts and metrics using customizable JSON payloads with Go template variables.

  8 minute read  

Overview

The Webhook destination node sends alerts and metrics to a webhook endpoint using customizable payloads with Go template variables.

Note: This destination does not support log data. To send logs to a webhook endpoint, use the HTTP destination instead.

This node requires Edge Delta agent version v0.1.72 or higher.

Supported Data Types:

  • Signals: Alerts generated by a threshold node on the edge, or triggered by a monitor via a webhook integration
  • Metrics: Any metric data routed directly to this destination from metric-producing nodes

Key Features:

  • Customizable JSON payloads using Go templates
  • Suppression windows for alerts (not applied to metrics)
  • Integration with any webhook-compatible service

See Send Data to a Webhook for detailed alert configuration instructions.

Integration Guides

GuideDescription
Send Alerts to Microsoft TeamsSend alerts to Teams using Adaptive Cards
Send Alerts to SlackSlack destination and webhook with Block Kit
Send Alerts to ServiceNowCreate ServiceNow incidents from alerts
Send Alerts to PagerDutyTrigger PagerDuty incidents using Events API v2
Send Alerts to OpsgenieCreate Opsgenie alerts for incident management
Send Metrics to a WebhookExtract metrics from logs and send to a webhook
Send Metrics to a Custom APISend metrics to custom dashboard APIs

Required Parameters

name

A descriptive name for the node. This is the name that will appear in pipeline builder and you can reference this node in the YAML using the name. It must be unique across all nodes. It is a YAML list element so it begins with a - and a space followed by the string. It is a required parameter for all nodes.

nodes:
  - name: <node name>
    type: <node type>

type: webhook_output

The type parameter specifies the type of node being configured. It is specified as a string from a closed list of node types. It is a required parameter.

nodes:
  - name: <node name>
    type: <node type>

endpoint

The endpoint parameter defines the webhook URL. It is specified as a string and is required. You may need to define egress permissions for your chosen endpoint.

nodes:
  - name: <node name>
    type: webhook_output
    endpoint: "<end point address>"
    payload: |
      <message template>      

payload

The payload parameter defines the message template. It is specified in JSON and is required.

nodes:
  - name: <node name>
    type: webhook_output
    endpoint: "<end point address>"
    payload: |
      <message JSON>      

Optional Parameters

headers

The headers parameter defines HTTP headers to include with the webhook request. It is specified as an array of header objects, each containing a header name and value.

nodes:
  - name: <node name>
    type: webhook_output
    endpoint: "<end point address>"
    headers:
    - header: Content-Type
      value: "application/json"
    - header: Authorization
      value: "Bearer ${API_TOKEN}"
    payload: |
      <message JSON>      

Template Variables Reference

The webhook node uses Go templates to create dynamic payloads. Variables are accessed through the .item object, which contains different fields depending on whether it’s an alert/signal or a metric.

Alert/Signal Variables

When processing alerts, the following variables are available:

Signal Fields (.item.signal)

VariableDescriptionExample Usage
titleAlert title{{ .item.signal.title }}
descriptionAlert description{{ .item.signal.description }}
signal_idUnique signal identifier{{ .item.signal.signal_id }}
nameMetric name that triggered the alert{{ .item.signal.name }}
valueCurrent metric value{{ .item.signal.value }}
threshold_typeType of threshold (e.g., “above”, “below”){{ .item.signal.threshold_type }}
threshold_valueConfigured threshold value{{ .item.signal.threshold_value }}
capture_flush_modeCapture mode for the alert{{ .item.signal.capture_flush_mode }}
capture_sizeSize of captured data{{ .item.signal.capture_size }}

Resource Fields (.item.resource)

VariableDescriptionExample Usage
ed.tagEdgeDelta agent tag{{ index .item.resource "ed.tag" }}
ed.source_nameSource name{{ index .item.resource "ed.source_name" }}
ed.source_typeSource type (e.g., “file”, “k8s”){{ index .item.resource "ed.source_type" }}
host.nameHost name{{ index .item.resource "host.name" }}
custom attributesAny custom resource attributes{{ index .item.resource "custom_key" }}

Additional Fields

  • Attributes (.item.attributes): Custom tags and labels as key-value pairs
  • Timestamp (.item.timestamp): ISO 8601 formatted timestamp

Metric Variables

When processing metrics, the following variables are available based on metric type:

VariableDescriptionExample Usage
.item.nameMetric name{{ .item.name }}
.item._stat_typeMetric type (gauge, sum, histogram){{ .item._stat_type }}
.item.resourceSame resource fields as alerts{{ index .item.resource "host.name" }}
.item.attributesMetric-specific attributes{{ index .item.attributes "environment" }}
.item.timestampMetric timestamp{{ .item.timestamp }}

Gauge Metrics:

VariableDescriptionExample Usage
.item.gauge.valueCurrent gauge value{{ .item.gauge.value }}

Sum Metrics:

VariableDescriptionExample Usage
.item.sum.value.sumCumulative sum value{{ .item.sum.value.sum }}
.item.sum.is_monotonicWhether sum is monotonic{{ .item.sum.is_monotonic }}

Histogram Metrics:

VariableDescriptionExample Usage
.item.histogram.countsArray of bucket counts{{ .item.histogram.counts }}
.item.histogram.minMinimum value{{ .item.histogram.min }}
.item.histogram.maxMaximum value{{ .item.histogram.max }}
.item.histogram.sumSum of all values{{ .item.histogram.sum }}
.item.histogram.countTotal count of values{{ .item.histogram.count }}

Template Functions

EdgeDelta supports all Sprig template functions plus standard Go template functions:

FunctionDescriptionExample
indexAccess map values with special characters{{ index .item.resource "ed.tag" }}
jsJavaScript escape strings{{ js .item.signal.description }}
defaultProvide default values`{{ .item.signal.value
toJsonConvert to JSON`{{ .item.attributes
ifConditional logic{{ if .item.signal }}...{{ end }}
rangeIterate over collections{{ range $k, $v := .item.attributes }}...{{ end }}

Routing Data to a Webhook

Routing Signals (Alerts)

Signals are generated when a threshold condition is met. To route signals to a webhook:

  1. Add a threshold node to your pipeline
  2. Connect the threshold node’s output to the webhook destination
  3. Configure the payload using signal template variables

See Send Data to a Webhook for step-by-step instructions.

Routing Metrics

To send metric data to a webhook, connect any metric-producing node directly to the webhook destination:

nodes:
  - name: system_metrics
    type: ed_system_stats_input

  - name: metrics_webhook
    type: webhook_output
    endpoint: https://api.example.com/metrics
    headers:
    - header: Content-Type
      value: "application/json"
    - header: Authorization
      value: "Bearer ${API_TOKEN}"
    payload: |
      {
        "metric": "{{ .item.name }}",
        "value": {{ if eq .item._stat_type "gauge" }}{{ .item.gauge.value }}{{ else if eq .item._stat_type "sum" }}{{ .item.sum.value.sum }}{{ else }}0{{ end }},
        "host": "{{ index .item.resource "host.name" }}",
        "timestamp": "{{ .item.timestamp }}"
      }      

links:
  - from: system_metrics
    to: metrics_webhook

Metrics do not use suppression windows—each metric data point is sent individually.

See Send Metrics to a Webhook for a complete example of extracting metrics from logs and sending them to a webhook.

Best Practices

Handling Special Characters

When field names contain special characters (dots, hyphens, etc.), use the index function:

# Correct - using index for fields with dots
{{ index .item.resource "ed.tag" }}

# Incorrect - will fail
{{ .item.resource.ed.tag }}

Escaping JSON Values

Use appropriate escaping for string values in JSON:

payload: |
  {
    "description": "{{ js .item.signal.description }}",  # JavaScript escape for JSON strings
    "value": {{ .item.signal.value }},                    # Numbers don't need quotes
    "is_alert": {{ if .item.signal }}true{{ else }}false{{ end }}  # Booleans without quotes
  }  

Default Values

Provide defaults for optional fields to prevent template errors:

"environment": "{{ index .item.attributes "environment" | default "production" }}",
"priority": "{{ index .item.attributes "priority" | default "P3" }}"

Suppression Windows

  • Alerts: Use suppression windows to prevent notification flooding
  • Metrics: Suppression is not applied to metric webhooks
  • Guideline: Start with longer windows (20-30m) and adjust based on alert frequency
suppression_window: 20m  # Prevents duplicate alerts for 20 minutes

Using Environment Variables

Store sensitive data like API keys in environment variables:

headers:
- header: Authorization
  value: "Bearer ${API_TOKEN}"  # Resolved from environment

Secure Testing Practices

Local Testing Methods

  1. Mock Server Approach: Use local mock servers to test webhook payloads

    # Using Python's built-in HTTP server to log requests
    python3 -m http.server 8080
    
    # Or use a dedicated mock server like WireMock
    docker run -p 8080:8080 wiremock/wiremock
    
  2. Docker-Based Testing: Create isolated test environments

    # docker-compose.yml for webhook testing
    services:
      webhook-receiver:
        image: mendhak/http-https-echo:latest
        ports:
          - "8080:8080"
    
  3. EdgeDelta Dry Run: Use EdgeDelta’s configuration validation

    edgedelta validate -c your-config.yaml
    

Testing with Mock Data

Always use synthetic test data, never production data:

# test-webhook.yaml - Safe test configuration
- name: test_webhook
  type: webhook_output
  endpoint: http://localhost:8080/webhook
  payload: |
    {
      "test": true,
      "alert": "SYNTHETIC_TEST_ALERT",
      "host": "test-host-001",
      "value": 99.9
    }    

Secure Tunneling (When External Access Required)

If you must expose a local endpoint for integration testing:

  1. Self-Hosted Solutions (Most Secure):

  2. Temporary Tunnels (Use with Caution):

    # Using Cloudflare Tunnel (recommended)
    cloudflared tunnel --url http://localhost:8080
    
    # Using Zrok (self-hosted option)
    zrok share public http://localhost:8080
    
    # Always authenticate and limit access
    

Security Checklist

  • ✅ Never use production credentials in test configurations
  • ✅ Always use mock/synthetic data for testing
  • ✅ Test locally first before any external exposure
  • ✅ Use environment variables for all sensitive data
  • ✅ Implement webhook signature verification when available
  • ✅ Log and monitor all webhook attempts
  • ✅ Use short-lived test endpoints that auto-expire
  • ✅ When using tunneling tools, prefer self-hosted solutions for sensitive environments

Troubleshooting

Common Issues

  1. Template parsing errors: Check for unescaped quotes in JSON strings
  2. Missing field errors: Use index function for fields with special characters
  3. Invalid JSON: Ensure proper escaping and no trailing commas
  4. Authentication failures: Verify environment variables are set correctly
  5. No webhook triggered: Check suppression window settings

Debug Tips

  • Enable trace logging to see rendered payloads
  • Test templates with sample data first
  • Use default values to handle missing fields gracefully
  • Validate webhook endpoints are accessible from EdgeDelta agents