Send Alerts to PagerDuty

Configure Edge Delta to trigger PagerDuty incidents using the Events API v2 for on-call alerting and incident management.

4 minute read

Overview

This guide demonstrates how to trigger PagerDuty incidents from EdgeDelta alerts using the Events API v2. PagerDuty’s Events API accepts machine-generated events and routes them to the appropriate on-call responders.

Prerequisites

PagerDuty Account: Admin access to create integrations
Integration Key: A routing key from a PagerDuty service
EdgeDelta Pipeline: A configured threshold node or monitor generating signals

Pipeline Flow

flowchart LR classDef icon-metrics fill:#E7EEFC,stroke:#2563EB,color:#0F2169; classDef icon-alert fill:#FDEAD7,stroke:#EA580C,color:#7C2D12; classDef icon-destination fill:#FCEADB,stroke:#EA580C,color:#7C2D12; classDef icon-notify fill:#FCE7E7,stroke:#DC2626,color:#7C1D1D; Metrics[" Metric Source"] Threshold[" Threshold Node"] Webhook[" Webhook Destination"] PagerDuty[" PagerDuty"] Metrics --> Threshold Threshold --> Webhook Webhook --> PagerDuty class Metrics icon-metrics; class Threshold icon-alert; class Webhook icon-destination; class PagerDuty icon-notify;

Diagram preview unavailable

View diagram source

flowchart LR
classDef icon-metrics fill:#E7EEFC,stroke:#2563EB,color:#0F2169;
classDef icon-alert fill:#FDEAD7,stroke:#EA580C,color:#7C2D12;
classDef icon-destination fill:#FCEADB,stroke:#EA580C,color:#7C2D12;
classDef icon-notify fill:#FCE7E7,stroke:#DC2626,color:#7C1D1D;
    Metrics["<span class='ph ph-chart-line-up'></span> Metric Source"]
    Threshold["<span class='ph ph-funnel'></span> Threshold Node"]
    Webhook["<span class='ph ph-paper-plane-tilt'></span> Webhook Destination"]
    PagerDuty["<span class='ph ph-bell-ringing'></span> PagerDuty"]
    Metrics --> Threshold
    Threshold --> Webhook
    Webhook --> PagerDuty
class Metrics icon-metrics;
class Threshold icon-alert;
class Webhook icon-destination;
class PagerDuty icon-notify;

Configuration

Step 1: Create a PagerDuty Integration

Log in to your PagerDuty account
Navigate to Services → Service Directory
Select an existing service or create a new one
Go to the Integrations tab
Click Add Integration
Select Events API v2
Copy the Integration Key (32-character routing key)

Step 2: Store the Routing Key

Store the routing key as an environment variable:

export PAGERDUTY_ROUTING_KEY="your-32-character-routing-key"

Step 3: Configure the Webhook Node

Create a webhook node that sends events to PagerDuty:

- name: pagerduty_alerts
  type: webhook_output
  endpoint: https://events.pagerduty.com/v2/enqueue
  headers:
  - header: Content-Type
    value: "application/json"
  suppression_window: 1h
  payload: |
    {
      "routing_key": "${PAGERDUTY_ROUTING_KEY}",
      "event_action": "trigger",
      "dedup_key": "{{ .item.signal.signal_id }}",
      "payload": {
        "summary": "{{ .item.signal.title }}",
        "severity": "error",
        "source": "{{ index .item.resource \"host.name\" }}",
        "component": "{{ index .item.resource \"ed.source_name\" }}",
        "group": "{{ index .item.resource \"ed.source_type\" }}",
        "class": "metric_threshold",
        "custom_details": {
          "description": "{{ .item.signal.description }}",
          "current_value": {{ .item.signal.value }},
          "threshold": {{ .item.signal.threshold_value }},
          "threshold_type": "{{ .item.signal.threshold_type }}",
          "timestamp": "{{ .item.timestamp }}",
          "agent_tag": "{{ index .item.resource \"ed.tag\" }}"
        }
      },
      "links": [
        {
          "href": "https://app.edgedelta.com",
          "text": "View in EdgeDelta"
        }
      ]
    }

Step 4: Connect to a Threshold Node

nodes:
  - name: cpu_threshold
    type: threshold
    filter: item.name == "system.cpu.usage"
    condition: value > 90

  - name: pagerduty_alerts
    type: webhook_output
    # ... configuration from above ...

links:
  - from: cpu_threshold
    to: pagerduty_alerts

Events API v2 Payload Reference

Required Fields

Field	Description	Example
`routing_key`	Integration key from PagerDuty service	`${PAGERDUTY_ROUTING_KEY}`
`event_action`	Action type: `trigger`, `acknowledge`, `resolve`	`trigger`
`payload.summary`	Brief description of the event	Alert title
`payload.severity`	Event severity: `critical`, `error`, `warning`, `info`	`error`
`payload.source`	The affected host or resource	Host name

Optional Fields

Field	Description	Usage
`dedup_key`	Unique identifier for deduplication	Signal ID prevents duplicates
`payload.component`	Affected component or service	Source name
`payload.group`	Logical grouping of the source	Source type
`payload.class`	Event class/type	`metric_threshold`
`payload.custom_details`	Additional context as key-value pairs	Metric details
`links`	URLs related to the event	EdgeDelta dashboard link
`images`	Images to display in the incident	Dashboard screenshots

Dynamic Severity Mapping

Map alert severity based on how far the metric exceeds its threshold:

payload: |
  {
    "routing_key": "${PAGERDUTY_ROUTING_KEY}",
    "event_action": "trigger",
    "dedup_key": "{{ .item.signal.signal_id }}",
    "payload": {
      "summary": "{{ .item.signal.title }}",
      "severity": "{{ if gt .item.signal.value (mul .item.signal.threshold_value 2) }}critical{{ else if gt .item.signal.value .item.signal.threshold_value }}error{{ else }}warning{{ end }}",
      "source": "{{ index .item.resource \"host.name\" }}",
      ...
    }
  }

Severity mapping:

critical: Value exceeds 2x the threshold
error: Value exceeds the threshold
warning: Value approaching threshold (if applicable)

Auto-Resolution

Create a separate webhook to automatically resolve incidents when metrics return to normal. This requires a monitor or threshold configuration that can detect recovery.

- name: pagerduty_resolve
  type: webhook_output
  endpoint: https://events.pagerduty.com/v2/enqueue
  headers:
  - header: Content-Type
    value: "application/json"
  payload: |
    {
      "routing_key": "${PAGERDUTY_ROUTING_KEY}",
      "event_action": "resolve",
      "dedup_key": "{{ .item.signal.signal_id }}"
    }

Note: The dedup_key must match the original trigger event for resolution to work.

Adding Images

Include dashboard screenshots or charts in the incident:

payload: |
  {
    ...
    "images": [
      {
        "src": "https://your-dashboard.com/chart?metric={{ .item.signal.name }}&host={{ index .item.resource \"host.name\" }}",
        "alt": "Metric Chart",
        "href": "https://app.edgedelta.com"
      }
    ]
  }

Testing

Test with PagerDuty’s Event Test Tool

Go to Services → Select your service → Integrations
Click on your Events API v2 integration
Use the Test button to verify connectivity

Test with a Local Endpoint

Verify your payload structure before connecting to PagerDuty:

endpoint: http://localhost:8080/webhook-test

Expected Response

Successful triggers return:

{
  "status": "success",
  "message": "Event processed",
  "dedup_key": "your-dedup-key"
}

Troubleshooting

Issue	Solution
`Invalid routing key`	Verify the integration key from PagerDuty
`Invalid JSON`	Check payload syntax and escaping
`Event rejected`	Ensure required fields are present
Duplicate incidents	Use consistent `dedup_key` values
Incidents not resolving	Verify `dedup_key` matches original trigger

Best Practices

Use deduplication keys: Prevent duplicate incidents with signal IDs
Set appropriate severity: Map metric values to meaningful severity levels
Include context: Custom details help responders understand the issue
Add links: Direct links to EdgeDelta reduce investigation time
Configure suppression: 1-hour windows work well for most infrastructure alerts
Test integration: Use PagerDuty’s test tools before production deployment