Send data to Datadog from Edge Delta

Prepare to send logs to Datadog.

  2 minute read  

Overview

The Datadog destination node send items to a Datadog destination. First you need to configure Datadog.

Configure Datadog

Create Measure Facets

You need to create Measures for sentiment_score and pattern_count for the pattern analysis panels. To create a facet:

  1. Search for @sentiment_score:* to find all negative events.
  2. Click any event, in the JSON section.
  3. Click sentiment_score and select Create Measure for @sentiment_score.
  4. Repeat the process for pattern_count.

See Measure Facets on the Datadog docs website. Alternatively, you can create processors to convert the sentiment_score and pattern_count fields from strings to integers.

If you add the include_pattern_info_in_samples: true parameter to the cluster processor, it will replace all the cluster_samples with patterns. You will need to also turn off the cluster pattern feature and turn on the cluster_sample feature. If you add the include_pattern_info_in_samples: true parameter you can skip the following steps: Create a Pipeline and Add a Grok Parser.

Create a Pipeline

Create a pipeline for @pattern_count.

  1. Click log - configuration.
  2. Add a new pipeline with filter @pattern_count:* to search patterns.
  3. Name the pipeline pattern.

See Create a pipeline on the Datadog docs website.

Add a Grok Parser

Add a Grok Parser processor with the following rule:

autoFilledRule1 %{regex(".*"):pattern}.*

Add a processor with the following attributes:

  • type: Grok Parser
  • log sample: * test
  • parsing rule: autoFilledRule1 %{regex(".*"):pattern}.*

In a log search, select edgedelta_datatype:cluster_pattern then click on a pattern. View the pattern event attribute, and click pattern - add as a facet.

See Add a Grok Parser on the Datadog docs website.

Configure the Edge Delta Agent

Finally, you configure the Datadog destination node using Visual Pipelines or the agent YAML configuration file. You can select an Existing Datadog Integration when creating a Datadog destination node. See the Datadog docs for the endpoints.

You can consider adding a key-value pair to the Datadog integration to send an identifiable attribute such as integration_name: edgedelta. This helps to to easily identify and isolate Edge Delta data so that you can create facets.