Recommendations

AI-powered recommendations help optimize your telemetry pipelines by analyzing patterns and suggesting relevant actions like filtering, sampling, or masking data.

  4 minute read  

Edge Delta’s Recommendations feature provides AI-powered insights to help you optimize your telemetry pipelines efficiently. The system analyzes your log patterns and pipeline configurations to suggest the most impactful processors, helping you reduce costs, improve data hygiene, and implement best practices.

How Recommendations Work

Recommendations are generated from two primary sources:

  1. Pattern Analysis: Edge Delta analyzes log patterns that represent at least 1% of traffic from each source node. These patterns are evaluated by AI to determine their value, with appropriate actions suggested based on the score.

Note: Only pattern data is sent to AI for analysis—your actual log content remains secure. The AI evaluates patterns to determine their importance and suggests appropriate pipeline actions based on verbosity and value assessment.

  1. Real-time Pipeline Analysis: While editing pipelines in the multiprocessor view, Edge Delta provides contextual recommendations based on detected source data characteristics.

Prerequisites

To receive pattern-based recommendations, your pipeline must include a Log to Pattern processor. This processor populates the Patterns page and drives pattern-based recommendation generation. Without this processor, you’ll only receive real-time multiprocessor recommendations.

The Recommendations Page

The Recommendations page serves as your central hub for all pipeline recommendations. Click Recommendations in the Navigation bar.

The recommendations feed displays all available recommendations with clear context including category, type, and value. The list is sortable (defaulting to value-based sorting for highest impact first) and includes robust filter capabilities in the Filters pane, along with search functionality to quickly find specific recommendations. Each recommendation provides a detailed preview of its impact, allowing you to evaluate changes before implementation.

Recommendations progress through four distinct states:

  1. Fresh recommendations are marked with a blue indicator dot
  2. Recommendations you’ve viewed but haven’t acted upon are listed in the Active tab.
  3. Recommendations you’ve implemented by creating processors are listed in the Configured tab.
  4. Recommendations you’ve chosen not to implement are listed on the Dismissed tab.

Recommendations remain in the Active state for up to 7 days before expiring. You can still accept dismissed recommendations if you change your mind.

Filter Pane

Filter recommendations using the left-side Filters pane by:

Categories

  • Cost Saving: High-impact recommendations for reducing data volume and costs
  • Data Hygiene: Suggestions for parsing JSON/XML, deduplicating logs, or removing empty fields
  • Best Practice: Implementation of recommended processors like Log to Pattern
  • Privacy: PII masking and data security recommendations

Types

  • Filter Out: Remove low-value data entirely
  • Sample: Reduce data volume while maintaining visibility
  • Mask: Protect sensitive information
  • Parse: Structure unstructured data for better analysis

Additional Filters

  • Pipeline: Filter by specific pipeline
  • Source: Filter by data source
  • State: View by recommendation state (New, Active, Configured, Dismissed)

Implement a Recommendation

To implement a recommendation:

  1. Navigate to the Recommendations page
  2. Review recommendations sorted by value (highest impact first)
  3. Use filters to focus on specific categories or types
  4. Click on a recommendation to view its details

You can

  • Configure: Implement the recommendation by creating the suggested processor
  • Dismiss: Remove from active view if not relevant to your use case
  • Review Later: Leave in Active state to revisit within 7 days
  1. Click Create [Processor Type] Processor to automatically:
    • Open the relevant pipeline in edit mode
    • Add the configured processor to the appropriate source node
    • Preview the effect using Live Capture
  1. Save and deploy if satisfied with the results

Log Patterns Page Recommendations

The Logs page’s Patterns tab includes a Recommendations column for patterns meeting the analysis threshold:

  1. Click Logs and select the Patterns tab.
  1. Look for the recommendation indicator icon next to qualifying patterns
  2. Click the icon to view the recommendation popup
  1. Click the popup to automatically:
    • Open the pipeline editor
    • Add the recommended processor to the source multiprocessor
    • Configure it with suggested settings
  1. Review and save changes

Pipeline Multiprocessor Recommendations

While editing pipelines, the multiprocessor view suggests relevant processors based on your data:

  1. Open a pipeline for editing
  2. Select a source node’s multiprocessor
  3. View recommended processors in the center processor pane
  1. Click to add and configure suggested processors
  2. Use Live Capture preview to validate changes
  3. Save and deploy when ready

Best Practices

  1. Start with High-Value Recommendations:

Focus on Cost Saving category items first for maximum impact. Recommendations are automatically sorted by value to prioritize the most impactful optimizations. Higher-value recommendations typically appear in the Cost Saving category and represent opportunities for significant telemetry optimization. The value calculation considers:

  • Potential cost savings from volume reduction
  • Data quality improvements
  • Security and compliance benefits
  • Operational efficiency gains
  1. Add Log to Pattern Processors: Enable pattern-based recommendations by ensuring pipelines include this processor
  2. Review Regularly: Check for new recommendations weekly as your data patterns evolve
  3. Test Before Deploying: Always use Live Capture preview to validate recommendation impact
  4. Consider Context: While AI suggestions are valuable, evaluate them against your specific observability requirements
  5. Batch Similar Changes: Group related recommendations for coordinated pipeline updates

Next Steps