Effectively Using Facets

Best practices for using facets to improve search performance and data structure across logs, metrics, and traces in Edge Delta.

  5 minute read  

Overview

A facet in Edge Delta is a field or attribute used to filter, group, or aggregate observability data across all telemetry types - logs, metrics, and traces. As data volumes grow exponentially with microservices and cloud-native architectures, facets become critical for maintaining fast and efficient search performance. This guide explains how to effectively implement facets to structure your data and improve search capabilities across your entire observability stack.

Edge Delta processes unstructured and semi-structured data from various sources. While the Telemetry Pipeline can handle any data format, searching through unstructured data presents challenges for user experience and performance. Facets solve this problem by creating indexed fields that enable rapid filtering and aggregation.

Edge Delta provides out-of-the-box facets for standard technologies including Kubernetes, AWS ECS, and Docker. These facets are automatically indexed to improve search performance. However, custom facets specific to your data patterns provide even greater value.

Practical Implementation Examples

Structuring Unstructured Security Logs

Consider unstructured logs from security devices like Palo Alto firewalls. The raw log body contains valuable information but lacks structure for efficient searching.

Apply a Processing Pack designed for Palo Alto logs to parse the unstructured data. The pack extracts fields like source IP, destination IP, action taken, and threat severity. Each extracted field becomes an attribute that you can convert to a facet.

Once structured, create facets for critical security fields:

  • source_ip for tracking traffic origins
  • threat_severity for prioritizing security events
  • action for filtering by firewall decisions

These facets enable rapid security investigations and threat hunting across millions of log entries.

Creating Targeted Facets for Unique Identifiers

Not all use cases require complete log parsing. When tracking specific transactions or user sessions, extract only the unique identifier fields.

Use OTTL to find patterns like UUIDs or transaction IDs within log bodies:

- type: ottl_transform
  statements: |
    set(attributes["transaction_id"], ExtractPatterns(body, "(?i)txn-[0-9a-f]{8}-[0-9a-f]{4}")) 
    where body matches ".*txn-.*"

This creates a transaction_id attribute without parsing the entire log structure. Convert this attribute to a facet for instant transaction lookup across your entire log volume.

Performance Optimization

Facet Selection Best Practices

Choosing which fields to convert into facets requires balancing search capabilities with system performance. Not every field should become a facet, particularly high-cardinality fields containing millions of unique values that can degrade search performance and increase indexing overhead.

The most effective facets are those that provide meaningful filtering capabilities aligned with your operational use cases. Fields that appear frequently in search queries deserve priority for faceting, as they directly impact user productivity. Consider how your teams actually search for data - if engineers constantly filter by service name, environment, or error type, these fields should be faceted. Conversely, fields like timestamps or unique request IDs, despite their presence in every log, make poor facet candidates due to their extreme cardinality.

Cardinality plays a crucial role in facet performance. Fields with thousands of unique values typically perform well as facets, while those with millions of values can slow down the entire search infrastructure. For example, a customer_id field in a multi-tenant application with millions of customers would create performance challenges as a facet. Instead, consider creating derived facets like customer_tier or customer_region that provide filtering capabilities with manageable cardinality.

Aggregation requirements also influence facet selection. Fields that appear in dashboards, power monitoring rules, or feed into reports gain additional value from faceting. The indexed nature of facets accelerates aggregation operations, making real-time dashboards more responsive and reducing the computational load of continuous monitors.

Managing Facet Growth

Observability environments naturally accumulate facets over time as new services deploy, additional attributes get logged, and monitoring requirements evolve. Without active management, facet proliferation can erode the performance benefits they’re meant to provide.

Regular review of facet usage patterns helps identify optimization opportunities. Edge Delta’s analytics show which facets are actively used versus those that consume resources without providing value. Unused facets that haven’t been queried in weeks or months should be removed to reduce indexing overhead and improve overall system performance. This cleanup process should become part of your regular observability maintenance routine.

Consolidation strategies can significantly reduce facet count while maintaining functionality. When you notice multiple facets serving similar purposes, consider combining them into a single, more versatile facet. For instance, rather than maintaining separate facets for error_code, error_type, and error_message, you might create a single error_category facet that captures the essential information for filtering and grouping. This consolidation requires updating your parsing logic to map multiple source fields to the unified facet, but the performance gains justify the effort.

Hierarchical faceting provides another approach to managing growth while preserving granularity. Instead of creating flat facets for every possible field combination, establish parent-child relationships that reflect your data’s natural structure. A hierarchical approach might use service.name as a parent facet with service.version and service.instance as child facets. This structure allows broad filtering at the service level while still enabling detailed analysis when needed, without the overhead of indexing every possible combination as separate facets.

Facets in Edge Delta

Monitors leverage facets for efficient threshold detection across all data types. Configure monitors to trigger based on facet values from logs, metrics, or traces, reducing the computation needed to evaluate conditions across large data volumes.

Dashboard variables use facets to create interactive visualizations. Users can filter dashboard data by selecting facet values, providing drill-down capabilities without complex queries. Facets work consistently whether visualizing logs, metrics, or trace data.

The Metrics Explorer uses facets as dimensions for grouping and aggregating metric data. Faceted fields enable multi-dimensional analysis, allowing you to slice metrics by service, environment, or any other indexed attribute.

In the Trace Explorer, facets enable filtering traces by service name, operation, status codes, or custom attributes. This makes it easier to identify problematic traces in distributed systems.

Combine facets with Edge Delta’s log-to-metrics capabilities to generate time-series data from structured logs. Faceted fields become dimensions in the resulting metrics, enabling historical analysis and trend detection.

Next Steps

After implementing facets, enhance your observability practice by:

  • Creating monitors that alert on facet-based conditions for faster incident detection
  • Building dashboards that leverage facets for interactive data exploration
  • Implementing log-to-metrics conversions for historical trend analysis
  • Configuring routing rules based on facet values to optimize data flow