Troubleshooting and Diagnostics
3 minute read
Overview
Edge Delta provides comprehensive debugging and diagnostic capabilities that help you identify and resolve issues across your pipelines and agents. From live data inspection to detailed agent logs, these tools enable rapid troubleshooting without disrupting production workloads.
Live Debugging
Edge Delta provides in-stream debugging capabilities without disrupting production. Live Capture allows you to:
- Capture live data matching specific criteria
- Inspect event structure and attributes
- Validate filter and processor logic
- Debug issues without log aggregation
Live Capture is particularly useful for:
- Validating that processors transform data correctly
- Confirming that filters match expected patterns
- Debugging parsing issues with new log formats
- Testing pipeline changes before deployment
Learn more: Live Capture In-Stream Debugging
Agent Logs
Access detailed agent logs for troubleshooting. Log levels include:
| Level | Description |
|---|---|
| INFO | Normal operational messages |
| WARN | Potential issues or degraded performance |
| ERROR | Failed operations requiring attention |
| DEBUG | Verbose logging during deep troubleshooting |
Accessing Agent Logs
Access agent logs using the following commands:
Kubernetes:
kubectl logs -n edgedelta <agent-pod-name>
Docker:
docker logs <agent-container-id>
Systemd:
journalctl -u edgedelta-agent
Common Troubleshooting Scenarios
Agent Connectivity Issues
When an agent cannot connect to the backend:
- Verify network connectivity to Edge Delta endpoints
- Check firewall rules and proxy configuration
- Validate API key configuration
- Review agent logs for specific error messages
High Resource Usage
When agents consume excessive CPU or memory:
- Review pipeline complexity and processor chains
- Check for high-cardinality metric extraction
- Consider disabling resource-intensive features like eBPF sources
- Profile the agent using pprof
Destination Connection Failures
When data is not reaching destinations:
- Verify destination credentials and endpoints
- Check network connectivity to destination
- Review circuit breaker status
- Inspect agent logs for retry or error messages
Configuration Errors
When pipeline configuration fails to apply:
- Validate YAML syntax
- Check for required fields in node configurations
- Review agent logs for specific validation errors
- Use the pipeline builder’s validation feature
Missing or Duplicate Data
When expected data is not appearing or appears multiple times:
- Verify source configuration and filters
- Check for overlapping source patterns
- Review processor conditions for conflicts
- Inspect Live Capture to trace data flow
Profiling and Performance Analysis
For deep performance analysis, Edge Delta supports profiling with pprof:
- Enable profiling in the Edge Delta UI for specific agents
- Analyze CPU and memory profiles
- Identify hotspots in pipeline processing
- Correlate with self-telemetry metrics
Troubleshooting Guides
For detailed troubleshooting procedures, refer to these guides:
Agent and Pipeline Troubleshooting
- Troubleshoot Edge Delta Agent with Helm - Kubernetes agent issues
- Troubleshoot Cloud Pipelines - Cloud pipeline connectivity
- Troubleshoot OTLP Destination - OTLP output issues
Performance and Resource Issues
- Reducing Agent Resource Consumption - Optimize agent resources
- Profiling Agents with pprof - Deep performance analysis
Configuration and Data Flow
- Live Capture In-Stream Debugging - Inspect live data
- Effective Pipeline Design - Design patterns and validation
Related Documentation
- Monitoring and Visibility - Track pipeline health
- Pipeline Dashboard - View pipeline status
- Agent Settings - Configuration options