ArgoCD Pack

Edge Delta Pipeline Pack for ArgoCD

Overview

The ArgoCD pack ingests and processes ArgoCD logs to provide insights into system operations and health. By extracting, classifying, and clustering logs, it enables quick identification and resolution of critical issues, facilitates proactive maintenance, and ensures comprehensive monitoring.

Pack Description

1. Data Ingestion

The data flow starts with the compound_input node, which is a Pack Input node. This node is the entry point for the pack, where it begins processing the ingested ArgoCD logs.

2. Field Extraction

Once the logs enter the pack, they are passed to the grok_extract_fields node, which is a Grok node. This node applies a predefined pattern to the log entries to extract and structure fields such as timestamp, level, message, application, and resource. This structuring process simplifies the logs, outputting these fields as individual attributes, making them easier to analyze.

3. Timestamp and Severity Standardization

The structured logs flow next to the patch_timestamp_and_log_level node, which is a Log Transform node. This node performs two critical transformations:

• It converts the timestamp to Unix Milliseconds format and inserts it into the item["timestamp"] field using the Edge Delta convert_timestamp macro.
• It copies the extracted log level to the item["severity_text"] field.

These transformations ensure consistent timestamp formats and severity level representations across all log entries, improving monitoring and alerting accuracy.

  - name: patch_timestamp_and_log_level
    type: log_transform
    transformations:
      - field_path: item["timestamp"]
        operation: upsert
        value:
          convert_timestamp(item["attributes"]["timestamp"], "2006-01-02T15:04:05.999999Z",
          "Unix Milli")
      - field_path: item["severity_text"]
        operation: upsert
        value: item["attributes"]["level"]

4. Log Classification

The logs are then routed to the level_router node, which is a Route node. This node classifies the logs based on their severity_text value:

• Logs with severity_text set to "warn" are routed to both the warning_logs and the log_to_pattern nodes.
• Logs with severity_text set to "error" are routed to both the error_logs and the log_to_pattern nodes.
• Logs that do not match warn or error conditions are routed to the other_logs node through the ‘unmatched’ path.

  - name: level_router
    type: route
    paths:
      - path: warn
        condition: item["severity_text"] == "warn"
        exit_if_matched: true
      - path: error
        condition: item["severity_text"] == "error"
        exit_if_matched: true

This classification enables you to focus on critical logs, facilitating quicker identification and resolution of significant issues.

5. Pattern Identification

For logs classified as warnings or errors, the log_to_pattern node, which is a Log to Pattern node, clusters similar log entries into patterns. By generating clusters of log patterns, you can recognize recurring issues and understand common log structures, thus enabling proactive maintenance and stability improvements.

  - name: log_to_pattern
    type: log_to_pattern
    num_of_clusters: 10
    samples_per_cluster: 5

6. Output of Clustered Log Patterns

The clustered patterns from the log_to_pattern node are outputted to the patterns_output node. Storing these patterns facilitates analyzing log trends and identifying anomalies, contributing to better system health insights.

7. Output of Warning Logs

Logs identified as warnings are captured by the warning_logs node. This node isolates warning logs, enabling you to monitor and address potential issues before they become critical problems.

8. Output of Error Logs

Logs classified as errors are routed to the error_logs node. By isolating these critical error logs, you can prioritize and resolve severe issues swiftly, ensuring system reliability and availability.

9. Output of Other Logs

Logs that do not match the warn or error criteria take the ‘unmatched’ path and are routed to the other_logs node. Retaining these logs ensures comprehensive log analysis, capturing unexpected issues that might not have been flagged as critical or warnings.

Sample Input

time=2024-09-20T19:08:26.629Z level=debug msg="Reconciliation completed" application=argocd/prod-cluster

time=2024-09-20T19:08:26.629Z level=fatal msg="GetRepoObjs stats" application=argocd/test-cluster