Search Logs in Edge Delta

Overview

Edge Delta’s Search is powered by the Common Query Language (CQL), a simple and powerful way to get answers from your logs.

Recap

Edge Delta Fleets send important logs and triggers to your configured data destinations, trigger endpoints, and archive destinations. To enable the Search function, it also archives logs in the Edge Delta back end, in customer specific s3 buckets. You can search this log archive using the search feature in Edge Delta. See here for more details about the architecture.

Flattening

When logs are archived, attributes are flattened to improve search performance. This may result in inconsistent application of a schema, particularly if custom field names containing dots have been used. Consider the following example:

"alpha": {
  "beta": "gamma"
}

This would be configured in the pipeline and testing as follows:

item["alpha"]["beta"]

Now consider a dot separated field name:

"delta.epsilon": "zeta"

This would be configured in the pipeline and testing as follows:

item["delta.epsilon"]

With flattening, these examples would be archived as follows:

alpha.beta: "gamma"
delta.epsilon: "zeta"

Sampling

From 3 hours of age, histogram and facet option counts are sampled so that log queries are performant. Logs themselves, however, are all stored and indexed.

Search for Logs

To search for logs you click Logs and select Search.

See Search Logs for more information.

Autocomplete suggests search strings based on your attribute names and values. By tabbing through the suggestions can quickly explore the data and see the frequency of values in the autocomplete suggestions.

You enter a search string in the Search box and click Search. You can cancel a query by clicking X.

See the Search Syntax.

You can also filter results using the Filters pane. Selecting all the checkboxes shows results from all the selected options. You can choose one specific filter, such as a Namespace to view results from only that namespace. If you select none of the filters, no results will be shown.

The query results are listed along with a chart. You can click and drag a time period on the histogram to filter the lookback period on it.

Search for Attributes

You can search for attributes using the @ character. For example, consider this log:

There is an attribute field anothernewfield. You can search for all logs with a particular value for this attribute by adding @anothernewfield:"parallelcluster-control-plane" to the search query:

Alternatively, you can click an attribute and select Add to Search: