Troubleshooting Persisting Cursor

Configure the Edge Delta Pipeline with a persisting cursor to maintain log file read positions across agent restarts.

  2 minute read  

Overview

A persisting cursor in Edge Delta is a feature that remembers the last read position in a log file even after an agent restart. This is useful in scenarios where data consistency is crucial, such as environments where data loss during agent restarts must be prevented.

Configuration

There are two methods of configuring persistent cursor. You can configure it for a single file source node, or for the entire pipeline.

You enable a persisting cursor with the enable_persisting_cursor parameter in a file source node to allow the system to continue reading from the last processed entry in a log file upon restart. This parameter is a Boolean with the default value of false.

nodes:
  - name: <node name>
    type: file_input
    path: "/var/logs/app/*.log"
    enable_persisting_cursor: true

Alternatively, you can configure a persisting cursor at the pipeline level in the agent settings section. This means all inputs that tail a log file will benefit from a persisting cursor. This option also gives you control over additional settings:

settings:
  tag: prod
  persisting_cursor_settings:
    path: /var/lib/edgedelta
    file_name: cursor_file.json
    flush_interval: 1m
  • path is the folder where the cursor file will be created.
  • file_name is the name of the cursor file.
  • flush_interval is the interval after which the file will be saved to from memory.

Kubernetes Path Requirements

When deploying Edge Delta agents in Kubernetes, the cursor file path must be within the mounted persistent storage volume for the cursor data to survive pod restarts.

The default Helm chart mounts:

Mount TypePath
Host mount path/var/lib/edgedelta
Container mount path/var/lib/edgedelta

For the cursor data to persist across pod restarts, the persisting_cursor_settings.path must be within the mounted path. Use /var/lib/edgedelta or a subdirectory such as /var/lib/edgedelta/pos.

Valid Kubernetes configuration

settings:
  persisting_cursor_settings:
    path: /var/lib/edgedelta
    file_name: cursor.json
    flush_interval: 1m

Invalid Kubernetes configuration

The following configuration will not persist cursor data because /var/edgedelta is not mounted to the host:

settings:
  persisting_cursor_settings:
    path: /var/edgedelta/pos  # Not within mounted volume
    file_name: cursor.json
    flush_interval: 1m

Custom mount paths

If you customize the mount paths using the Helm values persistingCursorProps.hostMountPath and persistingCursorProps.containerMountPath, ensure your persisting_cursor_settings.path is within the configured containerMountPath.

See Persistence Helm Values for details on configuring custom mount paths and Pipeline Settings for the GUI configuration options.