Troubleshooting Edge Delta on Linux

Troubleshooting the Edge Delta Fleet on Linux.

View the Fleet’s Configuration

Check the Fleet’s configuration file to ensure that the pipeline configuration does not contain any issues:

cat /opt/edgedelta/agent/config.yml

View the Process Configuration

Use the less command to view the configuration file.

  • For Upstart-based systems such as RHEL, Amazon Linux, etc.:
less /etc/init/edgedelta.conf
  • For Systemd-based systems such as Debian, Ubuntu, etc.:
less /etc/systemd/system/edgedelta.service

Get the Process Status

Use the status command to view the edgedelta process status:

  • For Upstart-based systems such as RHEL, Amazon Linux, etc.:
initctl status edgedelta
  • For Systemd-based systems such as Debian, Ubuntu, etc.:
systemctl status edgedelta
  • For older systems with init:
sudo /etc/init.d/edgedelta status
  • For older Ubuntu systems:
sudo service edgedelta status

Restart the Process

To restart the edgedelta process use the restart command:

  • For Upstart-based systems such as RHEL, Amazon Linux, etc.:
initctl restart edgedelta
  • For Systemd-based systems such as Debian, Ubuntu, etc.:
systemctl restart edgedelta

To ensure the process was restarted, first capture its PID:

pgrep edgedelta

or

ps aux | grep edgedelta

Run the command to restart the process and check the PID of the process again.

Start and Stop the Process

To start or stop the edgedelta process use one of the following commands:

  • For Upstart-based systems such as RHEL, Amazon Linux, etc.:
initctl start edgedelta
initctl stop edgedelta
  • For Systemd-based systems such as Debian, Ubuntu, etc.:
sytemctl start edgedelta
sytemctl stop edgedelta

Get the Process’ System Information

Information can be gathered by either using top or htop. The top command will give you an isolated view of the process:

top -p $(pgrep edgedelta)

If htop is installed: htop then press / and type edgedelta.

For Upstart-based systems such as RHEL, Amazon Linux, etc. Messages from the initialization script can give you more information. They are logged every time the system is reloaded, the process is started or restarted, and when the configuration is changed.

tail -F /var/log/messages

If there is not enough information check the log level set:

initctl log-priority

You can check available levels:

initctl log-priority help

For troubleshooting, you should use the finest level of detail:

$ initctl log-priority debug

SELinux Configuration in Enforcing Mode

If you encounter issues related to SELinux, ensure that the SELinux configuration for the Edge Delta agent is correctly set up:

sudo semanage fcontext -a -t bin_t -f f /opt/edgedelta/agent/edgedelta
sudo restorecon -v /opt/edgedelta/agent/edgedelta
sudo systemctl restart edgedelta.service

These commands ensure SELinux policies do not block the execution of the Edge Delta agent.

Restart the Operating System

For Upstart-based systems such as RHEL, Amazon Linux, etc. The operating system needs to be restarted for any change to the configuration to take effect. For Systemd-based systems such as Debian, Ubuntu, etc. use

systemctl daemon-reload

View the Agent Log File

Check the agent’s log file for any errors that may indicate an issue with the agent, configuration, or deployment settings:

cat /var/log/edgedelta/edgedelta.log

Troubleshoot Memory Limits

The edgedelta service will restart if it reaches its memory limits.

If you notice that the service restarts frequently, then you may want to increase the limits. If you do not want to change your memory limits, then you can enable memory profiling and contact support with a heapdump from the profiler.

Run the following command for the current agent version (replace x.x.xx) to enable a more aggressive Garbage Collection and pprof profiler on port 6060. If another service uses port 6060, then you must use another port number in the bash and curl commands.

`sudo ED_API_KEY=$API_KEY ED_ENV_VARS="GOGC=30,PROFILER_PORT=6060" bash -c "$(curl -L https://release.edgedelta.com/vx.x.xx/install.sh)"`

Run the following command to obtain heapdump from the profiler. GOGC is 30. The default value is 100 for Golang. This setting will initiate a garbage collection job once the heap size has grown 30% since the last garbage collection.

curl http://localhost:6060/debug/pprof/heap --output heap