Install Agent with kubectl
4 minute read
The Edge Delta Agent can be installed in a Kubernetes environment using kubectl. It is installed by default as a DaemonSet - a pod on every node. It analyses logs from each pod in each node as well as metrics from the cluster, and streams them to the configured destinations.
Install with Default Settings
You can install Edge Delta using kubectl without changing any default settings. You first create an Edge Delta agent configuration for Kubernetes if one doesn’t exist. Then you deploy the agent passing in the secret key to your Edge Delta web app configuration.
Create an Edge Delta Agent Configuration
Use the Kubernetes template option while following these steps:
- In the Edge Delta App, click Data Pipeline, and then click Agent Configs.
- Click Create Configuration.
- Select the appropriate template.
- Specify a tag to identify the agent and environment.
- Click Create Configuration.
- Optionally, modify and test the agent configuration then click Save Edits or Done.
The agent table is updated with a new agent configuration with the agent tag you specified.
Deploy an Edge Delta Agent
- In the agent table, click the hamburger button for the new agent configuration and click Deploy Instructions.
- Select Kubernetes.
- Follow the on-screen instructions.
These onscreen instructions step you through the process of creating a namespace for the Edge Delta agent, creating a kube secret to contain your API key, creating the Edge Delta agent DaemonSet, and confirming that the Edge Delta pods are up. The following example shows the commands in the instructions but with an example key 123456789.
kubectl create namespace edgedelta
kubectl create secret generic ed-api-key \
--namespace=edgedelta \
--from-literal=ed-api-key="123456789"
kubectl apply -f https://edgedelta.github.io/k8s/edgedelta-agent.yml
kubectl get pods -n edgedelta
Install with Alternative Settings
The default installation instructions apply the default agent manifest. However, there are other manifests from Edge Delta:
Manifest | Description | Location |
---|---|---|
Default | This manifest is the default agent DaemonSet. | https://edgedelta.github.io/k8s/edgedelta-agent.yml |
Persisting Cursor | This manifest is the agent DaemonSet with mounted host volumes to track file cursor positions persistently. | https://edgedelta.github.io/k8s/edgedelta-agent-persisting-cursor.yml |
Prometheus Metrics | This manifest is the agent DaemonSet that exposes port 6062 (metrics endpoint) in Prometheus format. See Prometheus Integration | https://edgedelta.github.io/k8s/edgedelta-prom-agent.yml |
On Premises | This manifest is the agent DaemonSet for locally managed or offline deployments. | https://edgedelta.github.io/k8s/edgedelta-agent-onprem.yml |
Install with Custom Settings
You can create your own custom manifest. To start, download the default manifest and add custom variables to it. Then apply the local file, in this example the custom-agent.yml file in the current folder is applied:
kubectl apply -f custom-agent.yml
mountPath
For custom Kubernetes deployments, you may need to update the mountPath to match the actual path of the container log folder.
For some Kubernetes distributions, /docker/containers is used, instead of the standard /var/lib/docker/containers. In these cases, you must update the the mountPath in the manifest file (edgedelta-agent.yml) to match the actual path of the container log folder.
SELinux
If you are running a SELinux-enforced Kubernetes cluster, then you need to add the following securityContext configuration into edgedelta-agent.yml manifest DaemonSet spec. This update will run agent pods in privileged mode to allow the collection of logs of other pods.
runAsUser: 0
privileged: true
OpenShift
In an OpenShift cluster, you need to also run the following commands to allow agent pods to run in privileged mode:
oc adm policy add-scc-to-user privileged system:serviceaccount:edgedelta:edgedelta
oc patch namespace edgedelta -p \
'{"metadata": {"annotations": {"openshift.io/node-selector": ""}}}'
Run on Specific Nodes
To run the agent on specific nodes in your cluster, add a node selector or nodeAffinity section to your pod config file. For example, if the desired nodes are labeled as logging=edgedelta, then adding the following nodeSelector will restrict the agent pods to nodes that have the logging=edgedelta label.
spec:
nodeSelector:
logging: edgedelta
To learn more, see this article on the kubernetes.io site.
In-Cluster Data Destinations
Edge Delta pods run in a dedicated edgedelta namespace.
If you want to configure an output destination within your Kubernetes cluster, then you must set a resolvable service endpoint in your agent configuration.
For example, if you have an elasticsearch-master Elasticsearch service in the elasticsearch namespace with port 9200 in your cluster-domain.example cluster, then you need to specify the Elastic output address in the agent configuration:
address:
- http://elasticsearch-master.elasticsearch.svc.cluster-domain.example:9200
To learn more, see this article on the kubernetes.io site.