Edge Delta ConfigMaps

You can manage a Kubernetes Edge Delta agent configuration locally using a ConfigMap. This is particularly useful if you want to secure the configuration and apply change control, for example in a GitOps workflow.

The agent polls the ConfigMap location every minute so changes to the configuration may take up to a minute to reflect.

Process Overview

Configuring the Edge Delta agent with a ConfigMap involves the following steps:

1. Create and deploy secrets for the API key and Organization ID.
2. Create a ConfigMap.
3. Modify the Edge Delta agent manifest to use a ConfigMap.
4. Deploy the manifest and the ConfigMap.

TL;DR

Example Files

• config.yml
• edgedelta-agent.yml

Example Commands

kubectl create namespace edgedelta
kubectl create secret generic ed-api-key \
    --namespace=edgedelta \
    --from-literal=ed-api-key="<your api key>"
kubectl create secret generic ed-org-id \
    --namespace=edgedelta \
    --from-literal=ed-org-id="<your org id>"
kubectl apply -f config.yml 
kubectl apply -f edgedelta-agent.yml

1. Create Secrets

The Edge Delta agent requires the Edge Delta API key and your Organization ID as secrets.

Create and name an Edge Delta agent configuration in the Edge Delta web app. This will create an agent with an agent ID, an API key, and an agent configuration. The agent configuration in the Edge Delta web app will be ignored because the configuration will be supplied locally as a ConfigMap to the cluster. However, you can use the interface to test your configuration. Use the following instructions with the Helm template option:

1. Click Pipelines.
2. Click New Fleet.
3. Select Edge Fleet.
4. Optionally, expand Advanced Fleet Configuration and select a pipeline with a configuration you want to duplicate.
5. Click Continue.
6. Select the appropriate template and click Continue.
7. Specify a name to identify the Fleet.
8. Click Generate Config.
9. Execute the installation commands, they include the unique ID for the Fleet.
10. Expand the namespaces and select the input sources you want to monitor.
11. Select the Destination Outputs you want to send processed data to, such as the Edge Delta Observability Platform.
12. Click Continue.
13. Click View Dashboard.

Note the key in the key column of the new agent. To see the Organization ID, click Admin - My Organization

Create the following secrets in your cluster. Replace 123456789 with your API key and 987654321 with your Organization ID.

kubectl create namespace edgedelta

kubectl create secret generic ed-api-key \
    --namespace=edgedelta \
    --from-literal=ed-api-key="123456789"

kubectl create secret generic ed-org-id \
    --namespace=edgedelta \
    --from-literal=ed-org-id="987654321"

2. Create the ConfigMap

You can copy the default agent configuration from the Edge Delta web app and use it as a basis for creating your custom local configuration. You can also use the Edge Delta web app to test your configuration using specific log files.

See Agent Configuration for more details about configuring the agent.

After creating a configuration, you convert it into a ConfigMap by adding the Kubernetes metadata, specifying a name and namespace, and specifying the ConfigMap filename that will be mounted.

To ensure that the ConfigMap data field parses correctly, you can pass your raw configuration yaml file (the one without any Kubernetes metadata) into kubectl to automatically create a well formatted ConfigMap file with the required Kubernetes metadata. In addition to the ConfigMap name and the namespace, you specify the source file that contains the raw configuration, and the destination file:

kubectl create configmap edgedelta-agent-config --from-file=raw_config.yml -n edgedelta --dry-run=client -o yaml  > config.yml

Note the output file is named config.yml to align with the expected filename specified in the Edge Delta agent environment variable commands configured next.

apiVersion: v1
kind: ConfigMap
metadata:
  name: edgedelta-agent-config
  namespace: edgedelta
data:
  config.yml: |
    workflows:    

See here for an example of a ConfigMap containing an agent configuration.

3. Modify the Edge Delta Agent Manifest

You can download the latest manifest from the Edge Delta repository and use it as a basis for your customized installation. The location is also listed in the Kubernetes agent installation instructions on the Edge Delta web app, in the kubectl apply -f step:

These changes must be made to the file to ensure it will work with the ConfigMap.

1. Configure the agent to not download a configuration from the Edge Delta back end by adding the ED_SKIP_CONF_DOWNLOAD environment variable value of 1.
2. Add environment variable commands to specify the location of the mounted configuration:

  command:
    - /edgedelta/edgedelta
    - -c
    - /config/config.yml

The configuration file name is config.yml to align the ConfigMap configured in the previous step.

3. Add a volume with a ConfigMap called edgedelta-agent-config, which is the name of the ConfigMap, then add a volumeMount with the mountPath of /config/. This adds the ConfigMap data to the directory specified in mountPath. This is required to ensure that updates made to the ConfigMap will be automatically applied without the need to restart the pod.
4. Create the ED_ORG_ID environmental variable with the secretKeyRef if it is missing.

Ensure that the secretKeyRef entries match the secret names defined in step 1: ed-api-key and ed-org-id.

See here for an example of a customized Edge Delta manifest for use with a ConfigMap.

4. Deploy the Configmap and the Agent

Finally, you deploy the ConfigMap and agent manifest.

kubectl apply -f config.yml

kubectl apply -f edgedelta-agent.yaml

If all the manifests you want to deploy are in one folder you can apply the entire folder with one command from the parent folder:

kubectl apply -f <foldername>

See Also:

GitOps Principles Deployment of Edge Delta

Samples