Edge Delta ConfigMaps

Rather than using the Edge Delta app to manage agent configurations, you can manage a Kubernetes Edge Delta agent configuration locally using a ConfigMap. This is particularly useful if you want to secure the configuration and apply change control, for example in a GitOps workflow.

The agent polls the ConfigMap location every minute so changes to the configuration may take up to a minute to reflect.

Process Overview

Configuring the Edge Delta agent with a ConfigMap involves the following steps:

1. Create and deploy secrets for the API key and Organization ID.
2. Create a ConfigMap.
3. Modify the Edge Delta agent manifest to use a ConfigMap.
4. Deploy the manifest and the ConfigMap.

TL;DR

Example Files

• config.yml
• edgedelta-agent.yml

Example Commands

kubectl create namespace edgedelta
kubectl create secret generic ed-api-key \
    --namespace=edgedelta \
    --from-literal=ed-api-key="<your api key>"
kubectl create secret generic ed-org-id \
    --namespace=edgedelta \
    --from-literal=ed-org-id="<your org id>"
kubectl apply -f config.yml 
kubectl apply -f edgedelta-agent.yml

1. Create Secrets

The Edge Delta agent requires the Edge Delta API key and your Organization ID as secrets.

Create and name an Edge Delta agent configuration in the Edge Delta web app. This will create an agent with an agent ID, an API key, and an agent configuration. The agent configuration in the Edge Delta web app will be ignored because the configuration will be supplied locally as a ConfigMap to the cluster. However, you can use the interface to tests your configuration. Use the following instructions with the Helm template option:

1. In the Edge Delta App, click Data Pipeline, and then click Agent Configs.
2. Click Create Configuration.
3. Select the appropriate template.
4. Specify a tag to identify the agent and environment.
5. Click Create Configuration.
6. A new default agent is created. Review the agent configuration in Visual Pipelines.

When you return to the Agent Configs page, the agent table is updated with a new agent configuration with the agent tag you specified.

Note the key in the key column of the new agent. To see the Organization ID, click Management - My Organization

Create the following secrets in your cluster. Replace 123456789 with your API key and 987654321 with your Organization ID.

kubectl create namespace edgedelta

kubectl create secret generic ed-api-key \
    --namespace=edgedelta \
    --from-literal=ed-api-key="123456789"

kubectl create secret generic ed-org-id \
    --namespace=edgedelta \
    --from-literal=ed-org-id="987654321"

Best practice is to use a secrets management solution.

2. Create the ConfigMap

Copy the default agent configuration from the Edge Delta web app and use it as a basis for creating your custom local configuration. Alternatively, configure an agent using Visual Pipelines then copy the yaml.

You can also Visual Pipelines to test your configuration using specific log files. See Agent Configuration for more details about configuring the agent.

To copy the agent YAML:

1. Click Data Pipeline and select Agent Configs.
2. Select the hamburger icon in the Actions column for the agent and select Edit.
3. Select the entire YAML configuration, copy and paste it in a local file named config.yaml.

Next, you convert the local file (config.yaml) into a ConfigMap by adding the Kubernetes metadata, specifying a name and namespace, and specifying the ConfigMap filename that will be mounted, in this instance, also config.yml.

To ensure that the ConfigMap data field parses correctly, you can pass your raw configuration yaml file (the one without any Kubernetes metadata) into kubectl to automatically create a well formatted ConfigMap file with the required Kubernetes metadata. In addition to the ConfigMap name and the namespace, you specify the source file that contains the raw configuration, and the destination file, which will in this example be in a child folder called parsed as it has the same name as the source file.

The names must match because the source filename is included in the configuration, and the output filename must align with the expected filename specified in the Edge Delta agent environment variable commands configured in the next step.

kubectl create configmap edgedelta-agent-config --from-file=config.yml -n edgedelta --dry-run=client -o yaml  > parsed/config.yml

See here for an example of a ConfigMap containing an agent version 3 configuration.

3. Modify the Edge Delta Agent Manifest

Download the latest manifest from the Edge Delta repository and use it as a basis for your customized installation. The location is also listed in the Kubernetes agent installation instructions on the Edge Delta web app, in the kubectl apply -f step:

The following changes must be made to the file to ensure it will work with the ConfigMap.

1. Configure the agent to not download a configuration from the Edge Delta back end by adding the ED_SKIP_CONF_DOWNLOAD environment variable value of 1.
2. Add environment variable commands to specify the location of the mounted configuration:

  command:
    - /edgedelta/edgedelta
    - -c
    - /config/config.yml

The configuration file name is config.yml to align the ConfigMap configured in the previous step.

3. Add a volume with a ConfigMap called edgedelta-agent-config, which is the name of the ConfigMap, then add a volumeMount with the mountPath of /config/. This adds the ConfigMap data to the directory specified in mountPath. This is required to ensure that updates made to the ConfigMap will be automatically applied without the need to restart the pod.
4. Create the ED_ORG_ID environmental variable with the secretKeyRef if it is missing.

Ensure that the secretKeyRef entries match the secret names defined in step 1: ed-api-key and ed-org-id.

See here for an example of a customized Edge Delta manifest for use with a ConfigMap.

4. Deploy the Configmap and the Agent

Finally, you deploy the ConfigMap and agent manifest.

kubectl apply -f config.yml

kubectl apply -f edgedelta-agent.yaml

If all the manifests you want to deploy are in one folder you can apply the entire folder with one command from the parent folder:

kubectl apply -f <foldername>

Samples