Configure Edge Delta with ConfigMaps

Configure Fleets using ConfigMaps.

  4 minute read  

You can manage a Kubernetes Edge Delta fleet configuration locally using a ConfigMap. This is particularly useful if you want to secure the configuration and apply change control, for example in a GitOps workflow.

The Fleet polls the ConfigMap location every minute so changes to the configuration may take up to a minute to reflect.

Process Overview

Configuring the Edge Delta Fleet with a ConfigMap involves the following steps:

  1. Create and deploy secrets for the Pipeline ID and Organization ID.
  2. Create a ConfigMap.
  3. Modify the Edge Delta Fleet manifest to use a ConfigMap.
  4. Deploy the manifest and the ConfigMap.

1. Create Secrets

The Edge Delta Fleet requires the Edge Delta Pipeline ID and your Organization ID as secrets.

Create and name an Edge Delta Fleet configuration in the Edge Delta web app. This will create a configuration with a Pipeline ID. This Fleet configuration in the Edge Delta web app will be ignored because the configuration will be supplied locally as a ConfigMap to the cluster. However, you can use the interface to test your configuration.

Use the following instructions with the Kubernetes template option:

  1. Click Pipelines.
  2. Click New Fleet.
  3. Select Edge Fleet.
  4. Optionally, expand Advanced Fleet Configuration and select a pipeline with a configuration you want to duplicate.
  5. Click Continue.
  6. Select the appropriate template and click Continue.
  7. Specify a name to identify the Fleet.
  8. Click Generate Config.
  9. Execute the installation commands, they include the unique ID for the Fleet.
  10. Expand the namespaces and select the input sources you want to monitor.
  11. Select the Destination Outputs you want to send processed data to, such as the Edge Delta Observability Platform.
  12. Click Continue.
  13. Click View Dashboard.

Copy the Pipeline ID and Organization ID

  1. In the Edge Delta App, click Pipelines.
  2. Select the Fleet.
  3. Copy the Pipeline ID for a later step.
  4. Click Admin - My Organization and copy your organization ID for a later step.

Create API and Org ID Secrets

Create the following secrets in your cluster. Replace 123456789 with your Pipeline ID and 987654321 with your Organization ID.

kubectl create namespace edgedelta

kubectl create secret generic ed-api-key \
    --namespace=edgedelta \
    --from-literal=ed-api-key="123456789"

kubectl create secret generic ed-org-id \
    --namespace=edgedelta \
    --from-literal=ed-org-id="987654321"

2. Create the ConfigMap

You can copy the default pipeline configuration from the Edge Delta web app and use it as a basis for creating your custom local configuration. You can also use the Edge Delta web app to test your configuration using specific log files.

See pipeline Configuration for more details about configuring the pipeline.

After creating a configuration, you convert it into a ConfigMap by adding the Kubernetes metadata, specifying a name and namespace, and specifying the ConfigMap filename that will be mounted.

To ensure that the ConfigMap data field parses correctly, you can pass your raw configuration YAML file (the one without any Kubernetes metadata) into kubectl to automatically create a well formatted ConfigMap file with the required Kubernetes metadata. In addition to the ConfigMap name and the namespace, you specify the source file that contains the raw configuration, and the destination file:

kubectl create configmap edgedelta-agent-config --from-file=/Users/me/rawconfig/config.yml -n edgedelta --dry-run=client -o yaml  > config.yml

Note the input and output files are named config.yml to align with the expected filename specified in the Edge Delta pipeline environment variable commands configured next.

apiVersion: v1
kind: ConfigMap
metadata:
  name: edgedelta-agent-config
  namespace: edgedelta
data:
  config.yml: |
    workflows:

3. Modify the Edge Delta Fleet Manifest

You can download the latest manifest from the Edge Delta repository and use it as a basis for your customized installation. The location is also listed in the Kubernetes agent installation instructions on the Edge Delta web app, in the kubectl apply -f step:

These changes must be made to the file to ensure it will work with the ConfigMap.

  1. Configure the fleet to not download a configuration from the Edge Delta back end by adding the ED_SKIP_CONF_DOWNLOAD environment variable value of 1.
  2. Add environment variable commands to specify the location of the mounted configuration:
  command:
    - /edgedelta/edgedelta
    - -c
    - /config/config.yml

The configuration file name is config.yml to align the ConfigMap configured in the previous step.

  1. Add a volume with a ConfigMap called edgedelta-agent-config, which is the name of the ConfigMap, then add a volumeMount with the mountPath of /config/. This adds the ConfigMap data to the directory specified in mountPath. This is required to ensure that updates made to the ConfigMap will be automatically applied without the need to restart the pod.
  2. Create the ED_ORG_ID environmental variable with the secretKeyRef if it is missing.

Ensure that the secretKeyRef entries match the secret names defined in step 1: ed-api-key and ed-org-id.

4. Deploy the Configmap and the Fleet

Finally, you deploy the ConfigMap and agent manifest.

kubectl apply -f config.yml

kubectl apply -f edgedelta-agent.yaml

If all the manifests you want to deploy are in one folder you can apply the entire folder with one command from the parent folder:

kubectl apply -f <foldername>

See Also:

GitOps Principles Deployment of Edge Delta