Deployment & Scaling

This page covers Helm values for deployment configuration, autoscaling, and pod distribution.

Deployment - Autoscaling and Replicas

The Deployment - Autoscaling section encompasses various configurations for enabling and fine-tuning the Horizontal Pod Autoscaler (HPA) behavior in an Edge Delta deployment. Autoscaling allows your deployment to dynamically adjust the number of replica processor pods based on resource utilization metrics such as CPU and memory usage, or custom metrics. The provided example demonstrates a comprehensive setup that integrates several autoscaling parameters.

Note: Changing the deployment kind to Deployment is a requirement for enabling Horizontal Pod Autoscaling (HPA). The default DaemonSet deployment kind does not support dynamic scaling out and in based on resource utilization because its primary goal is to maintain one pod per node rather than reacting to fluctuating loads.

Note: The rollup and Compactor Agents have their own autoscaling parameters.

Available in node chart only.

To install Edge Delta with deployment autoscaling:

helm upgrade edgedelta edgedelta/edgedelta -i --version v1.17.0 \
  --set secretApiKey.value=12345678987654321 \
  --set deployment.kind=Deployment \
  --set deployment.autoscaling.enabled=true \
  --set deployment.autoscaling.minReplicas=1 \
  --set deployment.autoscaling.maxReplicas=5 \
  --set deployment.autoscaling.targetForCPUUtilizationPercentage=80 \
  --set deployment.autoscaling.behavior.scaleDown.stabilizationWindowSeconds=300 \
  -n edgedelta --create-namespace

This command installs the Edge Delta agent with a Horizontal Pod Autoscaler (HPA) enabled, setting the minimum replica count to 1 and the maximum to 5. The HPA is configured to scale based on CPU utilization, targeting an 80% average usage. Additionally, it applies a 300-second stabilization window to prevent frequent scaling down due to temporary spikes.

To verify autoscaling:

kubectl get hpa -n edgedelta

kubectl describe hpa -n edgedelta

Each part of the autoscaling value is discussed next:

Deployment - Autoscaling - Enabled

Variable: deployment.autoscaling.enabled

Description: Enables the creation of a Horizontal Pod Autoscaler (HPA) for processor agents within the Edge Delta deployment. When set to true, the HPA is configured to manage the scaling of pods based on resource utilization.

Example:

--set deployment.autoscaling.enabled=true

Deployment - Autoscaling - External

Variable: deployment.autoscaling.external

Description: Set to true if using an external autoscaler like KEDA (Kubernetes Event-driven Autoscaling). This setting allows integrating with external scaling mechanisms outside of the standard HPA.

Example:

--set deployment.autoscaling.external=false

Deployment - Autoscaling - Min Replicas

Variable: deployment.autoscaling.minReplicas

Description: Specifies the minimum number of replica pods for the deployment. This ensures that the deployment maintains at least this number of replicas at all times.

Example:

--set deployment.autoscaling.minReplicas=1

Deployment - Autoscaling - Max Replicas

Variable: deployment.autoscaling.maxReplicas

Description: Specifies the maximum number of replica pods for the deployment. The HPA will not scale the deployment above this number of replicas.

Example:

--set deployment.autoscaling.maxReplicas=5

Deployment - Autoscaling - Target CPU Utilization Percentage

Variable: deployment.autoscaling.targetForCPUUtilizationPercentage

Description: Defines the target average CPU utilization percentage for the HPA to maintain across the pods of the deployment. The HPA will scale up or down to meet this target when the CPU usage crosses the specified threshold.

Example:

--set deployment.autoscaling.targetForCPUUtilizationPercentage=80

Deployment - Autoscaling - Target Memory Utilization Percentage

Variable: deployment.autoscaling.targetForMemoryUtilizationPercentage

Description: Defines the target average memory utilization percentage for the HPA to maintain across the pods of the deployment. The HPA will scale up or down to meet this target when the memory usage crosses the specified threshold.

Example:

--set deployment.autoscaling.targetForMemoryUtilizationPercentage=80

Deployment - Autoscaling - Custom Metric

Variable: deployment.autoscaling.customMetric

Description: Allows the use of custom metrics for autoscaling targets. This section can be used to configure other metric types beyond CPU and memory for the HPA to evaluate.

Example:

--set deployment.autoscaling.customMetric={type: "Pods", pods: {metric: {name: "packets-per-second"}, target: {type: "AverageValue", averageValue: "1k"}}}

Deployment - Autoscaling - Behavior - Scale Down Stabilization Window Seconds

Variable: deployment.autoscaling.behavior.scaleDown.stabilizationWindowSeconds

Description: Configures the stabilization window in seconds for scaling down the pods via the Horizontal Pod Autoscaler (HPA). During this window, the HPA delays scaling down actions to avoid unnecessary scale-downs due to temporary spikes in metrics.

Example:

--set deployment.autoscaling.behavior.scaleDown.stabilizationWindowSeconds=300

Deployment - Kind

Variable: deployment.kind

Description: The deployment.kind parameter defines how the Processing Agents within the Edge Delta pipeline are managed. This parameter can be either DaemonSet for deploying on each node or Deployment for a scalable set of agent pods. The default is to deploy the processor agent as a DaemonSet. See Installing as a Deployment.

Example:

helm upgrade edgedelta edgedelta/edgedelta -i --version v1.17.0 \
  --set secretApiKey.value=12345678987654321 \
  --set deployment.kind=Deployment \
  -n edgedelta --create-namespace

To verify:

kubectl get deployment -n edgedelta

kubectl get daemonset -n edgedelta

Note: Separate configuration sections and Helm values are provided for Rollup and Compactor Agents.

Deployment - Replicas

Variable: deployment.replicas

Description: Specifies the number of pods for the Processor Agents when deployment.kind is set to Deployment. This setting is mutually exclusive with autoscaling, which means it will not apply if deployment.autoscaling.enabled is set to true. The deployment.replicas parameter is especially useful for scenarios where a fixed number of Processor Agents is required, avoiding the complexity of autoscaling. This can simplify resource allocation and predictability in environments where the load is consistent and predictable.

Example:

--set deployment.replicas=3

To install Edge Delta Processor Agents with a static number of 3 replicas:

helm upgrade edgedelta edgedelta/edgedelta -i --version v1.17.0 \
  --set secretApiKey.value=12345678987654321 \
  --set deployment.kind=Deployment \
  --set deployment.replicas=3 \
  --set deployment.autoscaling.enabled=false \
  -n edgedelta --create-namespace

To verify the number of replicas:

kubectl get deployment -n edgedelta

kubectl describe deployment edgedelta -n edgedelta

Deployment - Topology Spread Constraints

The topologySpreadConstraints parameter allows you to define how Edge Delta Processor Agent pods should be distributed across different topology domains (e.g., zones, regions) within a Kubernetes cluster. This configuration ensures better availability and resiliency of the deployed agents by spreading them across various failure domains. The provided example demonstrates a comprehensive setup using several topology spread constraint parameters.

Note: The rollup and Compactor Agents have their own Topology Spread Constraints parameters.

To install Edge Delta with topology spread constraints ensuring that Processor Agent pods are spread across zones:

helm upgrade edgedelta edgedelta/edgedelta -i --version v1.17.0 \
  --set secretApiKey.value=12345678987654321 \
  --set deployment.kind=Deployment \
  --set deployment.topologySpreadConstraints\[0\].maxSkew=1 \
  --set deployment.topologySpreadConstraints\[0\].topologyKey=topology.kubernetes.io/zone \
  --set deployment.topologySpreadConstraints\[0\].whenUnsatisfiable=ScheduleAnyway \
  --set deployment.topologySpreadConstraints\[0\].labelSelector.matchLabels.'edgedelta\/agent-type'=processor \
  -n edgedelta --create-namespace

Note: The square brackets ([0]) in the helm command can cause shell interpretation issues.

By escaping the brackets with backslashes (), you prevented the shell from interpreting them, ensuring the command is passed correctly to helm. For example, --set deployment.topologySpreadConstraints[0].maxSkew=1 becomes --set deployment.topologySpreadConstraints\[0\].maxSkew=1.

Using a values file makes the Helm command cleaner and avoids issues with special characters interpretation by the shell. This approach is highly recommended for complex configurations and for maintaining clear and reusable deployment settings:

deployment:
  kind: Deployment
  topologySpreadConstraints:
    - maxSkew: 1
      topologyKey: topology.kubernetes.io/zone
      whenUnsatisfiable: ScheduleAnyway
      labelSelector:
        matchLabels:
          edgedelta/agent-type: processor

This command configures the deployment to spread the pods across different zones as evenly as possible with a maxSkew of 1.

To verify the topology spread constraints:

kubectl get deployment -n edgedelta

kubectl describe deployment edgedelta -n edgedelta

Each part of the topologySpreadConstraints value is discussed next:

Deployment - Topology Spread Constraints - Max Skew

Variable: deployment.topologySpreadConstraints[0].maxSkew

Description: This field specifies the maximum allowed difference in the number of pods across the specified topologies. It defines the degree of imbalance that is acceptable between nodes or other topology domains, based on the topologyKey provided. Setting maxSkew to a smaller number forces a more even distribution of pods.

Example:

--set deployment.topologySpreadConstraints\[0\].maxSkew=1

This sets the maxSkew to 1, aiming for a tight distribution of pods across zones.

Deployment - Topology Spread Constraints - Topology Key

Variable: deployment.topologySpreadConstraints[0].topologyKey

Description: This field identifies the key that the system evaluates when determining how to categorize nodes into topologies. Commonly used keys include topology.kubernetes.io/zone for spreading pods across physical zones or topology.kubernetes.io/region for larger geographical areas.

Example:

--set deployment.topologySpreadConstraints\[0\].topologyKey=topology.kubernetes.io/zone

This specifies that the nodes’ physical zones determine the domains over which the pods should be spread.

Deployment - Topology Spread Constraints - When Unsatisfiable

Variable: deployment.topologySpreadConstraints[0].whenUnsatisfiable

Description: This field determines what action to take if the pods cannot be spread as per the maxSkew definition. It manages the scheduling policy when it’s not possible to satisfy the skew criteria. Options include:

• DoNotSchedule: prevents the scheduler from scheduling the pod if doing so would violate the topology’s spread constraint.
• ScheduleAnyway: allows the scheduler to schedule the pods even if the spread constraints cannot be fully satisfied, thus, it prioritizes getting pods running over maintaining the spread constraint strictly.

Example:

--set deployment.topologySpreadConstraints\[0\].whenUnsatisfiable=ScheduleAnyway

This setting ensures that pods will still be scheduled even if the maxSkew cannot be exactly met, thus preventing pod scheduling failures due to strict spread constraints.

Deployment - Topology Spread Constraints - Label Selector Match Labels

Variable: deployment.topologySpreadConstraints[0].labelSelector.matchLabels

Description: This configuration allows selecting a subset of pods based on their labels. It is a key-value pair that matches against labels attached to objects (like pods). This criteria is used to determine which pods should be considered as part of the topology spread calculation. By configuring matchLabels, you can fine-tune the policy to only affect certain pods that match these labels, effectively applying the topology spread rules selectively based on the workload characteristics defined by labels.

Example:

--set deployment.topologySpreadConstraints\[0\].labelSelector.matchLabels.'edgedelta\/agent-type'=processor

This applies the constraint only to the pods that have a label edgedelta/agent-type=processor, ensuring only processor type agents are spread across the nodes as per the defined criteria.

Update Strategy

Variable: updateStrategy

Description: This dictates how updates to the application are rolled out. The updateStrategy.type: RollingUpdate strategy means that updates will roll out one pod at a time, rather than taking the entire application down and updating all at once. This provides high availability during updates. Specifically, updateStrategy.rollingUpdate.maxUnavailable: 1 means that during the update, at most one pod can be unavailable.