Amazon Elastic Container Service (ECS)

Installing the Edge Delta agent on Amazon ECS.

  3 minute read  

Step 1: Create a Configuration

Select the Kubernetes template option in the following steps:

  1. In the Edge Delta App, click Pipelines, and select Pipelines.
  2. Click New Pipeline.
  3. Select the appropriate template.
  4. Specify a tag to identify the agent and environment.
  5. Click Create Configuration.
  6. A new default agent is created. Review the agent configuration in Visual Pipelines.

When you return to the Pipelines page, the new agent configuration is listed.

Copy the agent API key from the Key column.

Step 2: Create an ECS Task Definition

  1. Download the https://release.edgedelta.com/edgedelta-ecs.json task definition.
  2. In the task definition, replace <YOUR_ED_API_KEY> with your copied API key.
  3. Create the agent task definition via the AWS CLI or via the AWS console.

AWS CLI:

aws ecs register-task-definition --cli-input-jsonfile://path_to_edgedelta-ecs.json

AWS Console:

  1. Navigate to the Elastic Container Service (ECS) section.
  2. Click Task Definitions.
  3. Click Create new Task Definition.
  4. Select a launch type compatibility, and then click Next.
  5. Click Configure via JSON.
  6. Paste the paste task definition JSON, and then click Save.
  7. Click Create.

Step 3: Run the Agent as an ECS Daemon Service

While you can use the agent as an ordinary ECS task, you will only collect logs from the containers running on the EC2 instance the task is running. To properly collect all logs from containers on all instances in your cluster, you need to run the agent as a Daemon Service.

  1. In the AWS Console, navigate to the Elastic Container Service (ECS) section, and then find your cluster.
  2. Under Services, click Create.
  3. Select EC2 Launch Type.
  4. In the drop-down menu, select the task definition you created.
  5. Select DAEMON service type, set a service name, and then click Next step.
  6. Select None for load balancer, click Next step.
  7. By default, auto scaling is disabled. click Next step.
  8. Review the summary, and then click Create Service.

Step 4: Verify Definitions and Configurations

  1. Verify that the container definition for other tasks or services does not have the logConfiguration.logDriver parameter. Without a logging driver, logs are written to standard output and collected by the agent service.
  2. In the Edge Delta App, click Pipelines - Pipelines. Click the configuration.
  3. Verify that you have the Docker input node with image=.* specified in the Docker Include field.

(Optional) Step 5: Set Up File Monitoring

As an optional step, you can monitor for additional log files on your EC2 instances in your ECS cluster.

To do so, update the mountpoints and volumes section in the edgedeleta-ecs.json.

The following example monitors log files in /var/log/ecs on an EC2 instance. Additionally, you must add a file input in the agent YAML configuration file to monitor the mounted path. In this example, the path is /host/var/log/ecs/.

"mountPoints": [
        {
          "containerPath": "/var/run/docker.sock",
          "sourceVolume": "docker_sock",
          "readOnly": true
        },
        {
          "containerPath": "/host/var/log/ecs/",
          "sourceVolume": "ecs_logs",
          "readOnly": true
        },
      ],
 ...

"volumes": [
  {
    "host": {
      "sourcePath": "/var/run/docker.sock"
    },
    "name": "docker_sock"
  },
  {
    "host": {
      "sourcePath": "/var/log/ecs/"
    },
    "name": "ecs_logs"
  }
],
...