Skip to main content

Automating Workflows with Docker CLI and APIs

Introduction​

Modern software development demands efficiency, repeatability, and integration. Automating Docker workflows helps eliminate manual steps, reduce errors, and integrate container management into larger DevOps pipelines. In this lesson, you'll learn how to automate Docker tasks using the powerful Docker CLI and the Docker Remote API, empowering you to script, schedule, and integrate container operations with other tools or systems.

Table of Contents​

  1. Why Automate Docker Workflows?
  2. Automating with the Docker CLI
  3. Automating with the Docker Remote API
  4. Common Automation Use Cases
  5. Common Mistakes and Pitfalls
  6. Summary
  7. Quiz

Why Automate Docker Workflows?​

  • Efficiency: Reduce repetitive manual tasks.
  • Consistency: Ensure the same steps are executed every time.
  • Integration: Fit Docker operations into DevOps pipelines and CI/CD workflows.
  • Scalability: Manage multiple containers/services at once.

Automating with the Docker CLI​

The Docker CLI (docker command) is scriptable, which means you can combine commands in shell scripts or batch files to automate complex workflows.

Scripting Common Docker Commands​

You can write scripts to automate tasks such as building images, starting containers, or cleaning up resources.

Example: Bash Script to Build and Deploy​

#!/bin/bash
# build-and-deploy.sh

IMAGE_NAME="myapp:latest"
CONTAINER_NAME="myapp-container"

# Build the image
docker build -t $IMAGE_NAME .

# Stop and remove any existing container
docker rm -f $CONTAINER_NAME 2>/dev/null

# Start the new container
docker run -d --name $CONTAINER_NAME -p 8080:80 $IMAGE_NAME

Usage:

chmod +x build-and-deploy.sh
./build-and-deploy.sh

Using docker-compose for Automation​

docker-compose can orchestrate multi-container applications, and is easily integrated into scripts.

Example: Bringing Up and Down a Stack​

docker-compose -f docker-compose.prod.yml up -d
# ... run tests or other steps ...
docker-compose -f docker-compose.prod.yml down

Scheduling Docker Tasks​

On Linux, use cron to schedule scripts:

0 2 * * * /home/user/build-and-deploy.sh

Runs build-and-deploy.sh every day at 2 AM.

On Windows, use Task Scheduler for similar results.

Automating with the Docker Remote API​

Introduction to the Docker API​

Docker exposes a RESTful API, allowing you to manage Docker programmatically. By default, this API listens on a UNIX socket (/var/run/docker.sock). You can also configure it to listen on a TCP port.

API Authentication and Security​

Warning: Exposing the Docker API over TCP is dangerous; always secure it with TLS and restrict access.

  • By default, access to /var/run/docker.sock is root-equivalent.
  • When enabling TCP, always use TLS certificates for authentication.
  • Limit access to trusted users and networks.

Example: Using curl to Interact with the API​

List All Containers​

curl --unix-socket /var/run/docker.sock http://localhost/containers/json

Start a Container​

curl --unix-socket /var/run/docker.sock -X POST http://localhost/containers/<container_id>/start

Automating with SDKs (Python Example)​

Docker offers official SDKs for several languages. Here's a Python example using the Docker SDK for Python:

import docker

client = docker.from_env()

# Build an image
image, logs = client.images.build(path=".", tag="myapp:latest")
for log in logs:
    print(log.get('stream', ''), end='')

# Remove existing container if exists
try:
    container = client.containers.get("myapp-container")
    container.stop()
    container.remove()
except docker.errors.NotFound:
    pass

# Run the container
container = client.containers.run(
    "myapp:latest", name="myapp-container", detach=True, ports={'80/tcp': 8080}
)
print(f"Started container {container.short_id}")

Common Automation Use Cases​

  • CI/CD Pipelines: Automated build, test, and deployment.
  • Scheduled Backups: Exporting volumes or databases from containers.
  • Resource Cleanup: Pruning unused images/containers on a schedule.
  • Scaling Services: Programmatically adjusting the number of running containers.
  • Monitoring and Health Checks: Automated health status queries and restarts.

Common Mistakes and Pitfalls​

  • Running Automated Scripts as Root: Always minimize privileges; avoid running scripts as root unless necessary.
  • Hardcoding Credentials: Never store sensitive information (like passwords or API keys) directly in scripts.
  • Ignoring Error Handling: Scripts or API calls should check for and handle errors (e.g., failed container start).
  • Unsecured API Endpoints: Never expose the Docker API to the network without proper TLS and authentication.
  • Resource Leaks: Always clean up containers/networks/images after automation tasks complete.

Summary​

Automating Docker workflows with the CLI and APIs unlocks powerful efficiencies for developers and operations teams. Whether scripting with the CLI, orchestrating with docker-compose, or integrating programmatically with the Docker API, automation is a cornerstone of scalable and reliable container management. Always apply best practices for security and error handling as you automate your Docker environments.

Quiz​

1. Which of the following is a secure way to access the Docker Remote API over TCP?

A) Exposing it without TLS on all interfaces
B) Using TLS certificates and restricting network access
C) Using plain HTTP from any machine
D) None of the above

Details

Answer B) Using TLS certificates and restricting network access

2. What command would list all containers using Docker's UNIX socket and curl?

A) curl http://localhost/containers/json
B) curl --unix-socket /var/run/docker.sock http://localhost/containers/json
C) docker ps
D) curl -X GET docker.sock

Details

Answer B) curl --unix-socket /var/run/docker.sock http://localhost/containers/json

3. Which tool allows you to define and manage multi-container Docker applications from a YAML file?

A) Docker CLI
B) Docker Compose
C) Docker API
D) Docker SDK

Details

Answer B) Docker Compose

4. What is a common pitfall when automating Docker workflows?

A) Always handling errors
B) Storing credentials securely
C) Exposing the Docker API without authentication
D) Cleaning up resources

Details

Answer C) Exposing the Docker API without authentication

5. In the Python Docker SDK example, what method is used to build a Docker image?

A) client.build()
B) client.images.build()
C) client.build_image()
D) client.images.create()

Details

Answer B) client.images.build()