Deutsche Version

Back to overview

Watchdog: Agent Setup

Watchdog has two ways of running checks:

• Dashboard checks run in the worker container of the Watchdog server. They test what is reachable from there over the network, such as HTTP, SSL, TCP or DNS.
• Agent checks run in a small container on the monitored machine. It has access to the filesystem, /proc, the process list and the Docker socket.

Which way a check type allows is listed in the check types overview.

The agent connects to the dashboard on its own and pulls its checks. No inbound port needs to be open on the monitored machine.

Step 1: Create the agent

1. Sign in and open Agents in the navigation.
2. Click + Register Agent and give it a name, for example host-server or customer-acme.
3. Copy the token that is shown right away. The UI shows it only once, afterwards only a hash exists in the database.

Each monitored machine gets its own agent and its own token.

Host agent in the Compose stack

The files compose.yml, compose.stage.yml and compose.live.yml already contain an agent service. It is built from the same Dockerfile (target agent), runs as user www-data and starts php bin/console watchdog:agent:run. It reaches the dashboard over the internal Docker network at http://app:80.

1. Put the token into the env file: WATCHDOG_AGENT_TOKEN=... in .env.stage or .env.live. In the local stack compose.yml reads the variable from the shell environment or from the .env file next to the Compose file, Docker Compose does not read .env.local.
2. Start or restart the service:

docker compose -f compose.stage.yml up -d agent

On live use compose.live.yml, locally docker compose up -d agent.

The service at a glance (stage):

agent:
  build:
    context: .
    target: agent
  environment:
    WATCHDOG_DASHBOARD_URL: http://app:80
    HOST_ROOT: /host/root
    TZ: Europe/Berlin
  env_file: .env.stage            # contains WATCHDOG_AGENT_TOKEN
  pid: host                       # for the process check
  volumes:
    - /:/host/root:ro             # for disk, file and log checks
    # - /var/run/docker.sock:/var/run/docker.sock:ro   # for Docker checks
  restart: unless-stopped

Docker checks on the agent

The Docker socket is commented out in the Compose files. Enable the line so that Docker and Docker Exec run on the agent. Because the agent runs as www-data, it must be allowed to read the socket. Worker and scheduler solve this in the Compose files with group_add using the socket's group ID (DOCKER_GID, default 995). The agent needs the same entry when the socket is only accessible to the Docker group.

Dashboard checks of type Docker run in the worker container, which already has the socket mounted.

Remote agent on another server

For another server (for example at a customer) there is a ready-made image that CI builds and publishes for the main branch and for release tags: ghcr.io/markus-michalski/watchdog/agent:latest (architectures amd64 and arm64).

Use docker-compose.agent.yml from the repository as a template:

services:
  watchdog-agent:
    image: ghcr.io/markus-michalski/watchdog/agent:latest
    restart: unless-stopped
    environment:
      WATCHDOG_DASHBOARD_URL: https://watchdog.example.com
      WATCHDOG_AGENT_TOKEN: your-agent-token-here
      HOST_ROOT: /host/root
      TZ: Europe/Berlin
    pid: host
    volumes:
      - /:/host/root:ro

docker compose -f docker-compose.agent.yml up -d

The template in the repository does not set HOST_ROOT. Without HOST_ROOT: /host/root the agent reads paths inside its own container instead of the host filesystem. Add the line as soon as you use disk, file or log checks.

The server must be able to reach the dashboard URL, preferably over HTTPS. WATCHDOG_DASHBOARD_URL has no trailing slash.

Environment variables

Requirements per check type

HTTP, HTTP Content and SSL Certificate are dashboard-only checks and cannot be assigned to an agent.

How it works

1. On startup the agent fetches its check list from /api/v1/agent/config.
2. Every 30 seconds (tick) it asks /api/v1/agent/run-now for manually requested runs and executes all checks that are due.
3. It pushes the results of a tick as one batch to /api/v1/agent/results.
4. Every 5 minutes the agent reloads the check list and so picks up new or changed checks.
5. If fetching the check list fails five times in a row (for example with a revoked token or an unreachable dashboard), the agent empties its list instead of continuing with stale configuration. As soon as the dashboard answers again it carries on automatically.

The agent shuts down gracefully on SIGTERM and SIGINT. Start a test run without continuous operation with --run-once:

docker compose -f docker-compose.agent.yml run --rm watchdog-agent --run-once

An agent only receives active checks of active clients that are assigned to it and whose runner is Agent (Local). Interval and daily time apply as for dashboard checks, but the agent evaluates them locally and starts a check at the earliest once the interval minus 5 seconds has elapsed.

Management

After an agent is deleted its checks keep running in the dashboard's worker container. Agent-type checks such as disk or process then measure the dashboard container instead of the monitored machine. Assign them to another agent or deactivate them before deleting.

The agent's status in the UI is Online when it made contact within the last 10 minutes. Watchdog sends no notification when an agent fails, see Notifications.

For notes on permissions see Security and Privacy, for failure patterns see Troubleshooting.