Skip to content

dozzle logs

secrets

i make my secrets from my own secret store, which only fits my setup. the docker secret create commands here are plain swarm: use them, or however you normally make secrets.

one dozzle UI shows the logs of every container on every docker host, fed by an agent on each host. the hub (the UI) has no docker socket, so every host reaches it through an agent, docker01 included.

piece where port
hub, the UI swarm, one replica pinned to docker01 8888, host mode, at 192.168.1.41:8888
agent swarm, mode: global, one per node 7007, host mode
agent truenas1, syn02, pi-zwave01, each its own stack 7007
compose.yml: the swarm, the hub and one agent per node, 65 lines, 4 notes

each in the code opens a note on that line. download compose.yml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
services:
  dozzle:
    image: amir20/dozzle:v11.1.0@sha256:7c4fb7f8124f5dea15ade881535cc19cc9a03e66639fd28154ff43a93b78944b
    environment:
      - TZ=America/Los_Angeles
      - DOZZLE_REMOTE_AGENT=192.168.1.41:7007|Docker01|Swarm,192.168.1.42:7007|Docker02|Swarm,192.168.1.43:7007|docker03|Swarm,192.168.1.86:7007|truenas1|TrueNAS,192.168.1.31:7007|syn02|Other,192.168.1.96:7007|pi-zwave01|Other
      - DOZZLE_CERT=/run/secrets/dozzle_agent_cert_v1
      - DOZZLE_KEY=/run/secrets/dozzle_agent_key_v1
    volumes:
      - type: bind  # (1)!
        source: /usr/share/zoneinfo
        target: /usr/share/zoneinfo
        read_only: true
      - data:/data
    secrets:
      - dozzle_agent_cert_v1
      - dozzle_agent_key_v1
    ports:  # (2)!
      - target: 8080
        published: 8888
        mode: host
    deploy:
      mode: replicated
      replicas: 1
      placement:
        constraints:
          - node.hostname == Docker01
      labels:  # (3)!
        - homepage.group=Monitoring
        - homepage.name=Dozzle
        - homepage.icon=dozzle.png
        - homepage.href=https://dozzle.mydomain.com
        - homepage.description=Container logs, all six hosts

  dozzle-agent:
    image: amir20/dozzle:v11.1.0@sha256:7c4fb7f8124f5dea15ade881535cc19cc9a03e66639fd28154ff43a93b78944b
    command: agent
    environment:
      - TZ=America/Los_Angeles
      - DOZZLE_CERT=/run/secrets/dozzle_agent_cert_v1
      - DOZZLE_KEY=/run/secrets/dozzle_agent_key_v1
    volumes:
      - type: bind
        source: /usr/share/zoneinfo
        target: /usr/share/zoneinfo
        read_only: true
      - /var/run/docker.sock:/var/run/docker.sock:ro
    secrets:
      - dozzle_agent_cert_v1
      - dozzle_agent_key_v1
    ports:  # (4)!
      - target: 7007
        published: 7007
        mode: host
    deploy:
      mode: global

secrets:
  dozzle_agent_cert_v1:
    external: true
  dozzle_agent_key_v1:
    external: true

volumes:
  data:
  1. the dozzle image has no time zone data, so TZ alone changes nothing. the hub and every agent mount the host's /usr/share/zoneinfo read-only.
  2. in host mode, 8888 answers only on docker01, the node the hub is pinned to. through the ingress mesh, every node would answer on it.
  3. the tile on the homepage dashboard, under deploy.labels because homepage reads labels from the swarm's service specs. the agents have no UI, so they carry no labels.
  4. in host mode, 7007 on a node's own address is that node's agent, so DOZZLE_REMOTE_AGENT can name each one. through the ingress mesh, a connection could reach any node's agent.

before you deploy

  1. make your own certificate pair. the hub and the agents authenticate each other with it:

    docker run --name dozzle-certgen amir20/dozzle:v11.1.0 \
  generate-certs --cert-out /dozzle_cert.pem --key-out /dozzle_key.pem
docker cp dozzle-certgen:/dozzle_cert.pem .
docker cp dozzle-certgen:/dozzle_key.pem .
docker rm dozzle-certgen
    • the image has a pair built in, and every copy has the same one. anyone who can reach port 7007 with a stock image can read every log on that host
    • the pair is valid for five years. replace it on the hub and every agent at the same time

  2. on a swarm manager, make the pair two docker secrets:

    docker secret create dozzle_agent_cert_v1 dozzle_cert.pem
docker secret create dozzle_agent_key_v1 dozzle_key.pem
    • the hub and the swarm's agents read them through DOZZLE_CERT and DOZZLE_KEY

  3. copy the pair to each standalone host, into the directory its agent binds from:

    host directory
    truenas1 /mnt/fast/configs/dozzle
    syn02 /volume1/docker/dozzle
    pi-zwave01 /docker-data/dozzle
    • docker creates a directory at a missing bind source, and the agent then fails its handshake
    • a host that loses its copy gets the same pair back. a new pair has to reach the hub and every agent at once

  4. deploy the agents on truenas1, syn02 and pi-zwave01 first, then the swarm's stack, which brings up its agents and the hub together

    • a hub with no agents answering is an empty UI

state considerations

  • the hub keeps /data in data, a plain named volume with no driver_opts, so docker keeps it on docker01, where the hub is pinned
  • the certificate pair is two docker secrets, dozzle_agent_cert_v1 and dozzle_agent_key_v1, which the hub and the swarm's agents mount. the standalone hosts bind the same pair from a directory on the host, as in before you deploy
  • the agents keep nothing

network considerations

  • the hub publishes the UI's 8080 as 8888 in host mode, so it answers on docker01 only, at 192.168.1.41:8888. its dashboard tile links to it by name, https://dozzle.mydomain.com, which is behind oauth on the lan
  • each agent publishes 7007 in host mode, so a node's own address reaches that node's agent. the hub names every agent by that address in the agent list. see stack conventions
  • the stack joins no shared overlay network

placement considerations

  • the hub is one replica pinned to docker01 (node.hostname == Docker01), so its address stays put instead of following the task between nodes
  • the swarm's agents run mode: global, one per node, and a node added to the swarm gets one

the agent list

the hub lists every agent by the host's own address, with a name and a sidebar group:

swarm/dozzle/compose.yml
      - DOZZLE_REMOTE_AGENT=192.168.1.41:7007|Docker01|Swarm,192.168.1.42:7007|Docker02|Swarm,192.168.1.43:7007|docker03|Swarm,192.168.1.86:7007|truenas1|TrueNAS,192.168.1.31:7007|syn02|Other,192.168.1.96:7007|pi-zwave01|Other

don't use the keepalived VIP. it moves, and the hub would show one node's logs under another's name.

the agents on the standalone hosts

truenas1, syn02 and pi-zwave01 are not in the swarm, so none of them can mount a swarm secret. each runs the agent as a stack of its own. it binds the pair read-only from the directory in before you deploy to /dozzle_cert.pem and /dozzle_key.pem, where dozzle looks by default.

compose.yml: the agent on truenas1, 19 lines

download compose.yml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
services:
  dozzle-agent:
    image: amir20/dozzle:v11.1.0@sha256:7c4fb7f8124f5dea15ade881535cc19cc9a03e66639fd28154ff43a93b78944b
    environment:
      - TZ=America/Los_Angeles
    command: agent
    restart: unless-stopped
    volumes:
      - type: bind
        source: /usr/share/zoneinfo
        target: /usr/share/zoneinfo
        read_only: true
        bind:
          create_host_path: false
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - /mnt/fast/configs/dozzle/dozzle_cert.pem:/dozzle_cert.pem:ro
      - /mnt/fast/configs/dozzle/dozzle_key.pem:/dozzle_key.pem:ro
    ports:
      - 7007:7007
compose.yml: the agent on syn02, 20 lines

download compose.yml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
services:
  dozzle-agent:
    image: amir20/dozzle:v11.1.0@sha256:7c4fb7f8124f5dea15ade881535cc19cc9a03e66639fd28154ff43a93b78944b
    environment:
      - TZ=America/Los_Angeles
    command: agent
    container_name: dozzle-agent
    restart: unless-stopped
    volumes:
      - type: bind
        source: /usr/share/zoneinfo
        target: /usr/share/zoneinfo
        read_only: true
        bind:
          create_host_path: false
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - /volume1/docker/dozzle/dozzle_cert.pem:/dozzle_cert.pem:ro
      - /volume1/docker/dozzle/dozzle_key.pem:/dozzle_key.pem:ro
    ports:
      - 7007:7007
compose.yml: the agent on pi-zwave01, 20 lines

download compose.yml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
services:
  dozzle-agent:
    image: amir20/dozzle:v11.1.0@sha256:7c4fb7f8124f5dea15ade881535cc19cc9a03e66639fd28154ff43a93b78944b
    environment:
      - TZ=America/Los_Angeles
    command: agent
    container_name: dozzle-agent
    restart: unless-stopped
    volumes:
      - type: bind
        source: /usr/share/zoneinfo
        target: /usr/share/zoneinfo
        read_only: true
        bind:
          create_host_path: false
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - /docker-data/dozzle/dozzle_cert.pem:/dozzle_cert.pem:ro
      - /docker-data/dozzle/dozzle_key.pem:/dozzle_key.pem:ro
    ports:
      - 7007:7007

adding a host

  1. copy the pair to the new host
  2. add its agent stack, as for syn02 or the pi
  3. add the host to the hub's DOZZLE_REMOTE_AGENT list, which redeploys the hub

checking it

the hub's page lists the hosts, and whether it can reach each agent:

curl -s http://192.168.1.41:8888/ | grep '"hosts"' | jq -r '.hosts[] | "\(.name) \(.available)"'

Docker01 true
Docker02 true
docker03 true
pi-zwave01 true
syn02 true
truenas1 true

false is an agent the hub can't reach or that refuses its certificate. gatus checks each agent's port on each host's own address.