Skip to content

traefik

traefik is my reverse proxy for every web interface, inside the lan and outside. it runs as one replica on the swarm. i chose it over nginx proxy manager, which it replaced, so every route is a line in git, reviewed in a pull request and checked before it deploys.

every request reaches an app through the VIP and traefik:

flowchart LR
  lan["lan"] -->|443| vip["swarm VIP"]
  net["internet"] --> cf["cloudflare"] --> router["router"]
  router -->|"443 to 1444"| vip
  router -->|"80 to 80"| vip
  vip --> traefik["traefik"]
  traefik -->|"oauth names"| o2p["oauth2-proxy"]
  traefik --> apps["the apps"]

the routes reach traefik from git, and homepage and gatus read traefik's API over a private network:

flowchart LR
  git["deploy branch in git"] --> sync["config-sync on every node"]
  sync -->|"routes.yml"| traefik["traefik"]
  hp["homepage"] -->|"traefik-api, 8080"| traefik
  gatus["gatus"] -->|"traefik-api, 8080"| traefik
compose.yml, 121 lines, 6 notes

each in the code opens a note on that line. download compose.yml

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
services:
  traefik:
    image: traefik:v3.7.13@sha256:24841fe2de7304c149343d877d2923b4c8800a38ba015dea9174c23b20e344a0
    environment:  # (1)!
      - TZ=America/Los_Angeles
      - CF_DNS_API_TOKEN_FILE=/run/secrets/traefik_cf_token_v1
    ports:  # (2)!
      - 80:80
      - 443:443
      - 1444:1444
    configs:
      - source: traefik_static
        target: /etc/traefik/traefik.yml
    secrets:
      - traefik_cf_token_v1
    volumes:
      - acme:/acme
      - type: volume
        source: dynamic
        target: /dynamic
        read_only: true
    networks:
      - default
      - traefik-api
    healthcheck:
      test: ["CMD", "traefik", "healthcheck", "--ping"]
      interval: 30s
      timeout: 5s
      retries: 3
      start_period: 20s
    deploy:
      mode: replicated
      replicas: 1
      update_config:  # (3)!
        order: start-first

  config-sync:
    image: registry.k8s.io/git-sync/git-sync:v4.7.1
    environment:
      - TZ=America/Los_Angeles
      - GITSYNC_REPO=ssh://git@ssh.github.com:443/<you>/<repo>.git
      - GITSYNC_REF=deploy/swarm/traefik
      - GITSYNC_PERIOD=60s
      - GITSYNC_ROOT=/git
      - GITSYNC_LINK=repo
      - GITSYNC_SSH_KEY_FILE=/dev/shm/key
      - GITSYNC_SSH_KNOWN_HOSTS_FILE=/known_hosts
      - GITSYNC_EXECHOOK_COMMAND=/sync-routes.sh
      - GITSYNC_FILTER=blob:none
      - GITSYNC_SPARSE_CHECKOUT_FILE=/sparse-checkout
      - GITSYNC_HTTP_BIND=:8080
    entrypoint:
      - /bin/sh
      - -c
      - |
        umask 077
        printf '%s\n' "$$(cat /run/secrets/gitsync_ssh_key_v1)" > /dev/shm/key
        [ -s /dev/shm/key ] || { echo "config-sync: could not write the SSH key to /dev/shm" >&2; exit 1; }
        mkdir -p /tmp/.ssh && cp /ssh_config /tmp/.ssh/config || { echo "config-sync: could not install the ssh config" >&2; exit 1; }
        exec /git-sync
    volumes:
      - type: bind
        source: /usr/share/zoneinfo
        target: /usr/share/zoneinfo
        read_only: true
      - dynamic:/dynamic
    configs:
      - source: traefik_known_hosts
        target: /known_hosts
      - source: traefik_ssh_config
        target: /ssh_config
      - source: traefik_sparse_checkout
        target: /sparse-checkout
      - source: traefik_sync_routes
        target: /sync-routes.sh
        mode: 0555
    secrets:
      - gitsync_ssh_key_v1
    deploy:  # (4)!
      mode: global

configs:
  traefik_static:
    file: ./traefik.yml
    name: traefik_static_v2
  traefik_known_hosts:
    file: ./known_hosts
    name: traefik_known_hosts_v2
  traefik_ssh_config:
    file: ./ssh_config
    name: traefik_ssh_config_v2
  traefik_sparse_checkout:
    file: ./sparse-checkout
    name: traefik_sparse_checkout_v1
  traefik_sync_routes:
    file: ./sync-routes.sh
    name: traefik_sync_routes_v1

networks:
  traefik-api:
    external: true

secrets:
  traefik_cf_token_v1:
    external: true
  gitsync_ssh_key_v1:
    external: true

volumes:
  acme:  # (5)!
    driver: local
    driver_opts:
      type: none
      device: "/mnt/docker-cephFS/traefik_acme"
      o: bind
  dynamic:  # (6)!
    driver: local
    driver_opts:
      type: tmpfs
      device: tmpfs
      o: "uid=65533,gid=65533,mode=0755,size=1m"
  1. traefik's ACME library, lego, reads any of its settings from a file named by <NAME>_FILE, so the cloudflare token comes from a docker secret and never enters the service spec.
  2. published through the ingress mesh, so every node's ports reach the one replica, and 1444 is the external listener the router's 443 forward points at. the mesh hides the client's address, as it does for npm.
  3. the new task starts before the old one stops, because every name behind the proxy goes down while it restarts.
  4. one per node, each writing into that node's own dynamic volume. traefik reloads when a file there changes, and on cephfs a write from another node never tells it, so the writer sits beside traefik wherever swarm puts it.
  5. the ACME account and the certificate, on cephfs so they follow traefik to any node. create it with mode 700 before the first deploy.
  6. in memory, one per node, and created owned by git-sync's own user, so its hook can write without a chown. it only empties when both services on a node stop, and the next sync refills it.
traefik.yml: the static configuration, 71 lines, 5 notes

each in the code opens a note on that line. download traefik.yml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
global:
  checkNewVersion: false
  sendAnonymousUsage: false

log:
  level: INFO

accessLog: {}

entryPoints:
  web:
    address: ":80"
    http:
      aliasHeadersStrategy: delete  # (1)!
      encodedCharacters: &encoded  # (2)!
        allowEncodedSlash: true
        allowEncodedBackSlash: false
        allowEncodedNullCharacter: false
        allowEncodedSemicolon: true
        allowEncodedPercent: true
        allowEncodedQuestionMark: true
        allowEncodedHash: true
      redirections:
        entryPoint:
          to: websecure
          scheme: https

  websecure:
    address: ":443"
    http:
      aliasHeadersStrategy: delete
      encodedCharacters: *encoded
      tls: &tls
        certResolver: letsencrypt
        domains:  # (3)!
          - main: mydomain.com
            sans:
              - "*.mydomain.com"

  external:
    address: ":1444"  # (4)!
    http:
      aliasHeadersStrategy: delete
      encodedCharacters: *encoded
      tls: *tls

  traefik:
    address: ":8080"
    http:
      aliasHeadersStrategy: delete
      encodedCharacters: *encoded

ping: {}

api:
  dashboard: true

providers:
  file:
    directory: /dynamic
    watch: true

certificatesResolvers:
  letsencrypt:
    acme:
      storage: /acme/acme.json
      dnsChallenge:
        provider: cloudflare
        resolvers:  # (5)!
          - "1.1.1.1:53"
          - "1.0.0.1:53"
  1. drops headers like X_Auth_User, which PHP or nginx backends read as X-Auth-User, so a client can't forge the identity headers a sign-in proxy adds.
  2. encoded null and backslash have no use in a path and are rejected. the rest stay allowed, as npm allowed them, because every route matches on the host name and never on the path.
  3. one wildcard certificate for every name. per-host certificates would put each internal name in the public certificate transparency logs, and adding a service would wait on issuing one.
  4. the external listener, a separate entrypoint from 443, so a route is reachable from outside only when the registry gives it an external mode.
  5. my domain is also my active directory domain inside the lan. left to the container's resolver, lego would look for the challenge's TXT record on the windows DNS servers and time out, since the record only exists in cloudflare.

before you deploy

  1. create a cloudflare API token with Zone / Zone / Read and Zone / DNS / Edit, limited to your zones, and store it as a docker secret. a name outside your domain needs its zone in the token too:

    docker secret create traefik_cf_token_v1 -
    • paste the token, then press Ctrl-D
    • traefik reads the token from the secret's file, and the service spec never holds it
    • in cloudflare, set each zone's encryption mode to full (strict), so cloudflare checks the certificate traefik serves. traefik also takes cloudflare's post-quantum key agreement to the origin, which needs TLS 1.3

  2. create the certificate folder on the cephfs mount:

    sudo mkdir -m 700 /mnt/docker-cephFS/traefik_acme
    • without the folder the bind fails and traefik doesn't start. that's better than starting with no certificate state

  3. create gitsync_ssh_key_v1, the read-only deploy key config-sync uses to read the repo, as in gatus's steps. gatus and homepage use the same key, so skip this if either is deployed

    • the stack declares the key as an external secret. without it the deploy fails with secret not found

  4. create the traefik-api overlay on a manager, with a fixed subnet:

    docker network create -d overlay --attachable --scope swarm --subnet 10.0.200.0/24 traefik-api
    • traefik's API router only answers this subnet, so the subnet has to match the registry. pick one clear of the /24s docker hands out in order from 10.0.0.0/8
    • no stack owns the overlay, so removing traefik, homepage or gatus leaves it for the other two

  5. check nothing on the swarm already listens on 80, 443 or 1444

state considerations

  • the certificate and the let's encrypt account are in /mnt/docker-cephFS/traefik_acme, a named bind on cephfs, see stack conventions. they follow traefik to whichever node it starts on
  • the routes are in a small in-memory volume on each node, filled by that node's config-sync. it holds nothing that isn't in git
  • there is one replica. traefik's free edition can't share its let's encrypt state between instances

network considerations

  • traefik publishes three ports on the ingress mesh, so each one answers on every node and on the keepalived VIP:

    port for
    80 redirects every name to https on the same name, keeping the path. the router forwards its 80 here
    443 the lan
    1444 outside. the router forwards its 443 here

  • the mesh replaces each client's address with its own, so traefik's logs show 10.0.0.x. nothing here decides anything on the client's address

  • homepage and gatus read traefik's read-only API on the unpublished 8080, over the traefik-api overlay. the dashboard's name is behind oauth, which a widget or a health check can't pass
    • 8080 also listens on the ingress network that every published service shares, so the router for the API only answers traefik-api's subnet
    • traefik doesn't join the discovery overlay. traefik faces the internet, and discovery carries the docker socket proxy

placement considerations

  • traefik can run on any node
  • config-sync runs on every node (mode: global). traefik reloads when a file in its folder changes, and a write on one node's cephfs mount never reaches a watcher on another, so each node writes its own copy of the routes

the registry

services.yaml has one entry per name. my script, tools/traefik_render.py, renders it into dynamic/routes.yml, which traefik reads. it also writes a gatus check for every route, and refuses an unknown field or a name used twice. CI fails if the files disagree with the registry, then starts the pinned traefik on them and fails if any route reports an error.

a service that already serves 443 with a valid certificate stays direct on the lan. home assistant and the unifi gateway keep their own names and certificates. traefik is for everything that lacks one or both.

adding a service

  1. add an entry to services.yaml:

    services.yaml
      jellyfin:
    upstream: http://192.168.1.86:8096
    internal: open
    native:
      auth: "password"
      mfa: "none"
    • the key is the name, so this serves jellyfin.mydomain.com
    • an app on the swarm is reached on its published port at the VIP. its own port keeps working and its stack doesn't change

  2. render the routes, and commit what the script writes

  3. merge. every node's config-sync picks up the routes within a minute, and traefik reloads without restarting

  4. add the name to the internal DNS, as a CNAME to the VIP's name. mine is windows DNS:

    Add-DnsServerResourceRecordCName -ZoneName "mydomain.com" -Name "jellyfin" -HostNameAlias "swarm.mydomain.com"
    • the name is the same inside and outside, so an app on a phone needs one address
    • never give a service a host's own name. truenas1 and frigate also carry ssh, smb and video, and pointing either at traefik breaks those

the fields

field does
upstream where traefik sends requests. a list gets a health check on / and a sticky cookie; proxmox uses all three nodes
upstream_tls: insecure skips the check of a backend's certificate, for one that is self-signed or expired
tls_name checks a backend's certificate against this name, for a backend reached by address
host the name, when it isn't the key under the domain. a name outside the domain's wildcard gets a certificate of its own from the same resolver
internal open, or oauth for oauth2-proxy in front. leave it out and the name isn't served on the lan
external oauth, own-mfa or public. leave it out and the name isn't served outside
mfa with own-mfa, a line saying how the app enforces MFA, so a reviewer sees the claim
native the app's own login and MFA, and its own address when it differs from upstream. required
external_oauth_paths, external_oauth_headers oauth in front of part of an own-mfa or public name only
cross_site_paths refuses, with 403, every request another site started, except to these exact paths. browsers say who started a request in the Sec-Fetch-Site header, which a page can't set, so a link or redirect elsewhere can't reach the rest of the app
redirect the name only redirects, keeping the path
root_redirect the front page redirects, and every other path is served
external_redirect the name is served on the lan, and outside every path redirects, keeping the path
services.yaml: the registry, one entry per name, 438 lines

download services.yaml

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
domain: mydomain.com

vip: 192.168.1.45

oauth:
  host: auth.mydomain.com
  upstream: http://192.168.1.45:4180

other_endpoints:
  - name: Frigate API
    address: "192.168.1.86:5000"
    auth: "none: the whole API, config and camera control included"
    mfa: "none"
  - name: MQTT (mosquitto)
    address: "192.168.1.45:1883"
    auth: "none: allow_anonymous true, so anyone can publish, Zigbee2MQTT's command topics included"
    mfa: "none"
  - name: Docker API proxy (truenas1)
    address: "192.168.1.86:2375"
    auth: "none: read-only Docker API"
    mfa: "none"
  - name: Ollama
    address: "192.168.1.86:30068"
    auth: "none"
    mfa: "none"
  - name: FlareSolverr
    address: "192.168.1.86:8191"
    auth: "none"
    mfa: "none"
  - name: Portainer agent (swarm nodes)
    address: "192.168.1.41, .42, .43:9001"
    auth: "none: no AGENT_SECRET, so any Portainer that reaches it can adopt the node (owner accepted, 2026-09-22)"
    mfa: "none"
  - name: Portainer agent (truenas1, pi-zwave01)
    address: "192.168.1.86:9001, 192.168.1.96:9001"
    auth: "unknown: not defined in this repo, so whether AGENT_SECRET is set is unchecked"
    mfa: "none"
  - name: Dozzle agents
    address: "192.168.1.41, .42, .43, .31, .86, .96:7007"
    auth: "mutual TLS with the shared dozzle agent certificate"
    mfa: "none"
  - name: oauth2-proxy
    address: "192.168.1.45:4180"
    auth: "it is the Entra sign-in; nothing is behind it but a 202"
    mfa: "Entra"

services:

  traefik:
    upstream: api@internal
    internal: oauth
    native:
      auth: "none: its read-only API has no login"
      mfa: "none"
      direct: "no port of its own; the API is also on :8080, for traefik-api only"

  apprise:
    upstream: http://192.168.1.45:8050
    internal: oauth
    native:
      auth: "none"
      mfa: "none"

  bentopdf:
    upstream: http://192.168.1.45:8091
    internal: open
    native:
      auth: "none"
      mfa: "none"

  blog:
    upstream: http://192.168.1.45:8180
    internal: open
    external: public
    native:
      auth: "WordPress password login (wp-login.php)"
      mfa: "unknown: check for a 2FA plugin"

  apex:
    host: mydomain.com
    upstream: http://192.168.1.45:8180
    internal: open
    external: public
    external_redirect: https://www.mydomain.com
    native:
      auth: "WordPress password login, at a custom address (wp-login.php answers 404); the network admin is here"
      mfa: "unknown: check for a 2FA plugin"

  mydomain1:
    host: mydomain1.com
    upstream: http://192.168.1.45:8180
    external: public
    native:
      auth: "WordPress password login, at a custom address (wp-login.php answers 404)"
      mfa: "unknown: check for a 2FA plugin"

  dozzle:
    upstream: http://192.168.1.41:8888
    internal: oauth
    native:
      auth: "none"
      mfa: "none"

  gatus:
    upstream: http://192.168.1.45:8085
    internal: open
    native:
      auth: "none"
      mfa: "none"

  homepage:
    upstream: http://192.168.1.45:3000
    internal: open
    external: oauth
    native:
      auth: "none"
      mfa: "none"

  ha:
    upstream: https://192.168.1.63:443
    tls_name: homeassistant.mydomain.com
    external: own-mfa
    mfa: Home Assistant's authenticator-app (TOTP) module, which every user must have turned on
    native:
      auth: "password"
      mfa: "authenticator-app module: not yet confirmed for every user"
      direct: "192.168.1.63:443 (homeassistant.mydomain.com)"

  infinitude:
    upstream: http://192.168.1.45:4000
    internal: open
    native:
      auth: "none"
      mfa: "none"

  npm:
    upstream: http://192.168.1.45:181
    internal: open
    native:
      auth: "password (NPM's admin)"
      mfa: "none"

  omni-tools:
    upstream: http://192.168.1.45:8090
    internal: open
    native:
      auth: "none"
      mfa: "none"

  portainer:
    upstream: http://192.168.1.45:9000
    internal: open
    external: own-mfa
    mfa: Portainer's own OAuth sign-in to Entra, where Conditional Access requires MFA; its password login and API keys need the proxy's Entra sign-in outside
    external_oauth_paths:
      - /api/auth
    external_oauth_headers:
      - X-API-Key
    native:
      auth: "native oauth (Entra); the initial admin's password (always on); API keys"
      mfa: "Entra; the password and API keys have none"
      direct: "192.168.1.45:9000, :9443"

  unifiapibrowser:
    upstream: http://192.168.1.45:8010
    internal: oauth
    native:
      auth: "none: its login is turned off (NOAPIBROWSERAUTH=1), and it holds a UniFi login"
      mfa: "none"

  www:
    upstream: http://192.168.1.45:8180
    internal: open
    external: public
    native:
      auth: "WordPress password login (wp-login.php)"
      mfa: "unknown: check for a 2FA plugin"

  adguard1:
    upstream: http://192.168.1.5:80
    internal: open
    native:
      auth: "password"
      mfa: "none"

  adguard2:
    upstream: http://192.168.1.6:3000
    internal: open
    native:
      auth: "password"
      mfa: "none"

  bazarr:
    upstream: http://192.168.1.86:6767
    internal: oauth
    native:
      auth: "password (form)"
      mfa: "none"

  grafana:
    upstream: http://192.168.1.86:30037
    internal: open
    native:
      auth: "native oauth (Entra), grafana's Azure AD setting; password form"
      mfa: "Entra; the password form has none"

  jellyfin:
    upstream: http://192.168.1.86:8096
    internal: open
    native:
      auth: "password"
      mfa: "none"

  open-webui:
    upstream: http://192.168.1.86:31028
    internal: open
    native:
      auth: "password (sign-up off)"
      mfa: "none"

  profilarr:
    upstream: http://192.168.1.86:6868
    internal: oauth
    native:
      auth: "password (native oauth supported, not set up)"
      mfa: "none"

  prometheus:
    upstream: http://192.168.1.86:30104
    internal: open
    native:
      auth: "none"
      mfa: "none"

  prowlarr:
    upstream: http://192.168.1.86:9696
    internal: oauth
    native:
      auth: "password (forms, required for all)"
      mfa: "none"

  qbittorrent:
    upstream: http://192.168.1.86:8080
    internal: oauth
    native:
      auth: "password (subnet whitelist off)"
      mfa: "none"

  radarr:
    upstream: http://192.168.1.86:7878
    internal: oauth
    native:
      auth: "password (forms, required for all)"
      mfa: "none"

  sabnzbd:
    upstream: http://192.168.1.86:8081
    internal: oauth
    native:
      auth: "password"
      mfa: "none"

  searxng:
    upstream: http://192.168.1.86:30053
    internal: open
    native:
      auth: "none"
      mfa: "none"

  seerr:
    upstream: http://192.168.1.86:5055
    internal: oauth
    external: oauth
    native:
      auth: "password (Jellyfin or local accounts)"
      mfa: "none"

  sonarr:
    upstream: http://192.168.1.86:8989
    internal: oauth
    native:
      auth: "password (forms, required for all)"
      mfa: "none"

  versity:
    upstream: https://truenas1.mydomain.com:30355
    internal: open
    native:
      auth: "S3 access key and secret"
      mfa: "none"

  glances-docker01:
    upstream: http://192.168.1.41:61208
    internal: open
    native:
      auth: "none"
      mfa: "none"

  glances-docker02:
    upstream: http://192.168.1.42:61208
    internal: open
    native:
      auth: "none"
      mfa: "none"

  glances-docker03:
    upstream: http://192.168.1.43:61208
    internal: open
    native:
      auth: "none"
      mfa: "none"

  glances-pi-zwave01:
    upstream: http://192.168.1.96:61208
    internal: open
    native:
      auth: "none"
      mfa: "none"

  glances-pve1:
    upstream: http://192.168.1.81:61208
    internal: open
    native:
      auth: "none"
      mfa: "none"

  glances-pve2:
    upstream: http://192.168.1.82:61208
    internal: open
    native:
      auth: "none"
      mfa: "none"

  glances-pve3:
    upstream: http://192.168.1.83:61208
    internal: open
    native:
      auth: "none"
      mfa: "none"

  glances-syn02:
    upstream: http://192.168.1.31:61208
    internal: open
    native:
      auth: "none"
      mfa: "none"

  glances-truenas1:
    upstream: http://192.168.1.86:61208
    internal: open
    native:
      auth: "none"
      mfa: "none"

  nvr:
    upstream: https://192.168.1.86:8971
    tls_name: frigate.mydomain.com
    internal: open
    native:
      auth: "password"
      mfa: "none"

  proxmox:
    upstream:
      - https://pve1.mydomain.com:8006
      - https://pve2.mydomain.com:8006
      - https://pve3.mydomain.com:8006
    internal: open
    native:
      auth: "native oauth (Entra), realm 'Azure AAD'; password realms pam, pve and AD"
      mfa: "Entra; root@pam: TOTP everywhere, and a WebAuthn passkey on proxmox.mydomain.com only (the nodes' :8006 fails Proxmox's origin check, which needs port 443; seen in pve1's log 2026-09-27); recovery keys held; other pam, pve and AD users: unknown"
      direct: "pve1-3.mydomain.com:8006"

  pbs:
    upstream: https://pbs1.mydomain.com:8007
    internal: open
    native:
      auth: "native oauth (Entra), realm 'EntraID'; password realms pam and pbs"
      mfa: "Entra; the password realms: unknown (per-user TOTP)"

  pbs-restore:
    upstream: https://192.168.1.45:8008
    upstream_tls: insecure
    internal: open
    cross_site_paths: [/, /login, /login/oidc/callback]
    native:
      auth: "Proxmox's sign-in, per user: native oauth (Entra) through the realm 'Azure AAD'; the pam, pve and AD password realms also appear"
      mfa: "Entra, its only MFA: the portal can't do Proxmox's TOTP. The password realms: none"

  syn01:
    upstream: https://syn01.mydomain.com:5101
    upstream_tls: insecure
    external: own-mfa
    mfa: DSM's 2-step verification (one-time codes), on for every account (owner, 2026-09-27)
    native:
      auth: "password"
      mfa: "one-time codes (owner)"
      direct: "192.168.1.30:5101 (syn01.mydomain.com)"

  syn02:
    upstream: https://syn02.mydomain.com:5101
    external: own-mfa
    mfa: DSM's 2-step verification (one-time codes), on for every account (owner, 2026-09-27)
    native:
      auth: "password"
      mfa: "one-time codes (owner)"
      direct: "192.168.1.31:5101 (syn02.mydomain.com)"

  unifi:
    upstream: https://192.168.1.1:443
    tls_name: unifi.mydomain.com
    external: own-mfa
    mfa: UniFi OS login, with MFA on the account (owner, 2026-09-27)
    native:
      auth: "UniFi OS login"
      mfa: "MFA on the account (owner)"
      direct: "192.168.1.1:443 (efg.mydomain.com)"

  watch-your-lan:
    upstream: http://192.168.1.31:8840
    internal: open
    native:
      auth: "none"
      mfa: "none"

  zigbee2mqtt:
    upstream: http://192.168.1.96:8080
    internal: oauth
    native:
      auth: "unknown: auth_token not checked (the Pi's SSH host key is not trusted here)"
      mfa: "none"

  zwave-js-ui:
    upstream: http://192.168.1.96:80
    internal: oauth
    native:
      auth: "password (owner)"
      mfa: "none"
dynamic/routes.yml: generated from the registry, never edited by hand, 600 lines

download routes.yml

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
http:
  routers:
    adguard1:
      rule: Host(`adguard1.mydomain.com`)
      entryPoints:
      - websecure
      service: adguard1
    adguard2:
      rule: Host(`adguard2.mydomain.com`)
      entryPoints:
      - websecure
      service: adguard2
    apex:
      rule: Host(`mydomain.com`)
      entryPoints:
      - websecure
      service: apex
    apex-external:
      rule: Host(`mydomain.com`)
      entryPoints:
      - external
      service: noop@internal
      middlewares:
      - apex-external-redirect
    api-private:
      rule: PathPrefix(`/api`) && (ClientIP(`127.0.0.1`) || ClientIP(`10.0.200.0/24`))
      entryPoints:
      - traefik
      service: api@internal
    apprise:
      rule: Host(`apprise.mydomain.com`)
      entryPoints:
      - websecure
      service: apprise
      middlewares:
      - oauth
    bazarr:
      rule: Host(`bazarr.mydomain.com`)
      entryPoints:
      - websecure
      service: bazarr
      middlewares:
      - oauth
    bentopdf:
      rule: Host(`bentopdf.mydomain.com`)
      entryPoints:
      - websecure
      service: bentopdf
    blog:
      rule: Host(`blog.mydomain.com`)
      entryPoints:
      - websecure
      service: blog
    blog-external:
      rule: Host(`blog.mydomain.com`)
      entryPoints:
      - external
      service: blog
    dozzle:
      rule: Host(`dozzle.mydomain.com`)
      entryPoints:
      - websecure
      service: dozzle
      middlewares:
      - oauth
    gatus:
      rule: Host(`gatus.mydomain.com`)
      entryPoints:
      - websecure
      service: gatus
    glances-docker01:
      rule: Host(`glances-docker01.mydomain.com`)
      entryPoints:
      - websecure
      service: glances-docker01
    glances-docker02:
      rule: Host(`glances-docker02.mydomain.com`)
      entryPoints:
      - websecure
      service: glances-docker02
    glances-docker03:
      rule: Host(`glances-docker03.mydomain.com`)
      entryPoints:
      - websecure
      service: glances-docker03
    glances-pi-zwave01:
      rule: Host(`glances-pi-zwave01.mydomain.com`)
      entryPoints:
      - websecure
      service: glances-pi-zwave01
    glances-pve1:
      rule: Host(`glances-pve1.mydomain.com`)
      entryPoints:
      - websecure
      service: glances-pve1
    glances-pve2:
      rule: Host(`glances-pve2.mydomain.com`)
      entryPoints:
      - websecure
      service: glances-pve2
    glances-pve3:
      rule: Host(`glances-pve3.mydomain.com`)
      entryPoints:
      - websecure
      service: glances-pve3
    glances-syn02:
      rule: Host(`glances-syn02.mydomain.com`)
      entryPoints:
      - websecure
      service: glances-syn02
    glances-truenas1:
      rule: Host(`glances-truenas1.mydomain.com`)
      entryPoints:
      - websecure
      service: glances-truenas1
    grafana:
      rule: Host(`grafana.mydomain.com`)
      entryPoints:
      - websecure
      service: grafana
    ha-external:
      rule: Host(`ha.mydomain.com`)
      entryPoints:
      - external
      service: ha
    homepage:
      rule: Host(`homepage.mydomain.com`)
      entryPoints:
      - websecure
      service: homepage
    homepage-external:
      rule: Host(`homepage.mydomain.com`)
      entryPoints:
      - external
      service: homepage
      middlewares:
      - oauth
    infinitude:
      rule: Host(`infinitude.mydomain.com`)
      entryPoints:
      - websecure
      service: infinitude
    jellyfin:
      rule: Host(`jellyfin.mydomain.com`)
      entryPoints:
      - websecure
      service: jellyfin
    npm:
      rule: Host(`npm.mydomain.com`)
      entryPoints:
      - websecure
      service: npm
    nvr:
      rule: Host(`nvr.mydomain.com`)
      entryPoints:
      - websecure
      service: nvr
    oauth2-proxy:
      rule: Host(`auth.mydomain.com`)
      entryPoints:
      - websecure
      - external
      service: oauth2-proxy
    omni-tools:
      rule: Host(`omni-tools.mydomain.com`)
      entryPoints:
      - websecure
      service: omni-tools
    open-webui:
      rule: Host(`open-webui.mydomain.com`)
      entryPoints:
      - websecure
      service: open-webui
    pbs:
      rule: Host(`pbs.mydomain.com`)
      entryPoints:
      - websecure
      service: pbs
    pbs-restore:
      rule: Host(`pbs-restore.mydomain.com`)
      entryPoints:
      - websecure
      service: pbs-restore
    pbs-restore-cross-site:
      rule: Host(`pbs-restore.mydomain.com`) && HeaderRegexp(`Sec-Fetch-Site`, `^(cross-site|same-site)$`) && !(Path(`/`) || Path(`/login`) || Path(`/login/oidc/callback`))
      entryPoints:
      - websecure
      service: noop@internal
      middlewares:
      - refuse
      priority: 20159
    portainer:
      rule: Host(`portainer.mydomain.com`)
      entryPoints:
      - websecure
      service: portainer
    portainer-external:
      rule: Host(`portainer.mydomain.com`)
      entryPoints:
      - external
      service: portainer
    portainer-external-oauth:
      rule: Host(`portainer.mydomain.com`) && (Path(`/api/auth`) || Path(`/api%2Fauth`) || HeaderRegexp(`X-API-Key`, `.+`))
      entryPoints:
      - external
      service: portainer
      middlewares:
      - oauth
      priority: 10110
    profilarr:
      rule: Host(`profilarr.mydomain.com`)
      entryPoints:
      - websecure
      service: profilarr
      middlewares:
      - oauth
    prometheus:
      rule: Host(`prometheus.mydomain.com`)
      entryPoints:
      - websecure
      service: prometheus
    prowlarr:
      rule: Host(`prowlarr.mydomain.com`)
      entryPoints:
      - websecure
      service: prowlarr
      middlewares:
      - oauth
    proxmox:
      rule: Host(`proxmox.mydomain.com`)
      entryPoints:
      - websecure
      service: proxmox
    qbittorrent:
      rule: Host(`qbittorrent.mydomain.com`)
      entryPoints:
      - websecure
      service: qbittorrent
      middlewares:
      - oauth
    radarr:
      rule: Host(`radarr.mydomain.com`)
      entryPoints:
      - websecure
      service: radarr
      middlewares:
      - oauth
    sabnzbd:
      rule: Host(`sabnzbd.mydomain.com`)
      entryPoints:
      - websecure
      service: sabnzbd
      middlewares:
      - oauth
    mydomain1-external:
      rule: Host(`mydomain1.com`)
      entryPoints:
      - external
      service: mydomain1
      tls:
        certResolver: letsencrypt
        domains:
        - main: mydomain1.com
    searxng:
      rule: Host(`searxng.mydomain.com`)
      entryPoints:
      - websecure
      service: searxng
    seerr:
      rule: Host(`seerr.mydomain.com`)
      entryPoints:
      - websecure
      service: seerr
      middlewares:
      - oauth
    seerr-external:
      rule: Host(`seerr.mydomain.com`)
      entryPoints:
      - external
      service: seerr
      middlewares:
      - oauth
    sonarr:
      rule: Host(`sonarr.mydomain.com`)
      entryPoints:
      - websecure
      service: sonarr
      middlewares:
      - oauth
    syn01-external:
      rule: Host(`syn01.mydomain.com`)
      entryPoints:
      - external
      service: syn01
    syn02-external:
      rule: Host(`syn02.mydomain.com`)
      entryPoints:
      - external
      service: syn02
    traefik:
      rule: Host(`traefik.mydomain.com`)
      entryPoints:
      - websecure
      service: api@internal
      middlewares:
      - oauth
    unifi-external:
      rule: Host(`unifi.mydomain.com`)
      entryPoints:
      - external
      service: unifi
    unifiapibrowser:
      rule: Host(`unifiapibrowser.mydomain.com`)
      entryPoints:
      - websecure
      service: unifiapibrowser
      middlewares:
      - oauth
    versity:
      rule: Host(`versity.mydomain.com`)
      entryPoints:
      - websecure
      service: versity
    watch-your-lan:
      rule: Host(`watch-your-lan.mydomain.com`)
      entryPoints:
      - websecure
      service: watch-your-lan
    www:
      rule: Host(`www.mydomain.com`)
      entryPoints:
      - websecure
      service: www
    www-external:
      rule: Host(`www.mydomain.com`)
      entryPoints:
      - external
      service: www
    zigbee2mqtt:
      rule: Host(`zigbee2mqtt.mydomain.com`)
      entryPoints:
      - websecure
      service: zigbee2mqtt
      middlewares:
      - oauth
    zwave-js-ui:
      rule: Host(`zwave-js-ui.mydomain.com`)
      entryPoints:
      - websecure
      service: zwave-js-ui
      middlewares:
      - oauth
  middlewares:
    apex-external-redirect:
      redirectRegex:
        regex: ^https?://[^/]+(.*)$
        replacement: https://www.mydomain.com${1}
        permanent: true
    oauth:
      forwardAuth:
        address: http://192.168.1.45:4180/
        trustForwardHeader: false
        maxResponseBodySize: 1048576
        authResponseHeaders:
        - X-Auth-Request-User
        - X-Auth-Request-Email
    refuse:
      ipAllowList:
        sourceRange:
        - 192.0.2.1/32
  services:
    adguard1:
      loadBalancer:
        servers:
        - url: http://192.168.1.5:80
    adguard2:
      loadBalancer:
        servers:
        - url: http://192.168.1.6:3000
    apex:
      loadBalancer:
        servers:
        - url: http://192.168.1.45:8180
    apprise:
      loadBalancer:
        servers:
        - url: http://192.168.1.45:8050
    bazarr:
      loadBalancer:
        servers:
        - url: http://192.168.1.86:6767
    bentopdf:
      loadBalancer:
        servers:
        - url: http://192.168.1.45:8091
    blog:
      loadBalancer:
        servers:
        - url: http://192.168.1.45:8180
    dozzle:
      loadBalancer:
        servers:
        - url: http://192.168.1.41:8888
    gatus:
      loadBalancer:
        servers:
        - url: http://192.168.1.45:8085
    glances-docker01:
      loadBalancer:
        servers:
        - url: http://192.168.1.41:61208
    glances-docker02:
      loadBalancer:
        servers:
        - url: http://192.168.1.42:61208
    glances-docker03:
      loadBalancer:
        servers:
        - url: http://192.168.1.43:61208
    glances-pi-zwave01:
      loadBalancer:
        servers:
        - url: http://192.168.1.96:61208
    glances-pve1:
      loadBalancer:
        servers:
        - url: http://192.168.1.81:61208
    glances-pve2:
      loadBalancer:
        servers:
        - url: http://192.168.1.82:61208
    glances-pve3:
      loadBalancer:
        servers:
        - url: http://192.168.1.83:61208
    glances-syn02:
      loadBalancer:
        servers:
        - url: http://192.168.1.31:61208
    glances-truenas1:
      loadBalancer:
        servers:
        - url: http://192.168.1.86:61208
    grafana:
      loadBalancer:
        servers:
        - url: http://192.168.1.86:30037
    ha:
      loadBalancer:
        servers:
        - url: https://192.168.1.63:443
        serversTransport: ha-tls
    homepage:
      loadBalancer:
        servers:
        - url: http://192.168.1.45:3000
    infinitude:
      loadBalancer:
        servers:
        - url: http://192.168.1.45:4000
    jellyfin:
      loadBalancer:
        servers:
        - url: http://192.168.1.86:8096
    npm:
      loadBalancer:
        servers:
        - url: http://192.168.1.45:181
    nvr:
      loadBalancer:
        servers:
        - url: https://192.168.1.86:8971
        serversTransport: nvr-tls
    oauth2-proxy:
      loadBalancer:
        servers:
        - url: http://192.168.1.45:4180
    omni-tools:
      loadBalancer:
        servers:
        - url: http://192.168.1.45:8090
    open-webui:
      loadBalancer:
        servers:
        - url: http://192.168.1.86:31028
    pbs:
      loadBalancer:
        servers:
        - url: https://pbs1.mydomain.com:8007
    pbs-restore:
      loadBalancer:
        servers:
        - url: https://192.168.1.45:8008
        serversTransport: insecure
    portainer:
      loadBalancer:
        servers:
        - url: http://192.168.1.45:9000
    profilarr:
      loadBalancer:
        servers:
        - url: http://192.168.1.86:6868
    prometheus:
      loadBalancer:
        servers:
        - url: http://192.168.1.86:30104
    prowlarr:
      loadBalancer:
        servers:
        - url: http://192.168.1.86:9696
    proxmox:
      loadBalancer:
        servers:
        - url: https://pve1.mydomain.com:8006
        - url: https://pve2.mydomain.com:8006
        - url: https://pve3.mydomain.com:8006
        healthCheck:
          path: /
          interval: 30s
          timeout: 5s
        sticky:
          cookie:
            name: proxmox_server
            secure: true
            httpOnly: true
    qbittorrent:
      loadBalancer:
        servers:
        - url: http://192.168.1.86:8080
    radarr:
      loadBalancer:
        servers:
        - url: http://192.168.1.86:7878
    sabnzbd:
      loadBalancer:
        servers:
        - url: http://192.168.1.86:8081
    mydomain1:
      loadBalancer:
        servers:
        - url: http://192.168.1.45:8180
    searxng:
      loadBalancer:
        servers:
        - url: http://192.168.1.86:30053
    seerr:
      loadBalancer:
        servers:
        - url: http://192.168.1.86:5055
    sonarr:
      loadBalancer:
        servers:
        - url: http://192.168.1.86:8989
    syn01:
      loadBalancer:
        servers:
        - url: https://syn01.mydomain.com:5101
        serversTransport: insecure
    syn02:
      loadBalancer:
        servers:
        - url: https://syn02.mydomain.com:5101
    unifi:
      loadBalancer:
        servers:
        - url: https://192.168.1.1:443
        serversTransport: unifi-tls
    unifiapibrowser:
      loadBalancer:
        servers:
        - url: http://192.168.1.45:8010
    versity:
      loadBalancer:
        servers:
        - url: https://truenas1.mydomain.com:30355
    watch-your-lan:
      loadBalancer:
        servers:
        - url: http://192.168.1.31:8840
    www:
      loadBalancer:
        servers:
        - url: http://192.168.1.45:8180
    zigbee2mqtt:
      loadBalancer:
        servers:
        - url: http://192.168.1.96:8080
    zwave-js-ui:
      loadBalancer:
        servers:
        - url: http://192.168.1.96:80
  serversTransports:
    ha-tls:
      serverName: homeassistant.mydomain.com
    insecure:
      insecureSkipVerify: true
    nvr-tls:
      serverName: frigate.mydomain.com
    unifi-tls:
      serverName: unifi.mydomain.com
sync-routes.sh: run by config-sync after each sync, 35 lines

download sync-routes.sh

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
#!/bin/sh
set -eu

src="./stacks/swarm/traefik/dynamic"
dst="/dynamic"

[ -d "$src" ] || { echo "config-sync: $src not found under $(pwd)" >&2; exit 1; }
[ -d "$dst" ] || { echo "config-sync: $dst is not mounted" >&2; exit 1; }

if ! ls "$src"/*.yml >/dev/null 2>&1; then
    echo "config-sync: no *.yml under $src, refusing to reconcile" >&2
    exit 1
fi

delivered=0
for f in "$src"/*.yml; do
    name="${f##*/}"
    tmp="$dst/.${name}.tmp"
    cp "$f" "$tmp"
    chmod 0644 "$tmp"
    mv "$tmp" "$dst/$name"
    delivered=$((delivered + 1))
done

removed=0
for g in "$dst"/*.yml; do
    [ -f "$g" ] || continue
    name="${g##*/}"
    [ -f "$src/$name" ] && continue
    rm -f "$g"
    removed=$((removed + 1))
    echo "config-sync: removed $name, no longer in git"
done

echo "config-sync: delivered $delivered, removed $removed, at ${GITSYNC_HASH:-unknown}"

oauth, for names with no login of their own

a name marked oauth asks oauth2-proxy about every request first. a browser that isn't signed in goes to entra, and comes back to the page it asked for. the sign-in lands on auth.mydomain.com and sets a cookie on the whole domain, so one sign-in covers every name. the oauth2-proxy page has the entra side.

  • on the lan, a name is open or oauth, chosen per name. oauth protects the name only: the app's own port still answers on the lan
  • an app whose users sign in with entra through its own OIDC setting, such as proxmox or portainer, needs no oauth in front of it
  • auth has every name's sign-in and MFA, on the lan and outside

what faces the internet

outside, every name needs MFA or has to be a public site. external has no open, and the script refuses a name that sets one.

  • oauth puts oauth2-proxy in front, so users sign in with entra first
  • own-mfa relies on the app's own login, and needs the mfa line
  • public is for a site meant for anyone, such as the blog

some apps with their own MFA have a second way in that skips it. portainer's first admin can always log in with a password, and an API key skips the login altogether. external_oauth_paths and external_oauth_headers put oauth in front of just those requests. each path is also matched with %2F in place of its inner slashes, because the app reads /api%2Fauth as /api/auth and a plain rule doesn't.

checking it

auth.mydomain.com is the one name with nothing in front of it. check it by name, as a browser reaches it:

curl -sS -w ' %{http_code} verify=%{ssl_verify_result}\n' https://auth.mydomain.com/ping

it should print OK 200 verify=0. then check the certificate, straight from the VIP:

openssl s_client -connect 192.168.1.45:443 -servername traefik.mydomain.com </dev/null 2>/dev/null | openssl x509 -noout -issuer -ext subjectAltName -enddate
  • the dashboard is at https://traefik.mydomain.com, behind oauth. it lists every route with its status
  • gatus checks every route through the VIP, with the name as the Host header and no DNS. a mis-wired route goes red on its own, and a DNS outage doesn't turn them all red
  • gatus also checks the certificate's expiry and goes red with 21 days left. traefik renews at 30