image updates with renovate

what it is

renovate reads the compose files in git, finds images with a newer version, and opens a pull request that changes the tag and digest. it never touches a running container.

why

• watchtower and shepherd updated running containers with nothing to review, and watchtower is archived upstream now
• a tag like latest means any restart can change what runs
• with renovate every update is a PR: a diff and a changelog link before anything moves, and i can refuse one or revert it
• swarm services and standalone containers are handled the same way, and nothing needs the docker socket

the cost is that updates are no longer automatic: i read and merge the PRs.

how an update flows

flowchart LR
    A[renovate<br>daily github action] -->|opens a PR| B[new tag and digest<br>in one compose file]
    B -->|ci passes, i merge| C[main]
    C -->|promote workflow| D[deploy/env/stack<br>moves forward]
    D -->|portainer polls, 5 min| E[that one stack<br>redeploys]

one PR touches one stack, so a merge redeploys that stack and nothing else, see stacks in git.

pin images by digest

    image: mysql:8.0@sha256:968e12b1fde035655c7a940db808b47372b70128293a38a3914e0b291c306e5e

• with the digest, a restart or a node failover pulls the same image that ran before. the version only changes when a renovate PR changes this line
• renovate updates the tag and the digest together
• pinDigests is off in my config, so renovate won't add digests by itself. i pin a stack when i adopt it, and renovate keeps the pins current from then on
• images tracking latest still get a digest; their updates arrive as digest-only PRs

set it up

your stacks need to be in git first.

pre-reqs

1. compose files in a github repo
2. admin on that repo (you are adding secrets and a workflow)
3. a github account you can create an app under

the repo is private and i want the credentials to be mine, so i run it as a scheduled github action, not the mend-hosted app.

create a github app

use an app, not a PAT. a PAT carries all your own permissions everywhere. a fine grained one expires within a year, and after that renovate stops opening PRs with no error to tell you. an app install doesn't expire.

1. github > settings > developer settings > GitHub Apps > New GitHub App
2. name it whatever, homepage url can be your repo
3. uncheck webhook active, you don't need it
4. set the repository permissions per the table below
5. create it, then Generate a private key, it downloads a .pem
6. note the App ID from the top of the app page
7. Install App > install it on just the one repo, not all repos

permission	access	why
Contents	read & write	it pushes the branch
Pull requests	read & write	it opens the PR
Issues	read & write	the dependency dashboard is an issue
Workflows	read & write	only needed if it bumps .github/workflows versions

the list in the UI is alphabetical, so Contents is nowhere near Pull requests and Workflows is right at the bottom. it's easy to miss one.

add the two secrets

repo > settings > secrets and variables > actions > New repository secret

name	value
RENOVATE_APP_ID	the app id number
RENOVATE_APP_PRIVATE_KEY	whole contents of the .pem, including the BEGIN/END lines

add the workflow

.github/workflows/renovate.yml

name: renovate

on:
  schedule:
    - cron: "23 14 * * *"       # daily
  issues:
    types: [edited]             # someone ticked a box on the dashboard
  workflow_dispatch:

concurrency:
  group: renovate
  cancel-in-progress: false

jobs:
  renovate:
    if: >-
      github.event_name != 'issues' ||
      (github.event.issue.title == 'Dependency Dashboard' &&
       github.event.sender.type == 'User')
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v5

      - name: mint an installation token
        id: app-token
        uses: actions/create-github-app-token@v3.2.0
        with:
          app-id: ${{ secrets.RENOVATE_APP_ID }}
          private-key: ${{ secrets.RENOVATE_APP_PRIVATE_KEY }}

      - uses: renovatebot/github-action@v46.2.3
        with:
          token: ${{ steps.app-token.outputs.token }}
        env:
          RENOVATE_REPOSITORIES: ${{ github.repository }}
          RENOVATE_PLATFORM: github
          RENOVATE_BASE_BRANCHES: main

do not use the built in GITHUB_TOKEN here. PRs opened with it don't trigger pull_request workflows, so your own validation never runs, and if main requires those checks the renovate PRs can never be merged.

• it runs when you tick a box. ticking a checkbox on the dashboard edits the issue, and that starts a run, so the PR turns up in a couple of minutes rather than at the next scheduled run
• the if: stops a loop. renovate rewrites the dashboard at the end of every run with the app's token, and events from an app token do trigger workflows. the app is a Bot sender and you are a User, so only your edits start a run
• the daily run is a safety net. it catches a tick whose event got missed, and it keeps open PRs current. renovate.json's schedule still decides when new update PRs open

add renovate.json

it goes in the repo root. this is a cut down version of mine.

{
  "$schema": "https://docs.renovatebot.com/renovate-schema.json",
  "extends": ["config:recommended"],
  "timezone": "America/Los_Angeles",
  "schedule": ["* * * * 1"],
  "prConcurrentLimit": 5,
  "dependencyDashboard": true,
  "ignorePaths": ["_attic/**"],
  "commitMessageSuffix": " [{{packageFileDir}}]",
  "packageRules": [
    {
      "matchCategories": ["docker"],
      "pinDigests": false
    },
    {
      "matchUpdateTypes": ["patch"],
      "groupName": "patch updates {{packageFileDir}}"
    },
    {
      "matchUpdateTypes": ["digest"],
      "groupName": "digest updates {{packageFileDir}}"
    },
    {
      "matchUpdateTypes": ["major"],
      "dependencyDashboardApproval": true
    },
    {
      "matchPackageNames": ["adguard/adguardhome", "jc21/nginx-proxy-manager"],
      "dependencyDashboardApproval": true
    },
    {
      "matchPackageNames": ["portainer/portainer-ee", "portainer/agent"],
      "dependencyDashboardApproval": true
    }
  ]
}

with the layout from the gitops page that gives PR titles like:

Update jc21/nginx-proxy-manager Docker tag to v2.12.6 [stacks/swarm/npm]
patch updates stacks/pi-zwave01/zwave-js-ui

merging the first one moves deploy/swarm/npm and nothing else.

what each setting does:

• commitMessageSuffix puts the stack directory in the commit subject. with dozens of stacks that keeps the git log readable
• groupName with {{packageFileDir}} gives you one PR per stack instead of one per image, which is a lot less noise
• dependencyDashboardApproval puts the update on the dashboard with a checkbox and waits for me, instead of opening a PR. i use it for major bumps, anything with a database in it, anything home assistant has to stay compatible with, portainer and its agent, and anything thats bitten me before
• enabled: false for images you never want it to touch
• ignorePaths for junk drawers, otherwise it raises PRs for stacks you aren't running
• if you want a whole stack left alone rather than an image, match the directory instead:

{
  "matchFileNames": ["stacks/<env>/<stack>/**"],
  "enabled": false
}

run it

run it by hand the first time, don't wait until monday:

repo > Actions > renovate > Run workflow

on its first run it opens an issue called Dependency Dashboard listing everything it found, and opens whatever PRs it is allowed to.

tick a checkbox on the dashboard and a run starts, and the PR appears a couple of minutes later. a tick overrides the schedule, so this works any day of the week.

merge a PR and your normal deploy path does the rest.

lock the branches down, last

do this after renovate has opened its first PR. you cannot require a status check until a run has reported one, and the name you enter has to match the check's name.

on main:

rule	why
require a pull request before merging	without it, anything can be pushed straight to main
require status checks, tick your validate job	a PR can't merge until ci passes
block force pushes	main's history can't be rewritten

the check name in the ruleset is the job name: from your workflow, not the file name and not the workflow name. rename the job later and every PR waits forever on a check that never reports, with no error to tell you why. change both together or not at all.

on deploy/*, block force pushes so a deploy branch can only move forward. mind the ** trap from the gitops page: a ruleset targeting refs/heads/deploy/** matches nothing, and shows as active while enforcing nothing.

do not add renovate's app to any bypass list. its PRs go through the same gate as yours.

rolling one back once its merged is a revert on main, promoted forward, see rolling back a compose.

notes

• if renovate dies with FORBIDDEN mentioning ["repository","issues"], you missed the Issues permission. the dashboard is an issue
• if it does all your compose files fine and then fails only on a .github/workflows bump, thats the Workflows permission
• the dashboard issue regenerates, so closing it does nothing. don't bother tidying it
• give the schedule a whole day, not an hour. github starts scheduled workflows late, often by hours: a monday cron meant for 5am ran at 10 or 11 and missed a before 6am on monday window every week. nothing fails, and new updates sit in Awaiting Schedule. * * * * 1 is all of monday in your timezone. use cron syntax, since renovate's text syntax is deprecated
• the dashboard doesn't say which host a stack is on, only the directory, so name your directories usefully
• check your actions minutes. billing rounds each job up to a whole minute, so five jobs finishing in nine seconds each bill as five minutes, not one. my validation workflow was using ~300 minutes a day, which on a private repo costs real money. i collapsed it into a single job, which took it from ~5 billed minutes a run to 1. the work itself takes under twenty seconds

Comments