Signing container images is the easy half

Adding image signing to a pipeline is a one-line change. cosign sign after the push, a key in the CI secret store, and the build log now says something reassuring.

Nothing is protected yet.

A signature is a claim sitting in the registry next to the image. Until something checks it at the moment a pod is scheduled, an unsigned image — or one signed by a key nobody recognises — still starts normally. The build produced evidence. Nobody is reading it.

Where the enforcement lives

In Kubernetes that check belongs in admission control. For my thesis platform I used Kyverno with a verifyImages rule:

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: verify-image-signatures
spec:
  validationFailureAction: Enforce   # not Audit
  webhookTimeoutSeconds: 30
  rules:
    - name: verify-harbor-images
      match:
        any:
          - resources:
              kinds: [Pod]
      verifyImages:
        - imageReferences:
            - "harbor.internal/*/*"
          attestors:
            - entries:
                - keys:
                    publicKeys: |-
                      -----BEGIN PUBLIC KEY-----
                      ...
                      -----END PUBLIC KEY-----

Two fields carry the entire weight of the policy.

validationFailureAction: Enforce. The default in most examples is Audit, which logs the violation and admits the pod. Audit mode is the right place to start and a terrible place to stay — it produces exactly the dashboard that makes everyone believe the control is on.

imageReferences. The pattern only covers what it matches. harbor.internal/*/* says nothing about docker.io/library/redis, so a policy that looks comprehensive can leave every third-party image unverified. Either enumerate the registries you trust and deny everything else, or accept that you have verified your own builds only — but know which one you did.

The failures you find immediately

Turning Enforce on breaks things, and the breakages are informative.

System workloads. kube-system, the CNI, the CSI driver, the monitoring stack — none of those images are signed by your key. They need explicit exclusions, and writing those exclusions is the first honest inventory of what actually runs on the cluster.

Mutable tags. Signatures bind to a digest. A deployment pinned to :latest verifies whichever digest that tag resolved to at admission time, which may not be the digest that was signed an hour ago. Enforcing signatures pushes you toward digest pinning whether or not that was the plan.

Webhook availability. The admission webhook is now on the path of every pod creation. If Kyverno is down and the failure policy is Fail, nothing schedules — including Kyverno’s own replacement pods. If it is Ignore, the control silently switches off exactly when something is wrong. Neither answer is comfortable; pick deliberately, run more than one replica, and know which way it fails.

SLSA L3 is mostly about the builder

Signing proves who published an image. SLSA Level 3 asks the harder question: can the build process itself be trusted not to tamper with what it produced?

That is why the pipeline uses Kaniko rather than a Docker socket. A build container with /var/run/docker.sock mounted can reach the daemon, and anything that can reach the daemon can change other builds. Kaniko builds in userspace, unprivileged, with no shared daemon — so the provenance Cosign attaches actually describes an isolated build.

The in-toto attestation then records the source commit and the builder identity, and Kyverno can verify that too: not merely “this image is signed”, but “this image was built by that pipeline, from that commit”.

The one-line version

Signing is provenance. Admission control is enforcement. Only the second one stops anything, and it is the half that requires you to find out what is already running.