TIL not to use resource reservations on frequent Kubernetes CronJobs
February 6, 2025
1 min read
Kubernetes
If you have a Kubernetes CronJob that runs every minute and a job fails, the failed pod sticks around with its resource reservations still held. With a new pod spawning every minute, your cluster resources can be exhausted very quickly.
Consider either removing resource requests from frequent CronJobs or setting a strict failedJobsHistoryLimit:
apiVersion: batch/v1
kind: CronJob
metadata:
name: my-job
spec:
schedule: "* * * * *"
failedJobsHistoryLimit: 1
jobTemplate:
spec:
template:
spec:
containers:
- name: my-job
image: my-image
# no resources.requests here
Without this, a single downstream outage can cascade into cluster-wide resource pressure just from piling up failed pods.