K3s Docker pull through registry
A cluster local Docker pull through registry can speedup your deployments. The registry checks if an image exists locally, if not, it pulls it from dockerhub and caches it locally.
You can apply (something like) the following Kubernetes deployment to run the registry in your cluster, and expose it on NodePort 30555. If you want to try this deployment, make sure you adapt the persistent volume claim for your cluster settings.
---
apiVersion: v1
kind: Namespace
metadata:
name: docker-registry
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: docker-registry-pvc
namespace: docker-registry
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 5Gi
storageClassName: managed-nfs-storage
---
kind: Service
apiVersion: v1
metadata:
name: docker-registry
namespace: docker-registry
spec:
type: NodePort
ports:
- port: 80
targetPort: http
nodePort: 30555
selector:
app: docker-registry
---
kind: Deployment
apiVersion: apps/v1
metadata:
name: docker-registry
namespace: docker-registry
spec:
replicas: 1
selector:
matchLabels:
app: docker-registry
template:
metadata:
labels:
app: docker-registry
spec:
containers:
- name: docker-registry
image: 'thomasdegraaff/registry-pull-through:2.7.1'
# env:
# - name: REGISTRY_USERNAME
# value: user
# - name: REGISTRY_PASSWORD
# valueFrom:
# secretKeyRef:
# name: docker-registry-password
# key: password
ports:
- name: http
containerPort: 5000
protocol: TCP
livenessProbe:
httpGet:
port: http
initialDelaySeconds: 60
readinessProbe:
httpGet:
port: http
initialDelaySeconds: 10
volumeMounts:
- mountPath: /var/lib/registry
name: docker-registry-data
volumes:
- name: docker-registry-data
persistentVolumeClaim:
claimName: docker-registry-pvc
The image used is based on the original Docker registry image, with the addition of a pull through registry config file. Optionally one can add environment variables when one needs to access a private dockerhub registry. See the Dockerfile for the build specs.
Once the registry is running, you can check if it is ok:
If everything is ok, the result should be something like this:
Now you need to configure k3s to use the pull through registry. On each node add the following file and restart k3s. Use the mirror name docker.io to overrule the default dockerhub the cluster uses.
Now deploy a test app to check if the cache is working.
---
apiVersion: v1
kind: Namespace
metadata:
name: learning
---
apiVersion: v1
kind: Service
metadata:
namespace: learning
name: hello-nodeport
spec:
type: ClusterIP
ports:
- protocol: TCP
port: 80
selector:
run: hello-world
---
apiVersion: apps/v1
kind: Deployment
metadata:
namespace: learning
name: hello-world
spec:
selector:
matchLabels:
run: hello-world
replicas: 1
template:
metadata:
namespace: learning
labels:
run: hello-world
spec:
containers:
- name: hello-world
image: nginxdemos/hello:0.2
ports:
- containerPort: 80
protocol: TCP
Now check the registry again for repositories.
If the output contains a nginxdemos/hello repo, success! This means that the pull through registry has cached the nginxdemos/hello image.When the test fails, and you want to test again, make sure the nginxdemos/hello image is removed from the cluster. Remove the hello world deployment, and then remove the image on your cluster node(s).
I had some trouble debugging when testing because I thought that the setting imagePullPolicy: Always
ment that the image was always pulled from Dockerhub. But this is not true. When a image exists in the cluster node with the exact same digest as the current image on Dockerhub, then the local image is used.