Because Kubernetes is a complex system, you will occasionally deal with your applications not deploying correctly or not working as planned. Solving such problems, though, can be difficult.
Troubleshooting issues in a Kubernetes cluster requires a clear understanding of Kubernetes and its error codes. This will reduce the time invested in debugging and correcting issues in order to achieve a faster, more precise debug mechanism.
In this article, you will learn what it means to get a `container not found` error code during your Kubernetes deployment and how that error can be resolved.
You’ll need the following prerequisites to use this guide:
- You should have a good knowledge of Kubernetes.
- You’ll need some knowledge of the kubectl command line tool for orchestrating Kubernetes.
- You should have a Kubernetes cluster running. You can use minikube to experiment with Kubernetes locally by creating a virtual cluster on your personal hardware.
What Is `container not found`?
Containers provide a standardized format for packaging and shipping all the components needed to run the containerized application. They eliminate the issue of the same application behaving differently in different environments by ensuring that every time the container is deployed, the containerized application runs in a consistent environment.
Managing a fleet of containers can be tedious and prone to errors. This is why Kubernetes was created. It helps you manage your containers and keep them running according to the desired state you have configured for the container.
In some instances, though, Kubernetes can’t get the objects to match the desired state configured and it throws an error code indicating what went wrong. The `container not found` error code generally indicates that kubectl can’t establish communication with the pod or container you are trying to interact with. This could be because the pod isn’t ready to receive commands yet or because it doesn’t exist in the node.
For example, if the pod scheduled to run on your node fails to start due to a network error during the scheduling process, when you try to orchestrate the pod with kubectl, you’ll get a `container not found` error code.
How Do You Solve `container not found`?
To resolve `container not found`, you need to identify what pod the container is running in. Pods are the smallest unit of deployment in Kubernetes. Kubernetes can’t directly run containers; instead, one or more containers are scheduled to run on a pod. When debugging deployment issues with a container, it makes sense to interact directly with its pod since that’s the closest unit of deployment to the container.
Once you identify the pod, you can determine whether the pod is in a state where it is ready to accept connections. This could be due to a pod eviction resulting from lack of resources. It could also be because of network issues or because the pod doesn’t exist.
You can explore these issues in more detail below, as well as how they can be resolved.
Get Information about Your Pod
There may be several reasons why your pod is not in the expected state, so you should see what the pod is reporting before you proceed. Gathering more information about the failing pod will give you better insights.
Run the following command to list all the pods across all namespaces:
Kubectl commands would be run against objects in the default namespace. The pod you are interested in may exist in a different pod, or you might want to see all the pods at a glance. Adding the `–all-namespaces` flag to the command instructs kubectl to list the pods available across all namespaces. Passing the `-o wide` flag prints the pods’ data with more information instead of just the default.
If you find the failing pod on the list, use the `kubectl describe pod <pod-name>` command to get more data. You can use `kubectl describe pod <pod-name>` to retrieve detailed information about your pod.
For example, you can get more information about the pod `nginx` with the below command:
You can also print the pod’s configuration in a YAML format using the command below:
From the output of both commands, you can gather more insights into the lifecycle of your pod. You should be able to determine whether the container crashed or whether there were network issues, which should help you quickly spin up the deployment again.
Determine the container status by checking the status property of the output from the `kubectl describe pod <pod-name>` command and the phase property of the `kubectl get pod <pod-name> -o yaml` command.
You should expect the status to have a state of running that denotes that the containers in the pod have been mounted and properly bound to a node in the cluster.
If the status is pending, then the container has not been set up and is not ready to run. During this phase, the pod could be waiting to be scheduled, or it’s busy fetching the container images and needs time to finish.
An unknown status indicates that the pod’s status could not be obtained. This typically occurs because of an error in communicating with the node where the pod is expected to be running. You can redeploy the object to fix this.
Check the logs using the `kubectl logs nginx` command to look for anything that might help. Watch out for pod crashes, network issues, and other Kubernetes error codes like `imagepullbackoff`.
The command below displays the logs for an `nginx` pod:
You should have a similar output representing all the events that took place on the pod and their timeframe.
Does the Pod Exist?
Checking against this before applying can save you a lot of debugging time. There are times when you are specifying a pod that doesn’t exist or the name was not specified correctly.
One way to determine if the pod exists is by listing all pods on the cluster. You’ll see details on the pod status and its ready state.
Run this command to see the list of pods running on your node:
As you can see above, the command returns all pods running on your cluster along with details you need to debug this error code, like the pod’s [status](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.23/#podstatus-v1-core) and its [ready status](https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/#pod-conditions) that indicates if the pod is ready to serve requests. Check the output of this command and see if it includes the pod you are running your command against.
Does the Pod Exist in Another Namespace?
Kubectl commands would run against objects in the default namespace if there is no namespace specified. If your pod exists in another namespace, be sure to specify the namespace in your command by using the `--namespace` flag.
For example, you can retrieve the pods running in the `nginx` namespace using the following:
Passing the `--namespace nginx` flag tells kubectl to get the pods that exist in the `nginx` namespace.
If you are unsure of the namespace, you can get a list of all namespaces that exist in your cluster using the command below:
Otherwise, you can simply retrieve a list of pods in all namespaces.
Append the `--all-namespaces` flag to the previous command, as shown below:
Kubernetes is complex, and the same is true for debugging it. As you’ve seen, you have multiple options for diagnosing a problem. The `container not found` error code may initially be frustrating, but using these methods, you’ll be better able to debug and redeploy your application.