What does working with Kubernetes look like?

We’ve been working on spreading Kubernetes (k8s) knowledge among our team, and the question keeps coming up “How is this different?” or “How do you get things done?”

There are hundreds of articles that talk about WHAT k8s is, but this talks about HOW to use it.

In general, interacting with Kubernetes means running commands from the command line. Kubernetes comes with a command line tool called kubectl, which interacts with the k8s back end via an API endpoint.

Prerequisites

Before we can set up kubectl, we need to get the config file that sets up

  1. The user
  2. What clusters it has access to.

If we’re setting up a local environment, it should be in the instructions near the end of the process. If you’re accessing a remote cluster, you can get the kubeconfig file from the administrator. If we’re using one cluster, we can just copy the file to ~/.kube/config. If we need to access more than one cluster, we can see how to manage that here.

Once the config file is in place, we can start using kubectl with a command like this:

kubectl cluster-info

If that returns a couple of lines, we’re set. If not, see if the config files and kubectl executable is in place.

Structure of Commands

In general, the kubectl commands use the format

kubectl <action> <resource type> <name> <options>

The action is something like get, create, delete. The resource type can be a pod, deployment, or service. The options are usually prefixed by two dashes, and can specify specific attributes for this command.

Getting Started

Now were starting to get started. First, let’s see what’s been deployed

kubectl get pods

If it’s a new cluster, you probably won’t have anything displayed. Let’s fix that by deploying the simplest kubernetes app: nginx.

kubectl run nginx –image=nginx

Now if you run the get pods, you’ll see something like this

nginx-56f766d96f-l8z8n 1/1 Running 1 332d

Actions

The most common actions are

Get – Get a brief description of a resource

Describe – Get detailed information about a resource

Create – Create a new resource object in the cluster. Most of the time, you’ll want to create objects from files, so you can re-use them later. This is called declarative management. If you want something quick, and won’t need to check in a copy, you can create objects directly from the command line using imperative management.

Delete – Similar to create, in that it can be used as declarative or imperative. Be aware that when we delete a pod that has been created for a deployment, Kubernetes will see that the pod isn’t running and try to start it back up. This may or may not be what you want. If pods keep getting created, look for deployments that you can “describe” to see what pods they are starting.

There are many others, but these are the most common ones.

Troubleshooting

Most of the time, our actions will look something like this.

When we’re troubleshooting, it’s good to start with pods. Since they act as a unit, it’s handy to see which ones have been created or have failed to start.

kubectl get pods

NAME                          READY     STATUS    RESTARTS   AGE
hello-node-7f5b6bd6b8-tqqjn 1/1 Running 1 327d
nginx-56f766d96f-l8z8n 1/1 Running 1 332d

The important parts here are the name and the status. Regular pods start with the name of the container and add a random string to it. We need that string to reference a specific pod. Let’s see what’s going on with our nginx node.

kubectl describe pod nginx-56f766d96f-l8z8n

Name:           nginx-56f766d96f-l8z8n
Namespace: default
Node: minikube/10.0.2.15
Start Time: Sat, 16 Feb 2019 14:36:40 -0700
Labels: app=nginx
pod-template-hash=1293228529
Annotations:
Status: Running
IP: 172.17.0.3
Controlled By: ReplicaSet/nginx-56f766d96f
Containers:
nginx:
Container ID: docker://cf767ded2f705d2fd2560750b030a5b1723a45bb536001271090ec226d041144
Image: nginx
Image ID: docker-pullable://nginx@sha256:8aa7f6a9585d908a63e5e418dc5d14ae7467d2e36e1ab4f0d8f9d059a3d071ce
Port:
Host Port:
State: Running
Started: Wed, 15 Jan 2020 09:45:12 -0700
Last State: Terminated
Reason: Error
Exit Code: 255
Started: Sat, 16 Feb 2019 14:37:12 -0700
Finished: Wed, 15 Jan 2020 09:42:10 -0700
Ready: True
Restart Count: 1
Environment:
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-bdd9p (ro)
Conditions:
Type Status
Initialized True
Ready True
PodScheduled True
Volumes:
default-token-bdd9p:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-bdd9p
Optional: false
QoS Class: BestEffort
Node-Selectors:
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal SuccessfulMountVolume 25m kubelet, minikube MountVolume.SetUp succeeded for volume "default-token-bdd9p"
Normal SandboxChanged 25m kubelet, minikube Pod sandbox changed, it will be killed and re-created.
Normal Pulling 25m kubelet, minikube pulling image "nginx"
Normal Pulled 24m kubelet, minikube Successfully pulled image "nginx"
Normal Created 24m kubelet, minikube Created container
Normal Started 24m kubelet, minikube Started container

That is a lot of information! The most interesting bits for troubleshooting are:

Status – is it running, or is there a problem?

Containers – a description of what containers this pod will try to start. This is a list, so be aware that while there’s USUALLY one container per pod, there CAN be more than one.

Conditions – Is the pod scheduled to run on a specific node, and if so, is it ready?

Events – This is a list of recent events that pertain to the node. If there is a problem scheduling, downloading, or starting the node, it should be listed here.

Fixing Things

When it comes to changing things in kubernetes, there are a couple of options.

  1. If you need to change or create an object, get the source yaml file for the object in question, modify the yaml as needed, and delete/recreate the object as needed.
  2. If you need to kill a pod that is part of a deployment (stuck, going crazy with memory, etc.) you can use kubectl delete <pod> and it will be terminated. If it’s part of a deployment, k8s will bring up a new one. If not, you’ll need to recreate it.

Finally

Kubernetes is complicated. There are thousands of details, dozens of big concepts, and more than one way to do almost everything.

However, when you’re just getting started it’s easy to get stuck on the details. This gives you an idea of how most of the work managing a kubernetes cluster looks like.