As Showmax, we are slowly moving toward future in the clouds. Among many changes we are introducing related to migration to AWS (which will be topic of many articles to come, do not worry), we also decided to leave behind our workload orchestrator - Nomad. Instead, we will shift towards Kubernetes.
As I was responsible for the AWS migration for my team (CMS), I wanted to make some simple guide showing the absolute basics. And because internet is about 50 % cat pictures and 50 % kubernetes memes, I ended up with this meme-driven Kubernetes guide for developers.
Just to stress out, this is not by any means enough to fully work within the new environment or claim you understand anything. Whole point was to show our developers easy way to dip their feet, so they do not have to be afraid of all the new things.
With that, here is the guide itself, hope you enjoy it - or at least have a good laugh.
What is this guide?
This guide is meant to help new and new-ish users of Kubernetes orient themselves somehow in this bizarre, wild world. We will walk you through common use cases, and most common issues that you may encounter with Kubernetes.
What is this “Kubernetes” thing?
Actual cheat sheets
Most Common Commands
In this part, we will show some of the basic commands you use while interacting with your service running in Kubernetes.
As an example, I am using service CMS, which is living in cms namespace.
Where are my containers? Are they running?
kubectl -n <namespace of your service> get pods
CMS Example
kubectl -n cms get pods cms
Tell me more about the containers!
kubectl -n <namespace of your service> describe pod <pod_id>
CMS example
kubectl -n cms describe pod cms-7cb45996cf-6lhth
The screenshot above is the pod ID we took from the previous command.
There you can see environment variables, deployed versions, labels, commands used to run the thing, etc.
How are they feeling, any complaints?
kubectl -n <namespace of your service> logs <pod_id> <container>
CMS Example
kubectl -n cms logs cms-7cb45996cf-6lhth admin
I want to create some meaningful connection with those pods!
kubectl -n <namespace of your service> exec -it <pod_id> -c <container> -- /bin/bash
CMS Example
kubectl -n cms exec -it cms-7cb45996cf-6lhth -c admin -- /bin/bash
This is the container name we took out of the pod description.
Except for the namespace and container (because otherwise you work with pods), this is the same as with our good old friend Docker. You almost knew that one already, admit it!
I wanna talk to it, directly. #NoFilter
kubectl-n <namespace of your service> port-forward <pod id> <pod port>:<your port>
Now localhost is talking directly to the pod on the specified port.
What’s interesting here is that you do not need to specify a container. In Kubernetes, all containers in a pod share a network namespace.
CMS Example:
kubectl -n cms port-forward cms-7cb45996cf-6lhth 3000:3000
We can take the port from the pod description.
Oh, I need to change something in the current deployment!
kubectl-n <namespace of your service> edit deployment <name of deployment>
Here’s a CMS example, if you really want to edit deployment:
kubectl -n cms edit deployment cms
Something is going wrong, what do I do?
I deployed a new version, it says CrashLoopBackOff:
Your pod is likely spinning up and the startup script is returning some kind of error. That is making the server restart itself, crash again, and so on, until it stops with CrashLoopBackOff. There could be other reasons, but basically you kept restarting and encountering an error until Kubernetes said “Enough!”
It gets easier if you just follow these 13,546 easy steps:
Get logs:
kubectl -n <namespace of your service> logs <pod id> <container>
Is there anything useful? Did you forget to declare some environment variables? If so, you should see that there.
If it is there, you can edit the deployment (see above)
No log to be found? Ok, your app is probably not logging enough. You should get to that later. For now, let’s just fix it somehow.
Often, you can put together crucial info from the description of the pod state. To get that, use:
kubectl describe pod -n <namespace of your service> <pod id>.
At the bottom you should see “Events:” and quite often that will tell you what is actually happening.
I did not do such a great job, how do I roll back?
Easy enough! Just change the manifest, part “version” and apply it again. As we have our Kubernetes manifests committed, you should even be able to find previously-deployed versions in git history.
I almost did a great job, can I just restart the pods?
Sure, go for it. This might help in some cases, or when you have no idea what is happening. Sometimes you just want to turn it off and on again. To do that, just run:
kubectl -n <namespace> rollout restart <deployment name>
CMS example:
kubectl -n cms rollout restart cms
What are those probes in the pods and why do we need 2?
Great question!
In Kubernetes, we have two different probes specified. Or, that’s what we should have.
- Readiness probe
- Liveness probe
We have both because they serve different purposes and do different things when the response is not 200.
The readiness probe reflects the state of your health endpoints, and you should do your checks for mounted file systems and databases there. If it fails more often than the threshold, the pod is considered sick and will be taken out of service by your Varnish or something similar.
The most basic check you can imagine belongs in your liveness probe. Just return 200. If this fails, the pod will be killed and restarted.
K9s - I am lazy and I do not want to change that
If you don’t enjoy using kubectl directly, consider K9s. K9s is a kubectl-like TUI interface that is really easy to use (extremely so, if you’re used to Vim). It makes it really easy to browse the contents of the cluster. Here are a few screenshots:
One minor note: while K9s is really one of the most useful tools I have used, PLEASE try to take some time to read through the actual cheat sheet. It will help you understand what K9s are actually doing and what commands are being called — it will help tremendously when you use that tool.
Not as lazy as K9s, but I could still use some help!
Consider shell tab completion for kubectl. It will suggest commands and resources. E.g. kubectl -n cms logs will give you all the pods from cms namespace.
You can get shell completion for kubectl
Or if you use zsh
Where to profound my knowledge?
The Illustrated Children’s Guide to Kubernetes - more wide explanation of the basics as a children book, love the idea
online course - this is the one I did, but you can find a lot of online courses for Kubernetes everywhere.
There sources can give you much better overview of what kubernetes are and how to use them. Just beware it does not become your sole focus.
