How to create Kubernetes YAML files
Introduction
This article is intended as a guide for creating Kubernetes manifest files.
YAML stands for YAML ain’t markup language is used to create Kubernetes manifests. According to the definition:
YAML is a human friendly data serialization language for all programming languages.
Indeed reading YAML is rather straightforward, once you realize that indentation is the way to build data structures. It is especially easy if you are familiar with Python, F# or other indentation-aware languages.
Writing YAML, on the other hand, is much less fun. If you are familiar with schema-less REST API development, think about Kubernetes YAML in the same way as JSON payload for an API. There is no schema to validate against and all the knobs are exposed as data in the payload.
It is relatively straightforward to create a simple YAML file, but for production use, many more settings are required. Here is an example of a Kubernetes Deployment manifest. If you know Kubernetes a bit, most of the settings would make sense when you read the file. Now imagine that you need to create a few of those from scratch (ingress, service, config map, secrets…)
As a developer, you will need to create several different types of manifests. Here is a list of Kubernetes resources, * indicates the resources typically created/maintained by developers. Please note that this only applies if you have a dedicated Kubernetes administrator who can take care of the rest. The list is abbreviated only to common resources and in production, scenarios will likely include various CRDs (Custom Resource Definitions).
If you are not confused, you are not paying attention.
Tom Peters
What you will learn
Let’s try to alleviate some of the confusion around writing YAML files and make the process easier and less painful.
After reading this article, you will learn:
- different ways to create Kubernetes YAML files
- how to automatically detect and correct errors during the development process
- what development tools to use to make creating YAML files easier
- learn about online tools specializing in generating different kinds of YAML files
Prerequisites
If you would like to follow along and get your hands dirty with YAML, you will need:
- VS Code with remote development extension
- docker desktop or any of the alternatives (need to be able to run containers)
- clone this repository
Please note that the repository uses a devcontainer with all the tools and configuration needed to run the examples build in. Because of this, the image is large, so please modify it accordingly before running the container.
Create vs generate
Initially, you might be tempted to generate as much of the boilerplate as possible. My recommendation is, don’t! Especially if you are new to Kubernetes or YAML, experiment, copy-paste from Kubernetes docs, but don’t use generators on day one.
Once you are familiar with the basics, progressively add tools that will make your life easier. There is good news; you will understand the basics pretty fast.
A good way to know if you are familiar enough with the YAML content of a specific resource if it is getting, well … boring. From here now, you should dive headfirst into the world of generators and helpers to keep your sanity and make your life easier.
#1 YQ
The first tool I want to talk about is yq. Yq is not a Kubernetes specific, it’s rather a “jack of all trades” of YAML. Learning this tool will help you query and manipulate YAML files directly from the command line. It helps with tasks, such as:
- filtering YAML file for a specific value, for example retrieving an image name from a deployment file
Selecting values from YAML files is useful, but mastering yq will help mostly with bulk operations on multiple files and more complex transformations.
#3 Kubectl
It is easy to get started with generating YAML files for most of the resources with kubectl. You can use “ — dry-run=client -oyaml > yaml_file.yaml” flag on the “kubectl create” or “kubectl run” commands to generate most of the resources.
For example, to generate a YAML file for an nginx pod you can run:
kubectl run nginx — image=nginx — port=8080 — env=env=DEV — labels=app=nginx,owner=user — privileged=false — dry-run=client -oyaml > nginx-pod.yaml
This command will generate the following YAML:
The file needs to be cleaned a bit, but it is a good starting point.
Now you could create a deployment using:
kubectl create deployment my-dep — image=nginx — dry-run=client -oyaml > deployment.yaml
and use yq to merge the two files.
This process can get complicated fast, but it’s easy to use shell scripts to automate most of the tasks.
Using the combination of kubectl and yq is great for starting a simple one-off project as well as help automate things in between.
If you are interested in kubectl tips & tricks, I have a growing list of useful commands in this gist.
#3 Docker-compose
Do you have a docker-compose.yaml file in your project? Generating Kubernetes manifests from the docker-compose file is possible using a tool called kompose.
Let’s see this in action. We will use a docker-compose file from the awesome-compose repository.
Here is a sample docker-compose file:
Now, let’s generate Kubernetes manifests using kompose:
This command takes the docker-compose as input and outputs generated Kubernetes YAML into the k8s-manifests folder.
Using kompose is a good option if you already have a docker-compose file. There are often tweaks needed, but it takes you one step closer to having a decent starting point.
#4 VS Code with plugins
VS Code has 2 plugins that help with creating YAML files. Big thanks to Avi Nehama for suggesting it.
The template enables quick scaffolding of any Kubernetes resource.
Create yaml file, start typing the name of Kubernetes resource and hit TAB to insert a template. Keep cycling with TAB to fill the names in the required fields.
This extension from Red Hat runs a YAML server in the background and adds context aware smart completion to any Kubernetes resource.
Remember to activate it in the settings and reload VS Code. Add this line to the settings to enable Kubernetes completion on all YAML files.
"yaml.schemas": {
"Kubernetes": "*.yaml"
}
#5 CDK8s
Moving on from command line to programming territory. If you have to author a lot of YAML, but happen to know Python, Typescript, JavaScript, Java or Go, you can harness the power of a programming language to make the process of writing YAML much easier.
Introducing CDK8s
cdk8s is an open-source software development framework for defining Kubernetes applications and reusable abstractions using familiar programming languages and rich object-oriented APIs. cdk8s apps synthesize into standard Kubernetes manifests which can be applied to any Kubernetes cluster.
CDK8s works by exposing Kubernetes resources objects and using an object called constructs to further abstract and automate YAML files creation.
The real power behind this approach is the ability to:
- create reusable components and abstractions that capture your requirements
- use native programming language constructs to automate, test and validate the process of creating YAML
#6 NAML
If you happen to know Go and don’t like YAML at all and want to avoid it at all costs, this project might be something for you!
A very interesting approach designed by Kris Nova (Github profile) is a Go-centric tool called naml which works by creating Kubernetes manifests directly in Go and installing them on the cluster via CLI install command.
This tool can produce YAML similarly to CKD8s but works only with Go.
#7 Online Tools
Now and then a frustrated developer will create a simple Web UI with a form to gather input for generating YAML files. Those projects are usually short-lived and not maintained very well, so my advice is to stay away from online Kubernetes YAML generators.
There is one exception, an online editor by cilium specialized in creating Kubernetes resource called Network Policy
Closing Thoughts
We’ve seen various ways of creating Kubernetes YAML files, some of them using simple command-line tools, auto-generating files. Others exposing full programming languages.
This is great, but how do we make sure that our YAML is correct? What are some of the best practices for validating the content of the file? How can those practices be automated?
Stay tuned for the next article if you would like to know answers to those and similar questions.
