Write Your Kubernetes Infrastructure as Go Code — Getting Started With Cdk8s | by Abhishek Gupta | Jul, 2022

cdk8s.io

Infrastructure as Code (IaC) is a well-established paradigm and refers to the standard practice of treating infrastructure (network, disk, storage, databases, message queues etc.) in the same way as application code and applying general software engineering practices including source control versioning, testing and more. For example, Terraform and AWS CloudFormation are widely-adopted technologies that use configuration files/templates to represent the infrastructure components.

Imagine you have an application that comprises of a Serverless function fronted by an API Gateway along with a NoSQL database as the backend. Instead of defining it in a static way (using JSON, YAML etc.), one can represent these components using standard programming language constructs such as classes, methods, etc. Here is a pseudo-code example:

DBTable table = new DBTable("demo-table");
table.addPrimaryKey("email", Type.String);
Function function = new Function("demo-func");
function.addEnvVars("TABLE_NAME", table.Name());
APIGateway apigw = new APIGateway();
apigw.addFunctionIntegration(function);

Notice the (hypothetical) classes DBTable, Function and APIGateway and the way they are used. For eg a function can reference the table object and get its name – all this comes to life during the program runtime and taken care of by the implementation details of the underlying framework/platform.

Thankfully, you don’t have to write pseudo-code for your production infrastructure!

… thanks to existing solutions such as cdk8s, AWS CDK, Pulumi, CDK for Terraform (cdktf) etc. Almost all these solutions follow a similar approach – write code to define infrastructure, then convert that into configuration, for eg Kubernetes manifest (YAML), AWS CloudFormation template, HCL config etc., which can then be applied using standard tooling.

While we are on this topic, it’s hard not to mention the Go programming language and its ubiquitous presence in the cloud services and infrastructure domain. It combines the safety of a compiled language with the speed of interpreted language (like Python), has a robust standard library and compiles to a single binary. These and many more qualities have led to lots of cloud-native software (IaC, monitoring, observability etc.) written in Go, such as Prometheus, Terraform, Grafana, Jaeger, etc.

“In fact, over 75 percent of projects in the Cloud Native Computing Foundation are written in Go.”

Over the course of multiple blog posts, I will cover how Go developers can use the cdk8s (Cloud Development Kit for Kubernetes) project for defining Kubernetes resources. It’s an open-source framework (also part of CNCF) that provides high-level abstractions which can be composed into larger Kubernetes applications. Instead of adopting YAML or other configuration/template-driven approach, cdk8s supports multiple programming languages, which means you can work with Kubernetes resources using familiar concepts such as classes, methods, etc. Ultimately, cdk8s generates Kubernetes manifests which you can apply using kubectl – business as usual!

At the time of writing, cdk8s supports Go, Typescript, Python and Java

This blog post will start things off and provide a gentle yet hands-on intro to cdk8s. By the end of it, you will be familiar with the key concepts and understand how to use cdk8s Go APIs to define a Kubernetes application, deploy (using kubectl) and test it out.

Make sure you have Go (v1.16 or above) and cdk8s CLI installed. Also, you need to have access to a Kubernetes cluster. For learning and experimentation I would recommend using a single-node cluster running locally – such as minikube, kind, etc.

I generally use minikubeso setting up a cluster is as simple as minikube start

To install cdk8s CLI.

You can choose from the below options:

#homebrew
brew install cdk8s
#npm
npm install -g cdk8s-cli
#yarn
yarn global add cdk8s-cli

Although this blog post will provide step-by-step instructions, you can always refer to the complete code on Github

cdk8s makes it really easy for you to get started and bootstrap your application. You don’t need to guess and figure out how to structure your project, setup dependencies etc. since the cdk8s init command does it for you!

cdk8s init go-app#output
....
Your cdk8s Go project is ready! cat help Prints this message
cdk8s synth Synthesize k8s manifests to dist/
cdk8s import Imports k8s API objects to "imports/k8s"
Deploy:
kubectl apply -f dist/

Once completed, you will get a directory structure as such:

.
├── cdk8s.yaml
├── dist
│ └── test.k8s.yaml
├── go.mod
├── go.sum
├── help
├── imports
│ └── k8s
│ ├── internal
│ │ └── types.go
│ ├── jsii
│ │ ├── jsii.go
│ │ └── k8s-0.0.0.tgz
│ ├── k8s.go
│ ├── k8s.init.go
│ └── version
└── main.go

Update the generate go.mod file, and replace it with the following – this is to make things simpler for you.

Feel free to use the latest version of the modules if needed.

module getting-started-with-cdk8s-gogo 1.16require (
github.com/aws/constructs-go/constructs/v10 v10.1.42
github.com/aws/jsii-runtime-go v1.60.1
github.com/cdk8s-team/cdk8s-core-go/cdk8s/v2 v2.3.29
)

The canonical Kubernetes “hello world” is to get a nginx server up and running. The easiest option is to use simply use kubectl run eg kubectl run nginx --image=nginx. But, since this is imperative, we switch to a declarative way where we define our desired state (in a yaml file) and ask Kubernetes to figure things out.

For eg we can write a Deployment manifest and submit it to Kubernetes using kubectl apply -f <name of the yaml file>.

apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
spec:
replicas: 1
selector:
matchLabels:
app: hello-nginx
template:
metadata:
labels:
app: hello-nginx
spec:
containers:
- image: nginx
name: nginx-container
ports:
- containerPort: 8080

So, open the main.go file and copy the below Go code. Don’t worry, I will walk you through it!

package mainimport (
"getting-started-with-cdk8s-go/imports/k8s"
"github.com/aws/constructs-go/constructs/v10"
"github.com/aws/jsii-runtime-go"
"github.com/cdk8s-team/cdk8s-core-go/cdk8s/v2"
)
type NginxChartProps struct {
cdk8s.ChartProps
}
func NewNginxChart(scope constructs.Construct, id string, props *NginxChartProps) cdk8s.Chart {
var cprops cdk8s.ChartProps
if props != nil {
cprops = props.ChartProps
}
chart := cdk8s.NewChart(scope, jsii.String(id), &cprops)
selector := &k8s.LabelSelector{MatchLabels: &map[string]*string{"app": jsii.String("hello-nginx")}} labels := &k8s.ObjectMeta{Labels: &map[string]*string{"app": jsii.String("hello-nginx")}} nginxContainer := &k8s.Container{Name: jsii.String("nginx-container"), Image: jsii.String("nginx"), Ports: &[]*k8s.ContainerPort{{ContainerPort: jsii.Number(80)}}} k8s.NewKubeDeployment(chart, jsii.String("deployment"),
&k8s.KubeDeploymentProps{
Spec: &k8s.DeploymentSpec{
Replicas: jsii.Number(1),
Selector: selector,
Template: &k8s.PodTemplateSpec{
Metadata: labels,
Spec: &k8s.PodSpec{
Containers: &[]*k8s.Container{nginxContainer}}}}})
return chart
}
func main() {
app := cdk8s.NewApp(nil)
NewNginxChart(app, "nginx", nil)
app.Synth()
}

When writing cdk8s based code in any language, you will come across a set of common concepts/terminologies – these include Construct, App and Chart. I will explain these as we walk through the code.

Start with the main function first – we use cdk8s.NewApp to create an App.

Well, what exactly in an App? It’s is a construct, and you can think of constructs as higher-level building blocks to represent state. The key thing to note is that these constructs are composable. What that means is that you can define levels of these constructs (each level provides/exposes a different abstraction layer) and combine them to create your desired end state – in this case it happens to be a Kubernetes manifest with objects such as Deploymentbut it could be something else.

For eg an AWS CloudFormation template (if you were to use AWS CDKnot be confused with cdk8s)

Back to the App – so, an App is also a construct. In fact you can think of it as the root in a tree (hierarchy) of constructs. So what else is there in that tree? Look the second line in the main function – NewNginxChart(app, "getting-started", nil) – this invokes a function NewNginxChart that returns a cdk8s.Chart which is the next component in the hierarchy. AA cdk8s App can contain multiple charts and each chart can be converted (or in precise cdk8s terminology – synthesized) into a separate Kubernetes manifest file (you will see this action very soon).

Finally, draw your attention to the NewNginxChart function. It has a bunch of things, but notice the call to k8s.NewKubeDeployment function. This is where we actually define Kubernetes Deployment in code (in the next section, we will also add a Service to the chart.)

You can define multiple Kubernetes components in a chart, such a Pod, Service, Ingress, Job etc. – what ever you need for your application to work on Kubernetes.

To summarise, here is a visual representation of what I just explained — remember everything is a Construct (App, Chart etc.)

cdk8s — Tree of Constructs

Wait, what about the Kubernetes API dependencies??

If you’ve spent time working on accessing Kubernetes programmatically, this is an obvious (and great!) question. If you were to deal with k8s object using go, at the minimum you will need Kubernetes client-go, API machinery etc. Guess what, cdk8s has got you covered there too!

You actually don’t need to pull in these dependencies because cdk8s allows you to treat these Kubernetes API Objects as constructs – remember, everything is s construct! They are automatically imported to your project when you run the cdk8s init command, but you can do it explicitly using cdk8s import as well. The resulting API is available as part of the imports folder (yes, go ahead and check that again!). On the top of main.gocheck the package that is imported – its just refers to the imports folder.

There is more to cdk8s import though. But you will have to wait for other blog posts to see that in action – we are just getting started!

.. and continue with the practical bits. It’s time to generate some yaml — you can’t eliminate it, but at least you don’t have to write it by hand! To do so, simply run:

cdk8s synth

Once that completes (should be quick!), check the dist directory to check what cdk8s has generated.
To make it easier to understand, here is a diagram which has a one-to-one mapping (notice the labels 1, 2,3, etc.?) between the cdk8s code objects/properties to their respective counterparts in yaml eg spec.replicas, spec.selector, template.spec etc.

Go and YAML — side by side

You can now use good old kubectl to deploy this to Kubernetes since cdk8s is not going to do that for you, at least not yet 😉

kubectl apply -f dist/
kubectl get pods -w

Once the Deployment is ready, the Pod should be in Running state. Simply use port-forward to access the nginx container port locally:

kubectl port-forward <enter nginx pod name> 8080:80

To access nginx home page, navigate to http://localhost:8080 using your browser

You also use a CLI tool eg curl localhost:8080.

Instead of port forwarding, let’s use the standard Kubernetes way of accessing applications by defining a Service resource, which is typically defined like this:

apiVersion: v1
kind: Service
metadata:
name: nginx-service
spec:
ports:
- port: 9090
targetPort: 8080
selector:
app: hello-nginx
type: LoadBalancer

But you know the rule — no yaml writing by hand! So, in the NewNginxChart function in the main.go file, add this piece of code after the part you defined the Deployment:

k8s.NewKubeService(chart, jsii.String("service"), &k8s.KubeServiceProps{
Spec: &k8s.ServiceSpec{
Type: jsii.String("LoadBalancer"),
Ports: &[]*k8s.ServicePort{{Port: jsii.Number(9090), TargetPort: k8s.IntOrString_FromNumber(jsii.Number(80))}},
Selector: &map[string]*string{"app": jsii.String("hello-nginx")}}})

First, delete the existing Deploymentkubectl delete -f dist/. Then, run cdk8s synth again to create the new manifest in the dist folder.

Both the Service and Deployment are in the same file – this is because they are part of the same Chart.

How you access the service will depend on the Kubernetes cluster. If you are using a cloud provider, it will likely provision a Load Balancer service native to that cloud eg Application Load Balancer in AWS. Please adjust this as per your setup.

For minikubeyou can simply follow these instructions https://minikube.sigs.k8s.io/docs/handbook/accessing/#loadbalancer-access“Services of type LoadBalancer can be exposed via the minikube tunnel command.”

In a terminal, run this command (it runs as a separate process):

minikube tunnel

In another terminal, delete the existing Deployment and then apply the new manifest:

kubectl apply -f dist/
kubectl get pods -w

Check the Service:

kubectl get svc

To access the nginx server, navigate to the external IP (as per the Service). In the case of minikube, you can simply use localhost:9090 or 127.0.0.0:9090

Remember to use port 9090 since that’s the external port we specified in the Service configuration in our code

.. I want to highlight a couple of other useful things in cdk8s.

Reference and reuse existing manifests and Helm charts

Say you have a Service already defined in a service.yaml file. You can include it in your cdk8s as part of a larger application/chart that you may have. Here is an example:

cdk8s.NewInclude(chart, jsii.String("existing service"), &cdk8s.IncludeProps{Url: jsii.String("service.yaml")})

Similarly, you can also include Helm charts. Say you wanted to add bitnami/nginx:

cdk8s.NewHelm(chart, jsii.String("bitnami nginx helm chart"), &cdk8s.HelmProps{
Chart: jsii.String("bitnami/nginx"),
Values: &map[string]interface{}{"service.type": "ClusterIP"}})

Well, you do need to have helm installed locally and also add the repo first helm repo add bitnami https://charts.bitnami.com/bitnami

Another handy feature is…

… the ability to declare dependencies between any two cdk8s constructs. For instance, in the previous example, we had a Deployment and a Service. You could create a dependency as such:

deployment := k8s.NewKubeDeployment(...)
service := k8s.NewKubeService(...)
deployment.AddDependency(service)

Thanks to AddDependencythe resulting manifest will be such that the Service is placed before the Deployment object.

Dependency is not limited to individual constructs in a chart. If you have multiple charts as part of your cdk8s app, you can establish dependencies across charts as well.

Leave a Comment