The one source of trouble I have with Cloudflare Tunnels is that I cannot have a wildcard or a dynamic way to add and remove services. I needed to SSH into the server, edit the configuration file by hand and restart the service. Not done yet, I needed to copy the configuration changes and make the same on the other redundant server running on a backup ISP link. Still not done, I needed to add an entry on the Cloudflare Dashboard CNAMEing the hostname to the tunnel’s domain. The process was far too much of a pain to do every time I added or removed a service.
Automation with Kubernetes
At work, I have access to an OpenShift Kubernetes cluster having a non-standard resource called Route. It allowed me to, just by defining a Route, get a lot of heavy lifting done. I got:
- an Ingress equivalent
- A DNS name provisioned on the DNS server (similar to external-dns )
- An SSO client application provisioned on an internal tool
- A Configured Ingress controller to use the SSO client to provide secure access to whatever service I exposed through the Route resource.
The idea that one resource can do all of these logically meaningful tasks was a 💡 moment for me. I looked into what Kubernetes offers to make such customizations possible, and lo and behold, the land of Custom Resources and Operator Patterns was in front of me. The fact that I could make up a custom specification and write code to tell the almighty brains of Kubernetes on how to take action on it was a heavenly combination. I set myself on a journey to understand how these work. And while doing so, to build an operator myself.
Operator Framework v/s others
If we take a peek at the section on writing Operators by Kubernets , it lists a bunch of frameworks that one can use to build an Operator. Ignoring the outright rejects like KubeOps (using .NET, I have no experience or interest in it) and Shell-operator for event-driven scripts, I am left with some good candidates.
Charmed Operator Framework
Initial impressions matter a lot, and the feeling I got at first glance from Charmed Framework was something enterprises would use to build complex solutions. It not only works with Kubernetes but virtual machines too. That would be more complex than I am willing to put effort into learning. Hard pass on that one.
Kudo promises a couple of good things, being uncomplicated and declarative among them. But the who is it for section did not feel like me.
KUDO is for you if you are an application administrator who wants to run your application on Kubernetes without having to learn about Kubernetes internals.
The next one was enticing, but again, I wanted to get more hands-on with Go too, two birds with one stone and all that.
KUDO is for you if you are a developer who wants an easy way to write Operators without having to write thousands of lines of Go.
A pass on this one too.
Reading about Metacontroller was another 💡 moment and a shift in my thinking about extensibility in general. Metacontroller allows us to provide a bunch of webhooks triggered anytime an event relevant to us occurs in the system. We can write the simple webhook handler for data manipulation in any language, including Python, to define what we need the state to be. Metacontroller takes care of the rest. I picked this one and even wrote a couple of test controllers until I figured out the cost of making something easy was forgoing customizability. I could not get a simple idea, a basic one in my head atleast into action. That was the end of the road for me with a very innovative tool.
The cost of making something easy is forgoing customizability.
Let me preface this with a quote from their landing page :
Users of Kubernetes will develop a deeper understanding of Kubernetes through learning the fundamental concepts behind how APIs are designed and implemented.
The quote spoke to me in a way that made this feel very close to my heart. Even if this book contained details on an unrelated tool, I would have read it regardless.
Invested, I was halfway into the book, understanding in detail how operator patterns work. There is a cluster of boilerplate code to set the controller up to work in a reconciliation loop (a loop that reacts to changes to move current to the desired state). Being new, boilerplate code is always the one that looks the scariest. Although the concepts were becoming pretty clear, I was not in any position to start writing an operator from scratch.
Boilerplate code is always the one that looks the scariest to a beginner.
The build part of the Operator Framework is something called an Operator-SDK . Scaffolding and generating code to bootstrap a new project is what the tool does, which I needed at this point. I learned that it uses Kubebuilder, something I now was catching up on, under the hood for the Operator itself. It fits like a glove, perfect!
The example of a sample Memcached Go Operator was good enough for me to get started using this tool. Although the complete documentation could have been better, I got what I needed to work. So, no complaints there.
Operator-SDK’s quick start guide took me farther than most quick start guides tend to take you: a fully functioning Operator, directly customizable to be anything you want it to be. Now in a position to build what I imagined, I proceeded to that part next, imagining (planning).
I talk about the architecture for the Operator in my next post on How would an Operator for Cloudflare Tunnels look like .