Recently, I read another article that critiqued Kubernetes as having a steep learning curve. At conferences, I also hear a lot of people in the Kubernetes community talk about how we need to make it more easy to onboard people. While I think it’s a noble goal to make Kubernetes more usable, I don’t think we should do it at the expense of long term elegance in design. Here’s why.
My experience started in 1999 running things like HTTPD, DNS, NFS, HPC schedulers, MySQL, Perl/Python/Ruby, BGP, routers, switches, and firewalls – at places like NASA Glenn Research Center, Altel (now owned by Verizon), American Greetings E-Cards, and Eyemg – which shaped how I look at running things at scale. I have scaled applications vertically, and horizontally. I have ran giant (at the time) SGI boxes with 35 processors, and clusters of 1000 x86 servers. I have written thousands and thousands of lines of code, and participated in countless architecture meetings (we didn’t have the word DevOps until I worked at Eyemg). Now, I work at Red Hat and I think about Kubernetes and containers daily – and I talk to a ton of customers, ranging from telco and financial services, to public sector and auto manufacturers.
In all of this, you start to see the same problems come up, time and time again. Things like Unikernels, Serverless, and containers are all just novel ways to solve the same computing problems we have always had – and many of them are very specialized which means they only work well for a small subset of the problems we face in development and operations.
When I think of scale, I think of 1000 servers, but also 75 different IT systems. What I mean by that is – you don’t just run web servers – you don’t just run web apps. In a fully matured IT environment, there is much more than just the business facing web servers. There’s the back end network devices (routers, switches, firewall) – even if virtual in AWS. There’s the specialized utilities and tools that connect to the network devices, or manage them at scale. There are the Integrated Lights Out cards, the special software to manage the Oracle Database – the special software to help normalize the Oracle Database. There are the tools to pull some of the data out of Oracle and move it into MongoDB, and then on to Hadoop.
The front end web services are some of the easiest things to run. A PaaS is fine for that. A PaaS is a Ferrari – it’s red, fast, and sexy. It has two seats and it’s great for showing off in, or taking through the mountains every now and then. It’s specialized. It’s terrible when you go to the grocery store. It’s even worse when you go to the home improvement store. It’s absolutely useless with three kids, and a dog.
IT systems are no different. Pivotal Cloud Foundry is a sexy, red Ferrari. It demos great. It has really sticky tires, so it performs well on the track. It’s near useless for stateful applications, and completely useless for EVERYTHING else in your IT environment. You don’t run glue programs in PCF, you don’t run tools, you don’t run HPC workloads, you don’t run network scanning tools, etc. It’s very specialized and focused on web apps. It’s good at that, but not much else. Adding stateful storage is like adding a pop-up trailer to a Ferrari. Never seen that.
Other tools are more flexible. CF Engine is 20 ton dump truck, it’s pretty useful for carrying a lot of different materials, but it’s hard to park in city streets. Ansible, Chef, and Puppet, are 10 ton dump trucks that are more manageable – Ansible can even manage network devices. Actually, Ansible is a Swiss Army Chainsaw (TM) and people love it because of that 🙂 People also like flexible tools because they can learn one, and get a TON of mileage of it.
Kubernetes – well, Kubernetes is a 10 ton dump truck, but it handles pretty well on the track at 200mph (321 kph). Yes, you might need a Commercial Drivers License (CDL) to legally drive it, but we are all professionals here. Race car drivers have special licenses to get on race tracks. Truck drivers have special licenses to drive on public roads. Crane operators have certifications. The point is, yes – there is a learning curve for really powerful tools. Kubernetes is very powerful and elegant – and that’s why I love it. It can be used to run a broad swath of workloads in an IT environment. It can contribute to standardizing, as opposed to having “another” management system.
I am tremendously excited by containers and Kubernetes because they have the potential to run almost anything. DNS, no problem. Crazy, five way replicated Galera MySQL, no problem. Network scanning tools, no problem. HPC workloads, yeah, that sounds do-able. Java, no problem. LAMP, no problem. Specialized Netflow analysis tool that you compiled because your tool can’t create the report you need? No problem. You name it and Kube can run it – and even easier than before. That Netflow tool you compiled in February 2017? It will still run two years later when you haven’t touched it since you compiled it – because it was built as a container image.
So, in short, yes, Kubernetes has a learning curve, but that’s why we love it. We have had PaaS’s for years, and containers still took off. Why? If PaaS is a panacea, then containers shouldn’t have taken off right? They took off because they are flexible, powerful, and elegant. With that, I will leave you with a drawing I made while developing my Linux Container Internals Lab. I created it shortly after I had an epiphany of how powerful Kubernetes is. It allows you to model an entire IT environment. All of the basic patterns are already baked into Kubernetes. It’s opinionated – just enough. Hopefully, this drawing will help with that learning curve people keep talking about….