The Systems Administrator’s Lab

Recently, I listened to an O'Reilly webcast called "The Myths of Innovation" where Scott Berkun discussed the concept of a lab. He showed a picture of Edison's lab which showed wooden tables, lamps, and beakers. Systems administrators are also inventors.  We are required to script, program, and configure exotic servers and equipment. To discover new solutions, we need a lab. This is especially true with cloud computing and virtual infrastructure where machines are created and destroyed in a very transient manner.  You need a lab to track all of the successful and failed experiments.

KVM Virtualization 101

Background For those of you who are familiar with virtualization, KVM is a hypervisor which is built into the Linux Kernel. This allows a Linux desktop or server to simulate multiple pieces of hardware, which can then have different operating systems installed. KVM uses the QEMU virtual machine format. QEMU may be familiar to those

Bootstrapping and Rooting Documentation: Part 2

Background In Bootstrapping and Rooting Documentation: Part 1, I laid out a blueprint for using documentation as the bootstrap for entry into an operations environment. In this article I will dig into the three main concepts mentioned in Part 1. In Part 3, I will demonstrate our use case for a data center of about

Uptime: Does it matter with the modern web service?

Last week Ksplice announced free no-reboot kernels for Fedora which sparked off some interesting conversation about uptime over at the Standalone Sysadmin. Honestly, I ran across Ksplice a while back and I thought to myself, huh that might be useful for a certain niche, I added to a wiki page that I use to track

Bootstrapping and Rooting Documentation: Part 1

Bootstrapping and rooting are two concepts often associated with computing, but not the documentation of computing environments. Sometimes concepts such as these are called design patterns and I would like to try and introduce a new pattern called Documentation Bootstrapping. I think this is a critical pattern for the creation of any successful and useful documentation.

Unix/Linux Signals 101

Background Unix/Linux allows a user to have control over a program that they are running by sending what are called signals. These signals are then normally handled by the program in a way that is compliant with Unix/Linux standards. Two of the most important signals that are commonly sent to a program are called SIGTERM

Unix/Linux Filesystem Permissions 101

Background Standard Unix filesystem permissions are less complex than Windows file system permissions and Linux ACLs. Though, this lacks flexibility which is sometimes needed, In many cases it can be leveraged as an advantage. Often the complexity of ACLs can allow administrators to create file system permissions which are cumbersome to audit and document. The

Centralized Logging System, Analysis, and Troubleshooting

Background Building a feature complete centralized logging system that provided the ability to trouble shoot problems and pro-actively find new issues before they became service outages was a top priority when I first started at ( I call it feature complete because it has successfully done both for us without spending too much time of

DevOps Toolchain: Problems with Automated Deployment, Data & Workflow

Background Automated deployment is obviously not new, but until this point, there was not much push in the open source world. Recently, the idea of DevOps or Ops 2.0 is gaining ground. We are starting to think of our deployment and provisioning methods more like software engineers. We are developing tools to help us provision