Systems Administrator’s Lab: OpenSSH MaxStartups

Background When performing automation using OpenSSH/Cron you will inevitably run into concurrency problems. Recently, we had a problem where one machine was receiving 21 ssh connection within one second. This is because the standard cron daemon only has a granularity of one minute. In this article, I am going to quickly elaborate on how we

System’s Administrator’s Lab: Testing

Today, I got an email from the Fedora package manager, Red, who let me know that there was a problem with Petit. I don’t think he knew it, but it was actually my fault that the whole thing got screwed up, so I felt kind of bad. Well, to make a long story short, when

DevOps Culture: An Ethnography

Background An ethnography is a holistic study of another culture conducted by an anthropologist. During the study, the anthropologist lives among the members of the foreign culture, takes notes, and collects data. From this data, the anthropologist develops theories and tests them cross culturally to determine if the source is genetic/biological in nature or enculturated.

Systems Administrator’s Lab: Cacti Development

Background Today, I finally took the time to update a Cacti Data Query which I wrote a while back. When I took a look at it, I found out that it was never actually working for other people strait out of the box. This data query graphs BGP prefixes, messages received and messages sent. I

The Systems Administrator’s Lab

Recently, I listened to an O'Reilly webcast called "The Myths of Innovation" where Scott Berkun discussed the concept of a lab. He showed a picture of Edison's lab which showed wooden tables, lamps, and beakers. Systems administrators are also inventors.  We are required to script, program, and configure exotic servers and equipment. To discover new solutions, we need a lab. This is especially true with cloud computing and virtual infrastructure where machines are created and destroyed in a very transient manner.  You need a lab to track all of the successful and failed experiments.

DevOps Toolchain: Problems with Automated Deployment, Data & Workflow

Background Automated deployment is obviously not new, but until this point, there was not much push in the open source world. Recently, the idea of DevOps or Ops 2.0 is gaining ground. We are starting to think of our deployment and provisioning methods more like software engineers. We are developing tools to help us provision

Scripting & Automation: The Qualitative to Quantitative Workflow

Background Scripting & Automation has been a goal since the beginning of Unix and, let me state, that I believe that it is possible to achieve a Fully Automated Provisioning system in our production environments. In fact, I think it is essential that we develop fully automated provisioning systems to keep up with the rate

Introduction to DevOps

I will be giving an introduction to DevOps (( and the DevOps Toolchain (( at the Akron Linux Users Group (ALUG) ((, held at the New Era Restaurant at 10 Massillon Rd. Akron Ohio (See map below) Often development and operations seem to have competing goals. Development is the gas, while operations is often perceived

Science in Systems Administration

Abstract This presentation was created for the Akron Linux Users Group in September 2009. It is a brief overview of several home grown tools and several from the wild. All of the tools used in this presentation are freely down-loadable and all but DejaClick are Open Source. This presentation enumerates philosophy and applied techniques in