Name: Nathan, aka "nathan"
Posts by nathan:
Getting a Basic Cobbler server going on CentOS.
March 26th, 2012
I love Cobbler.
Cobbler + Chef in my environment means that I can go from bare metal to an active cluster node in moments with little effort.
It is a powerful system for managing kickstart profiles, pxeboot, power, dhcp, dns etc..
Below are some notes to help get you going with just the basic feature set. It is a system you can easy go nuts with to automate a lot of your infrastructure.
Making a local CentOS mirror.
March 25th, 2012
Following up from my last post on creating a simple yum repository, here is how to setup a local CentOS mirror.
Making a simple Yum repository.
March 24th, 2012
Here is some low hanging fruit to improve your RHEL environment and simplify your work… setup a simple Yum repository.
Now Hiring Now, Right Now We’re Hiring Now.
March 19th, 2012
If you follow this blog, you can see I get to play with all sorts of awesome stuff.
Y U JELLY? No need to be.
There is so much awesome stuff to do… I don’t have time to do all the awesome stuff there is to do.
Come work with me and do awesome stuff.
It’ll be awesome.
Awesome Web Operations Engineer
Awesome Junior Web Operations Engineer
Director of Awesome IT
I do not apologize for my gratuitous use of the word awesome. If you can’t take this much awesome, you probably couldn’t handle working at Outbrain.
The Dell C6220 and an Ode to my Dell DCS Team.
March 19th, 2012
I’m pretty hard on vendors.
I can be down right mean sometimes when arguing for what I think is right. When I am fed buzzwords and technobabbel or smell BS I call them out on it.
Free drinks, swag or fancy steak dinners won’t get my business. Price, performance and responsiveness to my input and problems will. (Though, I’ll certainly let you guys keep trying since I love steak).
When I find technologies or vendors I like I latch on to them and nerdgush about them on this blog or to my peers. The C-Series servers are a good example. They’re pretty cool and allow me to hit my goals for pricing, density and other features, but the real value is the Dell Data Center Solutions team. Other vendors may have good-enough products and better pricing, but they don’t have anything like these guys.
The Dell C6100 and IPMI / DRAC / MegaRAC
March 18th, 2012
Over the last year Google Analytics says I’ve been getting a lot of hits from search that indicate there are some folks who want to know how the C-series DRAC works.
It is easy enough to setup like any other IPMI/DRAC system.
First you’ll need to plug the IPMI/Management Ethernet port into your network (preferably an our of band (OOB) network seperate from your production network). In the BIOS, make sure the management port is set to ‘Dedicated’, earlier ones shipped with it set to ‘Shared’ by default which precluded the dedicated IPMI port.
Keepalived for MySQL High Availability on CentOS
March 18th, 2012
We have a pretty normal single master MySQL setup.
Since we have a read heavy application it makes sense. Everyone writes to the master and reads from a large pool of read-only slaves.
But, with more and more slaves it becomes hard to manage what nodes read from what slaves. It can get unmanageable pretty quick when configuring the app servers.
If we lose a MySQL slave, we have to redirect all of those servers to the new one… which descends into a bunch of temporary app config or DNS changes that sometimes are not temporary :/
The stuff in this article isn’t my bit of magic, but it is what we have been using in one of our three datacenters for about a year now and am hoping to migrate the others to the scheme. My boss and an ex co-worker set it up an I think it is pretty nice.
Installing Graylog2 0.9.6, ElasticSearch 0.18.7, & MongoDB 2.0.3 on CentOS 5 (With RVM)
March 17th, 2012
Gorilla Party Rocking your logs like an open-source mogul.
Graylog2‘s moto should be LMFAO (logging my freaking apps off).
Graylog2 is lovely little Splunk-like server that collects your logs and provides a nice interface for searching and analyzing them.
From the site
Graylog2 is an open source log management solution that stores your logs in ElasticSearch. It consists of a server written in Java that accepts your syslog messages via TCP, UDP or AMQP and stores it in the database. The second part is a web interface that allows you to manage the log messages from your web browser.
They have lovely screen shots here.
The only problem with it is it has quite a few moving parts that need to be installed that are not traditionally easy to get going on CentOS.
So, here is my guide.
Keepalived 1.1.20 RPMs for CentOS 5
March 17th, 2012
Keepalived is a very handy piece of ops-sauce. Dash some on your operations project and it adds a bit of tangy high availability and an aroma of robust fail-over.
It implements a VRRPv2 stack to handle LVS director failover and acts as a userspace daemon for LVS cluster nodes healthchecks and LVS directors failover.
While trying to reverse engineer how a previous co-worker setup a MySQL load balancing scheme using keepalived I discovered how difficult it was to find rpms for it (I found 1.1.10 out there). I’ll be posting later on the MySQL HA scheme later.
I tried building the latest version (1.2.2) which continually broke in RHEL5 (despite there being a RHEL6 rpm)… so I gave in and built the latest version of the previous release (1.1.20).
Here we go…
A new Cassandra Chef Cookbook for handling multiple clusters.
March 15th, 2012
I find myself running more and more Cassandra clusters and when we were on Chef 0.9.8 I was being lazy and just cloning my Cassandra cookbook per cluster. Not exactly a way to scale the manageability of your config 
Now I’ve refactored the cookbook to allow me to manage multiple clusters by extracting the cluster_name and initial_token from a databag. Once we start implementing the new Environments feature in Chef 0.10 I’ll be able to simplify this further. 
I’m debating having the cookbook auto-generate tokens and assign them as well as re-generate/nodetool move/re-balance when I’ve added another node with that cluster specified in the databag. That’s a big project and for now I’m too much of a control freak to automate that, but I’m thinking on it.
I’ve also made it so the cookbook auto-generates the cassandra-topology.properties for the PropertyFileSnitch based off of location info stored in the databag.