Following up from my last post on creating a simple yum repository, here is how to setup a local CentOS mirror.
Making a local CentOS mirror.
2
Category Archives: Uncategorized
The Dell C6220 and an Ode to my Dell DCS Team.
I’m pretty hard on vendors.
I can be down right mean sometimes when arguing for what I think is right. When I am fed buzzwords and technobabbel or smell BS I call them out on it.
Free drinks, swag or fancy steak dinners won’t get my business. Price, performance and responsiveness to my input and problems will. (Though, I’ll certainly let you guys keep trying since I love steak).
When I find technologies or vendors I like I latch on to them and nerdgush about them on this blog or to my peers. The C-Series servers are a good example. They’re pretty cool and allow me to hit my goals for pricing, density and other features, but the real value is the Dell Data Center Solutions team. Other vendors may have good-enough products and better pricing, but they don’t have anything like these guys.
Keepalived for MySQL High Availability on CentOS
We have a pretty normal single master MySQL setup.
Since we have a read heavy application it makes sense. Everyone writes to the master and reads from a large pool of read-only slaves.
But, with more and more slaves it becomes hard to manage what nodes read from what slaves. It can get unmanageable pretty quick when configuring the app servers.
If we lose a MySQL slave, we have to redirect all of those servers to the new one… which descends into a bunch of temporary app config or DNS changes that sometimes are not temporary :/
The stuff in this article isn’t my bit of magic, but it is what we have been using in one of our three datacenters for about a year now and am hoping to migrate the others to the scheme. My boss and an ex co-worker set it up an I think it is pretty nice.
Installing Graylog2 0.9.6, ElasticSearch 0.18.7, & MongoDB 2.0.3 on CentOS 5 (With RVM)
Gorilla Party Rocking your logs like an open-source mogul.
Graylog2‘s moto should be LMFAO (logging my freaking apps off).
Graylog2 is lovely little Splunk-like server that collects your logs and provides a nice interface for searching and analyzing them.
From the site
Graylog2 is an open source log management solution that stores your logs in ElasticSearch. It consists of a server written in Java that accepts your syslog messages via TCP, UDP or AMQP and stores it in the database. The second part is a web interface that allows you to manage the log messages from your web browser.
They have lovely screen shots here.
The only problem with it is it has quite a few moving parts that need to be installed that are not traditionally easy to get going on CentOS.
So, here is my guide.
Keepalived 1.1.20 RPMs for CentOS 5
Keepalived is a very handy piece of ops-sauce. Dash some on your operations project and it adds a bit of tangy high availability and an aroma of robust fail-over.
It implements a VRRPv2 stack to handle LVS director failover and acts as a userspace daemon for LVS cluster nodes healthchecks and LVS directors failover.
While trying to reverse engineer how a previous co-worker setup a MySQL load balancing scheme using keepalived I discovered how difficult it was to find rpms for it (I found 1.1.10 out there). I’ll be posting later on the MySQL HA scheme later.
I tried building the latest version (1.2.2) which continually broke in RHEL5 (despite there being a RHEL6 rpm)… so I gave in and built the latest version of the previous release (1.1.20).
Here we go…
I wield the MySQL hammer of Thor! Mass killing queries by host.
Just a quick protip. I’m not a DBA, but I’m about to whack you with the clue hammer.
I had an errant application node hammering a MySQL slave with metric butt loads (engineering term) of junk queries that were locking the slave up.
If you find yourself in a similar position without your own surly DBA around, I’ll save 20 minutes of googling:
Adjust your slab! Memcached 1.4.12 RPMs on CentOS 5.7.
So, memcached 1.4.11 lets you rebalance and reassign slab memory!
This is epic!
Info why this is epic here.
Info on the implementation is in the release notes
From the release notes, please remember that the slab reassignment feature is in beta and is subject to some changes.
I just took the regular spec file I found for the project elsewhere and modified it a little. I disabled the SASL stuff in my spec file since we don’t use it and I didn’t want to mess with building it.
EDIT: Actually, this article has revised for less yak shaving. With the help of Dormando and Justin Lintz. I was able to shed some unneeded dependencies.
So here you go:
Cassandra NYC 2011 Talk on YouTube
I’ll be talking Tuesday, 12/6/11, at the Cassandra NYC Conference.
Heads up, I’m going to be giving my Cassandra for Sysadmins’s talk at Cassandra NYC on Tuesday, December 6th.
Come by and say hello!
Building RPMs for and setting up StatsD and Graphite on CentOS.
A while back Etsy opensourced a little node.js daemon called StatsD that makes it easy for you to ‘Measure All the Things.’
In my current environment setting up graphs for the folks on the business team and on the dev team is difficult and time consuming as it has to funnel through ops. We’re a bottleneck 
I’m hoping to implement StatsD to make graphing a service that most anyone can directly interact with and remove me and my team as the bottleneck.
Below are my notes for setting it up.