Gorilla Party Rocking your logs like an open-source mogul.  

Graylog2‘s moto should be LMFAO (logging my freaking apps off).

Graylog2 is lovely little Splunk-like server that collects your logs and provides a nice interface for searching and analyzing them.

From the site

Graylog2 is an open source log management solution that stores your logs in ElasticSearch. It consists of a server written in Java that accepts your syslog messages via TCP, UDP or AMQP and stores it in the database. The second part is a web interface that allows you to manage the log messages from your web browser.

They have lovely screen shots here.

The only problem with it is it has quite a few moving parts that need to be installed that are not traditionally easy to get going on CentOS.

So, here is my guide.

These steps were borrowed liberally from Joe Miller‘s blog here where he setup a previous version, as well as from each product’s documentation.

Installing ElasticSearch:

Grab and place ElasticSearch.

cd /opt
curl http://cloud.github.com/downloads/elasticsearch/elasticsearch/elasticsearch-0.18.7.tar.gz | tar zxv 
ln -s elasticsearch-0.18.7/ elasticsearch

Install the ES service wrapper.

curl -k -L http://github.com/elasticsearch/elasticsearch-servicewrapper/tarball/master | tar -xz
mv *servicewrapper*/service elasticsearch/bin/
rm -Rf *servicewrapper*
/opt/elasticsearch/bin/service/elasticsearch install

Give yourself a control script.

 
ln -s `readlink -f elasticsearch/bin/service/elasticsearch` /usr/bin/elasticsearch_ctl

Give the ES cluster a unique name.

sed -i -e 's|# cluster.name: elasticsearch|cluster.name: graylog2|' /opt/elasticsearch/config/elasticsearch.yml

Fire it up.

 
/etc/init.d/elasticsearch start

Test it out.

 
curl -XGET 'http://localhost:9200/_cluster/health?pretty=true'

Installing MongoDB:

Grab MongoDB from 10Gen:

 
wget http://downloads-distro.mongodb.org/repo/redhat/os/x86_64/RPMS/mongo-10gen-2.0.3-mongodb_1.x86_64.rpm
wget http://downloads-distro.mongodb.org/repo/redhat/os/x86_64/RPMS/mongo-10gen-server-2.0.3-mongodb_1.x86_64.rpm
rpm -Uvh *mongo*.rpm

Fire it up

 
/etc/init.d/mongod start

Setup MongoDB and auth:

mongo

Dump this into the shell (after updating the credentials as you see fit.

  
use admin
 db.addUser('admin', 'adminpass')
 db.auth('admin', 'adminpass')
use graylog2
 db.addUser('grayloguser', 'graylogpass')
 db.auth('grayloguser', 'graylogpass')
exit

Install the Graylog2 server:

Grab and place it:

 
cd /opt 
curl  http://cloud.github.com/downloads/Graylog2/graylog2-server/graylog2-server-0.9.6.tar.gz | tar zxv 
ln -s graylog2-server-0.9.6/ graylog2-server

Setup the config file and remove mongo authentication (couldn’t get it to work with auth enabled):

 
cp /opt/graylog2-server/graylog2.conf{.example,}
cd /etc
ln -s /opt/graylog2-server/graylog2.conf graylog2.conf
cd -
sed -i -e 's|mongodb_useauth = true|mongodb_useauth = false|' /opt/graylog2-server/graylog2.conf

Drop an init script:

vim /etc/init.d/graylog2-server
In it put this:

 
#!/bin/sh
#
# graylog2-server:   graylog2 message collector
#
# chkconfig: - 98 02
# description:  This daemon listens for syslog and GELF messages and stores them in mongodb
#
CMD=$1
NOHUP=`which nohup`
JAVA_HOME=/usr/java/latest
JAVA_CMD=$JAVA_HOME/bin/java
GRAYLOG2_SERVER_HOME=/opt/graylog2-server
 
start() {
    echo "Starting graylog2-server ..."
    $NOHUP $JAVA_CMD -jar $GRAYLOG2_SERVER_HOME/graylog2-server.jar > /var/log/graylog2.log 2>&1 &
}
 
stop() {
        PID=`cat /tmp/graylog2.pid`
    echo "Stopping graylog2-server ($PID) ..."
        kill $PID
}
 
restart() {
    echo "Restarting graylog2-server ..."
        stop
        start
}
 
case "$CMD" in
    start)
        start
        ;;
    stop)
        stop
        ;;
    restart)
        restart
        ;;
    *)
        echo "Usage $0 {start|stop|restart}"
        RETVAL=1
esac

Prime and load:

chmod +x /etc/init.d/graylog2-server
chkconfig --add graylog2-server
chkconfig graylog2-server on

Fire it up:

 
/etc/init.d/graylog2-server start

Grab the Web Interface:

 
cd /opt
curl  http://cloud.github.com/downloads/Graylog2/graylog2-web-interface/graylog2-web-interface-0.9.6.tar.gz | tar zxv 
ln -s graylog2-web-interface-0.9.6 graylog2-web-interface

Install RVM with Ruby 1.9.2:

 
sudo yum install -y gcc-c++ patch readline readline-devel zlib zlib-devel libyaml-devel libffi-devel openssl-devel curl-devel
echo insecure >> ~/.curlrc
bash -s stable < <(curl -s -k https://raw.github.com/wayneeseguin/rvm/master/binscripts/rvm-installer)
rvm install 1.9.2

Setup a Graylog2 user:

 
useradd graylog2 -d /opt/graylog2-web-interface -G rvm
chown -R graylog2:graylog2 /opt/graylog2-web-interface*

Give root RVM access:

 
usermod -G rvm root

Bundle:

 
rvm use 1.9.2
cd /opt/graylog2-web-interface
gem install bundler
bundle install

Setup the web interface’s mongo config:

vim /opt/graylog2-web-interface/config/mongoid.yml

Make sure the production setting looks like this:

 
production:
  host: localhost
  port: 27017
  username:
  password:
  database: graylog2

Fire it up and set a password:

 
su - graylog2
rvm use 1.9.2
RAILS_ENV=production script/rails server

Browse to http://$HOSTNAME:3000 and setup your first user.

Once done, CTRL-C to stop the server, then CTRL-D to go back to root.

Install and configure Passenger:

 
yum -y install curl-devel
rvm use 1.9.2
gem install passenger
gem install file-tail

Drop an init script:

vim /etc/init.d/graylog2-web
In it put this:

 
#!/bin/bash
#
# graylog2-web:   graylog2 web interface
#
# chkconfig: - 98 02
# description:  Starts graylog2-web-interface using passenger-standalone. \
#       Uses RVM to use switch to a specific ruby version.
#
# config
USER=graylog2
APP_DIR=/opt/graylog2-web-interface
RVM_RUBY=1.9.2
ADDR=0.0.0.0
PORT=3000
ENVIRONMENT=production
LOG_FILE=/var/log/graylog2-web-interface.log
 
# --

if [ ! -d $APP_DIR/tmp ]; then
   mkdir $APP_DIR/tmp
fi

if [ ! -d $APP_DIR/log ]; then
   mkdir $APP_DIR/log
fi

chown -R $USER:$USER $APP_DIR/{log,tmp}
chmod -R 777 $APP_DIR/{log,tmp}

CMD_START="cd $APP_DIR; rvm use $RVM_RUBY; passenger start -d \
                    -a $ADDR \
                    -p $PORT \
                    -e $ENVIRONMENT \
                    --user $USER"
CMD_STOP="cd $APP_DIR; rvm use $RVM_RUBY; passenger stop -p $PORT"
 
CMD_STATUS="cd $APP_DIR; rvm use $RVM_RUBY; passenger status -p $PORT"
 
. /lib/lsb/init-functions
case "$1" in
  start)
    echo "Starting graylog2-web-interface"
    su - $USER -c "$CMD_START"
    ;;
  stop)
    echo "Stopping graylog2-web-interface"
    su - $USER -c "$CMD_STOP"
    ;;
  status)
   su - $USER -c "$CMD_STATUS"
   ;;
  *)
    echo "Usage: $0 start|stop|status"
    exit 3
    ;;
esac

Prime and load:

chmod +x /etc/init.d/graylog2-web
chkconfig --add graylog2-web
chkconfig graylog2-web on

Fire it up:

 
/etc/init.d/graylog2-web start

First time it runs it’ll grab some stuff for passenger, but will be fine every time after.