Finally upgraded to Chef 0.10.6 from 0.9.8.
Hot, sweet, environments and encrypted data bag action.
Except… well… the chef-client would ocassionally die… quetly.
No log, no debug output, no exit codes, just poof… no more chef-client daemon.
(This is not the point there you guys tell me I should use cron or runit or daemontools or something to run chef, I’ve heard it)
The lovely folks at Opscode said that running on ruby 1.8.7 rather than ruby 1.9.2 was the culprit and then drew my attention to the super-happy-awesome Opscode Chef Omnibus installer here (avaliable as rpms, debs, tgz, etc..)
It installs (almost) everything you need into /opt and lets Chef run in it’s own ‘embedded’ ruby 1.9.2 environment keeping my system ruby clean.
I grabbed the rpm for the RHEL version by poking around their S3 bucket and dumped it into my local yum repo:
wget http://s3.amazonaws.com/opscode-full-stack/el-5.7-x86_64/chef-full-0.10.6-1.x86_64.rpm
From there I can remove the rbel Chef and install the Omnibus package:
yum -y erase rubygem-chef
yum -y install chef-full
Everything is in /opt/opscode
[root@sandbox:~]# /opt/opscode/embedded/bin/ruby -v
ruby 1.9.2p180 (2011-02-18) [x86_64-linux]
chef-solo, chef-client, knife, and shef are live in /opt/opscode/bin and a link is put in /usr/bin
There were two issues with it I’ve found.
First, it doesn’t drop an init script. I just stole the one from the rbel rubygems-chef RPM and had Chef drop it in /etc/init.d/chef-client as a cookbook_file. Worked fine.
Second, I have a cookbook that sets a user password which requires the ruby-shadow gem.
Ordinarily you could have a cookbook install it in the embedded ruby path thus
gem_package "ruby-shadow" do
gem_binary "/opt/opscode/embedded/bin/gem"
action :install
end
But, there is some bug with mongrel and ruby 1.9.2 that causes some problems (seen here specifically in Omnibus, documented here on stack overflow, I chose the top voted solution, not the ‘accepted’ one).
The little hack below got me around it for now. Since I am still ‘a/b testing’ it I don’t have the Omnibus installer setup everywhere so I wrapped it in a hackish check.
if File.exist?("/opt/opscode/")
package "gcc" do
action :install
end
execute "installMongrelPre" do
command "/opt/opscode/embedded/bin/gem install mongrel --pre"
creates "/opt/opscode/embedded/lib/ruby/gems/1.9.1/gems/mongrel-1.2.0.pre2/setup.rb"
action :run
end
gem_package "ruby-shadow" do
gem_binary "/opt/opscode/embedded/bin/gem"
action :install
end
end
And from there, my chef-client run completes with no issues and the chef-client daemon does not mysteriously die.