I work for a major Fortune 1000 company, and recently had occasion to specify, purchase and install a network/systems monitoring server. I decided to go with Ubuntu 7.04 running on an HP DL360G5, using the Cacti, Zenoss and Smokeping FOSS software packages.
Cacti is great at graphing data – specifically in my case, bandwidth and disk space usage. Zenoss is a network monitoring suite, using snmp to monitor systems, networks and services, and sending alerts via email when various user-defined thresholds are reached. Smokeping is a simple tool that shows you network latency and packet loss over time. Together they give me a great view of what is going on in my network, and alerting me to problems ahead of time, so I working pro-actively and not reactively.
This entry will server more or less as ‘crib notes’ – there are plenty of instructions on the net on how to do what I just did, but this will detail the issues, problems, and solutions that I ran into. It also focuses mostly on the Zenoss install – the others were almost trivial to install and get up and running.
The install of Ubuntu on the HP DL360G4 went exceedingly well – simply popped the CD in the drive, and booted to the LiveCD. Once there, I ran the install program, chose the defaults for drive partitioning, and an hour or so later, I had a newly installed Ubuntu system. after setting my IP settings via the GUI, I had to drop to the shell for a small bit of work. Ubuntu doesn’t come with ssh installed, so:
sudo apt-get install ssh sshd
did the trick. I was then able to get the rest of my work done from my workstation, and not at the console in the server room.
Setting up Cacti first, i followed the directions found on Ubuntugeeks.com. The directions here were clear, and worked straight out of the box (which surprised me – kudos to Ubuntugeeks.com!)
I then installed Smokeping. This one is easy as pie:
sudo apt-get install smokeping
From there, I copied over my config from my old machine (located in /etc/smokeping), and restarted smokeping:
/etc/init.d/smokeping restart
I had to manually remove the .rrd files in the smokeping directory:
cd /var/lib/smokeping/Local
rm -rf *
Note you only need to do this on a new install, when you are copying over a config from another machine. It will rebuild the .rrd files.
I then started on the Zenoss install, using the decent instructions located on Zenoss.com. These instructions were great to get me started and pointing in the right direction, but were lacking in other areas. The instructions are targeted for Ubuntu 6.10 – I was installing on 7.04, and I’m not sure if that was was caused the errors or not.
Note that I skipped a few steps – my machine was already installed and on the network, so I skipped Steps 1-6 and Steps 8-10. I also skipped Step 15, as Mysql was already setup during the Cacti installation. Step 17 is where things got hairy. Dependencies. I hate dependencies. To start the Zenoss installation, you type:
./install.sh
And…an immediate error.
Zenoss requires python 2.3 or 2.4
Ok, so my system apparently comes with Python 2.5. A quick
sudo apt-get install python2.4 python2.4_dev
fixes that. Ok, then
make clean
./install.sh
And the install is started again. After a bit of a wait – Uh-oh, another failure. A quick
tail -n 15 zenbuild.log
showed me that the install requires something called ‘autoconf’. Ok,
sudo apt-get install autoconf
then
make clean
./install.sh
And…a long wait. And another failure. Tailing the zenbuild.log file again, I find I need something called ‘swig’:
sudo apt-get install swig
fixes that. Rerun the install again (after another make clean), and Viola! Zenoss has installed successfully! Almost. Seems that some of the services need to run as root (hence the sudoers edit in Step 14). Except that didn’t seem to work. So, I pulled in the big guns – one of our Unix experts (hi Dave!). After much gnashing of teeth, and wailing, we determine that we need to do the following:
sudo chown root:zenoss $ZENHOME/bin/zensocket
sudo chmod 04750 $ZENHOME/bin/zensocket
This allows zensocket to be run by anyone in the zenoss group (in this case the zenoss user) to run as if root ran it (this is the set uid you might hear about). Now, looking in the Zenoss application (once you log in via Step 19 of the instructions), you can go into the SETTINGS then the DAEMONS tab, and start the three services that wouldn’t start without this last step.
Finishing up with the instructions from the Zenoss.com website, the last step I had to do was setup the email notifications. To make a real long story short, I never did get this to work with Exim 4. Simply installing Sendmail, and all was happy:
sudo apt-get remove exim4
sudo apt-get install sendmail
After all this, it was a simple matter of going into Zenoss, and setting up the devices that you need. But thats for another article.
Popularity: 4% [?]
October 9th, 2007 at 9:36 am
apt-get install opennms and you get all that
… Actually, I’m one of the maintainers of OpenNMS but I followed your instructions on my Ubuntu VM and it looks like you’ve documented this complete and will be a nice monitoring solution.
Even I think Zenoss is pretty cool. No doubt. But, they really should have packaging… apt-get (or yum for RPMers) install zenoss… that would fix those nasty dependency problems.
I meet with a lot of companies that install multiple open source monitoring tools to cover their needs and there are tremendous advantages to doing so… such as rapid deployment and flexible configuration. When it comes time for a common framework, someone ends up writing a front-end to those tools and it begins to get much more difficult to grow and support.
just another hacker’s $.02
October 9th, 2007 at 11:41 pm
I have unsuccessfully tried installing both cacti and zenoss, I gave up after second step above. Instead I installed Hyperic and GroundworkOpensource. After a week of eval I am sticking with Hyperic. Nothing beats 10 minute install and auto-discovery of all the apps I am running in my production environment (Apache, JBoss, MySQL, postfix and many home grown java processes). On top of that I get all the basic monitoring (like disk space, network, snmp, control actions for controlling above apps, syslog monitoring). All in all in less then half an hour I was monitoring close to 400 services and collecting 1000+ metric data per minute running over 7 RHEL8 servers. After I was satisfied with performance I spent another hour configuring alerting (email and alerting) on critical metric data, put in place couple of escalation scheme for me, my manager and another co-worker.
For now Hyperic OSS is doing the job, some day when the company has profit we will see if we can buy the support but for now we are all set.
If anyone wants to see how it looks, please drop me a line and I will send some cool snapshots of the systems I am monitoring.
October 10th, 2007 at 1:27 pm
Kamal
Interesting that you’ve mentioned Hyperic – I was/am getting frustrated with Zenoss’ windows monitoring and specifically the graphing. Some systems work, some don’t, but any way you slice it the documentation is poor. I installed Hyperic about two hours ago, and you are right – very easy to install and get running. The need for client agents is a bit cumbersome, but I’m only monitoring a small branch office, so thats not a big deal.
I’m going to run Hyperinc and Zenoss in parallel for a while (along with Cacti, to get the graphing that Zenoss so far doesn’t provide me), and I’ll report back with a new article on how that goes.
David -
I tried OpenNMS a while ago, and for reasons that completely escape me, I turned away from it. Perhaps I’ll throw it on this box, also, and give it a good three-way test agains Zenoss and Hyperic
October 11th, 2007 at 8:02 am
cavtroop,
Hyperic’s a great product, too. All these products tools have advantages… the great thing about open source, pick the one that works for you. Good luck with your installs…
/me sends best wishes to Jonathon’s chance of going to Comic Con!
October 11th, 2007 at 8:03 am
cavtroop,
Hyperic’s a great product, too. All these products/tools have advantages… that’s the great thing about open source, pick the one that works for you. Good luck with your installs…
/me sends best wishes to Jonathon’s chance of going to Comic Con!
October 18th, 2007 at 10:14 am
just a followup. having a HELL of a time getting opennms to install via apt-get. the stable package seems to be completely broken. Unstable seems to install, but Postgresql is giving me hell.
Zenoss was a no-brainer to install, but doesn’t do all that I need it to. Opennms seems to do what I need it to, but is a bear to install. Still on the fence about Hyperic.
November 16th, 2007 at 11:54 am
Super!
Sample design…
I really enjoyed your page.
April 15th, 2008 at 3:34 pm
@ cavtroop: Any success on the latest stable? I am digging though the endless loops to get it working.
We are using Hyperic now. I’m trying to make i working in Vmware before presenting it before my manager.
April 15th, 2008 at 3:40 pm
OpenNMS? No, never had much luck. I’ve got Nagios running now, which was fairly easy to setup, especially for my fairly small environment.
July 21st, 2010 at 5:06 am
I followed these instructions. Fantastic. Many Thanks!
November 26th, 2010 at 8:17 am
Why do you need to experience all this kind of problems with stupid Linux systems, if you can pay some money and get a very well developed Windows 7 and not have such a lot of head aches.
December 30th, 2010 at 10:07 am
I don’t like this Linux stuff and can’t understand how people use it for personal purposes.
June 28th, 2011 at 9:46 am
Action resumes tomorrow at Hayward Field, and can be followed on a live webcast provided in partnership with RunnerSpace.