216 days, 17 hours, and one big fat panic
April 29th, 2008 by Dan Fuhry216 days and 17 hours. That is how long my server was up without interruption according to the uptime command as of this afternoon. Not bad for an area that gets power outages like crazy this time of year (it hasn’t happened in exactly that many days).
The important thing is that I beat Neal Gompa’s record, 200 days. We’ve been in friendly competition about it for quite a while now and I was quite excited when uptime counter finally reached 200 days. There is a good and valid reason that it’s over now.
Last night I was fiddling around with NFS shares trying to make the Ubuntu live CD bootable with PXE. (I have no life. :P) Unfortunately I messed something up and got rpc.mountd to deny all mount requests, thus locking me out of 80% of my home directory from Nighthawk, and messing up my network boot system which uses Pelswick as a TFTP server. A reboot looked more and more imminent as I could not seem to get nfsd to unload (it’s built as a kernel module). Today in the late afternoon I took Bigmomma to runlevel 1 and gracefully stopped enough services to be able to peacefully umount /home, after which nfsd unloaded.
The problem came when I remounted /home and realized that I had been meaning to mount it with extended attribute (xattr) support enabled. I realized that I had not done this and once again umounted the device. When I did so, I got a big fat kernel panic, the first one I’ve seen in kernel 2.6.14.
So I let it sit there for a minute thinking about all that the server had been through during the last 216 days and decided that Bigmomma was about due for maintenance. So I turned the system off, pulled out the CD burner and 3.5″ floppy drive that were only used during the installation, blew 7 months’ worth of dust out of the case, replaced the cover, and booted her back up. She’s happily serving files and web pages again.
I did end up with a few “double free or corruption” messages from rpc.mountd so NFS isn’t quite perfect yet. It’s still a little unsteady even after recompiling the nfs-utils SRPM. Still working that part out.
Posted in Uncategorized |