Skip to main content

Better is not always... better? HP Smart Array and Linux

I am currently working through an issue with my 3-node RAC clusters (RHEL 5.6 x86_64 and Oracle RAC 11g running on HP DL580-G7 servers). They seem to enjoy rebooting themselves at will. There is nothing glaring for a root cause, other than some messages in the syslog about blocking for more than 120 seconds. Anyhow - after quite a bit of research I have discovered some things I really like about Linux. They have made the disk scheduler modular (in a sense). Therefore you can utilize your disk access in one of 4 methods. The CCISS is the HP Smart Array driver which is loaded and should be consistent in most Linux releases.

If you initially look at the "scheduler" file - you can see your four options. The one in use is surrounded by brackets. I am hoping that by changing the access for ONLY the cciss device to noop, that my reboots go away - and I leave a positive legacy behind at my customer-site ;-)


root@dbslp0067:/root
# cd /sys/block/cciss\!c0d0/queue/
root@dbslp0067:/sys/block/cciss!c0d0/queue
# cat scheduler
noop anticipatory deadline [cfq]
root@dbslp0067:/sys/block/cciss!c0d0/queue
# echo noop > scheduler
root@dbslp0067:/sys/block/cciss!c0d0/queue
# cat scheduler
[noop] anticipatory deadline cfq
root@dbslp0067:/sys/block/cciss!c0d0/queue

After trying this work-around on my system, I'm disappointed to report it did not help my cause. I will leave this out there, as I may need to tune a system for this at a later point.


Turned out to be a bad "CPU on a SAN switch blade". Not sure why multipath didn't handle the event better than the box locking up and subsequently rebooting itself. Might have to investigate the Multipath tunables?


Labels:

Comments

Post a Comment

Popular posts from this blog

Sun USS 7100 foo

TIP: put ALL of your LUNs into a designated TARGET and INITIATOR group when you create them.  If you leave them in the "default" group, then everything that does an discovery against the array will find them :-( I'm struggling to recognize a reason that a default should even be present on the array. Also - who, exactly, is Sun trying to kid.  The USS is simply a box.. running Solaris .. with IPMP and ZFS.  Great.  If you have ever attempted to "break-in" or "p0wn" your IBM HMC, you know that there are people out there that can harden a box - then.. there's Sun.  After a recent meltdown at the office I had to get quite intimate with my USS 7110 and learned quite a bit.  Namely: there's a shell ;-) My current irritation is how they attempt to "warn you" away from using the shell (my coverage expired a long time ago to worry about that) and then how they try to hide things, poorly. I was curious as to what version of SunOS it ...
1 comment
Read more

PXE boot a LiveCD image

Summary: I have wanted to build a kickstart environment which hosted a "rescue CD" or LiveCD to allow you to boot over the network after you blew your stuff up and needed to repair a few things.  Today I have worked through a method of doing so, with the help of the people who published a succinct script with the Red Hat Enterprise Virtualization Hypervisor.  (the script will be at the bottom of this post - if I have somehow not followed the GPL, please let me know and I will correct whatever is necessary) NOTE/Warning: The boot will fail due the initrd being too large (645mb).  I'm not sure how to proceed.  This procedure worked for RHEVh, because it is quite a bit smaller.  Hopefully I can report back with progress on this? :-$ Procedure: download your LiveCD image to /export/isos/RESCUE/Fedora-16-i686-Live-Desktop.iso # cd /var/tmp # vi livecd-iso-to-pxeboot (populate the file with the script shown below) # chmod 754 ./livecd-iso-to-pxeb...
2 comments
Read more

"Error getting authority: Error initializing authority: Could not connect: No such file or directory (g-io-error-quark, 1)"

"Error getting authority: Error initializing authority: Could not connect: No such file or directory (g-io-error-quark, 1)" One issue that may cause this to arise is if you managed to break your /etc/fstab We had an engineer add a line with the intended options of "nfsvers=3" but instead added "-onfsvers=3" and it broke the system fairly catastrophically.
Post a Comment
Read more