Skip to main content

Multipath ...

Issue: Removing a device, or replacing a device with a different SAN LUN can be a bit challenging.

HISTORY: Take this completely hypothetical situation, which may or may not happen to me quite frequently...
your customer asked for 3 x 64 and 1 x 16 Gig LUNs... and for some reason the SAN admin assigns 4 x 64 Gig LUNs. You don't know this until you scan the bus and see them from the OS. At which point you have to tell the SAN admin that you need to replace 1 of the 64 Gig LUNs with a 16 Gig. And you also ask the admin to let you know when he/she removes the incorrect LUN so you can do your procedure to then remove the device from the OS view, and then he/she can proceed with adding the 16 Gig. Well, inevitably you will end up with syslog complaining about a SAN device no longer being available and the fun begins (this is because the admin removed the 64 and added the 16 probably in the same keystroke, or click of a button and this will not give you an opportuntiy to straighten things out...) Well, the only thing that makes this worse is when it happens to a 3-node RAC cluster and have to clean things up on 3 boxes ;-)

# tail -f /var/log/messages
# multipath -ll > /tmp/mpath.out
-- check that output for 'fail' and you should still see the "sd" aliases.
-- confirm the failed device has the same WWN as you were expecting

# echo offline > /sys/class/scsi_disk/2:0:0:44/device/state
# echo offline > /sys/class/scsi_disk/3:0:0:44/device/state
# echo 1 > /sys/class/scsi_disk/2:0:0:44/device/delete
# echo 1 > /sys/class/scsi_disk/3:0:0:44/device/delete
# echo "- - -" > /sys/class/scsi_host/host#/scan
# rescan-scsi-bus.sh
-- the rescan shell script is part of the sg3_utils package

I believe a point of contention is that the LUN being replaced occupies the same device path, although I have not devoted much time to proving this, yet...

#! /bin/sh
# return all offline scsi devices to the running state
for d in /sys/block/sd*/device/state; do if [ `cat $d` = "offline" ]; then echo running > $d; fi; done

Enable additional logging in the lpfc driver:
# echo 0x1f > /sys/module/lpfc/lpfc_log_verbose

Enable extra logging for the scsi-subsystem:
# echo 7 > /sys/module/scsi_mod/scsi_logging_level
~
Labels:

Comments

Post a Comment

Popular posts from this blog

P2V using dd for KVM-QEMU guest

Preface: I have certainly not exhaustively tested this process.  I had a specific need and found a specific solution that worked. Situation:  I was issued a shiny new laptop running Red Hat Enterprise Linux 7 (with Corp VPN, certs, Authentication configuration, etc...)  The image was great, but I needed more flexibility on my bare metal.  So, my goal was to P2V the corporate image so I could just run it as a VM. * Remove corporate drive and install new SSD * install corp drive in external USB-3 case * Install RHEL 7 on new SSD * dd old drive to a disk-image file in a temp location which will be an image which is the same size as your actual drive (unless you have enough space in your destination to contain a temp and converted image) * convert the raw disk-image to a qcow file while pushing it to the final location - this step should reduce the disk size - however, I believe it will only reduce/collapse zero-byte blocks (not just free space - i.e. if you de...
1 comment
Read more

Sun USS 7100 foo

TIP: put ALL of your LUNs into a designated TARGET and INITIATOR group when you create them.  If you leave them in the "default" group, then everything that does an discovery against the array will find them :-( I'm struggling to recognize a reason that a default should even be present on the array. Also - who, exactly, is Sun trying to kid.  The USS is simply a box.. running Solaris .. with IPMP and ZFS.  Great.  If you have ever attempted to "break-in" or "p0wn" your IBM HMC, you know that there are people out there that can harden a box - then.. there's Sun.  After a recent meltdown at the office I had to get quite intimate with my USS 7110 and learned quite a bit.  Namely: there's a shell ;-) My current irritation is how they attempt to "warn you" away from using the shell (my coverage expired a long time ago to worry about that) and then how they try to hide things, poorly. I was curious as to what version of SunOS it ...
1 comment
Read more

Extending SNMP to run arbitrary shell script

Why are we here... This is not likely something I would have pursued under normal circumstances.  I happen to be working for a customer/client who is not afforded a lot of flexibility to accomplish their goals.  In this case, the rigor is justified.  They have to sometimes be fairly creative with how they solve problems. In this case they would like to utilize an existing snmp implementation to execute a command (or shell script) on a remote system.  They came to me with the idea of using Net-SNMP extend. https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Deployment_Guide/sect-System_Monitoring_Tools-Net-SNMP-Extending.html NOTE:  This is NOT a good implementation strategy in the "real world"  it will simply allow you to test the functionality.  There are a TON of security implications which would need to be taken in to consideration. Implementation Steps: [root@rh7tst01 ~]# yum -y install net-snmp net-snmp-utils ...
Post a Comment
Read more