Multipath ...

Issue: Removing a device, or replacing a device with a different SAN LUN can be a bit challenging.

HISTORY: Take this completely hypothetical situation, which may or may not happen to me quite frequently...

your customer asked for 3 x 64 and 1 x 16 Gig LUNs... and for some reason the SAN admin assigns 4 x 64 Gig LUNs. You don't know this until you scan the bus and see them from the OS. At which point you have to tell the SAN admin that you need to replace 1 of the 64 Gig LUNs with a 16 Gig. And you also ask the admin to let you know when he/she removes the incorrect LUN so you can do your procedure to then remove the device from the OS view, and then he/she can proceed with adding the 16 Gig. Well, inevitably you will end up with syslog complaining about a SAN device no longer being available and the fun begins (this is because the admin removed the 64 and added the 16 probably in the same keystroke, or click of a button and this will not give you an opportuntiy to straighten things out...) Well, the only thing that makes this worse is when it happens to a 3-node RAC cluster and have to clean things up on 3 boxes ;-)

# tail -f /var/log/messages

# multipath -ll > /tmp/mpath.out

-- check that output for 'fail' and you should still see the "sd" aliases.

-- confirm the failed device has the same WWN as you were expecting

# echo offline > /sys/class/scsi_disk/2:0:0:44/device/state

# echo offline > /sys/class/scsi_disk/3:0:0:44/device/state

# echo 1 > /sys/class/scsi_disk/2:0:0:44/device/delete

# echo 1 > /sys/class/scsi_disk/3:0:0:44/device/delete

# echo "- - -" > /sys/class/scsi_host/host#/scan

# rescan-scsi-bus.sh

-- the rescan shell script is part of the sg3_utils package

I believe a point of contention is that the LUN being replaced occupies the same device path, although I have not devoted much time to proving this, yet...

#! /bin/sh

# return all offline scsi devices to the running state

for d in /sys/block/sd*/device/state; do if [ `cat $d` = "offline" ]; then echo running > $d; fi; done

Enable additional logging in the lpfc driver:

# echo 0x1f > /sys/module/lpfc/lpfc_log_verbose

Enable extra logging for the scsi-subsystem:

# echo 7 > /sys/module/scsi_mod/scsi_logging_level

~