Skip to main content

Prelink cache - friend or foe?

Ran in to a very bothersome issue over the weekend.

Pushed a package installation automatically via Satellite to ALL of my Linux hosts at 11pm on Sunday evening.  Around 9am on Monday I was invited to a bridge to discuss some Oracle issue where no new connections were allowed to the database on the Listener.  The fun begins...
So - we discuss what changes had been applied recently and my software push is mentioned.  I had tested this software push on all our non-prod hosts 8 weeks prior - and everything has been fine.  So.. what the hell happened.

As it turns out the number of hosts affected were less than 1% - However, the way the problem was communicated made the problem sound much worse.  So - at this point I assume that the approved deployment to ALL our hosts broke EVERY.SINGLE.BOX in our environment.

I start to review the actual output of the software push and I notice a package that had been updated which was not included in my testing 8 weeks ago.  PCRE...  So, I pull that RPM down and look at the scripts and I notice that it runs ldconfig.  I also check all files that were updated in /etc (using find) and notice that /etc/prelink.cache had been updated.  I review the prelink cache and everything appears fine, but.. that file is just a collection of "paths" in a binary/data file.

We find a Red Hat article indicating that if you have an Oracle host, with and SGA greater than 200GB... that you should disable prelink cache.  Huh?

I uninstall all the packages from the night before - problem persists.  Ugg..
I then disable the prelink cache
# prelink -ua
# ls -l /etc | grep prelink

Problem goes away.  Progress... sort of?
I recreate the cache...

# prelink -va
# ls -l /etc | grep prelink

Still no issue.  Hmm.. So, I re-install all the packages from the earlier deployment.  Still no issue.

Summary:  Somehow the /etc/prelink.cache file became invalid, it would seem.  I'm not sure if there was a special character in there.. or if the path was screwed up by a new package installation (making Oracle find a binary in the wrong order).

Anyhow - I recommend Google Searching whether prelinking is a good/bad thing.  It's quite a spectacle to see all the discussions out there.  I'm still on the fence - but I'm leaning towards thinking that the prelinking is a good thing.
Labels:

Comments

Post a Comment

Popular posts from this blog

P2V using dd for KVM-QEMU guest

Preface: I have certainly not exhaustively tested this process.  I had a specific need and found a specific solution that worked. Situation:  I was issued a shiny new laptop running Red Hat Enterprise Linux 7 (with Corp VPN, certs, Authentication configuration, etc...)  The image was great, but I needed more flexibility on my bare metal.  So, my goal was to P2V the corporate image so I could just run it as a VM. * Remove corporate drive and install new SSD * install corp drive in external USB-3 case * Install RHEL 7 on new SSD * dd old drive to a disk-image file in a temp location which will be an image which is the same size as your actual drive (unless you have enough space in your destination to contain a temp and converted image) * convert the raw disk-image to a qcow file while pushing it to the final location - this step should reduce the disk size - however, I believe it will only reduce/collapse zero-byte blocks (not just free space - i.e. if you de...
1 comment
Read more

Sun USS 7100 foo

TIP: put ALL of your LUNs into a designated TARGET and INITIATOR group when you create them.  If you leave them in the "default" group, then everything that does an discovery against the array will find them :-( I'm struggling to recognize a reason that a default should even be present on the array. Also - who, exactly, is Sun trying to kid.  The USS is simply a box.. running Solaris .. with IPMP and ZFS.  Great.  If you have ever attempted to "break-in" or "p0wn" your IBM HMC, you know that there are people out there that can harden a box - then.. there's Sun.  After a recent meltdown at the office I had to get quite intimate with my USS 7110 and learned quite a bit.  Namely: there's a shell ;-) My current irritation is how they attempt to "warn you" away from using the shell (my coverage expired a long time ago to worry about that) and then how they try to hide things, poorly. I was curious as to what version of SunOS it ...
1 comment
Read more

Extending SNMP to run arbitrary shell script

Why are we here... This is not likely something I would have pursued under normal circumstances.  I happen to be working for a customer/client who is not afforded a lot of flexibility to accomplish their goals.  In this case, the rigor is justified.  They have to sometimes be fairly creative with how they solve problems. In this case they would like to utilize an existing snmp implementation to execute a command (or shell script) on a remote system.  They came to me with the idea of using Net-SNMP extend. https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Deployment_Guide/sect-System_Monitoring_Tools-Net-SNMP-Extending.html NOTE:  This is NOT a good implementation strategy in the "real world"  it will simply allow you to test the functionality.  There are a TON of security implications which would need to be taken in to consideration. Implementation Steps: [root@rh7tst01 ~]# yum -y install net-snmp net-snmp-utils ...
Post a Comment
Read more