Tuesday, February 16, 2016

Supermicro IPMI

Hope you have more luck with Supermicro IPMI interfaces than I, they're not old school iLO or DRAC like in reliability for me, and I guess there aren't many developers working on keeping the few models updated.  Not quite Enterprise grade is what I'm saying I guess.  Like so many products if they could just increment it so support current Java it would be less painful.

Remote power management is vital though, as is remote console and media. I really, really don't want to drive (or fly) to a colo just to reboot a purple screen or do an O/S reinstall.

The first of my recent issues was unusual as anything I touch normally is running vSphere ESXi rather than Linux, in this case I was trying to recover IPMI access on a box running Ubuntu on the bare metal.
The card was in some sort of funky state, where is didn't respond on 443, but on 22 I could see the SSH banner (with telnet), but not login with ADMIN ADMIN with SSH.

Some Googling later I downloaded and installed the Supermicro IPMIcfg utility,
ipmicfg_1.23.0_general_20151106.zip, stuck the 64 bit binaries on the affected host, ran:

modprobe ipmi_si
modprobe ipmi_msghandler
modprobe ipmi_devintf

These load the kernel modules to enable the utility to talk to the hardware.  Then you're ready to run useful commands:
ipmicfg-linux.x86_64 -m   # Which lists the IPMI IP and MAC
ipmicfg-linux.x86_64 -user list   # Which gives a list of users and privilege levels
ipmicfg-linux.x86_64 -r   # Which performs a reboot of the card

It allowed me to confirm the IP address and username was correct at least, then do a reset of the card, after which access was restored.

The second issue was with my lab, somehow I've created users that don't work and deleted ADMIN.  I can login to the web interface as me and it shows 'simon / administrator' but all the options say 'You have no permission to view this section.' with the exception of a few read only ones.
I then managed to purple screen (I presume as I couldn't get to the remote KVM) one of my hosts so really wanted to issue a reset.
IPMIcfg is no use as I'm neither running Linux nor have a running O/S, so I had a long play with another tool, SMCIPMITool.  Also downloaded as a static binary only this time it is used against a remote target card.

SMCIPMITool 10.10.10.1 ADMIN "ADMIN" ipmi sensor   # for example reads the sensor status

I found I couldn't do anything useful with the command in this form, but when I tried the shell option,

SMCIPMITool 10.10.10.1 ADMIN "ADMIN" shell

I eventually found that though 'power cycle' 'power reset' etc. returned an error (possibly permissions due to my weird account) 'power off' and 'power on' did work and I got my server back without venturing into the LA traffic.

Upon even further experimentation I also discovered that I can create new user accounts, so one quick,
user add 4 testuser testpassword 4
later, and I have an account which works properly in the GUI**.

Though getting the virtual KVM to work in modern Java on a model O/S was a huge pain - the applet is signed with a key that's both short and MD5, so even after whitelisting the IPs of my IPMIs I have to edit java.security to reenable MD5 certs and permit 256 bit certs.

No comments:

Post a Comment