Infiniband – HP ConnectX Card Firmware upgrade from ESXI

Today I spent a couple hours trying to figure out why my 3rd host won’t connect to the Cisco SFS700D 20GB infiniband switch. vCenter was reporting the connection as down:

ConnectXCard_DOWN
I am using the HP Connect X cards (Part Number:448397-B21) in all my host machines. My other two hosts connected without any issues. So this seems to be specific to this card. Possibly a firmware issue? In order to check firmware versions I needed to install Mellanox Firmware Tool (MFT) on the Host(s) found here.

Installing MFT on ESXI Host and check Firmware versions:

  1. Uploaded both vib files to ESXI01 and ESXI03 and put them in the tmp directory using WinSCP
  2. Installed both vibs by running (will require a reboot):
  3. Checked firmware version of HP ConnectX card in ESXI03:

    Firmware_Version
  4. Compared to ESXI01’s HP ConnectX card:
    ESXI01_Firmware_Version

Updating Firmware Version with MFT:

  1. cd to the following directory before running the below commands:
  2. Then needed to find out the PSID by running:

    HP_ConnectX_PSID
    Downloaded the HP OEM firmware 2.6 from here.
  3. Copied firmware to /tmp/fw-25408-2_6_000-448397-B21.bin on ESXI03
  4. Before applying the firmware needed the PCI Device Name (/dev/mt25418_pci_cr0). Running the ./mlxfwmanager –query command provided this last bit of information. 
  5. Apply the firmware: (./flint -d [device name] -i [firmware location] burn)

    ESXI03_Firmware_Update
  6. Reboot and VOILA!!! Confirmed firmware applied by running the ./mlxfwmanager –query and jump into vCenter to see 20000 Full.
    Voila

Hope this helps other Infiniband users!

Leave a reply:

Your email address will not be published.

Site Footer

Sliding Sidebar

Subscribe to Blog via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.