If you are having trouble getting OpenMPI on Linux to work via a Mellanox Infiniband fabric, than this thread may be of interest to you.

Long story short, you may need to set this parameter in your execd environment in order to avoid memory issues:

execd_params H_MEMORYLOCKED=infinity