Silence the Dell PowerEdge 2950

Update 2015-08-15: This article describes how to add resistors to the fans to reduce the fans’ speed. It’s a great alternative especially if you have difficulties finding compatible fans.

We got a couple of Dell PE 2950 for a test environment, they work great and are really cheap in China. The only downside is the Power Consumption which can be a problem for long term use.

Well, I was satisfied of my test environment except that my desk was located near the server bay and that the noise was unbearable for a prolongation period of time. A workmate measured the noise of the Dell PowerEdge 2950 to be around 70 dB. As loud as a busy street traffic!

Brent Ozar’s article on silencing a Dell PE 1900 motivated us to replace the fans.

At the time I did it, not so many articles were available on the web about silencing the 2950.

There are risks in modifying your vendor products.

Flashing your firmware could make your hardware unusable

I hold no responsibility for any damage you could do to your equipment!

Step1: Choose new fans

SUNON PMD1206PMB3-A for Dell PE 2950

SUNON PMD1206PMB3-A used to silence a Dell PE 2950. That’s the amount of dust you get in 1 year in Beijing without filters to your server room!

After a little research, our choice went for this fan: SUNON PMD1206PMB3-A

Specifications:

Size60*60*38mm
Max rotation speed6000 RPM
Max air flow34.4 CFM
Noise at max speed47 dB
Power consumption3.4W

In comparison, the original Delta fans, Dell P/N YW880:

Size60*60*38mm
Max rotation speed12,000 RPM
Max air flow67.8 CFM
Noise at max speed61.5 dB
Power consumption?

It’s very important that the new fans have 4 wires (PWM feature). If the BMC is not able to able to read the speed and modulate the power, it will report errors.

If the specs may appear well bellow the original fans but keep in mind that most of the time fans are not blowing full speed!
In case you didn’t open you Dell 2950 yet, there is 4 fans to replace!

Step2: Replacing the fans

Replace PowerEdge 2950

The plug needs to be changed and the wires remapped.

The plug for of the original Dell fan is specific to Dell so we need to cut off the plug of both new and old fans and solder the Dell plug onto the new fans.
The mapping of the wires goes like this:

Original colorSUNON color code
RedRed
BlackBlack
YellowYellow
BlueGreen

 You really don’t want to take any chance to have a shortcut in your server.

Solder the wires and have heat shrink tube to cover it. Try to move the heat shrink tube to make sure it’s not going anywhere. Be sure not do have sharp edges to your solder

Note that obviously, you also need to move the orange fan casing from the original to the new fans.

Step3: Replacing the BMC firmware

The BMC has speed warning and failure thresholds. Replacing our fans by slower silent fans means that the thresholds will be triggered frequently. Generating alerts and blowing the fans full speed.

The solution is to modify the BMC to lower the threshold for it now to generate alerts.

Apparently not all 2950 need to have their BMC firmware replaced

I silenced 3 PE 2590:

  • 2x PE 2950 II
  • 1x PE 2950 III

Both the version II had to have their firmware replaced but not the version III.
You may want to try your new fans and see if the fan speed alerts are triggered in the BMC.

The process to change the firmware is detailed in this blog article.

In case that blog goes down, the process goes likes this:

  1. Boot from Ubuntu live (CD/DVD or USB stick)
  2. Install IPMI and required tools
    sudo apt-get install freeipmi freeipmi-tools openipmi ipmitools python wget rpm
  3. Run ipmi-sensors to get the system information and record them.
  4. Download the latest BMC firmware to the server:
    wget http://downloads.dell.com/esm/BMC_FRMW_LX_R202152.BIN
  5. Fix and extract .BIN package
    sudo sed -i 's/#!\/bin\/sh/#!\/bin\/bash/' BMC_FRMW_LX_R202152.BIN
    sudo chmod 755 BMC_FRMW_LX_R202152.BIN
    sudo mkdir bmc_firmware
    sudo ./BMC_FRMW_LX_R202152.BIN --extract bmc_firmware
    cd bmc_firmware
  6. Patch the firmware:
    sudo wget http://projects.nuschkys.net/downloads/dell-adjust-fan-thresholds.py
    sudo chmod 755 dell-adjust-fan-threscholds.py
    sudo ./dell-adjust-fan-thresholds.py payload/bmcflsh.dat
  7. Flash the firmware:
    sudo LD_LIBRARY_PATH=./hapi/opt/dell/dup/lib:$LD_LIBRARY_PATH ./bmcfl32l -i=payload/bmcflsh.dat -f

    If you get this error: FRMW_LX_R204810.BIN: 52: typeset: not found.
    Run:

    sudo LD_LIBRARY_PATH=./hapi/opt/dell/dup/lib:$LD_LIBRARY_PATH ./bmcfl32l -i=payload/bmcflsh.dat -f

 

You now should have a silent server!

Step4: Make sure everything is running smoothly

Check your machine status using your DRAC or OpenManage:

temp-probe

fan-probe

If you don’t have alerts configured… Well, you should!

Modifying the cooling should be an extra reason for you to configure SMTP, SMS or SNMP with your DRAC or OpenManage. Seriously, do it now.

Sources:

Thanks to them for their great blogging!

53 Comments

  1. tim July 25, 2014 7:50 am  Reply

    thinking about taking your path to silencing my 2950s, any issues after the fact?

    • Martin DENIZET August 8, 2014 6:33 pm  Reply

      Hi Tim,
      No issues, in my company we did it on 3 2950, 2 version II and 1 version III.
      The version II were hot but not enough to trigger any alarm even during the summer in Beijing without AC.
      No regrets at all, and I wish you the same!
      Cheers!

  2. Nikolay Konstantinov November 8, 2014 12:04 am  Reply

    What if instead replace fans we put a resistors ?

    • Martin DENIZET November 8, 2014 12:29 am  Reply

      Hi Nikolay,
      That’s a very valid question! I have to admit I didn’t even give it a thought.
      I don’t know if there is enough space to but a resistor big enough and heat from the resistor could be a problem.
      It worth considering that you may still have to patch the BMC to prevent low RPM alerts.

      If you try it, please let me know how it went 🙂

      Cheers,

    • Iago Schlup November 8, 2014 3:31 am  Reply

      I did exactly that. I found a blog explaining step-by-step (noob friendly) how to put resistors in each fan, and after that patching the BMC. Very easy. Respecting Martin’s post, if he authorize, I can send here the address where I found the other solution. Regards.

  3. Fred J. March 3, 2015 12:54 pm  Reply

    Hello, im working on getting my Dell PE2950 iii fans changed out and im kinda new the whole using Ubuntu thing and such. When i get into Ubuntu and the terminal open, i run the command to install IPMI, but it comes back and tells me that Unable to locate package freeipmi and unable to locate package impitool. Do i need my server connected to the internet for this? Or what? I used Ubuntu 13.04 Desktop.

    • Martin DENIZET October 27, 2015 3:17 am  Reply

      Yes, that’s probably your problem. If you cannot get internet access, you could have more luck using a DVD version (which includes more packages than the CD version).

  4. Tim C August 10, 2015 3:42 am  Reply

    Hi Martin.

    I followed these instructions, thankyou very much, BUT, the fans have not slowed down.

    I set the fans to a speed of 15 (1125 rpm) and flashed the bios, no change. I went back and changed it to 10, 20, and finally back to default of 27.

    The fans themselves show a lower threshold of 2025 rpm (or whatever value I set them to), but all 4run at a constant 13025rpm! Yes, 13 thousand!

    Any ideas on what may have possibly gone wrong?

    • Martin DENIZET August 12, 2015 8:41 pm  Reply

      Hello Tim,

      Could it be that one of the temperature probes reached its warning value?
      Out of curiousity, which fans did you use?

      • Tim C August 17, 2015 1:04 am  Reply

        Hi Martin

        Thanks for responding so quickly.
        They are the original (stock) fans that came with the server, I believe they’re Sunon branded.

        I dont know where the temperature probe is, but when I run a sensor reader, it all comes back as a reasonable ambent 24 centigrade.

        This server is always run at home and on both power supplied. The room never gets about about 28 degrees.

        I’ve also run it outdoors in a tent for a cyber cafe and under no load at all and an air temperature of around 12 degrees at night, the fans were still running full blast (again, with both PSU plugged in).

        The only other thing I canthink of is that the RAID battery was dying and has now died completely, would that affect it do you think?

  5. Morgan \'ARR!\' Allen August 15, 2015 11:09 am  Reply

    The alternative command you give if receiving the `typeset not found` error is the same as the previous.

    • Morgan \\\'ARR!\\\' Allen August 15, 2015 11:11 am  Reply

      Oh also, the solution is to edit ./hapi/disneyinstall.sh and change `sh` to `bash` on the first line.

      but now this is where I\’m stuck.

      `
      Baseboard Management Controller Flash Utility Version: 1.33
      Copyright 2003-2007 Dell Inc. All Rights Reserved.
      Failed to Load Library
      `

      • Martin DENIZET August 15, 2015 4:41 pm  Reply

        Hello Morgan,

        I suggest you make sure the libraries paths are correct.
        These paths are relative paths so you need to be in the correct directory to run the command (Probably $HOME/bmc_firmware).

        Cheers

        • Erik October 3, 2018 8:07 am  Reply

          Hey,

          I\’m in the bmc_firmware directory but it also gives me the failed to load library error. What should I do?

  6. BarrelRoll September 17, 2015 10:54 pm  Reply

    I ran strace on the load and it appears to be looking for a Dell library that doesn\’t exist:

    open(\”/lib/i386-linux-gnu/libdchbas32.so\”, O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
    open(\”/usr/lib/i386-linux-gnu/libdchbas32.so\”, O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
    open(\”/lib/libdchbas32.so\”, O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
    open(\”/usr/lib/libdchbas32.so\”, O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

  7. Josh B October 8, 2016 3:00 pm  Reply

    Hello Martin
    I\’ve followed all the steps and everything went smoothly untill I go to flash it. When I flash it I get this

    Sudo: unable to execute ./bmcfl32l: no such file or directory.

    I know this page is a little old but any help would be greatly appreciated.

    • Martin DENIZET October 8, 2016 4:39 pm  Reply

      Hello Josh,
      When you run ./BMC_FRMW_LX_R202152.BIN --extract bmc_firmware it should have created a directory bmc_firmware.
      The file bmcfl32l should be there.
      Maybe you forgot to cd to this directory to run the flashing command?
      Eventually run ls -lah in the bmcfl32l and post the output here.
      Cheers,

      • Josh B October 8, 2016 5:09 pm  Reply

        Many thanks. I’ll try when I get home.

      • Josh B October 9, 2016 3:58 pm  Reply

        My firmware directory is called bmc. I cd bmc then tried to flash it. Here is a photo of ls -lah

        http://imgbox.com/gbTxHH6R

  8. Josh B October 9, 2016 4:32 pm  Reply

    I tried again following your eveyday step but end up with the same result.
    http://imgbox.com/3jTYETOL

    Thankyou for your help

    • Martin DENIZET October 13, 2016 12:00 am  Reply

      Is it me or the last 2 characters of the line you typed are if instead of -f?
      Maybe you could try to run just ./bmcfl32l --help, that should run, incorrectly but it should run.
      You could also try to use the full path of the script such as /root/bmc_firmware/bmcfl32l.
      Let me know how it goes.
      Cheers,

  9. Josh B October 12, 2016 11:46 am  Reply

    How are you?

    • Martin DENIZET October 13, 2016 6:54 pm  Reply

      I meant to use the absolute path of the file.
      On linux, an absolute path starts with a / (not a dot).
      For comparison, on Windows an absolute path is something like C:\folder\file.ext
      Run pwd to get the absolute path of your current directory and append your file name.
      Using an absolute path should bypass an eventual loading path issue.
      To make sure you got your absolute path right, you can test it beforehand doing something like:
      ls -lah /absolute/path/you/found
      If it displays the details of your file, you got the absolute path right and you can test it in the command.
      Make sure not to preppend a dot.
      Cheers!

  10. Josh B October 14, 2016 7:39 am  Reply

    Here are my results. http://imgbox.com/bRGB0Mbs
    Still no luck. Should I use a different version of CentOS or different version of the bmc firmware?

  11. Josh B October 20, 2016 6:43 pm  Reply

    Any ideas?

  12. Josh B October 23, 2016 7:02 pm  Reply

    I’m thinking about getting another server. Maybe a different motherboard will do the trick.

    • Martin DENIZET October 23, 2016 9:41 pm  Reply

      Excuse me for not replying earlier. I’m quite clueless as of what could go wrong.
      Are you using CentOS live?
      You could eventually try using Ubuntu Live like on the tutorial?
      Maybe this could have something to do with the temporary filesystem the live distro is using?
      I don’t think changing hardware would help because it’s an OS-level error (path not found).

  13. Josh B October 26, 2016 7:54 pm  Reply

    Hello again.
    I’ve successfully flashed the bmc firmware so I don’t get anymore fan rpm errors but I can hear the fans surging up and down in speed. Is this normal? I thought they were meant to maintain a constant speed.
    Many thanks

    • Juan October 16, 2017 3:44 am  Reply

      How did you manage to do it? I am having the same problem as you did.

  14. Jordan H. January 7, 2017 4:14 am  Reply

    Josh B,

    I am stuck at the bmcfl32l does not exist error as well. What did you do?

  15. maurice January 29, 2017 2:41 pm  Reply

    I investigated driving the pwm pins from an arduino with uln2003 driver it worked fine and gives the micro almost full control. What i have learned is very interesting the minimum fan speed that can be set is 3000rpm ie cutting the blue wire and connecting the blue wire from the fan to the black wire sets the fan speed at 3000rpm. Which is above the minimum required by the cpu. still alittle noisy but functional with no parts required.

  16. hayden patterson February 28, 2017 2:18 pm  Reply

    Hey so i created the .dat file and edited it to 1150 rpm and no change. the fans are constantly revving up and down and i get error 1313. i couldnt flash the dat file through linux since i kept getting errors although when i \”update\” the firmware it says im going back to 2.37. any help would be awesome

  17. Juan October 15, 2017 6:27 pm  Reply

    Hi. Thank you for this article.

    When I extract bmc_frmw_lx_r295871.bin the file in the payload directory is firmimg.d6. When I run the py script I get Error: invalid/unexpected file header found!

    Any advice? Thank you in advance.

  18. Brian November 1, 2017 5:03 pm  Reply

    Im realy having a hard time with this. I get to the flashing part and get unable to execute ./bmcfl32l: No such file or directory

    Thanks for the help

  19. Ras August 29, 2018 4:24 am  Reply

    Hi,

    I get the following error at the last step:

    root@2950:/home/rsts/bmc_firmware# sudo LD_LIBRARY_PATH=./hapi/opt/dell/dup/lib:$LD_LIBRARY_PATH ./bmcfl32l -i=payload/bmcflsh.dat -f

    Baseboard Management Controller Flash Utility Version: 1.33
    Copyright 2003-2007 Dell Inc. All Rights Reserved.
    Failed to Load Library
    root@2950:/home/rsts/bmc_firmware#

    I\’m fairly new to Linux so hopefully an easy fix for someone who knows what they\’re doing… I hope! 😀

    Thanks in advance for any info!!

  20. Douswan October 18, 2018 1:42 am  Reply

    Hey there,

    I know this is old so it will be a total shot in the dark but if someone could give me a hand that would be appreciated. I\’ve followed all the steps and everything works great right up until the moment I go to flash. Once I flash I get the following error.

    \”Baseboard Management Controller Flash Utility Version: 1.33
    Copyright 2003-2007 Dell Inc. All Rights Reserved.
    Failed to Load Library\”

    I\’ve also gone ahead and attached a screenshot.
    https://puu.sh/BMm0C/9667a94952.png

    If someone out there could advise that would be so greatly appreciated.

    Thanks,

    Douswan

    • Martin DENIZET October 18, 2018 3:34 am  Reply

      Hello Douswan,
      Which OS did you use to do that?
      I suspect the tool doesn’t work when using a recent OS with up to date libs.
      Cheers,

      • Douswan October 18, 2018 10:09 am  Reply

        Hey Martin,

        Holy crap you responded I’m so happy to hear from you if you help me figure this out I will finally get a days rest after 3 days of trying to figure this out hahaha. I’ve tried in Ubuntu 15.04, I’ve tried in Ubuntu 18.04 both in 64bit, also the latest OS I’ve tried was 16.04.5 32bit. I will try using the 15.04 again now in 32bit while I wait a response.

        Thanks,

        Douswan

        • Douswan October 18, 2018 11:53 am  Reply

          Hey Martin,

          I tried and failed with the same results. I’ve gathered more information and took a screenshot listed below… Hopefully this can help you help me…. If you require more information please let me know I really appreciate it.

          https://puu.sh/BMSh4/d56784d778.png

          Thanks,

          Douswan

          • Martin DENIZET October 18, 2018 1:05 pm 

            I will need to look into more details. I will do that before next week. Cheers!

          • Douswan October 26, 2018 1:42 am 

            Hey Martin,

            I’m just wondering if you managed to get some time to look into this issue or if you managed to find a fix as I haven’t heard from you in a while. I’ve tried all of these OS’s and I still get to the same spot and keep getting the exact same problem. If you could let me know or I can provide any more information that would be useful it would be so greatly appreciated. If you need to contact me via email please feel free.

            https://puu.sh/BQR5E/38aafc9019.png

            Thanks,

            Douswan

          • Martin DENIZET October 26, 2018 5:47 am 

            Hi Douswan,
            I will look into it over the weekend.
            Cheers

          • Martin DENIZET October 28, 2018 4:25 am 

            So I did strace to understand which library is not found.
            It seems it’s not working because it cannot find libdchbas32.so
            That lib is found in a package called srvadmin-hapi.
            I could only find the 64 bits version (libdchbas.so.9.1.0) of the lib in http://linux.dell.com/repo/community/openmanage/910/stretch/pool/main/s/srvadmin-hapi/srvadmin-hapi_9.1.0-1_amd64.deb
            I found a 32 bits version of the lib there: http://linux.dell.com/repo/hardware/Linux_Repository_14.12.00/platform_independent/rh50/srvadmin-i386/srvadmin-hapi-7.4.0-4.28.2.el5.i386.rpm but it is for redhat… I will try again later, need more googling.
            Cheers!

          • Martin DENIZET October 28, 2018 6:19 am 

            Seems I got the right libs (tho they needed to be renamed) from https://linux.dell.com/repo/community/deb/OMSA_6.4/i386/srvadmin-hapi_6.4.0-1_i386.deb (don’t remember if it’s the exact package).
            There is still a lib the flasher tries to open but cannot fing: libdcdiom.so
            I could not find it yet. But it seems to be related or part of the Dell ESM.
            I’ve no idea how comes the patch was working without adding these libs. I’m probably missing something.

          • Martin DENIZET October 28, 2018 3:59 pm 

            I’m not running my tests on an actual Dell PowerEdge server (I just have a C6100 on hands and I suspect they don’t work the same due shared cooling). Maybe the libs needed could be included in the BMC’s firmware?
            If you have access to a PowerEdge, you might try a strace to see what’s going on there

          • Martin DENIZET October 28, 2018 3:55 pm 

            I tried, doesn’t seem like it’s going to do it because it is 64 bits and the flash utilities looks for 32 bits libs.

          • Douswan October 29, 2018 12:33 pm 

            Hey Martin,

            Actually the OMSA71 worked like a charm. I really appreciate it I can finally get some rest after trying to get this working for over two weeks straight and trying multiple times with at least over 8 different OS’s. I knew I was taking a shot in the dark responding to an old thread but you’re a wizard my friend and you really pulled through for me. Again, thanks for all your time and hard work much appreciated.

            Thanks,
            Douswan

Leave a Reply