Update 2015-08-15: This article describes how to add resistors to the fans to reduce the fans’ speed. It’s a great alternative especially if you have difficulties finding compatible fans.
We got a couple of Dell PE 2950 for a test environment, they work great and are really cheap in China. The only downside is the Power Consumption which can be a problem for long term use.
Well, I was satisfied of my test environment except that my desk was located near the server bay and that the noise was unbearable for a prolongation period of time. A workmate measured the noise of the Dell PowerEdge 2950 to be around 70 dB. As loud as a busy street traffic!
Brent Ozar’s article on silencing a Dell PE 1900 motivated us to replace the fans.
At the time I did it, not so many articles were available on the web about silencing the 2950.
Flashing your firmware could make your hardware unusable
I hold no responsibility for any damage you could do to your equipment!
Step1: Choose new fans
SUNON PMD1206PMB3-A used to silence a Dell PE 2950. That’s the amount of dust you get in 1 year in Beijing without filters to your server room!
After a little research, our choice went for this fan: SUNON PMD1206PMB3-A
Specifications:
| Size | 60*60*38mm |
| Max rotation speed | 6000 RPM |
| Max air flow | 34.4 CFM |
| Noise at max speed | 47 dB |
| Power consumption | 3.4W |
In comparison, the original Delta fans, Dell P/N YW880:
| Size | 60*60*38mm |
| Max rotation speed | 12,000 RPM |
| Max air flow | 67.8 CFM |
| Noise at max speed | 61.5 dB |
| Power consumption | ? |
It’s very important that the new fans have 4 wires (PWM feature). If the BMC is not able to able to read the speed and modulate the power, it will report errors.
If the specs may appear well bellow the original fans but keep in mind that most of the time fans are not blowing full speed!
In case you didn’t open you Dell 2950 yet, there is 4 fans to replace!
Step2: Replacing the fans
The plug needs to be changed and the wires remapped.
The plug for of the original Dell fan is specific to Dell so we need to cut off the plug of both new and old fans and solder the Dell plug onto the new fans.
The mapping of the wires goes like this:
| Original color | SUNON color code |
|---|---|
| Red | Red |
| Black | Black |
| Yellow | Yellow |
| Blue | Green |
You really don’t want to take any chance to have a shortcut in your server.
Solder the wires and have heat shrink tube to cover it. Try to move the heat shrink tube to make sure it’s not going anywhere. Be sure not do have sharp edges to your solder
Note that obviously, you also need to move the orange fan casing from the original to the new fans.
Step3: Replacing the BMC firmware
The BMC has speed warning and failure thresholds. Replacing our fans by slower silent fans means that the thresholds will be triggered frequently. Generating alerts and blowing the fans full speed.
The solution is to modify the BMC to lower the threshold for it now to generate alerts.
Apparently not all 2950 need to have their BMC firmware replaced
I silenced 3 PE 2590:
- 2x PE 2950 II
- 1x PE 2950 III
Both the version II had to have their firmware replaced but not the version III.
You may want to try your new fans and see if the fan speed alerts are triggered in the BMC.
The process to change the firmware is detailed in this blog article.
In case that blog goes down, the process goes likes this:
- Boot from Ubuntu live (CD/DVD or USB stick)
- Install IPMI and required tools
sudo apt-get install freeipmi freeipmi-tools openipmi ipmitools python wget rpm - Run ipmi-sensors to get the system information and record them.
- Download the latest BMC firmware to the server:
wget http://downloads.dell.com/esm/BMC_FRMW_LX_R202152.BIN - Fix and extract .BIN package
sudo sed -i 's/#!\/bin\/sh/#!\/bin\/bash/' BMC_FRMW_LX_R202152.BIN sudo chmod 755 BMC_FRMW_LX_R202152.BIN sudo mkdir bmc_firmware sudo ./BMC_FRMW_LX_R202152.BIN --extract bmc_firmware cd bmc_firmware - Patch the firmware:
sudo wget http://projects.nuschkys.net/downloads/dell-adjust-fan-thresholds.py sudo chmod 755 dell-adjust-fan-threscholds.py sudo ./dell-adjust-fan-thresholds.py payload/bmcflsh.dat - Flash the firmware:
sudo LD_LIBRARY_PATH=./hapi/opt/dell/dup/lib:$LD_LIBRARY_PATH ./bmcfl32l -i=payload/bmcflsh.dat -fIf you get this error: FRMW_LX_R204810.BIN: 52: typeset: not found.
Run:sudo LD_LIBRARY_PATH=./hapi/opt/dell/dup/lib:$LD_LIBRARY_PATH ./bmcfl32l -i=payload/bmcflsh.dat -f
You now should have a silent server!
Step4: Make sure everything is running smoothly
Check your machine status using your DRAC or OpenManage:
If you don’t have alerts configured… Well, you should!
Modifying the cooling should be an extra reason for you to configure SMTP, SMS or SNMP with your DRAC or OpenManage. Seriously, do it now.
Sources:
Thanks to them for their great blogging!
thinking about taking your path to silencing my 2950s, any issues after the fact?
Hi Tim,
No issues, in my company we did it on 3 2950, 2 version II and 1 version III.
The version II were hot but not enough to trigger any alarm even during the summer in Beijing without AC.
No regrets at all, and I wish you the same!
Cheers!
What if instead replace fans we put a resistors ?
Hi Nikolay,
That’s a very valid question! I have to admit I didn’t even give it a thought.
I don’t know if there is enough space to but a resistor big enough and heat from the resistor could be a problem.
It worth considering that you may still have to patch the BMC to prevent low RPM alerts.
If you try it, please let me know how it went 🙂
Cheers,
I did exactly that. I found a blog explaining step-by-step (noob friendly) how to put resistors in each fan, and after that patching the BMC. Very easy. Respecting Martin’s post, if he authorize, I can send here the address where I found the other solution. Regards.
Hi Lago,
Yes, please go ahead, I’m looking forward reading that 🙂
Thanks!
A tutorial for adding resistors to fans can be found at http://www.ratzblog.com/2014/08/reducing-dell-poweredge-pe-295029002800.html
Hello, im working on getting my Dell PE2950 iii fans changed out and im kinda new the whole using Ubuntu thing and such. When i get into Ubuntu and the terminal open, i run the command to install IPMI, but it comes back and tells me that Unable to locate package freeipmi and unable to locate package impitool. Do i need my server connected to the internet for this? Or what? I used Ubuntu 13.04 Desktop.
Yes, that’s probably your problem. If you cannot get internet access, you could have more luck using a DVD version (which includes more packages than the CD version).
Hi Martin.
I followed these instructions, thankyou very much, BUT, the fans have not slowed down.
I set the fans to a speed of 15 (1125 rpm) and flashed the bios, no change. I went back and changed it to 10, 20, and finally back to default of 27.
The fans themselves show a lower threshold of 2025 rpm (or whatever value I set them to), but all 4run at a constant 13025rpm! Yes, 13 thousand!
Any ideas on what may have possibly gone wrong?
Hello Tim,
Could it be that one of the temperature probes reached its warning value?
Out of curiousity, which fans did you use?
Hi Martin
Thanks for responding so quickly.
They are the original (stock) fans that came with the server, I believe they’re Sunon branded.
I dont know where the temperature probe is, but when I run a sensor reader, it all comes back as a reasonable ambent 24 centigrade.
This server is always run at home and on both power supplied. The room never gets about about 28 degrees.
I’ve also run it outdoors in a tent for a cyber cafe and under no load at all and an air temperature of around 12 degrees at night, the fans were still running full blast (again, with both PSU plugged in).
The only other thing I canthink of is that the RAID battery was dying and has now died completely, would that affect it do you think?
The alternative command you give if receiving the `typeset not found` error is the same as the previous.
Oh also, the solution is to edit ./hapi/disneyinstall.sh and change `sh` to `bash` on the first line.
but now this is where I\’m stuck.
`
Baseboard Management Controller Flash Utility Version: 1.33
Copyright 2003-2007 Dell Inc. All Rights Reserved.
Failed to Load Library
`
Hello Morgan,
I suggest you make sure the libraries paths are correct.
These paths are relative paths so you need to be in the correct directory to run the command (Probably $HOME/bmc_firmware).
Cheers
Hey,
I\’m in the bmc_firmware directory but it also gives me the failed to load library error. What should I do?
I ran strace on the load and it appears to be looking for a Dell library that doesn\’t exist:
open(\”/lib/i386-linux-gnu/libdchbas32.so\”, O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open(\”/usr/lib/i386-linux-gnu/libdchbas32.so\”, O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open(\”/lib/libdchbas32.so\”, O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open(\”/usr/lib/libdchbas32.so\”, O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
Hello Martin
I\’ve followed all the steps and everything went smoothly untill I go to flash it. When I flash it I get this
Sudo: unable to execute ./bmcfl32l: no such file or directory.
I know this page is a little old but any help would be greatly appreciated.
Hello Josh,
When you run
./BMC_FRMW_LX_R202152.BIN --extract bmc_firmwareit should have created a directory bmc_firmware.The file bmcfl32l should be there.
Maybe you forgot to cd to this directory to run the flashing command?
Eventually run
ls -lahin the bmcfl32l and post the output here.Cheers,
Many thanks. I’ll try when I get home.
My firmware directory is called bmc. I cd bmc then tried to flash it. Here is a photo of ls -lah
http://imgbox.com/gbTxHH6R
I tried again following your eveyday step but end up with the same result.
http://imgbox.com/3jTYETOL
Thankyou for your help
Is it me or the last 2 characters of the line you typed are
ifinstead of-f?Maybe you could try to run just
./bmcfl32l --help, that should run, incorrectly but it should run.You could also try to use the full path of the script such as
/root/bmc_firmware/bmcfl32l.Let me know how it goes.
Cheers,
How are you?
Here are my results. Still no luck. http://imgbox.com/Smy5yLZE
I’m sorry this is Turing into such a problem.
I meant to use the absolute path of the file.
On linux, an absolute path starts with a
/(not a dot).For comparison, on Windows an absolute path is something like
C:\folder\file.extRun
pwdto get the absolute path of your current directory and append your file name.Using an absolute path should bypass an eventual loading path issue.
To make sure you got your absolute path right, you can test it beforehand doing something like:
ls -lah /absolute/path/you/foundIf it displays the details of your file, you got the absolute path right and you can test it in the command.
Make sure not to preppend a dot.
Cheers!
Here are my results. http://imgbox.com/bRGB0Mbs
Still no luck. Should I use a different version of CentOS or different version of the bmc firmware?
Any ideas?
I’m thinking about getting another server. Maybe a different motherboard will do the trick.
Excuse me for not replying earlier. I’m quite clueless as of what could go wrong.
Are you using CentOS live?
You could eventually try using Ubuntu Live like on the tutorial?
Maybe this could have something to do with the temporary filesystem the live distro is using?
I don’t think changing hardware would help because it’s an OS-level error (path not found).
Hello again.
I’ve successfully flashed the bmc firmware so I don’t get anymore fan rpm errors but I can hear the fans surging up and down in speed. Is this normal? I thought they were meant to maintain a constant speed.
Many thanks
How did you manage to do it? I am having the same problem as you did.
Josh B,
I am stuck at the bmcfl32l does not exist error as well. What did you do?
I investigated driving the pwm pins from an arduino with uln2003 driver it worked fine and gives the micro almost full control. What i have learned is very interesting the minimum fan speed that can be set is 3000rpm ie cutting the blue wire and connecting the blue wire from the fan to the black wire sets the fan speed at 3000rpm. Which is above the minimum required by the cpu. still alittle noisy but functional with no parts required.
Nice hack!
Thanks for sharing!
Hey so i created the .dat file and edited it to 1150 rpm and no change. the fans are constantly revving up and down and i get error 1313. i couldnt flash the dat file through linux since i kept getting errors although when i \”update\” the firmware it says im going back to 2.37. any help would be awesome
Hello,
I suggest you have a BMC firmware more recent that the one used in this article.
Did you try getting the latest version of the BMC from http://downloads.dell.com/esm/ ?
That should be: http://downloads.dell.com/esm/bmc_frmw_lx_r295871.bin
Note that I didn’t using another firmware that the one mentioned in the article. You might you to double check before trying that.
Cheers!
Hi. Thank you for this article.
When I extract bmc_frmw_lx_r295871.bin the file in the payload directory is firmimg.d6. When I run the py script I get Error: invalid/unexpected file header found!
Any advice? Thank you in advance.
Im realy having a hard time with this. I get to the flashing part and get unable to execute ./bmcfl32l: No such file or directory
Thanks for the help
Hi,
I get the following error at the last step:
root@2950:/home/rsts/bmc_firmware# sudo LD_LIBRARY_PATH=./hapi/opt/dell/dup/lib:$LD_LIBRARY_PATH ./bmcfl32l -i=payload/bmcflsh.dat -f
Baseboard Management Controller Flash Utility Version: 1.33
Copyright 2003-2007 Dell Inc. All Rights Reserved.
Failed to Load Library
root@2950:/home/rsts/bmc_firmware#
I\’m fairly new to Linux so hopefully an easy fix for someone who knows what they\’re doing… I hope! 😀
Thanks in advance for any info!!
Hey there,
I know this is old so it will be a total shot in the dark but if someone could give me a hand that would be appreciated. I\’ve followed all the steps and everything works great right up until the moment I go to flash. Once I flash I get the following error.
\”Baseboard Management Controller Flash Utility Version: 1.33
Copyright 2003-2007 Dell Inc. All Rights Reserved.
Failed to Load Library\”
I\’ve also gone ahead and attached a screenshot.
https://puu.sh/BMm0C/9667a94952.png
If someone out there could advise that would be so greatly appreciated.
Thanks,
Douswan
Hello Douswan,
Which OS did you use to do that?
I suspect the tool doesn’t work when using a recent OS with up to date libs.
Cheers,
Hey Martin,
Holy crap you responded I’m so happy to hear from you if you help me figure this out I will finally get a days rest after 3 days of trying to figure this out hahaha. I’ve tried in Ubuntu 15.04, I’ve tried in Ubuntu 18.04 both in 64bit, also the latest OS I’ve tried was 16.04.5 32bit. I will try using the 15.04 again now in 32bit while I wait a response.
Thanks,
Douswan
Hey Martin,
I tried and failed with the same results. I’ve gathered more information and took a screenshot listed below… Hopefully this can help you help me…. If you require more information please let me know I really appreciate it.
https://puu.sh/BMSh4/d56784d778.png
Thanks,
Douswan
I will need to look into more details. I will do that before next week. Cheers!
Hey Martin,
I’m just wondering if you managed to get some time to look into this issue or if you managed to find a fix as I haven’t heard from you in a while. I’ve tried all of these OS’s and I still get to the same spot and keep getting the exact same problem. If you could let me know or I can provide any more information that would be useful it would be so greatly appreciated. If you need to contact me via email please feel free.
https://puu.sh/BQR5E/38aafc9019.png
Thanks,
Douswan
Hi Douswan,
I will look into it over the weekend.
Cheers
So I did strace to understand which library is not found.
It seems it’s not working because it cannot find libdchbas32.so
That lib is found in a package called srvadmin-hapi.
I could only find the 64 bits version (libdchbas.so.9.1.0) of the lib in http://linux.dell.com/repo/community/openmanage/910/stretch/pool/main/s/srvadmin-hapi/srvadmin-hapi_9.1.0-1_amd64.deb
I found a 32 bits version of the lib there: http://linux.dell.com/repo/hardware/Linux_Repository_14.12.00/platform_independent/rh50/srvadmin-i386/srvadmin-hapi-7.4.0-4.28.2.el5.i386.rpm but it is for redhat… I will try again later, need more googling.
Cheers!
Seems I got the right libs (tho they needed to be renamed) from https://linux.dell.com/repo/community/deb/OMSA_6.4/i386/srvadmin-hapi_6.4.0-1_i386.deb (don’t remember if it’s the exact package).
There is still a lib the flasher tries to open but cannot fing: libdcdiom.so
I could not find it yet. But it seems to be related or part of the Dell ESM.
I’ve no idea how comes the patch was working without adding these libs. I’m probably missing something.
I’m not running my tests on an actual Dell PowerEdge server (I just have a C6100 on hands and I suspect they don’t work the same due shared cooling). Maybe the libs needed could be included in the BMC’s firmware?
If you have access to a PowerEdge, you might try a strace to see what’s going on there
I think there’s something to try: the Dell OMSA live DVD: https://linux.dell.com/files/openmanage-contributions/omsa-71-live/ if you wan to take a last swing at this… 🙂
I tried, doesn’t seem like it’s going to do it because it is 64 bits and the flash utilities looks for 32 bits libs.
Hey Martin,
Actually the OMSA71 worked like a charm. I really appreciate it I can finally get some rest after trying to get this working for over two weeks straight and trying multiple times with at least over 8 different OS’s. I knew I was taking a shot in the dark responding to an old thread but you’re a wizard my friend and you really pulled through for me. Again, thanks for all your time and hard work much appreciated.
Thanks,
Douswan