CPU and chassis fan replacements attempt to reduce Supermicro SYS-5028D-TN4T mini-tower's already low noise levels while increasing M.2 NVMe SSD cooling
I now have measurements of watt burn and dB levels for each of the SuperServer Xeon D form factors, seen near in TinkerTry's exclusive comparison table. The mini-tower is still by far the quietest and most versatile. The faster and cheaper Samsung 960 M.2 NVMe drives run cooler than the 950 PRO, so throttling is even less likely during my normal everyday use. See also various (warranty voiding?) ceramic heat sinks for M.2 NVMe that come bundled with various PCIe to M.2 adapters. I've given up worrying about replacement fans and heat sinks, at least for the two already-well-designed mini-towers that are currently the only Xeon Ds that I own, with even better cooling on the Xeon D-1567 12 core version, available here. Original article below.
I'm excited about how it's going so far, making this a system that can run not just in a home basement or closet, but perhaps right there in a home office.
This post is a work-in-progress, orignally drafted Feb. 22nd, see also Intel's Xeon D-1500 thermal documentation. Parts and process not yet finalized, for details, jump to the latest update below.
I demonstrated my fan adapter prototype with the M.2 cooling duct to a VMUG crowd this past Tuesday, along with a brief FLIR One thermal imaging demonstration. I was showing how the Xeon D-1541 motherboard components stayed cool even with the chassis cover off. Even under heavy load:
The configuration used for that brief demo was VMware ESXi 6.0 U1b running on a USB thumb drive seen in the front of the mini-tower, with a Windows Server 2016 Technical Preview 4 VM on my Samsung 950 PRO M.2 NVMe 512GB SSD. The VM was given 16 vCPUs, and 125GB of RAM. Good old Prime95 was used to abuse all 16 vCPUs, quickly and easily.
I haven't yet worked out my favorite combination that cools the CPU at least as well as the stock Delta fan, while keeping things quiet. It's looking like replacement 4 pin PWM fans may run a bit slow, so the Noctua FLX 3 ping models that run at full speed might be what I settle on, stay tuned.
Why just the CPU fan, what about the chassis fan?
I unplugged the chassis fan while the system was at idle and under load, and the perceived difference in noise emitted from the system was minimal. Can't go messing with the power supply fan, which appears to be pretty quiet anyway. That leaves the CPU fan. Yep, big noticeable volume difference noticed, when trying out a variety of 50mm and 60mm fans on it.
What about 1U?
Small fans mean higher RPMs, so it's unlikely to find a 1U system that's pleasant to live near.
Extra fan header on the new PBC Rev. 2.00 for Xeon D-1541
Note that the X10SDV-TLN4F now ships with the newer Rev. 2.00 PCB for the new Xeon D-1541, and it has an extra fan header, discussed here, and seen at bottom left of the above motherboard photo. Sliding the motherboard out the back of the chassis is rather easy, and takes only about 2-3 minutes, once it's become a familiar process to you. Howto videos on the way!
Why not cool the M.2 occupant just a bit, while I'm at it?
A nice bonus might be successfully keeping the Samsung 950 M.2 NVMe slightly cooler as well, with preliminary tests already showing the 3D printed duct can cause it to easily handle long benchmarks at full speeds, with no throttling, even with the BIOS fan speed settings at default normal settings. While the chassis top fan speed can easily keep the 950 PRO cool no matter how long you abuse it, I'm not interested in that increased noise 24x7. I'm not saying thermal throttling is really an issue for me at all during normal use, given a full cloning of a Windows 10 VM takes only 18 seconds. I'm just saying that it's possible that creative use of a $2 hunk of plastic (that can easily hold up to the mild heat of the CPU heat sink) may also be able settle that question once and for all, even for heavy benchmarking, without resorting to warranty-threatening NVMe sticker removal for heat sinks.
This approach of using an inexpensive 3D printed adapter was inspired partly by years of rave Noctua fan reviews, coupled with the recent Noctua fans featured in the cool Lego encased Xeon D-1540 system I wrote about back on January 19 2016:
- How cool is that Hypercube Lego Server, featuring the powerful Xeon D-1540 on tiny and familiar Supermicro X10SDV-F motherboard
where I also noticed that Noctua doesn't make a straight 50mm fan replacement, only 40mm and 60mm sizes.
I hope to be able to achieve good results with simple inexpensive parts, that can withstand the rigors of travel. I have such a journey coming up actually, to give a live demo in Las Vegas for my first Experts 2 Experts Virtualization Conference.
TinkerTry's Intel Xeon D-1500 approach is motherboard maker agnostic
If this part works out, it could make it possible for such a duct/fan combo to be useful in other vendor's Xeon D-1500 offerings, such as ASRock and Gigabyte. That's why the title of this article doesn't mention Supermicro.
I created my first 3D printout ever by first by modifying a thingiverse.com designs in tinkercad.com. Good name for a site, no? ;-) Special thanks to my older son for getting my exported .STL file printed at Tufts University's new Jumbo’s Maker Studio @ CLIC. My first prototype is what you see pictured in this article. He used a Creator Pro FDM 3D Printer with red ABS filament. I may also try out a local public library, using a Makerbot Replicator Mini, 5th generation. I'm not done with tweaking this prototype yet, as I'm not even sure what fan size I'll settle on. More temperature tests through IPMI and FLIR will be needed, especially since I may need to reposition that Supermicro black plastic internal air baffle a bit, that this CSE-721TQ-250B chassis comes with. Of course I'll be sharing whatever I come up with.
STH's approach to noise reduction
This is all a very different approach than what Patrick Kennedy coincidentally and independently published today, doing his usual great work here:
If you're reading this article and live anywhere near California, you may also be interested in his first STH meetup coming up in Mountain View, CA:
The way I see it, the more people interested, and tinkering with this motherboard, the better! Use whatever solution works for your use-case. Preserving your warranty, and ensuring things keep cool and safe, are your responsibility. Probably good to mention that Intel processors poweroff the system abruptly if your cooling modification efforts go horribly wrong and PCU temperatures are allowed to climb to the insanely high thermal limits. You may want to set your IPMI to alert you to issues via email. Just saying.
I suspect over time we'll see more and more folks getting creative with the Xeon D-1500 series of CPUs, much like we have for the Intel NUC. For many of my readers, it's especially exciting that the Supermicro SuperServer SYS-5028D-TN4T system is the very first home lab system on the VMware Hardware Compatibility Guide!
This beloved platform is going to be around a while. What's not to love about very few dollars and a few minutes spent to obtain a much quieter system that still runs cool, can run benchmarks at full speeds without throttling, and can be left running efficiently in your living space? Noise, heat, and cost were exactly the reasons the smaller Intel NUC and Mac Mini did so well for home lab virtualization enthusiasts. With Intel Xeon D-1500, we can take a huge step up from NUC with 4x the cores, 3x the cache, 4x the memory, and 4x the networking, all integrated nicely into a compact mini-tower with much faster storage, at about thrice the price. Nice!
All contents below will be modified and updated as I progress with my testing, eventually including a parts list.
What kind of improvement?
How does the following set of improvements sound, while the system's NVMe and CPU is heavily stressed?
- Increase CFM - from 13.57 CFM to TBD, something higher CFM
- Reduce noise levels - from TBD dB to TBD dB
- Reduce RPM - from 6,500 RPM to TBD, 3,000 to 4,000 range likely
- Decrease M.2 - from 140F to TBD
Challenges
There are some factors at play here that didn't make doing the research any easier, including:
- 50mm-50mm isn't a very common dimensional size for a CPU fan (with 40mm x 40mm hole spacing)
- there are no aftermarket integrated heat-sink cooling fans for the Xeon D-1500 series FCBGA1667 CPU design, but that's ok, I don't really want to recommend removing the thermal paste anyway, or delidding
- Supermicro doesn't seem to list the 50mm x 50mm CPU fan that's included with the X10SDV-TLN4F motherboard in their System Fan Matrix
- While I can find the listings for this Delta fan at Digi and AVNET, none of them are in the 4 wire PWM (Pulse Width Modulation) configuration that Supermicro uses
- unsure of dB test result measurement techniques and distance, and I don't have an anechoic chamber
My Tentative Test Plan
No problem. I don't need anything fancy. I can just configure my test environment the same way I had it running for my recent dB tests, seen in the video below
Once the environment is configured, I see if I get the same dB readings, at the same 30" distance. Once that's squared away, I'll retest once the fan has been replaced.
That's why it's all the more important to record a baseline, carefully measuring the CPU temperature at idle and under load, and the dB range, at a given distance.
Next, replace the CPU fan with a Noctua fan and have another identical look, measuring the CPU temperature at idle and under load, and the dB range, at a given distance.
I'm also in the midst of quieting down my original SuperServer Workstation design's low profile 65 watt Visiontek 7750 GPU, read about this vSphere 6 Datacenter/Workstation Bundle 1 combo here:
Reference Material
Stock Fan - Digi Part # (3 wire)
Digi-Key Part Number 603-1583-ND
Manufacturer Delta Electronics
digikey.com/product-detail/en/EFB0512HHA-R00/603-1583-ND/2560706
Manufacturer Part Number
EFB0512HHA-R00
Air Flow 13.6 CFM (0.385m³/min)
Static Pressure 0.232 in H2O (57.8 Pa)
Stock Fan - Delta Part # (3 wire) at AVNET
Manufacturer Part #: EFB0512HHA-R00
DELTA FMBG
50X50X10MM 12V DC FAN W/ SPEED SENSOR (LOCK)
Available at AVNET Express
EFB0512HHA-R00 Datasheet (March 05 2010)
Maximums: 6500 RPM / 13.57 CFM / 38.5 dB
data.datasheetlib.com/pdf1/12/69/126962/efb0512hha-r00_3962ffc302.pdf
Noctua NF-A6x25 PWM (4 wire)
Noctua NF-A6x25 PWM
550 RPM to 3000 RPM
noctua.at/en/nf-a6x25-pwm.html
Airflow 29,2 m³/h
Airflow with L.N.A. 22,6 m³/h
Acoustical Noise 19,3 dB(A)
Acoustical Noise with L.N.A. 13,7 dB(A)
The Noctua is 17.2 cfm max, see
convertunits.com/from/cubic+m/hr/to/cfm
I'm investigating using other size Noctua fans as well.
Frozen CPU 50 mm fans
frozencpu.com/cat/l3/g36/c15/s71/list/p1/Fans-12_Volt_Fans-50mm_x_10mm_Fans-Page1.html
Gelid Solutions Silent 5, 50mm Quiet Case Fan
One simple and affordable option I'm also investigating can be seen at quietpc.com/50mmfans:
- Amazon - $12.00 (Apr 12 2016)
- Newegg - $32.80 (Apr 12 2016)
- Coolerguys - $6.95 (Apr 12 2016)
- QuietPC - £5.98 (Apr 12 2016)
Mar 06 2016 Update
It's proving to be difficult to find a way to use a Noctua fan that will also keep the CPU as cool as the stock fan, but I'm getting close. Trying multiple sizes and adapters, that static pressure rating is important. If you're looking for a straight fan swap with just 8 screws to get the job done, the Gelid 50mm seems to work almost as well as the stock fan, with less noise, installed in the picture below.
I have more 3D adapter prototype tests coming up. These "prints" take a while, as do my tax return preparations. It could be a few more days before my next article update. Life, and work, currently intervening.
Mar 07 2016 Update
Keep in mind that ServeTheHome article is a 35 Watt CPU. What I'm finding for the Xeon D-1541 is that the 60 mm Noctua doesn't cool as well as the stock Delta fan, or as well as the Gelid 50mm Quiet Case Fan. That's one of the reasons that my search continues.
Have a look at that page 3-1 of the:
- SC721 CHASSIS SERIES SC721TQ-250B USER’S MANUAL
Fans
The chassis includes a 9cm rear fan with 25db whisper quiet operation, and an
optional 8cm front fan
Not sure which of the many 80mm Supermicro fans this is referring to, in the Supermicro:
Mar 11 2016 Update
More 3D printed prototypes (50mm to 80mm adapter), and some more brands in there. As for striking the right balance between noise reduction and effective cooling, the Gelid 50mm continues to be the champ, handling even the worst abuse Prime95 can dish out, cooling nearly as well as the stock fan.
You'll also see I'm messing around with VT-d/GPUs again, trying out different ways to reduce those tiny fan's noise as well. PNY NVIDIA Quadro K1200 is the newest, most powerful GPU that fits in a low profile PCI slot, with 4 mini DisplayPort 4K outputs. Looking good, with very little sound. VT-d passthru testing under ESXi 6.0U1b, TBD, still working on that...
As for testing and refining my final solution for my home, my testing is very much still underway. CyberTech RPM measuring device seen at top right, and toying with various combinations of adapters and fan settings which aren't always reported by the IPMI for pin configuration reasons. Remember, these aren't exactly OEM fans. PWM's 4 pins for IPMI controlled fan speeds matters less when full fan speed is barely audible.
Mar 24 2016 Update
I'm not sure yet whether I'll have time to get install and testing videos published soon, but I did want to get my findings published as soon as possible, for those of you who have been waiting. These are still preliminary findings, but it's looking likely that these recommendations will be my final ones, even before I complete all my thermal tests. These two fans will likely work out well for your need for increased quiet, even if my finalized solution winds up changing things a bit.
If you were to ever need to send your Supermicro system in for in-warranty repairs, I would suggest replacing these aftermarket fans with the original fans before shipping the unit in. If you are not comfortable with minor PC surgery, I would not recommend you replace any fans. If you are on the fence, I would recommend waiting until I'm able to publish how to videos on these straight forward fan swaps. No thermal paste needed, the CPU heat sink stays put. It's just philips head screwdriver work, and about 15 minutes of your time.
Turns out only one of the many CPU fans I tried kept the CPU very nearly as cool as the factory CPU fan, even when left running Prime95 for an extended duration. It's very important for the fan to keep its RPMs up even when there's backpressure, from either pumping a large area of air into smaller hole (using my 3D printed prototypes), or obstacles nearby (CPU heat vanes).
STEP 1 - Suitable CPU Fan Replacement - designed in UK
Gelid Solutions Silent 5, 50mm Quiet Case Fan
The only fan that handled cooling well while eliminating the stock fan's high pitch fan noise is the Gelid Solutions Silent 5 50mm Quiet Case Fan. It's currently priced somewhere between £4.98 (UK) and $32.80 (US), with Newegg currently offering the lowest prices in the US. Simple 4 screw drop-in replacement, no adapters required.
Specs and product information on all 50mm CPU fans at quietpc.com/50mmfans where you'll find only 2 models listed, with purchase links for the Gelid here:
- Amazon - $12.00 (Apr 12 2016)
- Newegg - $32.80 (Apr 12 2016)
- Coolerguys - $6.95 (Apr 12 2016)
- QuietPC UK - £5.98 (Apr 12 2016)
STEP 2 - Suitable Chassis Fan Replacement - designed in Germany
- SILENT WINGS 2 | 120mm
Keeps that M.2 SSD cooler too!
Specs and product information at bequiet.com/en/casefans/259, currently at Amazon for $34.95, and Newegg for roughly $10.95:
- Amazon - US
- Newegg - US
- Where to Buy reseller locator
If you're really into keeping things quiet and cool, you can also get your Samsung 950 PRO M.2 NVMe SSD temperatures to stay low enough to avoid thermal throttling. Unfortunately, the 3D printed adapters got this done, but at the expense of 10 to 15 degrees Celsius higher CPU temps, which isn't really acceptable. Even with the bypass duct (hole) blocked, all CPU fan reducing adapter scenarios meant a CPU running at noticeably higher temperatures when under load. Static pressure capability matters, and higher RPM/smaller fans did better.
The answer for me was not to just swap out the CPU fan, but to also swap out the chassis fan, using the be quiet! brand.
The exact model I chose to test first is NOT the PWM version. It's the 3 pin model that will keep its RPMs at the rated 1500rpm at all times, moving 50.5 cfm / 85.8 m3/h. Even at this fixed speed, it's still a little quieter than the stock fan it replaced at any speed. This be quiet! German manufacturer is also known for power supplies, and claim a life expectancy of 34 years of continuous operation, because of the Fluid Dynamic Bearing (FDB). This fan manages to improve airflow at all times, and preliminary results seem to indicate it can also prevents M.2 thermal throttling as well, at least for my 4-in-a-row tests of ATTO Disk Benchmark.
I will still be trying the SILENT WINGS 2 | 120mm PWM model as well, to determine if it keeps the system cool enough. It just hasn't arrived yet. It's currently available for $10.95 at Newegg here, and various Amazon sources here.
Product Video
Apr 12 2016 Update
I've discovered that PWM fans from be quiet! tend to hunt when used with this system. By hunt, I mean that they cycle between higher and lower RPMs roughly every 8 seconds, never settling on any one RPM. The 120 version for example would sometimes go as low as 500 rpm, which would signal a warning. So non-PWM 3 wire be quiet! variants seem to be the best choices.
I've also noticed that Patrick's scheme for using a front-mounted Noctua 60mm fan does allow you to run the Samsung 950 M.2 drive at full speeds without throttling, but at the cost of some extra noise.
I'm actually still experimenting with various combinations of fans that give the quietest experience under system stress while still providing excellent cooling and a 4+ year life expectency.
Jun 18 2016 Update
I've now published an unboxing video for the Supermicro SNK-C0057A4L Supermicro Heat Sink with retention back plane, which includes the Delta fan attached to the fan bracket that's attached to the CPU heat sink assembly, with thermal paste pre-applied.
These 4 tiny screws that hold the fan bracket onto the CPU heat sink are a little bit tricky to remove, if you wish to avoid removing the whole motherboard to get to the underside, where the CPU retention back plate lives. I spoke with Supermicro parts, and they unfortunately can't (or won't) sell the screws separately. So you'll want to take extra care to avoid stripping them.
Jul 13 2016
There are yet more options out there. There's a non-PWM fan I'll be trying soon by be quiet! called the
- Shadow Wings SW1 120mm 2200rpm
Fan speed @ 12V (rpm)2200
Air flow @ 12V (cfm, m3/h)76.2 / 129.1
Air pressure @ 12V (mm H2O)2.70
Noise level @ 12V (dB(A))29.7
And there's the 4 pin PWM fan that is likely to cycle RPM speeds like all the rest, that's this Corsair fan called the
- ML120 PRO 120mm Premium Magnetic Levitation Fan
Airflow 12 - 75 CFM
Static Pressure 0.2 - 4.2 mmH20
Sound Level 16 - 37 dBA
Speed 400 - 2400 RPM
Power Draw 0.225 A
Aug 11 2016
This has been a long and winding road, but now that I'm reasonably comfortable the PWM fan speed issue with hunting may be fixable (keep reading), it would seem that aftermarket fans are becoming a more viable choice.
It's complicated though, and with my recently Xeon D-1541 SYS-5028D-TN4T I've stuck with stock cooling, to give me a chance to measure watt burn and noise levels at factory defaults, as I compare the various tiny newcomers to the incumbents at:
The Mini Tower SYS-5028D-TN4T is still far and away the champ as far as noise levels. Replacing the CPU fan without removing the heat sink is a bit trickier surgery than I would have liked, with minimal noise reduction gains. Using the fan speed attenuator from the Noctua kits to quiet the too-high-RPM GPU fan in the Bundle 1 (SuperServer VMware Datacenter/Windows 10 Workstation combo) has been the most successful finding so far. Not much to show, from months of on-again/off-again tinkering.
I'm feeling more and more like the average user is rarely going to encounter throttling conditions in everyday use, thus, the stock fans and the stock IPMI RPM calibrations are going to be hard to beat, and supporting after market solutions is likely to be difficult.
All that aside, these little Xeon D-1500 CPUs run fine at high (65C and up) temps without throttling, so I regret if folks feel this is some sort of flaw or problem. This whole tiny chipset runs in an incredibly small space quite nicely, and the new SYS-E200-8D model will really put this notion to the test!
As I turn my attention to my VMworld 2016 demo, I'll be back-burner-ing these fan swaps for a while, out of necessity to get things done.
I think the technical aspects of this long convoluted story are summed up well in this recent dialogue from right here at TinkerTry, special thanks to Walter Franken and his notion of tweaking IPMI:
Oct 14 2016
- Copper cooling for the Supermicro X10SDV Xeon D motherboards
Oct 10 2016 by Patrick Kennedy at STHWarning
We wanted to start this post off with a warning: this will likely void your warranty if you damage your system. Furthermore, since the Xeon D package is a BGA package affixed to the motherboard, you will risk rendering your expensive motherboard and processor unusable, especially if you fail to tighten properly. This modification will be extremely easy to see for anyone checking motherboards for warranty returns so you will need to accept that as a risk before proceeding.
I'm not a fan of this approach, knowing that the decrease in noise levels for the SYS-5028D-TN4T are likely to be minimal, with the risk outweighing the benefit for most. But if you sit near your server, and you frequently use it under load and want to leave the factory default chassis fan speeds rather than max speeds, then an article like the one above may be of considerable value to you.
I did Google searches for what Patrick has come up with a few months back, but came up mostly empty, and I never had an extra system to work with that I was willing to risk violating the warranty on. Let's be thankful Patrick found some options, for those of you that really want an even quieter system.
My focus was more on what can be done to quiet the new mini 1U systems, such as the SYS-E300-8D and SYS-E200-8D systems. I couldn't find any simple solutions that ran as cool as Supermicro's stock fans, or were easy enough to install with only some light Dremel tool mods of the fan assembly itself. Here's one example that almost fits as-is, but left me with temps at least 10C higher than the 2 1U included fans in the stock servers:
Oct 24 2016 Update
Excellent update on folks efforts to quiet down the related SYS-5028D-TN4T system, over in the STH forums, you'll want to read the whole source article:
- Forums > Hardware > DIY Server and Workstation Builds > Supermicro SYS-E300-8D
RPMS:
Fan2: 4.4K (prev. 6.6K)
Fan3: 2.7K
FanB: 4.6K (prev. 6.8K)ETA: I also managed to squeeze in 5 flash based drives (2 SATADOM, 1 M.2, 1 MSATA, and one SATA)
Nov 06 2016 Update
I now have a video available, for a close view of how my experiment with trying to replace the Supermicro's two FFB0412SHN brushless 40mm fan assembly with a single Dyantron® CF-003 fan:
Here's another video that shows what the surgery is like, should you decide to replace your entire CPU fan/heatsink assembly, at your own risk:
Jun 25 2016 Update
I've been asked about the direction of the CPU fan that comes built in, wish I had been more clear on that in the article above. The stock fan sucks cooler air from above downward, into the heatsink assembly, causing warmer air to blow out both sides of the CPU heatsink. Hope this helps!
See also at TinkerTry
-
First Look at the FLIR ONE Thermal Imaging camera for smartphones and tablets
-
How to locate your triple monitor (up to 2K) PC 20 feet away for less noise and more joy
- Intel Xeon Processor D-1500 Product Family Thermal/Mechanical Specification and Design Guide
See also
Wikipedia - Computer fan control
Wikipedia - Pulse-width modulation
Socket FCBGA 1667 aftermarket cooling??
Huge thx #CTVMUG #USERCON 4 great day, & @RobKlobukowski for https://t.co/LXQGD5ZZ7e! @vBrownBag @vMattK @MyVMUG pic.twitter.com/D8QDGwDJ1s
— Paul Braren (@paulbraren) March 2, 2016
@TinkerTry @paulbraren pic.twitter.com/HywAhlW4od
— Matt Kozloski (@vMattK) March 1, 2016
Disclosure
TinkerTry.com, LLC is an independent site, has no sponsored posts, and all ads are run through 3rd party BuySellAds. All equipment and software is purchased for long-term productive use, and any rare exceptions are noted.
TinkerTry's relationship with Wiredzone is similar to the Amazon Associates program, where a very modest commission is earned from each referral sale from TinkerTry's SuperServer order page. I chose this trusted authorized reseller for its low cost and customer service, and a mutual desire to help folks worldwide, including a new way to reduce EU shipping costs. Why? Such commissions help reduce TinkerTry's reliance on advertisers, while building a community around the Xeon D-1500 chipset that strikes a great balance between efficiency and capability.
I personally traveled to Wiredzone near Miami FL to see the assembly room first-hand, and to Supermicro HQ in San Jose CA to share ideas and give direct product feedback.
I'm a full time IT Pro for the past 23 years. I've worked with IBM, HP, Dell, and Lenovo servers for hands-on implementation work across the US. Working from home lately, I'm quite enjoying finally owning a lower-cost Supermicro solution that I can recommend to IT Pro colleagues, knowing it will "just work." That's right, no tinkering required.
All Comments on This Article (102)
Sorry, I don' tknow. Scrolling to the bottom of
https://www.supermicro.com/products/system/midtower/5028/SYS-5028D-TN4T.cfm
to open the Parts List section up, and I notice that unfortunately those screws aren't listed separately.
6 years later and I'm doing the exact same thing with a 3d printed adapter for a noctua fan.
Do you have any idea what the screws are that attach the stock fan to the stock heat sink? I want some shorter ones.
Thanks!
As far as I know, nobody I know has found a way to be a bit more quiet while staying just as cool. That said, the Xeon D itself can handle very high temps, all the way into the high 90s C, but I'm no as sure about the rest of the chips on the motherboard.
Hi Paul and everyone,
Has anyone successfully fixed the noise problem with the e200-8d? I've seen may posts and some people talking about the noctua fans. I tried two of them and the temp was just excessively high so I had to go back to the stock fans. Thank you!
Regards,
Luis M.
I made my E200-8D completely silent...But I think I lost warranty :-) :-) :-)
http://segesta.be/supermicro-e200-8d-frankenstein-edition/
Didn't fit as-is, and when I made it fit, it didn't cool nearly well enough, I'm sorry to report. So sorry for my delayed response, I've been away. I sure wish there were quieter options that were easy to install! Darn physics, 40mm is such a small area to move air through...
Hi, unfortunately video about Dyantron CF-003 installation is not avaliable for some reason.
Probably you can share results? What was the reason for failure? Didn't it feet SYS-200-8D chasis?
Wow, great article, so thorough, really appreciate your sharing the link here wazoo42!
The short version is the single fan option reduces temperature between 10 and 15 °C. The push-pull setup is even better and reduces the temperature differential by 15 to 25 °C. When it comes to fans, higher static pressure is definitely better than higher volumetric airflow.
The long version may be found here:
http://chemlinux.blogspot.com/2017/09/cooling-x10sdv-xeon-d-15401541.html
The stock HSF runs about 66 C at full load. These heatsinks with the duct (one has an 80 mm Delta and one has a 92 mm Vantec) range from 48 C to 52 C, though the vantec is not currently running at full load.
When it comes to noise, the Delta is not quieter but the Vantec is definitely quieter. I will post more as I try out different options (I also want to make a push-pull duct).
This is great, so glad you shared the pics here! So, how did it go, does cooling exceed the original, at lower dB?
Taking a page from your work Paul, here is my first stab at a fan duct for the above heatsinks. I made versions for 70 mm, 80 mm, and 92 mm fans.
https://uploads.disquscdn.com/images/4bceaf9a52f9d55ab99a37d71d96e20b7b8876fb58d53cc79e5ce6953fb716c2.jpg
https://uploads.disquscdn.com/images/676dd9eb03a9cc4b9d7d0782a40e28e4a471144f7dc6339166ebc8ad197ed19f.jpg
More pics inside the chassis here:
https://tinkertry.com/supermicro-superserver-sys-5028d-tn4t-xeon-d-1541-and-xeon-d-1567-have-different-heatsink-different-fan#sys-5028d-tn4t-12-core
and more about 3rd party CPU cooling solutions here for Xeon D:
https://www.servethehome.com/aftermarket-cooling-for-the-supermicro-x10sdv-xeon-d-motherboards/
but I see you're talking about a whole different chipset which I've not used. My focus has been on Xeon D and NVMe and 10GbE, admittedly.
Wow, thanks for your fast reply! That helps a lot.
I'm planning to use a ASRock Rack E3C236D2I in my FreeNAS build, because there is no SuperMicro X11 LGA 1151 Board in the MiniITX form factor. Therefore I'm not bound to the Supermicro proprietary heatsinks of the SOC-boards too.
If I'm interpreting your photo right, I'm able to insert a cooler with a maximum height of about 75mm (height of the removable tray) minus the height of the mainboard spacers (approx. about 70mm)?
Is there something inside the chassis that blocks this 70mm cooler? I'd like to use a vertical/angled fan on the heatsink to fit with the airflow inside.
About 1.75 inches / 45 mm needed for the motherboard assembly to slide out of the back of the chassis. Hope this helps? The photo isn't perfect, since the camera isn't an an infinite distance, so the ruler appears to not be lined up at the bottom properly, but it is.
FYI, the taller CPU fan/heatsink assembly featured in this article:
https://tinkertry.com/supermicro-superserver-sys-5028d-tn4t-xeon-d-1541-and-xeon-d-1567-have-different-heatsink-different-fan
isn't available, unfortunately. This is something I tried hard to get Supermicro to sell standalone, as have other TinkerTry visitors.
https://uploads.disquscdn.com/images/d502ac83db67475b83bd362dbe503b1383ba571314ef30be3b1b0528b7dc7774.jpg
Hey Paul, I really appreciated your article. Thank you!
I'm planning to build my own NAS based on the Supermicro SuperChassis 721TQ-250B too. I've done a lot of research on this case but I cant find an information about the maximum cooler height. Therefore I can't plan the cooler to use in my build.
I'm scared buying a too large one and don't getting the removable tray through the chassis opening.
Could you or somebody else measure the maximum height for me? That would help me a lot!
Sorry, I don't have anybody volunteering their email publicly, so unfortunately, it's still just mailto:support@Supermicro.com for contacting them so they can (hopefully) log your request.
I guess the only other options are some kind of DIY strap/tiedown or possibly 3D printing, or contacting Supermicro to request they offer the SNK-C0092A4L (or at least the plastic fan assembly parts). That would allow someone to purchase whichever passively cooled X10SDV motherboard they want (the majority of the X10SDV's that Supermicro sells are passively cooled), and since the heatsink on them appears to be identical to the one that comes with the SNK-C0092A4L, just having the few plastic pieces from the SNK-C0092A4L would allow an easy upgrade to active cooling. And the cooling would be superior to the 1U active cooling options they offer (and which they only offer on a very limited number of X10SDV motherboards).
I was curious if you have a recommended Supermicro email address to send requests for things like this? Maybe if myself and a few other readers contact them, they'll take note.
While it was happy news they were willing to make a special 12 core SKU for Wiredzone to resell as a mini-tower (because of TinkerTry votes and demand), they didn't make just the motherboard available separately, unfortunately. Those manufacture and distribution decisions were made independently of me. But it all sure beats no Xeon D-1567 in the product line at all.
I've heard back from Wiredzone, and unfortunately, Supermicro won't provide them the parts to resell, stating "this is MB material and not for re-sell."
I'm disappointed too.
Yeah, I've now heard they prefer to use info@wiredzone.com, sorry about that. I too am a bit disappointed that they weren't able to get that fan assembly into the catalog (I asked some weeks ago), will inquire again, just in case. Thanks JS!
Yeah, on that CPU fan assembly page, you mentioned:
"Meanwhile, you can see already view some initial test of the thermal characteristics of this new Xeon D-1567 system here." But that twitter link doesn't seem to bring me to any discussion on thermals. Anyway, looking forward to seeing the thermals on the compare table. Thanks.
On the E200-8D, a 3rd fan did improve things:
https://www.youtube.com/watch?v=feEh9VSwJ3U
I just never got around to writing an article discussing my findings, but I did write this about the new CPU fan assembly:
https://tinkertry.com/supermicro-superserver-sys-5028d-tn4t-xeon-d-1541-and-xeon-d-1567-have-different-heatsink-different-fan
As for adding temps to my https://TinkerTry.com/compare table you are absolutely right, I forgot to add that (handwritten notebook data) to my table, I do need to do that. Thanks for the reminder!
Thanks, pretty interesting how the 3rd fan made things worse, it shows that it's not always a "more is better" when it comes to fans.
I couldn't find the page where you showed the thermal comparison between the SNK-C0057A4L fan assembly and the SNK-C0092A4L fan assembly. What was the link to that?
Also, I was thinking it would also be great if you could (if you have the data) add the CPU temps to your Xeon-D comparison table (https://tinkertry.com/intel-nuc-compared-to-xeon-d-supermicro-superservers#intel-nuc6i7kyk-supermicro-xeon-d-spec-comparison-table).
For some more looks at fans inside, see:
https://www.youtube.com/watch?v=jSychSuwvvE
Sure, all of Supermicro's CPU fans featured in this article (Xeon D) blow air down onto the CPU heat sink.
Yeah, that's definitely beyond what I intend to do.
BTW, can you confirm which direction the cpu fan blows?
Not claiming it would fit, just having some fun. Product page here:
http://www.coolermaster.com/cooling/cpu-liquid-cooler/masterliquid-maker-92/
I would encourage you to not get too crazy, as the small 1U SYS-E200-8D and SYS-E300-8D do run hotter than the mini-tower SYS-5028D-TN4T, like 70C I believe under heavy sustained load, but no throttling, so does it really matter if this SoC is designed to handle such temps. Only time will tell, I suppose. I had a test machine loaner up to 96C at one point (briefly), and it still didn't shut itself down yet, and seemed to work fine still.
If you want to get crazy, see also this fun tweet
https://twitter.com/sebastianpeak/status/813419936529924096
The Supermicro SNK-C0092A4L looks like the perfect solution (especially the 4 plastic mounting posts are the key part and how they attach to the heatsink screws). Probably could even figure out a way to adapt it to use the Noctua 60mm NF-A6x25 PWM, and hopefully without any need to remove the stock heatsink. Thanks for pointing me to your article comparing the two SM stock active cooling solutions. So I'll definitely email Wiredzone.com to express my interest (and I'll let them know you referred me to them).
On a different note, one really basic question: which direction are the CPU fans configured to blow? Toward the heatsink/cpu or away from the heatsink/cpu? I'm assuming toward the heatsink/cpu based on the arrow on one of the fans in your photos, but I wanted to double-check.
Glad you found this site, JS! I'd have to say I've been enjoying just using the stock fans in the Bundles, but yeah, you might enjoy this article, if you're wiling to take some risk, and perhaps even see if this bracket will fit over the CPU cooler:
Supermicro SNK-C0092A4L
without removing it from the CPU, pictures and more at:
https://tinkertry.com/supermicro-superserver-sys-5028d-tn4t-xeon-d-1541-and-xeon-d-1567-have-different-heatsink-different-fan
Unfortunately, this Supermicro SNK-C0092A4L doesn't appear to be on Wiredzone.com or elsewhere. I've asked for them to include it a few weeks ago, and their closed for holidays now, but sending an email to wiredzone@hotmail.com expressing your interest wouldn't hurt.
Unfortunately, I no longer have any 1U Xeon D systems (that were on loan from supermicro), so my fan tinkering is largely behind me.
See also:
https://tinkertry.com/supermicro-superserver-sys-e200-8d-and-sys-e300-pictures
Wow, this is terrific information, I really appreciate your sharing it here. Consider also sharing at https://openhomelab.org/index.php?title=Supermicro_Xeon_D-1500 as well, and/or here:
https://forums.servethehome.com/index.php
as the more people that find your solution, the better.
Hi Paul,
Super useful blog. I'm looking at the X10SDV-4C-7TP4H motherboard (https://www.supermicro.com/products/motherboard/Xeon/D/X10SDV-4C-7TP4F.cfm), which is only sold with passive cooling. It looks lke it only comes with >1U (2U?) passive heatsink (the heatsink is definitely taller than the SNK-C0057A4L, as you can see in the photos on this page https://www.bussink.ch/?p=1787). I'm hesitant to remove the heatsink (given risks of damage and voiding warranty), but I'll be using this in a tower chassis, so I'm concerned that it won't have high speed air flowing through it, and probably needs an active fan.
So I was wondering if you have any thoughts on fan attachments to the existing >1U stock heatsink? How effective do you think it would be (perhaps might work better than in your experiments, since it's >1U to start with, albeit not copper)?
Also, how best to attach the fan? I've seen people just tying the fan down, but was wondering if you can think of a more elegant way (perhaps 3d printing something, but what would the anchor point)?
Thanks.
Well, did some experiences using the Supermicro SC721 and using a ITX-Mini Gigabyte GAB150N Phoenix WiFi main board..
1. The SC721 does not provide any cable management, so you have to deal with custom power on/system reset and IDE activate & power on LED's
2. In addition I use my old 6405E Adaptec raid controller for the RAID 1 disk's
3. I got some nightmare on MS 2102 Server and Intel I219-V NIC using GAB150N
Finally hacking the INF file (google on Intel Forum how to)
4. I used the be quiet Silent Wings 3 and connected it to the main board on SYS1
5. Using the low profile Arctic Freezer 11 LP on the I5-6500 was also not possible:
RAM & cap's space issues!
6. Connecting a Slim DVD needs a special Supermicro part only to provide a Floppy power connector and a SATA cable connector
Overall a new nice tinny Server :D
My apologies for not getting my long video of the hardware swap produced first. I have removed the motherboard easily many times, but have yet to share the videos, as those efforts resulted in a configuration where I know I can still do better. Still a work in progress, thanks for your patience, and hopefully this article provided some value.
Ah, so you have a Bundle 1 workstation, correct?
https://TinkerTry.com/superservers#Bundle1
I have a new way to quiet that VisionTek Radeon HD7750 down (NVIDIA K1200 didn't work out), and happy to provide you info on that. Just wanted to be sure of what you have first.
And to add, after just watching the video I do feel somewhat a dummy!, I was thinking it was the screws which hold the fan on need removed, I didn't realise it was the screws on the side of the heatsink!, so it will be fine the screwdrivers will fit.
Apologies for the confusion!
Thanks, perfect...I may just leave it for the time being, in light of the screw situation and after swapping the other fans out and removing the the GPU it is a lot quieter.
Hi Paul,
Below is a direct link to the screwdrivers I purchased, however on closer inspection the screws look very different...
http://www.ebay.co.uk/itm/Moody-Tools-58-0405-4Pc-Extended-Reach-JIS-Screwdriver-Set-/311614494721?hash=item488dabd801:g:ANEAAOSwZetXO3Ds
The screw heads kind of look like this...
http://w3.uwyo.edu/~jimkirk/fasteners/TRx-ExternalTorx-Z.gif
I will post up a picture this weekend of the screw and you will see what I mean.
I have now published a video of the CPU/heat sink assembly:
https://tinkertry.com/superserver-combined-cpu-and-m2-cooling-fan#jun-18-2016-update
This is currently the only way to get those 4 tiny exact screws, in case you stripped one of the ones you have, but $36USD is a bit steep if all you need is the screws, I realize.
Oh my, that is unfortunate, I would like to get to the bottom of this. Which exact screwdrivers did you buy?
It would be bad if Supermicro actually changed the screws that come with the heatsink assembly http://www.wiredzone.com/supermicro-components-cpu-processors-accessories-snk-c0057a4l-10022641?UTM_Source=TT&UTM_Campaign=10022641
and I still haven't found a way to buy just those little screws separately.
Hi Paul,
so the screwdrivers arrived, however they didn't fit....after closer inspection I have discovered they have changed the screws on the fan assembly. Just need to try and find the screwdriver for them now!
Thanks for sharing the picture.
The CPU/M.2 combo I tried (and now emailed to you) didn't work out as well as I would have liked, not cooling the CPU as well as the stock fan, and having marginal effects on the temperature of the Samsung 950 Pro in the M.2 slot.
If I do find a more elegant way, that's quieter than the 60mm front bezel mounted Noctua fan, I'll certainly let everybody know about it.
I guess I should mention that I have started trying to build a m.2 fan shroud, but as I'm very new to 3d printing, I'm not really any good at 3d modeling...
So far it looks like this:
Sorry for the delay, I've been sick unfortunately.
Mine a straight adapter, I didn't want to weaken the airflow going to the cpu.
I'm just using some copper "ram" sized heatsinks I had lying around from another project on the 950 pro.
My Moody Tools 58-0219 http://amzn.to/1UfpDpB just arrived, hope to have a chance to try this weekend...
So sorry I didn't get back to you. No, I never did publish my STL file, as I didn't get satisfactory results.
I did something similar to what you describe, pictured above. I found that hole in the side didn't cool my nearby Samsung 950 PRO enough. Is yours a straight adapter, or do you get creative with NVMe cooling ducting too?
Just in case anyone else finds this page looking for an aftermarket cooling option, better fan options, cpu shrouds, etc...
I found the 3d prints at http://www.thingiverse.com/thing:735038/#files for tons of fan adapters.
I actually took the 50mm to 80mm adapter from this page, made it 50% flatter, and put an 80mm noctua on the 50mm heatsink.
Snap, thats my plan as well once the arrive, though being in the UK they are taken a little longer to arrive, the set I have ordered is slightly different so hopefully they will fit as well :)
Thanks Barry! Just ordered the JIS driver set http://amzn.to/1UfpDpB that jackofrost mentioned here:
https://tinkertry.com/superserver-combined-cpu-and-m2-cooling-fan#comment-2707135485
Nice, thank you jackofrost! I'm so glad you found a source and solution!
Particularly handy that Amazon Prime has it: http://amzn.to/1UfpDpB
Just placed my order with just an hour to spare, for next day Prime delivery. Right on time for my planned weekend tinkering with my new Xeon D-1541 SYS-5028D-TN4T, doing that Gelid fan swap.
Hi Paul,
After doing a little digging around regarding the CPU fan screws, you mentioned that the P00 didn't quite fit right, I think the screws used are JIS Type S 00 screws.
Japanse Industry Standard (JIS)
The are seemingly similar to philips head screws, but have a slightly different pitch to the philips, The info I found whilst searching around was on a RC helicopter site of all places!
I have ordered up a screwdriver, however it make take a few weeks to arrive into the UK, I will update once it arrives and see how it goes.
If it does turn out to be this screwdriver, then at least we will know the screws to get as well for spares just in case one gets stripped!
Hi Guys,
Thank you both for your help, yes it does help quite a bit, I thought I was going crazy as I couldn't see the screw heads, they are tiny!
I think I will order a spare heatsink/fan assembly up, for the cost its handy to have should I ever need to RMA the system.
I have the rev2 motherboard, and it came with a stock fan, but its driving me insane!, too loud to sit next to for any length of time :(
I think I'm going to put the Noctua NF-S12B redux back in Paul. I'd installed that even before turning the server on for the first time.
If I get time I'll play around with Gelid/Noctua, having already done Gelid/Silent Wings 2, and then stock fan/Silent Wings 2 and finally back to stock fan/Noctua. I have no equipment to measure the dbA so it would be by ear only and I'll do it all in one sitting saving me from over stressing those tiny screws.
It's not particularly warm over here in the UK just yet and nor am I'm stressing the server in any way as I'm only playing around with XenServer Dundee right now. I haven't even put the SSDs in. I'm simply using USB flash drives. Not that SSDs would add much to the temperature in the chassis or at least not with my expected usage pattern.
Oh and I am really impressed with the Silent Wings 2's design.
The heat sink/fan assembly URL is now back online, at a little higher price now:
http://www.wiredzone.com/supermicro-components-cpu-processors-accessories-snk-c0057a4l-10022641?UTM_Source=TT&UTM_Campaign=10022641
Good point, not sure of Barry's motherboard, possible he doesn't have a CPU fan at all.
The Supermicro kit has a heat sink and fan pre-assembled, with thermal paste pre-applied (now out of stock) was actually $30 per SNK-C0057A4L assembly, sorry about that (I had forgotten that ordered two, plus about about $7 shipping for 2 from FL to CT), here's what the URL was:
http://www.wiredzone.com/supermicro-components-cpu-processors-accessories-snk-c0057a4l-10022641?UTM_Source=TT&UTM_Campaign=10022641
That is the same SNK-C0057A4L that appears on this unrelated motherboard, for example:
http://www.supermicro.com/products/motherboard/Xeon/QM77/X9SPV-F-3217UE.cfm
This $30 kit is the only legit way to get those tiny PH00 screws that I know of.
Not sure why it's not on the proper product page:
http://www.supermicro.com/products/system/midtower/5028/SYS-5028D-TN4T.cfm
http://www.supermicro.com/products/motherboard/Xeon/D/X10SDV-TLN4F.cfm
which doesn't help make this easier for Barry.
I'll ask Wiredzone about that part availability, and if the URL will change.
Meanwhile, this sort of time consuming back and forth tends to make me lean more heavily toward turn-key solutions that are tested and bundled, and perhaps with (optional) quieter fans installed by Wiredzone, at a reasonable cost for the parts and (careful) labor.
My pleasure Paul :). Your posts have helped me tremendously so I'm simply paying it forward.
You mention a $75 heatsink kit. Have you found a 3rd party product that is a direct fit or do you mean a replacement from Supermicro is $75?
I started to wonder if I'd missed Barry stating he had one of the passively cooled X10SDV boards and as such would need the actively cooled HSF.
Dear smartconnectables.io:
Wow, I really appreciate your chiming in, to help Barry out, so nice of you!
Barry:
I would recommend continuing to hold off on the "surgery" for the CPU fan for now, at least until I can get a video out. For $75ish, you can buy a heat sink kit that contains everything, but now you're dealing with removing the motherboard, removing the 4 heatsink screws, removing the 4 tiny PH00 screws with a straight-on screwdriver angle (PH0 screwdriver works too), installing Gelid fan, putting those 4 tiny screws back in, cleaning off the thermal paste, then installing the new heat sink/fan assembly, (that has thermal paste pre-applied), then reinstalling motherboard.
Avoiding the $75 cost, you can remove the PH0 screws with the tool I mentioned, but it's tricky, and doing it more than once is likely to result in one stripped screw that you just don't bother using (not a big deal, it's not going anywhere with 2 or 3 screws fastening it anyway). The problem is the angle of the screwdriver, with things like a ifixit pro kit's PH0 or PH00 bits having too much of an angle when using their includes shafts, potentially leading to trouble when loosening/tightening (stripped head).
I have tried valiantly to get Supermicro RMAs/Parts to sell me just a bunch of the tiny screws, to no avail.
Does this help? I'm just not able to record and produce such a video in the next 2 weeks or so, with a lot of travel. If you could hold off, and watch the video I'll make, it will give you a much better idea if you should feel comfortable doing this or not, much better than my rushed words here.
If you ever needed RMA service on your system, frankly, you should also be prepared to put the original fan back on, so reducing noise really requires a commitment, and willingness to take on some risk. I'm glad you stopped when you reached an uncomfortable spot, and I also realize the Supermicro manual doesn't really help at all with what you were attempting. There's no manual with the heatsink kit either by the way, and I'll need to add big red disclaimers to this article above, and the video, now that I know it's rather easy to "oops, I stripped a screw" incidents to happen, very good that the PH0 is magetic tip, to keep any metal shards from falling onto the mobo.
I sincerely hope this helps, I'll need to do a big post update as soon as I get a chance, I guess my reply to you is a first draft ;-)
Since Paul mentioned he is travelling I'll offer some assistance. With regards to the screws this photo taken from the post gives you the clearest view.
https://cdn.tinkertry.com/content/articles/718-superserver-combined-cpu-and-m2-cooling-fan/IMG_2224.JPG
I'll echo Paul's suggestion of using the correct screwdriver for the job! Use a PH0, a PH00 worked fine for me too.
Going onto the cable. The connector will be keyed by an arrow pointing at pin 1 and would be colour-coded as per following picture.
Hi Guys, I just tried to do this mod but had to give up!, I couldnt figure out how to take the stock fan off the heatsink, it didn't seem to have screws with heads when looking down on it, kind of looked like the end of screws.
Any ideas/videos on how to remove this type of fan from the heatsinks?
Also a random question if anyone has this board can you take a close up picture of the 8 pin white cable which plugs into the front of the board, next to the ribbon cable as I forgot to take a pic before removing the board, so have plugged it together based on what I could see in a video, but would be good to be sure I have it the correct way.
Thanks guys :)
Hi Paul,
Great blog, thank you! After landing on tinkertry.com a month ago, I ended up moving from a 4 host 6th gen nuc based silent cluster to a single host based on SYS-5028-TN4T with D-1541 processor. I'd like to make this as quiet as possible, and I was wondering whether the fan on the visiontek 7750 can be replaced with a quieter option. One other thing, in terms of noisiness, how does the visiontek 7750's fan compare to K1200's fan?
If I manage to bring the noise down to an acceptable level (the host sits in my bedroom), I'll be getting another system based on the CSE-721TQ-250B chassis as well, but this time to build a NAS; keeping raid configuration, disk type and speed aside, which supermicro Xeon D-15xx -with 2*10Gbe + 2*1Gbe- motherboards will help me make effective use of a 10gbps point-to-point copper connection back to my esxi host? I'd also like to make the second system as quiet as possible..
Is there any way you can post the 3d print STL files for your shroud? Or maybe I missed them...
Thanks
I went for 3-pin non-PWM versions. Thanks for the suggestion of the PH0 driver to prevent stripping. I have a full set of Wera Micro drivers so that is covered but would have naively used their Zyklop on it, and with the more than ample mechanical power it adds given its size, plus given I'm already pretty heavy handed would have stripped them for sure so one mobo saved thanks to you!
No paste needed, but though to not strip the tiny heatsink fan bracket, this Amazon part helps: "Ouya H30203 Plastic Grip Phillips Screwdrivers, PH0 x 200mm"
(Sorry, traveling, need to do article/video but it could be a while) silent wings fan is easy 4 screws/swap, did u get PWM or regular 3 pin model?
Hi. I have both the SILENT WINGS 2 120mm and Gelid Silent 5 here ready and waiting. Just to confirm... does replacing the CPU fan requiring messing with thermal paste? Thanks (and for all your X10SDV-TNT4 posts).
Should be pretty easy, having learned the right parts to get recently.
So here's what you'd want to buy:
1) Supermicro snk-c0057a4l is likely available from Supermicro RMA department, contact support at:
https://www.supermicro.com/support/contact.cfm
and ask for parts.
If that doesn't work out, order at Wiredzone if they're in stock again, about $35 each, here's the URL:
http://www.wiredzone.com/supermicro-components-cpu-processors-accessories-snk-c0057a4l-10022641?utm_source=tt&utm_campaign=10022641
It's preassembled with the CPU fan already on there, just disconnect motherboard, cables, and the 5 screws in the back, and slide the whole motherboard tray out.
I haven't done the rest of the procedure yet, but yes, for your 1520 it likely has the 1U style passive heatsink, although your photo from Newegg does seem to have the same heatsink. If it is the same, you just need to remove the 4 tiny screws that hold the bracket over the heat sink, taking care when putting them back in that you don't strip them. Even better to replace the whole heat sink you have already, but that does mean you have to remove the 4 screws under the motherboard that hold the heat sink to the CPU, clean off the thermal paste, then install the new thisnk-c0057a4l kit.
If you wish for a quiter CPU fan, the Gelid CPU fan will work.
https://tinkertry.com/superserver-combined-cpu-and-m2-cooling-fan#mar-24-2016-update
it attached to the underside of the already mentioned bracket.
For those tiny screws that hold the fan to the heat sin, you'll need a PH00 or PH0 philips driver:
http://amzn.to/25aS2YS
You can't buy that fan/bracket assembly alone, it's part of the heat sink kit, with thermal paste pre-applied and all, so you'll likely be fine, if comfortable doing such work.
Remember though, that CPU can handle over 90C before thermal shutdown, so not a big deal if you're seeing 60s.
hey man i've got the xeon 1520 version of this motherboard and it doesn't come with a fan on the heatsink! I'm getting some pretty high temps that I want to bring down (http://www.newegg.com/Product/Product.aspx?Item=N82E16813182973&cm_re=xeon_1520-_-13-182-973-_-Product) . Do you have any ideas on how best to mount a fan? I tried looking at the pics of your bare heatsink and it doesn't look any different than mine. How does your oem fan even get mounted onto the heatsink?
Thanks for sharing!
Sorry for the long delay, work consumed me this weekend (and not in a good way).
Here are the pics and I should note that I have 6 extra heatsinks if anybody is interested.
As for the Sunon fan, sorry I have only owned it a short while and also have not been able to find any info regarding its life expectancy. That being said, at least it is a dual ball bearing model which should last longer than a sleeve variety.
Awesome, this is why I blog, (sort of) meeting people like you that I'd otherwise not be very likely to come across. Feel free to share pictures of what you've done right here in Disqus if you'd like, always good to compare methods and ideas.
That little Sunon does move a lot of air, 37 CFM:
http://www.frozencpu.com/products/2414/fan-05/Sunon_80mm_Fan_-_3-pin_KD1208PTB2.html?id=dSTLxScK&mv_pc=507
any chance you found claimed life expectancy?
Wonderful article! I ran into similar troubles with CPU temps on my 4 and 8 core units (full load temps 60 - 65 C). It was a bit of a kludge, but I made an adapter plate so I could use a 60 to 80 mm fan duct and paired that with a Sunon 80 mm fan.
http://www.frozencpu.com/products/2414/fan-05/Sunon_80mm_Fan_-_3-pin_KD1208PTB2.html?id=dSTLxScK&mv_pc=507
This dropped the full load temps a few degrees.
My next plan was to have custom 70 mm x 70 mm x 40 mm heatsinks made. The combination of these heatsinks and the Sunon fan led to full load temps of only 48 - 53 C.
My next step is creating a fan shroud similar to yours.
As for my fan choices, it's looking like GELID combined with "be quiet~" Silent Wings 120 works very well, see in updates above, here's the direct link: https://TinkerTry.com/superserver-combined-cpu-and-m2-cooling-fan#mar-24-2016-update
The only thing left is to document the process through a comprehensive video, and to determine if the Silent Wings 120, or the Silent Wings 120 PWM 4 pin, is the best choice, those tests still underway...
Here's a look at my SYS-5028D-TN4T with Xeon D-1540, at idle, running ESXi 6.0, 41C for each of my 2 32GB DIMMs, stock fan speed. See picture attached. I'll boot over to Windows 10 and hammer at DIMMs with Prime95 soon, and report back with my results.
About ~42° (today installed in final position, I'll check next days).
Fan speed default (stock for now), 128GB.
In a couple of days I should get the serial bracket, so I will power off the server and try to install another fan (if I find exactly how to extract the motherboard without breaking something ;-D).
At the moment, I have only one working server in the house, doing daily backup duties. This makes tinkering with alternative OSs more of a challenge for now, saving my pennies for my next Xeon D-1500 purchase...
What temperature do you start worrying about your RDIMM temps?
Default chassis fan speed or full speed?
I have a Noctua fan (as in servethehome guide), but currently not installed (I'm working hard with ESXi to migrate a customer).
After getting a "stable" software environment I'll try to reduce RDIMM temperature (~42° as reported by the hypervisor), a litte too high for my confidence.
How about a study on using FreeBSD's zfs mirror on two VMDK (different disks) to run a RAID-1 virtual server... without RAID card?
Now that I think I have the fans worked out, yes, I do plan to be doing just such a video, likely sometime this month. I can say that it was rather easy to detach the cabling by hand, then remove the screws on the back, then just slide the mobo out, took about 5 minutes the first time, faster ever since. Assembly takes another 2-3 minutes tops.
Curious, are you considering the Gelid/Silent Wings 2 swap?
Please do you have made a video on "dismantling" the superserver? How to extract the motherboard?
Thank you
Still testing, and found the be quiet! Silent Wings 2 120 fans for just $10.95 at Newegg, with article updated accordingly:
https://tinkertry.com/superserver-combined-cpu-and-m2-cooling-fan#mar-24-2016-update
Just one more test to go, the PWM model. Already I can say that Gelid/Silent Wings combo is looking very appealing, affordable, and fairly easy to install.
Yes, that's the red 3d printed 50mm to 80mm adapter is all about (installed under the Be Quiet fan sitting on the CPU at top left of the photo). It would seem the back pressure is enough to slow the RPMs and effective airflow down, which is why I'm developing a Prime 95 test with the test bed in open air, coupled with an RPM meter, a volt meter, and a dB meter. The only thing I'm lacking right now is the spare time to focus, dive in, and test it all. Any 3d reprints take days, so it's a slower process than I'd like. Still looking like Gelid is the right solution for most folks.
Wow, $30 is pricey! I got mine from:
https://www.scan.co.uk/products/80mm-be-quiet!-silent-wings-2-quiet-fan-max-speed-2000rpm-noise-100-145-db%28a%29
I do remember on an American hardware forum someone mentioning how expensive BeQuiet was in the US which is strange. Still worth keeping them in mind cause they really are fantastic products. I also have an 850W PSU by them and thats been solid too.
Interesting that the original fan is cooler than the Noctua (and other) fans you have tried out. I would have though the Noctua would have been streets ahead of the stock fan.
If theres one thing that bothers me with a system its NOISE!
Have you considered trying to fit a larger fan on the CPU (space permitting of course)? Larger the fan = less noise
Cheaper in the US here http://www.ncixus.com/search/?categoryid=0&q=be+quiet+fan (but slower shipping than Amazon Prime)
I agree, £12 / $17 USD is fine. Over here in the US, Newegg doesn't have them, and Amazon lifts the price, as the only easy to find reseller, $29.95 USD for the 80 mm Silent Wings 2:
http://amzn.to/1TLPw5c
Still worth it, but interesting to note the (current) price uplift here. What I'm also struggling with is keeping things as cool as the original fan, that's proving difficult with all of my 3D printed adapters (since they don't make 50mm)
Thought you'd like them! Been running the first batch of fans I bought for almost 4yrs now. Maybe its just me but I don't think £12 for a fan is pricey consdering the benefits you get (silence!).
Those BeQuiet fans sure are quiet (and pricey), lovely design. I'm so glad you tipped me off! As for what I wind up keeping, had to back-burner those tests for a bit longer, other life priorities right now...
To be honest I didn't look at their product range (its beena couple years since purchasing a BeQuiet fan) but I thought I'd still mention the company as I have 8 or 9 of their fans (case fans and CPU coolers) and they are excellent. They really are very quiet and feel very well made. Only reason I chose them over the Noctua was due to the black colour! Noctua is excellent too but I don't own any of their fans.
Awesome, with so many fans you will find the right one I am sure.
The gelid seems the one that fits me better until now , to be easily replaceable. But I am waiting until November when the warranty is over to perform the replacement
See pic added https://TinkerTry.com/superserver-combined-cpu-and-m2-cooling-fan#Mar-11-2016-Update (testing still underway)
They do have an 80mm http://www.bequiet.com/en/casefans/258
Awesome. I will wait for it.
Thanks for the heads up on Noctua and alternatives
Maybe you won't have to, if I find an affordable 3d print shop that ships worldwide, we'll see. Hang tight, it could be a while before I resolve all this, but I think it might be worth the wait.
Meanwhile, you could do the quick and easy and affordable CPU fan swap, The Gelid 50mm fan http://www.quietpc.com/50mmfans (available on Amazon at http://amzn.to/1Tz9U9U), since the Noctua 60mm fan doesn't seem to keep the temperature controlled quite as well.
Thank you, certainly an option to look into, but not seeing a 50mm fan there, only heat sink combination options that require removing the existing heat sink (not desirable from a warranty perspective), and don't state Xeon D-1500 compatibility. Am I missing something?
The fans I was referring to can be found at: http://www.bequiet.com/en
I am defintely interested as I run my server in the lounge in a corner so noise levels are extremely important for me. My servers so quiet I don't even know its on. The ultimate test was when I had family over and I had to sleep in the lounge, I could barely hear it at all ;-)
You pay a premium for BeQuiet fans but I think its so worth it.
The Gelid 50mm fan http://www.quietpc.com/50mmfans (available on Amazon at http://amzn.to/1Tz9U9U or Newegg at http://fave.co/1ThsWRA) is one option that I'm also testing, hopefully this weekend, stay tuned. Good to know there's interest!
Yes for you it is not the best option since you move a lot the server.
Mine is just sitting all day on the same place so maybe I will think about it.
Nevertheless I will keep an eye on your approach so that I can also get the most of the improved airflow on a future M.2 drive.
BTW nice way to get into the 3D printing world ehe, Haven't had the opportunity yet,
Yeah, I saw that, his simple approach (like the Lego guy's build) will work fine for most folks, and is very appealing. Unless the unit will get moved around a lot, as he acknowledges. My need to travel and show this publicly has me personally incented to continue to try to give a slight boost to airflow around the M.2 slot, which has me continuing my tinkering. Just in case it can actually work well. Ideally, with a 3D printed part that folks can have printed and shipped anywhere in the world affordably, hopefully. If nothing else, it's a way for me to practice motherboard removal, and trying out 3D printing software and printers, learning as I go. So much more fun this way.
I appreciate this open dialogue, and your comments. Differing approaches and ideas are always better for everybody, and it all adds up to field-testing that hopefully gets taken note of by motherboard vendors, if nothing else than to realize that dB matters for mini-towers that might wind up in a home and not a datacenter/rack. I've also added this link to the "See also at TinkerTry" section above FYI https://TinkerTry.com/intel-xeon-d-1500-design-guide (Yeah, I've been thinking about this for a while, that Dynamat testing video was actually recorded October 2nd of last year)
Ty Paul.
Patrick used this fan but not really firmly attached as I can see.
http://www.servethehome.com/near-silent-powerhouse-making-a-quieter-microlab-platform/
There is no 50mm Noctua fan, but I'm looking into other options as well, because yes, such a swap would be simpler, although you'd lose the side-benefit of a bit of extra M.2 cooling. This is very much a work in progress, stay tuned!
Hi Paul,
Nice article as usual.
On my SS 1540 Xeon, can I replace just the CPU FAN with a Noctua or other in order to reduce the noise (while at least maintaining the same temperatures as the stock fan) but without performing any custom mod like yours?
I mean just pop out the stock fan and insert one which improves gets the SS more silent?
Thanks a lot
Chris Gregson
Hi Paul - I have the same case as yours, but coupled with a X11SDV-8C+-TLN2F.
I'm looking to expand out my storage as I want to have a play with vCloud Director/NSX. I currently have a AOC-SLG3-2M2 with 2x 1TB Samsung 970 EVO Plus NVME's and thought I'd try going for a used or new P4510 8TB U.2 Intel SSD (depending on price) .
I'll be looking to install this in the spare 2.5" top bay, but wondered if you'd had much advice with respect to temps and this case as I've heard these Intel P4510's can run pretty hot under load.
My understanding is that the exhaust fan (FAN A) will be doing much of the work - Obviously I can increase the fan thresholds with ipmitool for ESXi, but want to keep the noise levels down.
My concern is buying the drive, only to find the temps are through the roof and having to up the exhaust fan to beyond acceptable noise levels.
I would opt for an 8TB Rocket NVMe, but have heard about some reliability problems.
Your thoughts would be very much appreciated!
Kind Regards
Chris Gregson