Netgear XS708T managed 8 port 10GbE switch unboxed, noise and power draw measured - suited for Xeon D cluster
What do you look for when you're trying to build a 3 node test/demo cluster, with each Xeon D system blessed with two copper 10GbE ports each?
How about a relatively quiet switch that doesn't burn more power than the servers its connecting!
Just so happens Netgear recently release just such a 10GbE switch, see also the very recent (and very good) review with lots of pictures over at Serve The Home here:
- Netgear ProSAFE XS708T Review: Quiet 8 port 10Gbase-T switch
JUL 07 2016 by PATRICK KENNEDY at STHThe Netgear ProSAFE XS708T is the first 10Gbase-T switch I would put in a SMB office or a home without reservation....The bottom line on this: it is the best low port count 10Gbase-T switch we have seen to date and is a watershed product in terms of SMB/ SOHO networking.
As you'll see in the brief first-looks video, it's apparently pretty efficient and not very loud. Admittedly, I haven't put it under load yet, that's what those NVMe M.2 drives in those new little SuperServers are for!
I'm using this switch to CAT6A or CAT7 wire up all my copper 10GbE ports for those on-loan compact Xeon D systems, and I may use it for future home lab projects as well.
It has two shared SFP+ ports. Sure could be helpful for folks who prefer SFP+ equipped Flex ATX Xeon D-1500 systems like the very-recently released Supermicro SYS-E300D-8D.
I personally prefer copper 10GbE 1000BASE-T in my home lab, so I don't have to buy pricier (~$200 USD per!) SFP+ tranceivers and cabling. Short lengths of CAT6A or CAT7 cable for under $5 each will do nicely, thank you.
This 8 10GbE port Netgear XS708T-100NES is currently experiencing somewhat limited availability, with larger and even more costly 12, 16, 28, and 48 port versions also available at Amazon and Newegg. Prices of the 8 port model on Amazon seem to actually be rising by about $30 in the week since I placed my order.
Closing thoughts
There's a lot of room for improvement here, with pricing way too high, higher than the cost of the smaller Xeon D SuperServers!. Looking inside this 1U switch, and it's clear that there's plenty of room in there to make a much smaller form factor using the same electronics, something sized more like the 1GbE Cisco SG112-24.
This new video features my unboxing, careful inspection of all design features, brief watt-burn testing, and my very first successful switched 10GbE connection in my home lab.
Yeah, 10GbE switches are still horribly overpriced for home labs, but at least we might finally be getting closer to practicality, and I'm so glad I waited until something approaching practicality arrived on the scene.
Can you smell that (temporary/borrowed) vSAN plan I'm cooking up?
Video
See also at TinkerTry
For even more affordable networking that I use to wire the rest of my home, see:
All Comments on This Article (22)
I found the stock fan on the XS708T to be far too noisy for my quiet office. So I modded it. Thought I'd share to help others who love silent hardware!
The Noctua NF-A4x20 PWM http://noctua.at/en/products/fan/nf-a4x20-pwm fan is absolutely perfect for this device. It's a drop in replacement and comes with all you need.
It has made the switch 100% silent for me while still keeping it cool, even under load.
Thanks Paul, I have the 10gig NICs visible in ESXi now. I assumed that update2 would have had the driver in their by now too, perhaps 6.5 will have it.
VMware hasn't added native X552/X557 support still, which is surprising. But the fix is pretty easy, although it does require a reboot:
https://TinkerTry.com/how-to-install-intel-x552-vib-on-esxi-6-on-superserver-5028d-tn4t#expert-short-instructions
I have the latest version, 1541, and when I install ESXi 6.0u2 the 10gig NICs still don't show up as available. Has anybody else seen this issue? Any help would be appreciated.
My Dell copper SFP+ patch cables listed here:
https://TinkerTry.com/supermicro-superserver-vsan-plan-for-vmworld-2016#parts-list-not-pictured
seem to work fine with this XS708T.
Thats good to hear!
Do you know how fussy the Netgear XS708T switch is when it comes to the brand of SFP+ cables you can use? What brand/model SFP+ cables do you use?
I was looking at the "Cisco 3M SFP+ Cable 10Gb 37-0961-01 SFP-H10GB-CU3" SFP+ cables but can't get a straight answer out of anyone as to if they will work with the 10Gb switch!
I have not seen such behavior, not even with my very thin Monoprice SlimRun Cat6a cables (that many visitors at VMworld 2016 loved), seen pictured here:
https://TinkerTry.com/supermicro-superserver-vsan-plan-for-vmworld-2016
but I will be on the lookout for this.
I've been reading that 10Gb using RJ45 cables/connectors can sometimes overheat. Have you experienced this with the SYS-5028D-TN4T and the Netgear XS708T switch?
I just closed my Netgear case yesterday as they did absolutely nothing to even collect logs or try to troubleshoot the issue. My Supermicro case is still open but also going no where. Once it magically came back up I didn't want to take the cluster node down for more testing as I suspect its either the driver or the switch. I did try a Cisco 4900 and couldn't get a link there either though when the problem is happening.
I have ordered a second Netgear switch so that I can split my NICs in the hopes that if one fails the other will remain up and not take out my cluster. If this still fails then its the driver or server hardware.
Sadly my 3rd cluster node lost link again this morning and is presently down for the last few hours. Last time it took over 24 hours to magically come back on its own. The rest of the switch is still functioning. So thus far this same issue has occurred on 2 of my nodes at different times. This is the second time on node #3.
Here's how things look when it's up and working properly, with a loaner Supermicro SYS-E300-8D connected to my Netgear XS708T, using two Dell 1.5' SFP+ to SFP+ copper cables http://amzn.to/2alff06
This 10GbE thing (built in to Intel Xeon System on a Chip design) is a developing area, see also https://tinkertry.com/close-look-at-netgear-xs708t-100nes-including-decibels#comment-2857219495
but I suspect that Intel/VMware/Supermicro will want to resolve all that very soon.
As for stand-alone 10GbE NICs, no, I have no experience with buying them, since I haven't had the need. So sorry for my very delayed response.
I just saw my first mystery disappearance of link from my Supermicro SYS-E300-8D to my Netgear XS708T https://TinkerTry.com/close-look-at-netgear-xs708t-100nes-including-decibels, rebooting of the 10GbE switch (for other reasons), and it came right back. Sounds like what William Adams said here:
https://TinkerTry.com/supermicro-superserver-xeon-d-bios-1-1b-released#comment-2838170718
My Ubiquiti Networks EdgeSwitch 16 XG (ES-16-XG) https://www.ubnt.com/edgemax/edgeswitch-16-xg/
is due to arrive from Amazon http://amzn.to/2bFcHNi tomorrow, but I suspect the same issue will appear there as well.
Still trying to reach out to Intel/VMware/Supermicro on this, but now that I'm able to start replicating the issues, it will be easier. Any chance you have a VMware Service Request # already that you can share, via private email to me at https://tinkertry.com/contact#email
I am trying these Dell copper SFP+ short cables:
https://tinkertry.com/supermicro-superserver-vsan-plan-for-vmworld-2016#parts-list-not-pictured
Has anyone tried using 10GBASE-T SFP+ Copper Transceivers like Prolabs is making to connect to the SFP+ ports on Cisco / Dell Powerconnect switches? Not really vSAN or clustering but I am using a Xeon D system to host several virtual DC's and it would be nice to be able to use the two copper 10GbE ports, I am building out the infrastructure for a classroom and can not justify the cost of the Netgear 10G. I saw where Ubuiqiti was releasing a 10G switch with a mix of SFP+ and copper ports in the $600 range so I may just wait.
GP
Hard to answer, but I'd say suited for home in rooms other than bedroom.
Found much better cables by the way
https://tinkertry.com/supermicro-superserver-vsan-plan-for-vmworld-2016?
Fixed, and thank you for the heads-up!
Makes sense to do that, absolutely! Yeah, 10GbE switching is still insanely expensive, as I acknowledge here: https://TinkerTry.com/supermicro-superserver-vsan-plan-for-vmworld-2016
I do direct connects with my QNAP to my single D1540. No point in buying a switch that costs as much as my ESX Server.
Paul, great write-up, thank you! So what are some good, reasonably priced 10gb RJ45 copper nics? I could really use the bump in speed, but 10gb still seems to be out of reach costwise, which is still hard to believe. I think there's a huge pent up market for these switches and nics and the first company to really drop the price would more than make up any short term profit by increased volumes.
Clicking on the image takes you to the Amazon page for the Cisco 24-port switch
Paul Braren | TinkerTry.com
Thank you for adding this very helpful info here! For those interested, here’s a (non-affiliate) Amazon shopping link https://www.amazon.com/gp/aw/d/B071W93333