Today's Hard|Forum Post
Today's Hard|Forum Post

AMD vs. Intel HEDT Platform Showdown Editorial

I’ve spent quite a bit of time with AMD’s Threadripper and X399 chipset and I thought I’d give our readers my impression of it and talk about the platform as well as giving interested consumers a general overview of the platform and what it has to offer. We compare it to Intel’s HEDT platform and give our take on this match up.


PCI-Express and NVMe Storage

The second bullet point is easily one of the biggest points of comparison and the most interesting aspect of this match up. Both platforms offer 68 PCIe lanes on paper. Both platforms reach these numbers in very different ways. AMD’s Threadripper has 64 PCI-Express lanes on the CPU’s integrated PCIe controller. Four of these are reserved for communication with the chipset bringing the usable total down to 60 lanes. However, AMD has up to three dedicated x4 links to M.2 / PCIe NVMe capable storage devices. This leaves up to 48 lanes for expansion cards. AMD gives you an additional 8x PCI-Express 2.0 compliant lanes on the chipset. This brings the total lane count to 68. I wish AMD had made those PCI-Express 3.0 compliant lanes, but AMD has often lagged Intel in areas like this. This was something we saw with AMD’s X370 chipset as well. I consider it a minor issue, as the things normally connected to the PCH don’t truly need to be PCIe 3.0 compliant and you still have the PCIe x4 link to the CPU as a bottleneck.

In contrast, Intel gives you 44 PCI-Express 3.0 compliant lanes on the CPU’s integrated PCIe controller. These are all available as a separate dedicated DMI 3.0 link (more or less PCIe x4) to the PCH or chipset as it’s generally known. The X299 PCH offers 24 PCI-Express 3.0 lanes. Due to the DMI link, you always have a bottleneck to the CPU. Between the chipsets, AMD and Intel are roughly equal, though there could be protocol efficiencies that favor Intel here. DMI itself has virtually no overhead. This isn’t something we can compare easily. In either case, AMD’s limited at the PCH level due to the x4 link to the CPU and having only PCI-Express 2.0 compliance. Intel is limited, not by its lanes but by the DMI 3.0 link. DMI 3.0 limits you to roughly 4,000MB/s. This is, in theory somewhat problematic as M.2 and U.2 devices go through the PCH and do not access the CPU directly. This can limit faster SSDs, especially when these are utilized in a RAID configuration. Network controllers and other devices go through the PCH, leaving even less bandwidth for NVMe drives. One NVMe device is fine and two are usually OK, although you’ll find the reads capped on the newer drives. Writes are generally unaffected due to being well below the 4,000MB/s threshold with a lot of drive combinations. However, three drives seems pointless. This gives AMD’s X399 and Threadripper an obvious advantage on the storage front, however, the reality is that these limitations only present themselves when you are running benchmarks or are purposefully trying to saturate the DMI bus specifically.

Speaking of which, we’ll take a look at some performance data. Here are benchmarks showing NVMe SSD performance on each platform.

NOTE: For all Intel X299 drive and subsystem testing, we used MSI’s X299 SLI Plus motherboard. We also used an Intel Core i7 7740X (4.3GHz / 4.5GHz Turbo) and 2x 8GB (16GB total) Kingston HyperX (3200MHz DDR4 18-19-19-39, 2T@1.35v) memory modules running at DDR4 2666MHz speeds (stock testing, up to 3200MHz overclocked) were used. For power, I used an XFX XTR 850watt unit. Our discreet graphics card needs were handled by an EVGA NVIDIA GeForce GTX 780Ti reference card. The CPU was cooled with a Koolance Exos 2.5 system and an Alphacool Eisblock XPX waterblock.

NOTE: For all X399 drive and subsystem testing, we used GIGABYTE’s X399 Aorus Gaming 7 motherboard. We also used an AMD Threadripper 1920X (3.7GHz / 4.0GHz Boost) and 4x 8GB (32GB total) Corsair Vengeance LPX (3600MHz DDR4 18-19-19-39, 2T@1.35v) memory modules running at DDR4 2666MHz speeds (stock testing, up to 3200MHz overclocked) were used. For power, I used the an XFX XTR 850watt unit. Our discreet graphics card needs were handled by a Diamond Multimedia Radeon HD 7970 GHz Edition reference card. The CPU was cooled with a Thermaltake FLOE Rising 360 TT Premium Edition

X299 vs. X399 NVMe Storage (Single Drive)

Intel X299

Article Image

AMD X399

Article Image

The drives used here were both Corsair MP500 256GB NVMe drives. As you can see here, both systems trade blows with a single NVMe drive. One shows better reads in one test, and worse in others. Particularly interesting is that AMD’s X399 shows very low 100MB sequential read numbers. This is something I’ve seen on more than one motherboard now. Oddly, the MSI X299 SLI Plus has very low 50MB sequential read numbers which isn’t something I’ve seen on all X299 motherboards. Again, I’d have RAID numbers for you if VROC were released, and I had NVMe capability on the GIGABYTE X399 Aorus Gaming 7 at the time of this writing. This is something we will likely revisit in the future. I’d like to compare Intel’s NVMe RAID via the PCH vs. VROC and compare those to AMD’s X399 NVMe RAID solution. Despite Intel’s DMI 3.0 limitations, there isn’t any perceivable impact from running a single NVMe device via the PCH.

Of course, there is more to storage than raw bandwidth. On that front, there are some trade-offs with each platform. Intel’s M.2 slots are behind the PCH, which is unfortunate. It does support VROC (Virtual RAID on CPU) which is an implementation that allows for creating RAID arrays and booting from these via the CPU’s PCIe lanes. Effectively, this only works for devices connected through expansion cards or adapters placed into the expansion card slots. Outside of the built in M.2 slots on an X399 motherboard, it’s pretty much the same thing. With AMD, it’s technically possible to connect more devices to the CPU than it is on an Intel system because of how their lanes are allocated. However, things get much less clear at this point. AMD has a 7-device limit. These 7 devices can’t exceed 60 PCIe lanes. This limit can be overcome with external clock generators. How frequently these will be employed is anyone’s guess at this point. I do not have an answer to that at this present. Secondly, AMD’s new NVMe RAID implementation only allows for 10 devices to be managed in a RAID array. Intel’s allows for 20 devices and has no device limits that I’m aware of beyond that. Of course, that’s more lanes in most cases than you could use via expansion cards.

Of course, Intel’s VROC technology is great in theory and has a stronger feature set with its established iRSTe software and hardware implementation. In addition to Intel supporting double the volume of devices, Intel supports more RAID stripe sizes and more RAID levels. Intel supports RAID 0, 1, 5, and 10. AMD only supports 0, 1, and 10. That said, RAID 5 isn’t something that works well without hardware dedicated to parity checking. There is a substantial performance hit when using it on anything other than enterprise class storage controllers with their own cache, BIOS, and dedicated processor. Of course, at this point, I’d love to show you some benchmarks and pit VROC RAID vs. AMD’s NVMe RAID solution using CPU based lanes on each platform. Sadly, VROC isn’t technically released, and AMD’s NVMe RAID requires a UEFI BIOS update and only ASUS seems to have the BIOS for this ready at the time of this writing. To make matters worse, Intel locks it’s VROC technology behind some paywall. This is where things are still less clear to most people, and where things get interesting in the way one has a fascination for train or car wrecks despite something horrible transpiring to create those.

Intel’s VROC technology has several technical requirements. First and foremost, you need a Skylake-X CPU. Kaby Lake-X cannot be used for this purpose. At a glance one might think it’s due to the limited PCIe lanes of Kaby Lake-X’s PCIe controller, but that’s only part of the reason. Skylake-X has a Volume Management Device embedded in it which handles VROC. This device isn’t present on Kaby Lake-X. VROC also has three licensing modes. Passthrough, standard, and premium. The passthrough mode only allows for the creation and management of RAID 0 arrays. RAID 1, and 10 require the standard license and RAID 5 requires the premium license. Passthrough licensing is part of the X299 PCH and therefore is available to all with a Skylake-X CPU. However, standard and premium licenses require a physical hardware key which can only be used on a VROC ready motherboard. That is, a motherboard with the port required to physically install the license key. The license keys are available at an additional cost. I haven’t found the premium key for sale, but the standard key was $89 when I checked. As if those restrictions aren’t enough, use of a bootable drive array with VROC technology requires Intel SSDs specifically. You cannot use another brand (although some have reported otherwise).

In contrast, AMD’s Threadripper and X399 platform doesn’t have any such requirements or shenanigans. All your M.2 slots go through the CPU’s PCIe controller and anything you install in the PCIe expansion slots can be used with AMD’s NVMe RAID implementation. As I said, RAID 5 wouldn’t work well on either solution, so AMD’s failure to include it isn’t really a problem. The device limit is potentially interesting, as it would only take 2x PCIe cards to hit the 7-device limit. I do not know if having four M.2 devices on a card like ASUS’ Hyper 4x card would count. Without a RAID BIOS or some way to make those appear as a single device to the motherboard, they should count individually. It wouldn’t take too much effort to hit the 10-NVMe device limit, budget considerations aside. You also must consider that the 7-device limit on the bus comes into play, as does the 10-device limit of the NVMe RAID solution. That said, I don’t think people will elect to use more than two graphics cards and a single Hyper 4x card with 4x M.2 devices onboard. So, while Intel’s more capable in some areas with fewer limitations, it’s artificial limitations cost considerably more and the lack of PCIe lanes comes into play much earlier. VROC limitations on Intel’s X299, do not apply to traditional PCH based M.2 devices. I created a bootable RAID 0 array with Corsair MP500 drives without any issues. So, this remains unchanged.