How Good Is The AMD EPYC 3251? Review of ASRockRack EPYC3251D4I-2T Mini-ITX motherboard

I was looking forward to the opportunity to test AMD EPYC 3000, and here's why: EPYC 7000 series processors, both on the first generation core (Napples) and on the second generation (Rome), are a real hit and a breakthrough in the world of cloud computing and virtualization ... But a person does not live in a single cloud, and for dedicated network gateways, NAS-s, various security systems or for edge computing, 32 cores is a clear overkill. In such tasks, rarely does anyone load more than 4 cores, and compactness and the ability to work well, literally under the feet of a system administrator, and not in a room with an air conditioner, are qualities for which customers pay money. Intel has a wonderful Xeon D processor for these tasks, on which it is possible to produce simply crazy compact, but very powerful systems.

And if Intel, being in a not very profitable situation, was able to make such a miracle as Xeon D, then all AMD needs is just to saw off one crystal from its EPYC 7000 chiplet, solder legs to it - and you get EPYC 3251: the same 16 threads, the same SoC, only this processor was born in a family where the word `` security '' - not an empty phrase, which means you are not afraid of attacks on the processor microcode and medications from them that slow down the host speed.

We have a motherboard that reveals the essence of an integrated processor with a TDP of 55 Wt. In Mini-ITX format, ASRockrack EPYC3251D4I-2T has a pair of 10 Gigabit copper interfaces, supports up to 256 GB of RAM , moreover, on regular RDIMMs, and not on expensive LRDIMMs. If you want to save even more, you can use desktop "non-ECC" memory.

As you can see, in the RAM area, the EPYC 3251 differs from the Xeon D-2143 very significant: AMD has fewer channels, but a higher frequency, plus the ability to use desktop RAM.

Network subsystem

ASRockRack has not pampered customers with a large number of network ports for a long time, believing that the built-in 20 Gigabit/s is enough for any need. In addition, using these ports in out-of-band mode, you can access the IPMI interface of the motherboard if for some reason you do not want to use a dedicated network port. Interestingly, a separate Intel X550-T2 controller is used for the network here, and not integrated into the AMD processor. This should improve compatibility with naughty software, especially in features like SR-IOV.

The network controller supports PXE and the first LAN port is NCSI. ASRockrack's arsenal has analogs of this board with a 1 Gigabit network on the Intel i350 chip. As a rule, cards with this controller have iSCSI boot, which is completely absent in 10 Gigabit versions.

Storage

For the compact enclosures that ASRockrack EPYC3251D4I-2T is designed for, there are plenty of options: 2 SATA-600 ports, one M.2 22110 (PCI-E 4x Gen3) and one Oculink switchable to 4 SATA or 1 PCI-E mode (Gen3 4x). When you install a SATA3 drive in an M.2 slot, one of the SATA ports will be disabled, so there is no point in that. That is, if you need more space - you can connect 6 hard drives + M.2 SSD PCI-E cache, if more speed - M.2 or PCI-E SSD. There is no hardware RAID on the board, so if you need more, use a PCI-E controller.

The AMD EPYC 3251 processor itself supports 32 PCI-E Gen3 lanes, and this number is enough to bring out a full-fledged PCI-E 16x slot that does not share the bus with anything, so even with the maximum complement of the motherboard, you can use the GPU or an interface controller.

Additional connections

Among the additional pins there are pads for connecting SATA SGPIO, COM1 port, TPM module, as well as two USB 3.1 ports.

In addition, there are pins for connecting monitoring buses IPMB and PSU SMBus.

Cooling and power

Despite the fact that the EPYC 3251 processor is considered energy efficient by modern standards, its power is up to 55 W, and this value requires good active cooling. However, only a passive aluminum heatsink is installed on the board, and the instructions say that if the air flow through it is below 2 CFM, then you need to install a fan. Without additional cooling, just lying on the table in the BIOS (when all energy-saving functions are turned off) the processor easily warms up to 97 degrees, and no matter how scary it looks, the critical temperature for it is 105 degrees Celsius. And yet the designers were right: the processor does not need much, and the slowest fan easily drops the temperature to 56 degrees in boot mode.

In accordance with modern trends in the design of motherboards, only 3 aluminum solid-state capacitors are installed here, and IC elements located on the back of the board are mainly used to filter the power supply. The fan connectors have LED indication of the cooler malfunction - a nice and damn convenient addition.

The board uses an interesting power connection scheme: well, firstly, in order not to waste precious space in the Mini-ITX format, the power input is 4-pin + 8-pin, and the bundle includes an adapter from ATX 24 pin to ATX12V 4 pin ... Secondly, if your power supply unit does not have SATA ports, then you can take power directly from the motherboard through an unusual cable from ATX12V 4 pin to SATA. No one will tell you why the power input and output on the board should be absolutely identical, but they will warn you that improper cable connections will lead to the inevitable death of the board.

IPMI/BMC Remote Management

For remote management, an ASPeed AST2500 chip is used with a dedicated 1 Gigabit network port, which is used only for BMC functions and nothing else. I would like to say that AST2500 is the best solution for today, having a modern HTML-5 interface, optimized for tablets and smartphones. You have access to fast and convenient console management, connecting a remote disk for installing the operating system both from a computer and through a NAS. There is a very convenient temperature monitoring and the ability to record POST tests.

Compared to Dell iDRAC, HPE iLO and Lenovo XClarity management systems, here you do not pay for remote management, but get almost the same capabilities, not to mention the fact that you can simply forget about IP-KVM like a bad dream ... Yes, and one more thing: if you do not use a dedicated gigabit network port, you can access the BMC through a 10-gigabit port: the board implements a kind of network switch, so that we can access the hypervisor web interface via one network wire, and in the BIOS via the console. Cool!

BIOS/UEFI

Of the interesting features of the AMI BIOS, it should be noted that the Watch Dog timer is not switched off, which has 2 server reboot modes: Reset and NMI, the choice of the operating mode for the Oculink SATA port, as well as the ability to adjust the processor thermal package and access to AMD Zen, which is usually server motherboards you won't find.

And in general, in terms of its equipment, the BIOS seriously stands out among the gray mass: here you have full hardware monitoring, and setting up memory controllers and PCI Express ... In general, those who like to delve into the parameters will be happy.

Compatible with operating systems (VMware, FreeNAS)

The weak point of the motherboard in question is the lack of declared compatibility with VMWare ESXi and FreeBSD. The latter is seriously depressing, since the most common free security gateways (PFSense, OPNSense) use a "free" kernel, and where there are compatibility problems, they are launched under ESXi. Well, that is, at least stop, at least fall, but in the list of supported OS there are only 64-bit RHEL/Centos, Ubuntu and Windows Server 2016. I turned to my friend, a VMWare representative with a question: when will ESXi support EPYC 3000, but I said , that he does not know and generally refused to comment, although he is usually very talkative. And although from my point of view it is an absolute nonsense to get hung up on one processor manufacturer, you can still understand VMware: they need to sell their hypervisor in expensive cloud environments, and EPYC 3000 is not their caliber.

I could not pass by such a challenge and check on my own experience what and how it works, and the first thing I did was install VMware ESXi 6.7 U3 from scratch on our test motherboard. Everything was installed without a hitch, as if it should be so, and after playing with the launch of my virtual machines, I decided to check if the EPYC 3000 is ready to fit into the AMD ecosystem? In our article "Migrating from Intel Xeon to AMD EPYC, avoiding pitfalls" ; we touched on the lack of live migration between Intel and AMD, but now I have an EPYC 7551p server under my desk, and an unsupported EPYC 3251 on my desk. What do you think?

When Linux Mint was launched, it migrated perfectly without stopping. That is, VMware does not have any technical issues related to poor EPYC 3000 support, and perhaps they simply could not agree on certification issues with AMD. Moreover, I ran all our tests under ESXi 6.7U3, and the only thing I noticed was the `` virtualization penalty '' in synthetic tests. For example, running on bare metal Cinebench R15 delivers 25% faster speed than with the same parameters, but in a virtual machine. For Xeons and EPYC 7000s, this figure is much lower, about 10%, so if VMware is optimized for EPYC 3000, it may work even faster.

FreeNAS 11.2 worked just as easily when installed on bare metal. I have already mentioned in other articles that some widely supported FreeBSD distributions work fine on AMD EPYC, although the official support for the "red" stuck somewhere at Athlon 64 level.

In total, we can say that if you need to integrate AMD EPYC 3000 into the EPYC 7000 ecosystem, then these processors are compatible with each other: you will have live migration and support in a virtual environment of any OS. Even ESXi works like a clock, although it does not openly admit it.

Testing

Testbed configuration:

• Motherboard: ASRock Rack EPYC3251D4I-2T
• Memory: 32 Gb RAM
• SSD: Transcend TS1TSSD230S 1 Tb
• Hypervisor: VMware ESXi 6.7 Update 3
• OS: Windows 10 x64, Oracle Linux 7
• Guest OS settings: 16 vCPU, 16 Gb RAM

Please note: we have a 2-channel memory configuration in the AMD stand and a 4-channel memory configuration in the Intel stand, in order to fully load the processor's capabilities. Before we go directly to the benchmarks, let's look at the CPU-Z output.

Now let's go through the synthetic tests from the guest Windows 10, remembering that the virtualization penalty on EPYC 3251 is higher than on Xeon 2143.

Our old Xeon E5-2603 V4 got into the test results clearly by mistake, but the very first benchmark showed that a serious test is being prepared for Xeon. AIDA64 will show the performance in more detail

Well, just excellent encryption speed and disastrous results in photo processing. Do not forget that this is just Windows 10 and full synthetics, in fact, everything could be different, and already the first real test, the non-relational database Redis, says that in relation to the Xeon D 2143, this is just `` beating babies ''. Here it is likely that the Xeon D 2143 has a worse performance by raising the core frequency at boot, because I see no other explanation for the 1.5-fold difference in AMD's favor.

Redis, requests per second, P=1

But again, as the threads increase, the difference is leveled, which again suggests the efficiency of power management.

Redis, requests per second, P=16

But it should be borne in mind that Redis during Set/Get operations is more strongly dependent on memory speed and to a lesser extent on some kind of internal processor architecture. A completely different matter is MySQL databases, where we create a table with 1 million rows for testing, and we run the test for 100 thousand rows, because in multi-threaded mode these processors do not need more. We use latency as a yardstick, which ideally should be less than 20 ms.

MariaDB, dependence of the latency on the number of threads

MariaDB, dependence of the latency on the number of threads

The results of this test speak for themselves: EPYC 3251 is on average 2 times faster than Xeon D 2143IT.

Power Consumption, Wt

Based on the power consumption of a board with 1 memory stick, we can say that on average, the EPYC 3251 will wind up the counter twice as slow as the Xeon D-2143IT.

Ordering Guide

Our tests show that the EPYC 3251 in real applications is twice as fast as the Xeon D-2143, and at the same time it consumes 2 times less electricity, which means it heats up less and does not require any special cooling. With these features, it is better suited for an edge computing server or for use as a virtualization host for a small company that houses a gateway, NAS, and domain controller. The dual-channel memory controller is no reason to be sad because the 4-channel Xeon is slower and the only thing worth worrying about is the software. Officially you only have Windows + Linux, but unofficially you can use VMware ESXi and FreeBSD.

On average, the considered motherboard based on EPYC 3251 will cost the customer $ 700-800, while the option based on Xeon D 2143 will cost about $ 1200-1300. The difference in price between platforms is true for products from other vendors, so even from the point of view of the wallet, the EPYC 3000 is great!

Choosing an EPYC 3251 motherboard, pay attention to the board from ASRock Rack we are considering: Intel X550-T2 network controller is the best that exists today for 10 Gigabit copper (which standard is better for a 10-gigabit network?). Connections to DC12V power supplies are the ability to power the control PC from the on-board network or batteries, bypassing the inverter. The deadliest 120mm fan blowing over the board reduces the CPU temperature in Idle mode to 25 degrees Celsius, and in the BIOS you can set the maximum TDP to reduce the maximum temperature, for example, in a Blade case on a Mini-ITX platform.

We recommend ASRockRack EPYC3251D4I-2T for use in security gateways, NAS-s and edge servers, that is, where you should have older Xeon D or lower Xeon E5.

Mikhail Degtyarev (aka LIKE OFF)
30/12.2019