Seagate Exos 10E2400 - testing a hybrid SFF drive with an extended method

Many analysts believed that 2.5-inch hard drives in the future should have disappeared altogether, since in the corporate segment they would not have found a place between voluminous 3.5-inch drives and ultra-fast solid-state drives of various types (NVMe, SSD). But if you carefully examine the lineup of the same SuperMicro, you can see that more than half of the models of rack servers have 2.5-inch slots for HDDs, and GPU computing platforms generally come with only 2.5-inch bays that take up less space. At this stage, analysts' forecasts did not come true - customers still have a demand for a large amount of local data on the server, which requires large drives.

Last year, Seagate rebranded its hard drives by merging and merging corporate models into the Exos series, which includes both 3.5 "and 2.5" HDDs with different capacities and different spindle speeds. The 2.4-terabyte 2.5-inch hard drive Exos E102400 (P / N ST2400MM0129) we are considering was previously called Enterprise Performance 10K v9, but the name change did not affect the technical characteristics or even the firmware, even the SKU number remained unchanged. As you can see from the previous name, we are dealing with the ninth generation of drives, of which the first six belonged to the Savvio 10K-10K.6 series, the seventh and eighth - to Enterprise Performance 10K, and the ninth - to both Enterprise Performance 10k and Exos. Starting with the eighth generation, these drives have been added with NAND read cache (TurboBoost technology), so it is more correct to call these drives hybrid.

Key features of Exos 10e2400 (Enterprise Performance 10k v9)

• Volume - 2.4 TB
• Interface - SAS-12
• Native 4K sector support
• Spindle speed - 10500 RPM
• Typical access time is 2.9 ms
• Buffer - 256 MB
• NVC cache - 8 MB
• 16GB Read Flash Buffer
• Size - 2.5 inches
• 4 plates, 8 heads
• Record type - PMR

Exos 10E2400 highlights

One of the distinguishing features of the 9th generation 10K series discs is support for 512E (Advanced Format) and 4Kn formats. At the physical level, the sector size is 4 KB, but by default the disk presents a 512-byte sector to the operating system due to simple emulation. Today, not every operating system or program supports a 4K logical sector. For example, even in VMware ESXi, support for 4K blocks appeared only in version 6.7, released in mid-2018, and all the same, 4K disks cannot be forwarded to the guest OS via RDM. So, when you make sure that all your programs work fine with 4K sectors, you just need to reformat the hard disk to the new block size. Seagate promises that if the partitioning software supports Fast Format, the conversion will take seconds.

If you install a hard drive in a storage system, then the storage system itself must know about Fast Format support, that is, these disks must be on the compatibility list. And of course, by default, the disks are shipped with a sector size of 512 KB in emulation mode (512E mode).

8MB NVC cache

Starting with the Seagate Enterprise Performance v8 series, hard drives have a small area of ​​the write buffer protected from power failure. This is a small NVC-cache (non-volatile cache) of 8 MB, consisting of eMLC memory cells, as in corporate SSDs. If you connect a hard drive with the Write Cache Enable = 0 (WCE = 0) parameter, you still use write buffering, but the cache size is limited to the specified 8 megabytes. In case of a sudden power outage, the residual energy of rotation of the plates is used, the motor switches to generator mode and feeds the HDD electronics to write a buffer to the NVC area, where data can be stored for 90 days, and when the disk is turned on, the contents of the NVC cache are automatically transferred to the magnetic media.

If the disk is working with the WCE = 1 parameter, the buffer size is not limited to 8 megabytes, and you can expect a higher write performance, although not every test will show this difference. In fact, NVC cache solves the age-old problem of disk-level write buffering in storage systems that almost always use a SAS controller with non-volatile memory. Now we can say that write caching in Exos 10E2400 disks is always used, just with WCE = 1 the buffer can exceed 8 MB, and with WCE = 0 it is not. Let's see if it makes sense at all to talk about write buffering in real-world tasks.

Our tests show that write buffering gives any noticeable performance increase only in video surveillance systems, with sequential multithreaded recording, and in the old Savvio 10K.6 drive it generally leads to a decrease in performance. I don't think anyone in their right mind would use such an expensive disk for VMS software, so from our point of view, you can forget about NVC cache and built-in write buffering and not use it in further tests.

TurboBoost Technology

Each Exos 7E2400 drive has a 16GB NAND cache on eMLC chips used to speed up reading hot data. This technology looks quite simple and understandable: the controller places the most frequently requested blocks in the random read mode in the NAND area, from where data can be read with a minimum delay compared to reading from magnetic platters. It is important to understand that TurboBoost is not used for write and sequential read operations, but in RAID arrays, the cache size will be summed up, since each disk reads something different. Naturally, this technology is completely hardware-based, and no drivers or software are needed for its operation.

Here you need to clearly understand that in the modern world of virtualization and Big Data, 16 GB is a drop in the ocean, and modern storage systems and servers are perfectly able to cache both read and write - both on SSD and in RAM. Of course, in arrays of 30 drives, the total amount of TurboBoost area is already comparable to one cheap 480 GB SSD, but still, from a practical point of view, it is not impressive. However, think about how much of the data in the general distribution is really `` hot ''? In typical databases it is about 10% or less, so the practical value of TurboBoost depends entirely on the infrastructure topology and the nature of the requests.

Of course, if you are using modern storage systems like which offer multilayer storage and SSD cache for free, Seagate's TurboBoost doesn't make sense. But if the storage system does not support Tiering and SSD cache, or you install hard drives in a machine running Windows Server or VMware ESXi, then, of course, TurboBoost can become the only way to transfer frequently requested data to NAND memory, and it is completely transparent for the operating system, without additional licenses. programs and activation keys.

To test Turbo Boost, we limited the size of the test area to 16, 32 and 96 GB.

The random access results are impressive. So much for the speeds unattainable for conventional hard drives, but let's increase the test area up to 32 GB and see what happens. Do not forget to fill the cache with a 3-hour random reading of an area of 500 GB in between tests.

The picture changes, and pronounced peaks and valleys appear. We clog the cache again with garbage and deploy the test in a 96 GB area.

As you might expect, as the volume of hot data grows, the effect of TurboBoost fades away. There is an interesting point: all Enterprise Performance 10K v9 disks have the same NAND area, so hot data can be transferred to Exos 10E2400 of any size, given the total amount of TurboBoost cache in the array.

Power Consumption

Winchesters of the Exos 10E2400 series have 4 grades of power consumption. Depending on the load, the drive can be in one of 4 states:

• Heads above the disk, controller power consumption is reduced (4.9 W, transition time to state - 100 ms)
• The heads are parked and the discs are spinning at maximum speed (3.78W)
• Heads parked, discs spinning at reduced speed (2.13W)
• Heads parked, motor stopped (1.24 W)

During read and write operations, disk power consumption varies from 6.98 to 8.75 W .

Reliability and thermal performance

Seagate Exos 10E2400 hard drives are designed for 24/7 operation at ambient temperatures below 26 degrees Celsius. The declared HDD resource is 2 million hours (228 years) and no less than 600,000 head parking operations. The documentation for the drive says that reliability indicators are calculated for the temperature of the HDD itself below 50 degrees Celsius, that is, for Exos 10E2400, 49 degrees and below is the norm, and the maximum heating, judging by SMART, is 60 degrees Celsius.

For some of their hard drives, for example, for the Ironwolf Pro series, the manufacturer indicates a normal load in terabytes per year, which includes read and write operations. There is no such parameter for the Enterprise Performance HDD series, and it is especially interesting how the NAND part of the hard drive will age. Seagate bypasses the issue of the chip resource used for TurboBoost technology in every possible way, only mentioning that the eMLC memory works in such a mode that its resource is several times higher than that of MLC chips in conventional SSDs and is designed for the entire life of the disk.

Be that as it may, the Exos 10E2400 series hard drives are provided with a 5-year warranty without any load restrictions.

Testing

As you have seen, the speed results of a hybrid hard drive will greatly depend on the size of the active area in which reading takes place. In our main benchmarks, we use the entire disk space to show the overall performance of the Exos 7E2400. Let's start with the traditional random access test.

Let's see how performance changes depending on the number of threads.

It is interesting how the new disk is significantly inferior to the old one in terms of the number of I/O operations, but with such a number of threads the access time delay is more important, see the diagrams.

The Exos 7E2400 is better balanced in IOPS / ms for random write operations. Let's check how the random access performance changes depending on the size of the transaction.

A huge drop in write performance when a transaction grows above 4Kb can be explained by its own 4KB disk sector size. This advantage lingers on transactions up to 64 KB and then dies out. In random read operations, in general, you cannot go against physics: a larger disk volume gives an advantage only in the area above 512 KB, that is, in the area of operation of VMware ESXi, which by default marks the VMFS6 file system with a sector size of 1 MB.

Let's take a look at sequential access.

Very good, but much better to see how sequential and random access work together in real life.

Random write operations are the weak side of this hard drive, so in all three parameters (speed, bandwidth, latency) it loses to its ancient colleague.

As I mentioned above in the text, no one in their right mind would buy 2.4 terabyte hard drives for video surveillance needs, but in a large installation, part of the array can be allocated for storing VMS data and initial analysis of recordings using AI. We saw that in the Macroscop test, the new product gives about 88 megabytes per second, which corresponds to 100 streams from 1080p cameras. Let's check the latency with different number of cameras.

As you can see from the diagram, in the region of 100 cameras, Exos 10E2400 gives approximately two-fold superiority in access latency. In simple terms, where on old hard drives VMS software starts to swear at an error accessing the disk system, new hard drives will work smoothly.

Capacity density and cost

In standard 2U enclosures designed for 24 drives, Exos 7E2400 drives achieve a density of 28.8 TB raw-capacity per unit. In the same JBOD case for 12 LFF drives, a 3.5-inch Exos X12 with a volume of 12 TB will give three times the density - 72 TB per 1 unit. One hard drive Exos 7E2400 (p/n ST2400MM0129) has a cost of $ 460, and a 12 terabyte 3.5-inch Exos X12 ST12000NM0027 - $ 450, so in a direct comparison, the difference in price per terabyte is just glaring, but let's see how the situation changes in an array, for which we take a 2U shelf and fill it with hard drives.

For a 24-disk SFF shelf, we will have the following configuration: RAID 6 for 22 hard drives + 2 hot-swap disks. This will give us 48TB of raw magnetic storage plus 352GB of SSD cache for a total of $ 11040.

We need to install 8 Exos X12 hard drives in a 16-disk LFF shelf for the same capacity, 6 of which will be in RAID 6 and 2 in Hot Spare. To add NAND cache for reading, we need to install a SAS SSD such as the Seagate Nytro ST400FM0233 with a capacity of 400 GB and a price of $ 743. The cost of this solution will be $ 4342.

From the point of view of volume, 2.5 "hybrid 10K hard drives are 2.5 times inferior to configurations from 3.5" HDD and SSD.

Conclusions

In addition to the volume, which is steadily increasing every year, we are also offered new technologies that slowly penetrate into hard drives. Based on the results of testing Exos 10E2400, we can draw the following conclusions: the disk shows excellent results for HDDs random read with a 4K transaction size and random write with a 1MB transaction size. But the patterns of real tasks do not confirm this assumption, and the only serious speed advantage over its distant ancestor Exos 10E2400 showed in 4K video import.

However, everything changes if the nature of the load changes. In real life, you will hardly keep a 2 TB database on one hard drive, and as soon as the number of disks in the array increases, and frequently requested data is indexed and cached in the NAND area, TurboBost will come into play, which can give a speed increase of 1.5 -2 times in mixed operations and 4-5 times in read operations, on any operating system and with any controller.

It is also worth noting that already on 8 disks in RAID 10, you can achieve a speed of 1 Gb/s, which will allow you to work with 4K video without compression in real time at a volume of 9.6 TB. In machines running AI video analytics applications and in media studios, this is one of those cases where you can put together a relatively inexpensive array at crazy linear speed.

There is one more point that I cannot but pay attention to: in the Enterprise Performance 10K v8 series disks with a volume of 0.9-1.8 TB had 32 GB NAND cache, and in 10K V9 for some reason the cache was reduced to 16 GB, so when choosing models from 900 to 1.8 TB for transactional load, it makes sense to look at the previous, eighth generation.

Mikhail Degtyarev (aka LIKE OFF)
08/11.2018