When it comes to enterprise TLC NAND (eTLC) vs. consumer TLC NAND (cTLC) many people get lost in the weeds fairly quickly. Quite honestly… this is by design. NAND manufactures really do not want to discuss a lot of the ‘differences’ as they really do not sound all that impressive outside of a press release / factoid form. This is because, in many ways… they really are not all that impressive. It still is the same underlying NAND tech, just tweaked for the higher demands/requirements of the enterprise environment.
So let’s start at the beginning and go over some history. So called ‘enterprise’ NAND (which sometimes is called ‘high endurance’ or ‘enhanced’ NAND) came about because the differences in cost of SLC NAND vs MLC NAD was getting bigger and bigger every generation… and it was getting darn difficult to offer a large SLC NAND package at a price people were willing to pay. Sadly, the endurance of MLC NAND was not much to write home about when compared to SLC NAND. This was/is especially true when dealing with early generation SSD controllers that ate NAND life for breakfast, dinner, lunch and five snacks a day in heavy, constant workload scenarios. So yes, NAND endurance was (and still somewhat is) a big deal. The same is true of raw performance of SLC vs MLC (as it is just plain faster to read or write to a cell when you only have to worry about ‘full’ and ’empty’ cell states).
(photo courtesy of http://www.focc-fiber.com/info/what-is-slc-mlc-tlc-nand-flash-38215894.htmland is for illustrative purposes only)
To minimize the perceived differences NAND manufactures came up with the brilliant idea of increasing the over-provisioning their standard MLC NAND product came with (as in they spec’ed and sold say 256MB NAND IC as a 200MB IC to the SSD manufactures – and before you send us hate mail over this we are exaggerating to make the general point clearer… and the reality is most of it comes via the differences between GiB and GB standards), offering ‘secret sauce’ optimizations to SSD controller manufactures to bake into their firmware, and increasing the operating temperature range the NAND was rate for (as heat kills NAND faster than a government lockdown kills ‘non-essential’ business). Thus ‘enterprise’ or ‘high endurance’ NAND was created and it resulted in a NAND product that was noticeably better / longer lived than the original NAND blueprint allowed for. Thanks to the optimizations it also resulted in an enterprise storage solution that may not be as ‘fast’ as the ‘consumer’ NAND but was much more consistent when it came to long-term performance.
If some of this sounds familiar… that is because it is. Over-provisioning and NAND specific firmware optimization has trickled down to the consumer NAND marketplace as SSD controller manufactures started baking in some of the enterprise over-provisioning specifications (and why you will see two models using the same controller and same amount of NAND onboard but offered in two different capacities), as well as (some of the) read/write optimizations into their ‘consumer’ firmware. Basically, beyond the increase operating range (thanks to higher factory binning, and longer QA/QC factory testing), at the end of the day both eNAND and cNAND variants start out life as the same NAND design. The eNAND is just more heat resistant and designed with long term performance in mind, not short-term. So it is better… but it may not be better enough to justify the added expense for your particular needs and scenarios.
With the advent of TLC NAND, the exact same issue started to crop up vis à vis price vs. durability vs. performance of MLC NAND and TLC NAND comparisons… and NAND manufactures once again took the eNAND formula and applied it to their TLC NAND products to create “eTLC” NAND. With Seagate’s Nytro 3332 this means Toshiba BiCS TLC NAND has gotten the enterprise treatment. Based on performance the specific BiCS TLC NAND in the Nytro 3×32 generation is most likely newer 96-layer BiCS 4… but when dealing with controllers running firmware that do not ‘do’ short term burst performance and instead are optimized for consistent long term performance do not take that as gospel. We just are 99.44 percent sure it is BiCS 4 and are hedging our bets a teeny tiny bit as not a single IC had a legible model number we could read.
To help on the temperature front Seagate has further helped things along via a rather conservative approach to NAND IC densities. As you can see this itty bitty “960GB” drive uses a whopping ten NAND ICs. Furthermore, Seagate does not just use heatpads to connect them to the metal exterior chassis they have included a secondary internal heat spreader (and a big reason why the 15 mm-thick 2.5-inch form-factor is being used) that is also connected to the exterior chassis via TIM. This combination provides a lot of mass for the NAND and controller to dump their heat into, and when used in server with typically high CFM airflow results in an SSD chassis design that is best described as a big heatsink. For example, when used in our supermicro 4U based NAS server this drive stayed surprisingly cool even when constantly hammered for days at a time. Of course, the key words are ‘high CFM’ and ‘server case’ as these drives do run warm inside workstation and/or ‘PC’ cases.
Moving on. We would love to tell you precisely what controller Seagate is using for their new Nytro 3332 series, but Seagate is mum on the exact specifications. There are actually few and far between SAS12 controllers out there but with the large ‘Avago’ on it, it is obviously made for Seagate by Avago (aka Broadcom aka LSI aka the company that loves to gobble up the competition and rebrand itself every couple years) and is (most likely) a Seagate inhouse design that came about via their SandForce IP acquisition. To be honest, the exact model and actual controller really does not matter all that much. These days even if it is a SAS controller used in a competitor’s product (rather unlikely), the firmware used matters just as much as the controller. Seagate does not ‘share’ their firmware with others and their low-level tweaks are proprietary.
Much like the recently reviewed IronWolf 510, just with the dial cranked to 11, the Nytro 3×32 series firmware is extremely refined. “Razors Edge” does not even come close how narrowly focused the firmware is on nearly each and every model. So while we can go over some of the basics, there will be vast differences between the various Nytro 3×32 model options.
What each and every model does share in common however is DuraWrite. In its simplistic form, Seagate’s DuraWrite technology (which grew out of their SandForce IP acquisition) means that all data is compressed before it is written to the NAND. The actual amount of compression will vary from data block to data block, but this compression does a few key things. The first is that as more of the drive’s capacity is ‘seen’ as being used the more over-provisioning it actually will have. Remember, before a NAND block can be reused for new writes, it has to be erased and then and only then can it be written to. The more free blocks in a ‘virgin state’ there are available, the longer a drive can sustain 100 percent I/0 demands before slowing down. This in conjunction with an extra 64GB’s worth set aside for Over-Provisioning means this drive can be hammered before ever slowing down and can spread the load out over more NAND resulting in longer lived NAND cells. It just will not be able to do so nearly as long as Nytro 3532 or 3732 which comes with 224GB set aside for OP.
The other major tweak to the firmware is precisely how fast the controller accepts new write requests. With typical consumer SSD’s the controller basically goes flat-out accepting as much as it can right up until its RAM buffer and all its virgin NAND blocks are used. Then and only then will it slow down on accepting new write requests. Considering few home users, or even workstation users for that matter, constantly hammer a drive the idle time is more than long enough for the drive to recover and be ready for the next short data burst to come its way. However, if you saturate the buffer and the OP’s ready to go free blocks… performance plummets. This is why you see ‘up to’ used in performance ratings with consumer (and even many S/MB) orientated drives.
Nytro’s never have and never will do things this way. Seagate knows the extra processor cycles dedicated towards compression and data verification are going to slow things down but they also tweaked the firmware so that writes never exceed what the NAND and controller can do under sustained loads. Yes, this means that short-term performance is going to be lower than the ‘other way’ of doing things, but it not only ensures the NAND is going to be cooler running, and longer lived… it allows them to give a minimum performance guarantee for the various Nytro variants. This may cause some confusion as on paper a Nytro 3332 and a 3532 will have somewhat similar sequential ratings (e.g. 3332 960Gb’er boasts 2150/1000MBps vs. a 3532 800GB’er with 2200/1300 r/w rating)… and one must be careful to look at the IOPs ratings to gain a clearer understanding of what a drive will typically do in the real world. Basically, a 333x model will have its write performance dialed way down so as to not ‘eat’ its NAND nor exceed its lowered amount of over-provisioning, whereas the 353x will not be as aggressively throttled as it comes with a lot more over-provisioning. This is an over-simplification as the firmware in a Nytro 333x will also be tweaked more towards read performance than that found on a Nytro 353x, but gives a decent general overview of how the firmware ‘works’.
For the power loss protection Seagate has opted for a Microchip Technology PIC24FJ48GA004 (aka “PIC24F GA”… aka ‘pick 24 value line’). This 16-bit, 32MHz MCU may be based on an older (circa 2004) design but has been used in a lot of industrial devices over the years. It is used so often as it is an extremely robust and proven design. One that works well, is known to work well, and downright sips power… and is rather reasonably priced at about $10,800 per 5K tray / $2.16 each.
(photo courtesy of https://www.atpinc.com/de/about/stories/microcontroller-SSD-power-loss-protection and is for illustrative purposes only)
This chip handles the power loss data protection by being a ‘man in the middle’ type deal where the voltage coming in from the server is routed through this controller and then routed to the NAND, RAM, and actual storage controller. This not only allows for another level of power condition / voltage regulation to occur, but the instant this Micro-Controller Unit senses a drop in power it immediately boosts the voltages back to their optimum level via the massive array of capacitors. If the voltage drop is a severe enough it also sends a flush cache request to the main controller, and then when the controller confirms all writes requests are not only written but parity checked…. does it ask the controller to do a graceful shutdown of the drive. The end result? No matter if your server has a catastrophic power supply array failure, or just ‘dirty’ power the Nytro 3332 will be able to complete all write requests sent to it and you can be 100 percent sure that nothing will be corrupted along the way. Put another way… this is the big beefy bouncer at the door that keeps Mister Murphy from getting into your party and at the end of the night pushes you into a cab if you are in no shape to drive home.
For the actual capacitors, Seagate has opted for fourteen, 16volt 330µF AVX Capacitors (as an aside the little ‘A’ at the start is AVX’s logo, ‘337’ stands for 33 with 7 zeros or 330 µF (and 336 would mean 33µF), the ‘C’ is voltage rating of 16v, and the next line is the shortened model code… of which AVX has a bajillion all optimized for different roles and scenarios).
For the RAM cache buffer this particular capacity variant (ironically) uses a pair of a Samsung DDR4-2400 1GB ICs (model number K4A8G085WB-BCRC), for a total onboard buffer capacity of 2GB.
The last noteworthy feature is the Nytro 3332 series offers four different variants per model. You have the bog standard version, a version with Secure SED (Self-Encrypting Drive – the versions sent to us for testing), Secure FIPS, and Instant Secure Erase only (aka ISE). In simplistic terms the differences in the three “Seagate Secure Storage” options above the standard version all come down to security and what they (and you) can do. All three options allow for “instant” cryptographic erasure which is so much easier and faster than the old school multi-full drive writes with random 0s and 1 method that is needed with the base model (and a big reason why base models are usually physically destroyed via drill/bullet/thermite/etc when taken out of service… as who has time to do it the long way). Where they differ is in their included feature set.
As the name suggests SED means the drive can self-encrypt and do not require external software for encryption/decryption. Furthermore you (or the IT admin at least) can manage how the security is configured and take ownership of the security (basically you can tell the drive that you are going to use say a fingerprint scanner and to accept that as a legit encryption key). SED w/ FIPS 140-2 adds in NIST (National Institute of Standards and Technology) certification for the cryptographic module. Once again you can manage the security and trigger crypto erasures as well as factory resets. It just is more secure and guaranteed to be meet minimum security specifications. ISE-only models do not allow ‘you’ to take ownership of the security. They handle it and all you can do is reset the drive or crypto erase it.
Typically, ISE-only are going to cost a bit more than the base model, SED costs a bit more again, and SED with FIPS certification will be the costliest version. Typically. There are exceptions and it really can come down to how many are made, what is selling, etc etc on which will cost more than others. So do not just use cost on deciding what option is ‘best’. Our rule of thumb is once we go beyond base models and into the realm of security options… when in doubt we pick the SED model. They typically are a good middle of the road option which may lack certification but are cheaper and still come with advanced configuration abilities… but YMMV and it depends on your requirements and specifications for a given server.
Overall the Nytro 3332 is not just a well-built Solid State Drive it is an overbuilt SSD where phrases such as ‘everything but the kitchen sink’ were thrown around during the design stage. This level of overkill is the way it should be on all enterprise grade drives. Even those intended to spend most of their lives pushing data off and not on to their NAND.
