Ryzen Infinity Fabric

Overview With the release of AMD's 'Zen' based Ryzen series of CPUs a very interesting side-effect has occurred in the PC marketplace. Namely what was once simply marketing spin has become a potential reality. You see a dirty little 'secret' of the PC industry is that RAM speed has very little impact on overall performance compared to putting that extra money towards a better CPU, GPU, SSD. So much so that for most consumers opting for anything much above what the CPU manufacturer designed the CPU around resulted in minor improvements. Oh sure synthetic benchmarks told a different story, but the net improvement for opting for uber-high performance, and uber-high priced, RAM is minor to say the least.

AMD's Ryzen CPU series actually changes all that… and makes RAM speed possibly critically important. This is because AMD made a very interesting design decision and tied their Infinity Fabric speed to the RAM speed. Specifically the Infinity Fabric runs at 1:2 clock speeds of the RAM (or to be even more specific at 1:1 of the IMC's speed setting as DDR4 is double data rate and 'DDR4-2600' is actually running at 1300mhz not 2600mhz as the name suggests). Since the Infinity Fabric is how the two CCX (4 core CPU Complex building block) talk to each other and the rest of the system the higher the transmission speed, the higher the bandwidth… and theoretically the higher the overall performance. This is because the shorter the period of time one block of CPU cores are waiting for data the happier they will be.

Today we are going to investigate exactly how big a deal this really is and what speed of RAM you should opt for… and put some of the savings over an Intel HEDT system towards. To do this we are going to use three different sets of 32GB (4x8GB) DDR4 RAM. Option 1 is going to HyperX Fury DDR4-2133 RAM (model HX421C14FB2K4/32), option 2 will be HyperX Fury DDR4-2400 (HX424C15FBK4/32), and the third is going to be HyperX Fury DDR4-2666 (HX426C15FBK4/32). We have chosen theses three sets of RAM as they are good value kits we have used in the past, offer great capacity, and look sexy as all get out.

Equally important all three sets of DDR4 RAM are 1.2v 4x8GB kits with good timings of C15 or better – so they are fast for their class but not so fast as to require much in the TLC department. Put another way these are low hassle, 'plug and play' kits that should just plain work while not skimping on capacity. As for prices, 32GB of HyperX Fury 2133 will set you back about $220 USD, the 2400 kit will cost you $250 USD, and the 2666 kit is still a very reasonable $290. While this is 'only; a $70 variance across all three sets it is almost enough of a difference but a Crucial MX300 275GB drive which costs about $90 (USD).

We have stopped at DDR4-2666 as beyond this people have to choose between capacity and speed… and honestly anything above DDR4-2666 is a bit of a crapshoot when it comes to working with Ryzen processors and large capacity RAM sticks. So let's warm up our kits, getting our Ryzen 1800X humming along… and see what the actual numbers are and if the added -albeit still very reasonable – expense is worth it or not.

What is Infinity Fabric and why should I care? Unlike most CPU designs AMD's Ryzen series is not a monolithic design. What we mean is that usually a CPU is one design with all cores and cache, and other miscellaneous parts, built into one large 'building'. This is the conservative approach that AMD used for generations – and worked well for them. Instead of taking this conservative approach AMD took a page from their APU teams and created a new way of building CPUs. A way that allowed for easily scaling up or down a 'CPU' core count, abilities, and basically allows them to simply 'swap out a floor' or even add 'another floor' instead of creating an entire new 'building' from the ground up every time they want to tweak the blueprint.

This certainly is a good thing for consumers and AMD alike as it allowed AMD the luxury of creating a 8-core (Ryzen 7) design and then easily scaling it down to Ryzen 5 (6 cores) and Ryzen 3 (4 cores)… and then simply modifying it to create an entire APU line instead of having to build a new blueprint every time they want to meet the demands of a new market. This in large part is why AMD was able to release a 8-core beast that costs only a smidge more than an Intel 4 core design.

The downside however to using this scalable building block method is that not all cores are interconnected. Instead, much like two separate apparent buildings, each 'CCX' block of four cores is internally homogenous but data has to use a 'highway' to get from one block to the other. Equally important is this highway is what connects the CCX blocks to the rest of the system.

This highway is what AMD calls Infinity Fabric. To remove the analogy, it is a 256-wide bi-directional crossbar that is used to simultaneously transport data for multiple buses to their final destination (the cores). Basically it is a next generation HyperTransport/FrontSideBus/internal core bus all rolled into one, and in very basic terms can be considered 'HyperTransport 2.0'. The internal – CCX to CCX – portion is called the 'Control Fabric' and the external – Memory bus, etc – portion is called the 'Data Fabric'. This multi-role responsibility is why it best described as a crossbar and not just bus per say… as it is multi-layered and interconnected bus much like a crossbar interconnector in networking architecture.

For those familiar with Intel designs, think of Infinity Fabric as AMD's version of 'uncore' or 'System Agent' on steroids… as this is the 'data highway' that connects the various major system components to the CPU (including the memory controller) and the 4-core CCX blocks to each other. Yes it is a big deal, and yes it is a critical component to Zen design. In fact you can say it is the key concept that allows Zen – and not just Ryzen – to be so scalable.

This linking of RAM to Infinity Fabric speed is most likely a major reason why AMD Ryzen CPUs (and probably APUs) have a tough time handling speeds in excess of DDR4-2666 with 32GB kits… as the Infinity Fabric is being overclocked right along with the memory. This however to us means that the faster the RAM the more efficient the CPU is going to be.

As one of its jobs is to tie each CCX together and allow them to talk to directly to each other, in certain instances one entire block of cores could be waiting on data to be delivered from the other block and cannot continue until it gets that data. Thus the less cycles that are wasted on waiting for data to 'cross the highway'… the better the overall performance. Now AMD in a rather interesting move linked the speed this connector works at to the speed of the RAM. Specifically, it runs at the exact same speed as the integrated memory controller. Yes, by 'overclocking your ram' you are actually overclocking this interconnect at the same time.

So let's talk specifics. When the Integrated Memory Controller's speed is set to DDR4-2133 this means the Infinity Fabric is running at 1066.5Mhz (as that is the actual clock speed the DDR4 RAM is running at). When using DDR4-2400 it is running at 1200Mhz, and when dealing with DDR4-2666 RAM it is running at a 'blistering' 1300Mhz. Taking one extreme to the other this is a 21.9% speed – and potential performance - difference. To put this in more understandable terms when using DDR4-2133 ram the Infinity Fabric is able to handle (an estimated) 68.26 gigabytes of data per second, whereas at DDR4-2666 it is able to push a whopping 83.2GB of data per second. This however is a lot less than the (estimated) 200GB/s that each core in a CCX block can share data with the others cores in the same CCX block. So yes this is a potential bottleneck and one that may help boost overall performance.

Of course, just because the Infinity Fabric can handle 83.2GB/s (at 2666) instead of 68.3GB/s (2133) does not mean it actually needs that extra speed. To make an(other) analogy if you have a ten-lane highway but only 6 cars are on the road… those extra lanes are not going to matter - as everything is humming along with no traffic jams or slowdowns. Basically if the two CCXs are not wasting enough cycles, and the RAM nor PCIe buses are not a bottleneck, the performance differences will be only noticed in synthetic tests and not the real world.

This is the crux of the matter and how this article came about… as there is a lot of synthetic data proving why X is better than Y… there is very little in the way of tangible real world results proving this potential issue is 'real' or not.

Before we move on the actual testing phase, please do not take any of this out of context. We are not being apologists for AMD here. AMD should never have linked the Infinity Fabric to the IMC speed. In all likelihood this was done so as to simplify the design and allow for easier building of new consumer 'CPUs' or 'APUs' based on what at its heart is a enterprise design – where Infinity Fabric is actually a very sophisticated answer to multi-socket interface. In other words, it was a design hack (in the original meaning of the term – and less forgiving people may go as far as saying 'kludge') that may come back to haunt AMD in the long-term… or not. Nothing says that as this new technology matures that 'Zen 2.0' will be as limited. It is possible that future Zen based models are capable of having the Infinity Fabric combined with the IMC running at uber-high speeds without issue. In the short term though this is a growing pain, and something consumers thinking about building a Ryzen based system need to be aware of and know how big – if at all – this issue really is.

AMD Ryzen 7 Testbed Ryzen Testbed:

• Motherboard: MSI X370 SLI Plus • CPU: AMD Ryzen 7 1800x • Cooling: EVGA CLC 280 AIO • Video Card(s): MSI GTX 980Ti Lightning • SSD: 1x Crucial MX300 2050GB, 2x Intel DC S3700 800GB SSDs • Power Supply: Corsair AX860i • RAM: HyperX 32GB (4x8GB): HX421C14FB2K4/32 DDR4-2133 HX424C15FBK4/32 DDR4-2400 HX426C15FBK4/32 DDR4-2666

Application Benchmarks Adobe Photoshop CC 2017

Adobe Photoshop is a raster graphics editor developed and published by Adobe Systems for macOS and Windows. To further make loading slower we have included a bunch of extra brushes that need to be loaded before the program is ready to use.

Adobe Premiere Elements 2015

Adobe Premiere Elements is a video editing software published by Adobe Systems. It is a scaled-down version of Adobe Premiere Pro and is tailored to novice editors and consumers. The entry screen offers clip organization, editing and auto-movie generation options. To stress the CPU we have disabled GPU rendering and instead Premier will use the CPU only.

CINEBENCH R15

CINEBENCH is a real-world cross platform test suite that evaluates your computer's performance capabilities. CINEBENCH is based on MAXON's award-winning animation software CINEMA 4D, which is used extensively by studios and production houses worldwide for 3D content creation.

GeekBench Benchmark

Geekbench is a cross-platform processor benchmark, with a scoring system that separates single-core and multi-core performance, and workloads that simulate real-world scenarios

POV-Ray Benchmark

The Persistence of Vision Ray Tracer, or POV-Ray, is a ray tracing program which generates images from a text-based scene description, and is available for a variety of computer platforms.

winRAR Benchmark

WinRAR is a popular file compression and decompression tool that also happens to be multi-threaded application. While it does have a built in benchmark we have opted to use a custom 5GB test consisting of small, medium, and moderately large files. We then time how long it takes to compress the files into one .rar file with all WinRAR settings left at their defaults.

x264 HD Benchmark

x264 HD Benchmark is a benchmark that allows you to measure how fast your PC can encode a 1080p video clip into a high quality x264 video file. It allows for an easy comparison because everyone running it will use the same video clip and software. The x264 video encoder has a fairly accurate internal benchmark (in frames per second) for each pass of the video encode and it also uses multi-core processors very efficiently. All these factors make the x264 HD Benchmark an ideal tool in comparing the video encoding performance of different processors and systems.

Gaming Benchmarks Assassins Creed: Syndicate Gaming Benchmark

Assassins Creed: Syndicate is an historical action-adventure video game, published by Ubisoft and released in October 2015. Like Battlefield 4 this games does not include an in-game benchmark. This makes it perfect for more real world gaming testing. To obtain repeatable results we have used FRAPs and recorded a custom run through Thames River section starting on St. Pual's Pier. An average of four runs was taken.

The settings used in the testing below are stock highest settings for quality, VSync disabled and a resolution of 1920x1080.

Battlefield 4

Battlefield 4 is first person shooter video game, published by EA Digital Illusions CE and released in October 2013. Unlike most, this games does not include an in-game benchmark. This makes it perfect for more real world gaming tests. To obtain repeatable results we have used FRAPs and recorded the first 90 seconds of the single player Tashgar level. An average of four runs was taken.

The settings used in the testing below are Ultra pre-set for quality, with VSync disabled and a resolution of 1920x1080.

Crysis 3 Gaming Benchmark

Crysis 3 is a first person shooter video game, published by Electronic Arts and released in February 2013. While older than some of the others it is one of the most visually stunning games released to date and puts a lot of demands on the GPU. This makes it perfect for more real world gaming testing. To obtain repeatable results we have used FRAPs and recorded 90 seconds of the single player ‘Post Human' level, starting as soon as soon as prophet is handed a Hammer II pistol by Psycho. An average of four runs was taken.

The settings used in the testing below are highest settings for quality, VSync disabled and a resolution of 1920x1080. Texture Quality, Game Effects, Objects, Particles, Post Processing, Shadows, Shading, Water, and System Specs all set to Very High. Motion Blur was set to High and Lens Flare was set to On. Anti-Aliasing was set to MSAA 8X and Antistrophic Filtering was set to 16x.

Grand Theft Auto 5 Gaming Benchmark

GTA V is an open world action-adventure video game published by Rockstar Games and released in April 2014 for the PC. The game has a benchmark component to it that mimics game play and an average of four runs was taken.

The settings used in the testing below are stock 'very high' settings for quality, VSync disabled and AA set to off, with a resolution of 1920x1080.

Metro: Last Light Gaming Benchmark

Metro: Last Light is a DX11 first-person shooter video game developed by Ukrainian studio 4A Games and published by Deep Silver released in May 2013. The game is set in a post-apocalyptic world and features action-oriented gameplay. The game has a benchmark component to it that mimics game play. Scene D6 was used and an average of four runs was taken.

The settings used in the testing below are Very High for quality and a resolution of 1920x1080. DirectX 11 is used, texture filtering is set to AF 16X, motion blur is normal, SSA turned on and tessellation is set to high.

Performance Summary As you can see that there is indeed a difference but it is not all due to Infinity Fabric speed. The performance increase is not linear. Some of the difference is due to different RAM frequency speed being used. RAM speed does impart a small percentage difference to overall CPU performance in real world scenarios and RAM timings (CAS Latency 14 vs 15) also imparts a small performance gain/loss.

This to us means there is two possibilities, but the end result is identical. The first option is that Infinity Fabric speed is not that big a deal when dealing with sub-DDR4-3000 32GB kits of RAM – Ryzen 7 is still fast enough to compete against the best Intel has to offer. The alternative is that Infinity Fabric is still being starved for bandwidth (and thus the two CCXs) regardless of the speed the IMC is running at, and only when dealing with uber-high kits of RAM will it possibly matter to the average consumer. Considering using DDR4-3000 (and higher) 32GB kits are hit or miss with Ryzen (at this time) the answer once again does not matter. It is a moot point as the Infinity Fabric is tied to the IMC.

Until AMD obtains more experience with DDR4 Integrated Memory Controller architecture design, or decouples these two critical CPU components, this is an interesting issue but should not be overly concerning to the average buyer. As such we would take a practical approach to this issue… and use the fastest RAM your budget can justify and will not cause issues.

Closing Statement As you can see from the previous pages results there is indeed 'something' to the whole 'RAM speed equals better Ryzen CPU performance' debate. Under certain scenarios overclocking the Infinity Fabric does indeed pay tangible dividends. However, this tangible increase in performance is not an across the board phenomena, and it really will depend on what you plan on doing with your shiny new AMD Ryzen 7 system.

For instance if you have just built a PC Gaming orientated Ryzen system the speed of the RAM will probably not make a noticeable difference; rather than getting faster RAM a better GPU or a SSD is a more optimal use of your hard earned dollars. The same holds true of typical home user scenarios such as application & OS load times where putting the extra cash towards a SSD makes more sense. Is there an improvement? Absolutely! Will most notice it? Probably not. Once again if it comes down to faster but less RAM or faster RAM but a slower GPU/SSD/etc. and the wise decision is to get the slower RAM. For these readers the HyperX Fury DDR4-2133 series is the best use of your budget. Buy it and enjoy knowing that you are not really giving up (much) perceptible real-world performance by going for the lower speed RAM kit. You just will just not be stressing enough cores - at the same time - to be concerning.

So who will see noticeable real world differences? Power users who not only push their systems to the max but are using applications that involve a lot of number crunching across a bunch of cores. Here the increased speed of both the RAM and Infinity Fabric does indeed pay noticeable, justifiable dividends. If you want to shave seconds or even minutes – depending on the file size – off your video transcode times the added expense of the HyperX Fury DDR4-2666 kit is more than reasonable and it is indeed a wise purchase decision. In fact it is a downright bargain as the investment in faster RAM will pay dividends immediately. The same holds true of those who are building a Ryzen 'budget is not a concern' build. If budget does not enter the equation… then it is a no-brainer. Get the higher speed RAM, dual GPUs, a NVMe Solid State Drive (or two)… and everything else on your wish list.

Of course the real world is not as black and white as we are making it out to be. Most users don't really know what they are going to be using their new rig for. Others do know but do not want a dedicated system for doing 'X'. Instead a lot of you reading this article are looking for an all-round PC that can do a little bit of everything and will be doing a bunch of different things depending on the day… or even time of day. For those of you who fall into this category we would recommend a compromise approach. Kits like the HyperX Fury DDR4-2400 are a bit more expensive than their DDR4-2133 counterparts but it is only $30. This is not going to bust many budgets nor require sacrifices in other departments (if it does then go for a slower kit). Yes the added speed boost is not going to be as great as if you had opted for the DDR4-2666 kit but it will be close enough for most… and only cost you $30 more than the 2133 option. Basically you will be hedging your bets so to speak. Spending a bit more than possibly necessary but not so much as to be a major budgetary concern.

So there you have it. A lot of people will be fine with DDR4-2133; others need DDR4-2666 (and will be wishing that even higher speed, high capacity DDR4 RAM kits 'just work' with Ryzen 7 processors); but when it doubt play it safe. Spend a little bit extra for DDR4-2400. Right now DDR4-2400 is probably the 'value sweet spot' for AMD Ryzen systems when it comes to 32GB kits… and what you probably should be looking for in your next Ryzen build.