As we are sure nearly everyone is well aware of the new RTX ‘Turing’ TU10x series is not like any video processing core NVIDIA has released previously. Instead of the entirety of the core being dedicated solely towards ‘old school’ video creation the new one takes a different approach. This is why a goodly chunk of the massive TU core is not dedicated towards CUDA cores, instead it is used for two other things. The first is Ray Tracing and the second is Tensor.
To understand Ray Tracing and why it is so different – and requires its own unquiet method of calculation of performance – let’s start with the older method: rasterization. To create seemingly three-dimensional objects in a 2D screen, game development software use polygons to create ‘3D’ objects. This creates a 3D ‘mesh’ of the environment and gives basic form to everything in the virtual world. The more polygons used on a given object… the more details it has. At this point the 3D world as form, but no color nor depth. To add realism the software then overlays things like color, texture, even fake 3D dimensionality via shading… all of which are based on how the various polygons are lined up against the ‘camera lens’ (i.e. your screen). All this data on how to build a given image is then sent to the video card to do a ton of math and spit out a single image of the virtual world. Every time the ‘camera’ moves in this 3D world or something in the 3D world moves the interaction of these virtual meshes change. Resulting in new calculations to create the new image.
With modern cards, and years of low-level work on software rasterization techniques, this results in pretty decent frame rates and pretty darn good-looking images (compared say Half Life 2 with a modern 2018 AAA title to see the advancements!). Where it falls down on the job is that many things we take for granted in the real world have to be manually added in and are thus static ‘blobs’ on the screen with no interaction of the things around it. This is especially true for light sources. For example, if you want a reflection of say flames on a car as it drives by a fire the software has to manually add in fake flames on the 3D mesh of the car, the building, and anything else the software dev team thinks it needs to add in reflections to. The same holds true of shadows – and why shadow effects are extremely GPU intensive and still can look rather fake compare to real world shadows.
Basically, it is like painting a picture from scratch every frame, with every tiny detail having to not only be painted in by the video card but told to be painted in by the software. The higher the resolution the more paint strokes it takes to paint that picture. Put another way it is a lot of math, and a lot of work for developers to get a virtual world that looks even halfway realistic.
Instead of doing things this way Ray tracing does things much like the real world and your eyes does things. Right now, wherever you are, there is countless beams of light bouncing around your environment, combining and then bouncing back into your eyes. This is massive amounts of bouncing light and its interaction is ray tracing in a nutshell.
How it does this is it basically considers your 2D screen as the lens of a camera that the software ‘shoots’ light beams through, around and out of and into the virtual world. These light beam paths are calculated (i.e. light ‘ray tracing’) and as they bounce off, or are blocked by the created ‘things’ in the virtual world (all of which are not painted over like in rasterization but rather have various software definitions on how they react when hit with light). On top of this it also calculates even how the light beams interact with other existing light sources in the virtual world (e.g. a light bulb in front of your avatar, a fire off to the side of the screen, etc.) before bouncing back to the screen. Rinse and repeat for every possible ray and its flight path that could reach your ‘eye’… and the resulting image appears on your screen.
The amount of math needed to do all this is insane – so much so it makes rasterization vs ray tracing look like a comparison of kindergarten math vs calculus. The upside is a lot less work has to be done by the developer to ‘get’ light reflections, shadows, and a more realistic looking image. The downside… is it is a ton of math. So much math even powerful GeForce GTX 1080Ti card cannot do even a single frame per second at higher resolutions (as it only offers 1.1 GigaRays/s). Even now a GeForce RTX 2080Ti is only capable of doing 10 GigaRays per second (or 10 billion light ray calculations per second, which may sound like a lot but really isn’t) and is going to be hard pressed at higher resolutions to provide a ‘smooth’ gaming experience with frame rates that people have come to expect with 1K plus video cards. As a ‘cheat’ ray tracing in its current form doesn’t really trace every light ray. Instead it only traces the ones that would reach the ‘camera lens’ (this is why we said RT is like beaming light out of the camera lens not in to the lens).
Needless to say, Ray Tracing is an incredibly difficult way of creating a video game image… and doing it at enough frames per second to be playable is even more so. The upside is that when done properly – and not overly ‘look at this! it is RAY TRACING!!!’ by the developer team – you can get much more ‘photo realistic’ images. This in turn can theoretically improves gaming immersion and overall game enjoyment. Yes, it is possibly a big deal… at some point in the future.
We say ‘future’ as at this time the number of video games which has been released with the low-level coding required to send the proper ‘data’ to the RT portion of the core… can be counted on one hand. Time will tell if RT takes off or not. In all likelihood it will, but by that time RTX 30 or even 40-series will be on the scene. Game developing is a multi-year process these days so it will be at least a year or two before enough games with Ray Tracing hit the market to even notice, and most likely will be even be further off before being even remotely considered the ‘new’ standard of game development.
Ray tracing is however only the first round in the magazine. The next is the Tensor cores that have been added. Tensor cores are actually not a new idea and in fact come from NVIDIA’s workstation sector. What these turning tensor cores do is rather complicated. But a short Coles Notes version is they allow the RTX 20-series the luxury of offloading a lot of work from the ‘other’ part of the TU100 series core to these new cores. Of the offloaded work one is a standout feature that may, or may not, be a game changer. This is DLSS or Deep Learning Super Sampling.
DLSS was previously called DLAA or Deep Learning Anti-Aliasing when it was first announced. Regardless of what NVIDA calls it now, DLSS is not ‘Deep Learning’ in the truest sense of the word. What it is, is pre-optimized algorithms for specific games that can be offloaded to the Tensor cores. These algorithms are not trained/learned on your GPU… as your shiny new GPU doesn’t have the horsepower for true Deep Learning. Instead DLSS algorithms are ‘trained’ or optimized via the use of a super computer at Nvidia.
While NV is mum on details what this most likely means is that they take 1080P, 1440P and 4K images and compare the upconverted image to the native image. It looks for where it guessed wrong, tweaks the alog and tries again. Rinse and repeat… a lot and then NV’s Deep Learning can distill this down to rules that will work on the tensor cores for that specific game. Then NVIDIA includes these tweaks for that specific game in the next graphic driver update. So while it really is not ‘Deep Learning’ (and you need not worry about your computer one day saying “I’m sorry Dave I cannot do that”… or even just judging you on your web browsing… ‘habits’) what it does however promise is to not only offload next gen Anti-Aliasing to the tensor cores, freeing the rest of the core up for other duties, but also allows for improved frames per second.
This improvement in FPS is due to two things. Firstly, DLSS is more of a pseudo super sampling technique compared to earlier generations in that DLSS algorithm knows when a given antialiasing level is required for a given chunk of a rendering (e.g. 8X or eight times subsampling) and when it is not (and uses a lower subsampling… say 2X). In addition to on the fly AA changes DLSS also allows either the CUDA core portion (for rasterization) or RT cores (ray tracing) to create a lower resolution image than what the game / monitor is set to. For example, during the NV keynote speech where the games were running ‘4k’ some of the DLSS portions of the workload were most likely being run internally at 1440P and then upconverted to 4K before being pushed into the buffer for output. This combination of ‘trained’ up-conversion with ‘on the fly’ changing of how much subsampling the core was doing is how the Key Note highlights were able to show such a massive FPS improvement over a 1080Ti’s ‘TAA at a given setting and resolution all the time’ performance results.
Is DLSS ‘cheating’? Yes and no. Yes, it is because NV was comparing apples to oranges. But not really when the final output looks as good as what the previous gen could do. Increased FPS is increased FPS. Pretty images are pretty images. The downside to when DLSS gets the settings wrong the output will look worse than what it would look like when not using DLSS. This is also where the where the machine learning comes into play as the algorithms should know how to keep it from looking too wrong.
Will DLSS end up like PhsyX or HairWorks in that it is a cool little idea but infrequently used by software developers and consumers? Only time will tell, but from a technical point of view it may help the RTX 20-series last longer during its twilight years (i.e. when the RTX 30 or 40-series is on the scene) by allowing the now ‘old’ card to play ‘modern’ games at higher resolutions and at playable framerates. To imagine this, think about trying to play a modern AAA 2018 game on max resolution and eye candy… but with a GeForce GTX 770. Then imagine trying to play that selfsame game on the selfsame video card but at 1080P. Sure frame rates will not be as great as a ‘modern’ card but they will be playable. That is what DLSS has to offer… if it catches on.
We say ‘if’ as it will take developers enabling it… even developers who are not interested in being part of NV’s “The Way It’s Meant To Be Played” program. This may be a hard sell as DLSS is proprietary and will not work with potentially half their customer base (AMD users). That is a lot of manhours for potentially little gain. DLSS could further balkanize the game industry and make choosing a video card even more difficult. So time will tell on how often DLSS is even an option for future games. It does however have the highest realistic chance of hitting critical mass compared to Ray Tracing during the RTX 20-series lifespan.
