Is 3D Gaussian Splatting the Next Evolution of Video Game Graphics or Another Niche?

Imagine playing a video game that looks so realistic that you can hardly tell the difference from a photograph. Imagine exploring a virtual world that captures every detail of a real scene, from the reflections of light to the textures of materials. Imagine doing all this with the same graphics card that you use for your current games.

This might sound like a distant dream, but it could soon become a reality thanks to a new graphic rendering technique called 3D Gaussian Splatting. This technique, developed by researchers from Inria and ETH Zurich, allows developers to produce photorealistic graphics in real time using millions of particles that mimic the behavior of light and materials in a realistic manner. Is it really the next evolution of gaming graphics, or will it turn into another niche in the sea?

How Does 3D Gaussian Splatting Work?

The technique uses 3D Gaussians, which are fuzzy spherical balls that exhibit a 3D Gaussian distribution, to represent a scene. Each 3D Gaussian has a position, a rotation, and a non-uniform scale in 3D space, as well as a color and a transparency. By projecting these onto a 2D image space, the technique can create realistic images that capture the appearance and occlusion of the scene.

The technique is based on the idea of radiance fields, which are continuous functions that describe the color and direction of light at every point in space. Radiance fields can be used to synthesize novel views of a scene from multiple photos or videos, but they are usually very expensive to compute and store. This problem is solved by using 3D Gaussians as a compact and efficient representation of radiance fields, which can be optimized and rendered on the GPU.

The Advantages of 3D Gaussian Splatting

One of the main advantages of this new technique is that it can handle unbounded and complete scenes, rather than isolated objects or bounded volumes. This means that it can render large and complex environments with high visual quality and low memory consumption.

Another advantage is that it can support anisotropic splatting, which means that the 3D Gaussians can have different shapes and orientations depending on the scene geometry and lighting. This allows it to handle complex lighting effects, such as reflections, refractions, shadows, and global illumination.

A third advantage is that it can be easily integrated with existing rendering pipelines and engines, such as Unreal Engine.

3D Gaussian Splatting can be used in video games to create immersive and realistic environments that can be explored from any angle and viewpoint. For example, it can be used to render scenes captured with multiple photos or videos, such as historical landmarks, natural landscapes, or urban settings. It can also be used to render synthetic scenes created with procedural or artistic methods, such as fantasy worlds, sci-fi scenarios, or abstract designs.

Of course the best way to give you an idea of how amazing this new technique will be for the gaming industry, is to look at actual tech demos.

Will 3D Gaussian Splatting Become Another Niche in the Gaming Industry?

3D Gaussian Splatting might not be an evolution in video game graphics, but rather a niche for certain types of games. One reason is that the technique is best suited for static or slow-moving scenes, where the particles do not change much over time. This might restrict its use to games that focus on exploration, simulation, or storytelling, rather than action, combat, or racing.

Moreover, graphics that are too realistic could actually take the fun out of some types of games. For example, some games rely on stylized or abstract graphics to create a distinctive aesthetic or mood, such as Limbo, Journey, or Minecraft. These games would lose their charm and identity if they were rendered with photorealistic graphics.

Furthermore, some games use unrealistic graphics to enhance the gameplay or the immersion, such as Portal, Superhot, or Half-Life: Alyx. These games would become less enjoyable or believable if they were rendered with 3D Gaussian Splatting. Therefore, realism is not always the ultimate goal of video game graphics, and different techniques might suit different genres and preferences.

3D Gaussian Splatting could be a game-changing technique that could revolutionize the way graphics look in video games forever. By using 3D Gaussians as a novel representation of radiance fields, it can achieve photorealistic graphics in real time with high fidelity and low cost. It can also enable new forms of artistic expression and creativity in video game design.

Imagine a sports game like NBA 2K, which currently has a very outdated graphics engine, using this technique to render NBA players that look truly like real life.