There's a future where you and your graphics card decide what a game looks like, not the developers

HardwareGraphics CardsThere’s a future where you and your graphics card decide what a game looks like, not the developersWhen you purchase through links on our site, we may earn an affiliate commission.Here’s how it works.

When you purchase through links on our site, we may earn an affiliate commission.Here’s how it works.

(Image credit:Bioshock with ultra-realistic graphics Gen-3 video to video Runway Artificial intelligence (youtube.com))

An image of Elizabeth from Bioshock Infinite generated by the Gen-3 video-to-video tool in Runway ML

Recent updatesUpdate October 12, 2024:Just wanted to jump in here and note that I’m aware of the negative reception this piece has received since its publication, and that I take it onboard. I do want to acknowledge that in my excitement over the continual evolution of the tech used to create and run PC games, I failed to balance that out with the wider context of the negative impact generative AI is having on the industry and the world at large. Creatives have lost, are losing, and will continue to lose their livelihoods due to the use and misuse of generative AI, and that is a live consequence of such excitement as mine in the AI industry. My only intention here was to extrapolate on where I see graphics and game technologies going with the almost inevitable expansion of AI usage within current technologies such as Freestyle, Frame Gen, and DLSS.

Recent updates

Update October 12, 2024:Just wanted to jump in here and note that I’m aware of the negative reception this piece has received since its publication, and that I take it onboard. I do want to acknowledge that in my excitement over the continual evolution of the tech used to create and run PC games, I failed to balance that out with the wider context of the negative impact generative AI is having on the industry and the world at large. Creatives have lost, are losing, and will continue to lose their livelihoods due to the use and misuse of generative AI, and that is a live consequence of such excitement as mine in the AI industry. My only intention here was to extrapolate on where I see graphics and game technologies going with the almost inevitable expansion of AI usage within current technologies such as Freestyle, Frame Gen, and DLSS.

Wait! Before you click away because of the merest mention of an AI generated gaming future—I know, I know, I’m largely generative AI fatigued, too—this is actually potentially super exciting when it comes to actually having games that look kinda real.

If your algorithm looks anything like mine (and what a weird turn of phrase that is) then you’ll have been seeing AI-generated gaming videos all over TikTok and YouTube, where traditional game capture is fed through a video-to-video generator and out the other end comes some seriously uncanny valley vids.

But they’re also pretty spectacular, both from how they look now and from what they could mean for our future. If this sort of AI pathway could be jammed into our games as some sort of final post processing filter that’s something to be excited about. These examples here are obviously not real-time AI filters—and that’s going to be the real trick—they’re just taking existing video and generating a more realistic version with that as the input reference point.

But when you look at what it does with Half-Life, Red Dead, and GTA, though, the results are pretty outstanding.

What’s going on right now is content creators, such asYouTuber Soundtrickwho produced these videos, are taking existing game footage and the running it through the Gen-3 video-to-video tool on Runway ML. It’s then generating its own video based entirely off that input video, and people are taking a ton of different games and feeding them into the AI grinder to come out with ‘remastered’ or ‘reimagined with AI’ footage of the same games.

And, while they often look janky af, it’s not hard to see the potential in this form of generative AI if it’s given enough processing power and is somehow able to be done in real-time. Think of what it would look like as a post-processing layer in a game engine, a layer that could interpret a relatively basic input and produce a photorealistic finished product.

The biggest gaming news, reviews and hardware deals

It’s actually something that’s kinda been bothering me as a football-liker and FIFA/FC player—why doesn’t it look like the real thing, and how do we ever get there? With AI fed on the ton of televised football matches shown every second of every day around the world, there’s more than enough data now to train a single model that could make the game look identical to a real-world football match.

Suddenly standard rendering gets turned on its head and ray tracing gets shown the door. Who needs realistically mapped light rays when you can get some artificial intelligence to make it up? I’ll happily go back to faked lighting if it actually looks more realistic.

The actual GPU component of your graphics card will become secondary, needing only to do some rough rendering of low-res polygons with minimal, purely referential textures, just for the purposes of character recognition, motion, and animations. The most important part of your graphics card will then be the memory and matrix processing parts, as well as whatever else is needed to accelerate the probabilistic AI mathematics needed to rapidly generate photorealistic frames once every 8.33 ms.

Half Life with ultra-realistic graphics Gen-3 video to video Runway ML Artificial intelligence - YouTubeWatch On

Half Life with ultra-realistic graphics Gen-3 video to video Runway ML Artificial intelligence - YouTube

OK, none of that is easy, or even definitively achievable when it comes to working with low-latency user inputs in a gameworld versus pre-baked video, but if that becomesourgaming reality it’s going to be very different to the one we inhabit now.

For one, your gaming experience is going to look very different to mine.

If a subsequent generation of DLSS starts using a generative AI post processing filter then that will no longer be the case. Nvidia would obviously stick with its proprietary stance, which would use specific model and training data for its version, and that would end up generating different visual results to a theoretical open source version from AMD or Intel, which operated on differently trained models.

Hell, it being generative AI, even if you were using the exact same GPU there’s a good chance you’d have a different visual experience compared with someone else.

And maybe that would be deliberately so.

Nvidia already has itsFreestyle filtersyou can use to change how your games look depending on your personal tastes. Take that a little further down the AI rabbit hole and we could be in a situation where you’re guiding that generative AI filter to really tailor your experience. And I’m kinda excited about what that would mean playing your old games with such technology, too.

GTA IV with ultra-realistic graphics Gen-3 video to video Runway ML Artificial intelligence - YouTubeWatch On

GTA IV with ultra-realistic graphics Gen-3 video to video Runway ML Artificial intelligence - YouTube

Wanna replay GTA V, but have a hankering for more of a Sleeping Dogs vibe? Tell the filter that you want it to be set in Hong Kong instead of Los Santos.

Because, theoretically—in my own made up future, anyway—you could layer the AI post processing over an older game and have yourself a pretty spectacular remaster. Or, more interestingly, have a completely different set of visuals. Wanna replay GTA V, but have a hankering for more of a Sleeping Dogs vibe? Tell the filter that you want it to be set in Hong Kong instead of Los Santos. Want everybody to have the face of Thomas the Tank Engine? Sure thing. Why not replace all the bullets with paint balls while you’re at it?

It’ll put modders out of business, but it’s also not going to go down well with game artists, either. Poor schmucks. But it’ll put us in control of at least one aspect of a gameworld if we want to.

This is a future which doesn’t feel all that far away from when I’m looking at photo-realised versions of Professor Kleiner or Roman Bellic layered over their own character models. Though there are obviously some pretty hefty hurdles to leap before we get there. Memory will be a biggie. The models, in order to achieve the sort of latencies we’re talking about here, would likely need to be local; stored on your own machine. And that means you’re going to need a ton of storage space and memory to be able to run them fast enough. Then there’s the actual AI processing itself. And as we get closer to such a reality there will probably be some even more janky, uncanny valley moments to suffer through, too.

But it could well be worth it. If only to replay Half-Life all over again.

TOPICSHardware

TOPICS

More about graphics cardsIs the new RTX 5070 really as fast as Nvidia’s previous flagship RTX 4090 GPU? Turns out the answer is yes. Kinda.It’s time for me to admit that AI-accelerated frame generation might actually be the way of the future and that’s a good thingLatestThe Last of Us season 2 trailer is a brief look at a story you should probably just play yourself in AprilSee more latest►

More about graphics cards

Is the new RTX 5070 really as fast as Nvidia’s previous flagship RTX 4090 GPU? Turns out the answer is yes. Kinda.It’s time for me to admit that AI-accelerated frame generation might actually be the way of the future and that’s a good thing

Nvidia RTX 5070 graphics card

Is the new RTX 5070 really as fast as Nvidia’s previous flagship RTX 4090 GPU? Turns out the answer is yes. Kinda.

Nvidia RTX 5090 Founders Edition rendered on a green background.

It’s time for me to admit that AI-accelerated frame generation might actually be the way of the future and that’s a good thing

LatestThe Last of Us season 2 trailer is a brief look at a story you should probably just play yourself in AprilSee more latest►

Latest

The Last of Us season 2 trailer is a brief look at a story you should probably just play yourself in April

the last of us season 2 trailer

The Last of Us season 2 trailer is a brief look at a story you should probably just play yourself in April

See more latest►

Most Popular

The Witcher 3’s now 2-year-old bonus quest is our first taste of the ‘vibe’ CD Projekt is going for in The Witcher 4

2024 was the year updates for old games beat out all the new ones for me

Train like you game with this adventure-inspired workout

‘It’s simply impossible to make a difficulty level that’s just right for all players’: How Final Fantasy 14’s lead battle designer has been playing a precarious balancing game for Dawntrail’s dungeons and raids

Please join me in getting super excited for all the cool looking survival games coming in 2025 (and beyond)

Competitive shooters are at a crucial crossroads in 2025: ‘sweaty’ teamplay vs. casual fun

Call of Duty’s $28 Squid Game skins are the perfect crossover for our capitalist dystopia, and Activision knows exactly what it’s doing

These are the 14 biggest upcoming RPGs of 2025—get ready for another amazing year for the genre

Five new Steam games you probably missed (January 6, 2025)

I’ve seen enough: No more forcing singleplayer studios to make mediocre live service games

HARDWARE BUYING GUIDESLATEST GAME REVIEWS1Best Steam Deck accessories in Australia for 2025: Our favorite docks, powerbanks and gamepads2Best graphics card for laptops: the mobile GPUs I’d want in my next gaming laptop3Best mini PCs in 2025: The compact computers I love the most4Best 14-inch gaming laptop: The top compact gaming laptops I’ve held in these hands5Best Mini-ITX motherboards in 2025: My pick from all the mini mobo marvels I’ve tested1Thank Goodness You’re Here! review2Shiren the Wanderer: The Mystery Dungeon of Serpentcoil Island review3WD Black SN850X 8 TB NVMe SSD review4Ikea Utespelare desk review5Asus ROG Harpe Ace Mini wireless mouse review

HARDWARE BUYING GUIDESLATEST GAME REVIEWS

1Best Steam Deck accessories in Australia for 2025: Our favorite docks, powerbanks and gamepads

A bunch of the best Steam Deck accessories on a blue background.

1Best Steam Deck accessories in Australia for 2025: Our favorite docks, powerbanks and gamepads

Best Steam Deck accessories in Australia for 2025: Our favorite docks, powerbanks and gamepads

2Best graphics card for laptops: the mobile GPUs I’d want in my next gaming laptop

Best graphics card for laptops: the mobile GPUs I’d want in my next gaming laptop

3Best mini PCs in 2025: The compact computers I love the most

Best mini PCs in 2025: The compact computers I love the most

4Best 14-inch gaming laptop: The top compact gaming laptops I’ve held in these hands

Best 14-inch gaming laptop: The top compact gaming laptops I’ve held in these hands

5Best Mini-ITX motherboards in 2025: My pick from all the mini mobo marvels I’ve tested

Best Mini-ITX motherboards in 2025: My pick from all the mini mobo marvels I’ve tested

1Thank Goodness You’re Here! review

Thank Goodness You’re Here review

1Thank Goodness You’re Here! review

Thank Goodness You’re Here! review

2Shiren the Wanderer: The Mystery Dungeon of Serpentcoil Island review

Shiren the Wanderer: The Mystery Dungeon of Serpentcoil Island review

3WD Black SN850X 8 TB NVMe SSD review

WD Black SN850X 8 TB NVMe SSD review

4Ikea Utespelare desk review

Ikea Utespelare desk review

5Asus ROG Harpe Ace Mini wireless mouse review

Asus ROG Harpe Ace Mini wireless mouse review

The biggest gaming news, reviews and hardware deals#

The biggest gaming news, reviews and hardware deals