Nvidia's eyeing up ways to beef up its future AI chips with GPUs on top of GPUs and the power of light

HardwareGraphics CardsNvidia’s eyeing up ways to beef up its future AI chips with GPUs on top of GPUs and the power of lightWhen you purchase through links on our site, we may earn an affiliate commission.Here’s how it works.

When you purchase through links on our site, we may earn an affiliate commission.Here’s how it works.

(Image credit: Nvidia)

Images of Nvidia�s Blackwell GPU from GTC.

I’m not usually one for extravagant tech and hardware predictions—things can change so much and so unpredictably that it’s difficult to put too much weight behind such premonitions. But when it comes to AI tech, things move so quickly that what seems distant might not actually be too far away. Combine this with Nvidia being the one outlining the futuristic vision, and I take it a little more seriously.

Nvidia’s latest prediction, as outlined at the IEDM 2024 conferenceaccording to Dr. Ian Cutress(viaTechPowerUp), is AI accelerators that are 3D-stacked and that use—at least in part—silicon photonics for data transmission. This is, as Cutress puts it, Nvidia’s vision of “the future of AI compute”.

The image provided in the post shows an AI accelerator (ie, a datacentre GPU) that’s split vertically into a substrate, integrated silicon photonics, GPU tiers, 3D stacked DRAM, and cold plate.

The two big innovations in this picture, insofar as they could be applied to AI accelerators, are silicon photonics and the vertical stacking for logic. The former uses photons (light) to transmit data to and from optical components, which is faster and uses less power for more bandwidth than traditional electrical data transmission.

Here’s @NVIDIA’s vision of the future of AI compute.Silicon photonics interposerSiPh intrachip and interchip12 SiPh connects, 3 per GPU tile4 GPU tiles per tierGPU ‘tiers’ (GPU on GPU?!?)3D Stacked DRAM, 6 per tile, fine-grainedFrom #iedm24. My guess, 2028/2029/2030… pic.twitter.com/5IsDkYSWT2December 8, 2024

However, TechPowerUp says these accelerators feature “12 SiPh [silicon photonics] connections for intrachip and interchip connections, with three connections per GPU tile across four GPU tiles per tier”. And “intra-chip connection” would seem to imply connection between each of these tiles within each tier, too.

The diagram says there’s anelectrical(not optical) interconnect from die-to-die and tier-to-tier, which would suggest it’s using more traditional Through-Silicon Via (TSV) tech to get the vertical stacking done.

The biggest gaming news, reviews and hardware deals

Silicon photonics is stillonly in its infancy. It might make more sense for Nvidia to use Through-Silicon-Via (TSV) technology for the vertical dimension, which essentially involves creating tiny tunnels as pathways between the stacked chips. This is the technology that allows theAMD Ryzen 7 9800X3D, for example, to have its processor sitting on top of its cache.

Though usually we see 3D stacked chips limited to cache on logic, i.e. L3 cache on cores as per AMD’s chip, rather than logic on logic, which is suggested here.

It looks like four GPU tiles will exist per GPU “tier”, and these tile tiers will be stacked vertically, too. Then on top of all of that, stacked DRAM. That all sounds like it’d get extremely toasty, and it’s not something I’d expect to be achieveable in the immediate future.

Whatever the case, it’s certainly an interesting picture of what might be to come, and if anyone’s able to do it it’ll be Nvidia. And while we shouldn’t infer too much about these technologies making their way across to gaming GPUs any time soon, it wouldn’t be unreasonable to assumesomeof it might, atsomepoint.

If the technology’s there and gets implemented in AI accelerators, the cheaper aspects (such as TSV stacking) might be worth adding to the consumer GPU mix. Gaming graphics doesn’t require the kind of bandwidth that AI processing does, though, so I think we can remove photonics from the gaming equation for the foreseeable future.

And these combined technologies won’t even be possible for AI accelerators in the near future, either. I think Cutress is right: “My guess, 2028/2029/2030 minimum.”

Best CPU for gaming: Top chips from Intel and AMD.Best gaming motherboard: The right boards.Best graphics card: Your perfect pixel-pusher awaits.Best SSD for gaming: Get into the game first.

Best CPU for gamingBest gaming motherboardBest graphics cardBest SSD for gaming

Best CPU for gaming: Top chips from Intel and AMD.Best gaming motherboard: The right boards.Best graphics card: Your perfect pixel-pusher awaits.Best SSD for gaming: Get into the game first.

TOPICSHardwareNvidia

TOPICS

More about graphics cardsIs the new RTX 5070 really as fast as Nvidia’s previous flagship RTX 4090 GPU? Turns out the answer is yes. Kinda.It’s time for me to admit that AI-accelerated frame generation might actually be the way of the future and that’s a good thingLatestMy Summer Car, the absurdly detailed Finnish life sim about vehicle maintenance and drinking in your underpants, smashes into 1.0 after nearly a decade in Steam early accessSee more latest►

More about graphics cards

Is the new RTX 5070 really as fast as Nvidia’s previous flagship RTX 4090 GPU? Turns out the answer is yes. Kinda.It’s time for me to admit that AI-accelerated frame generation might actually be the way of the future and that’s a good thing

Nvidia RTX 5070 graphics card

Is the new RTX 5070 really as fast as Nvidia’s previous flagship RTX 4090 GPU? Turns out the answer is yes. Kinda.

Nvidia RTX 5090 Founders Edition rendered on a green background.

It’s time for me to admit that AI-accelerated frame generation might actually be the way of the future and that’s a good thing

LatestMy Summer Car, the absurdly detailed Finnish life sim about vehicle maintenance and drinking in your underpants, smashes into 1.0 after nearly a decade in Steam early accessSee more latest►

Latest

My Summer Car, the absurdly detailed Finnish life sim about vehicle maintenance and drinking in your underpants, smashes into 1.0 after nearly a decade in Steam early access

The protagonist of My Summer Car smoking a cigarette outside while a truck in the background pumps out a septic tank.

My Summer Car, the absurdly detailed Finnish life sim about vehicle maintenance and drinking in your underpants, smashes into 1.0 after nearly a decade in Steam early access

See more latest►

Most PopularThe Sims begins its 25th anniversary celebration next week with a Behind The Sims episode of news and announcementsPath of Exile 2 numberlord spends 16 straight days killing rare monsters to prove that a stat that makes loot better makes better lootPalestinian developer raises more than $200,000 to make Dreams on a Pillow, a game about the horrors of the 1948 NakbaPiranha Games will lay off employees after Mechwarrior 5: Clans ‘performed below projections’Marvel Rivals’ latest update quietly killed the game’s burgeoning mod sceneSorry everyone, Minecraft 2 is cancelled, and it might be our faultTeam Fortress 2 just had the single biggest trade in its history, with ‘the absolute best hat you can get’ going for around $40KIs the new RTX 5070 really as fast as Nvidia’s previous flagship RTX 4090 GPU? Turns out the answer is yes. Kinda.Elden Ring: Nightreign is getting a closed network test, and I’m totally not fuming because PC players aren’t invited to the party, or anythingIf you’re a content creator and gamer looking for a CPU upgrade, this Ryzen 9 9900X deal will get you $90 offAMD is now reportedly making all-American Ryzen 9000 CPU dies at TSMC’s Arizona fab

Most Popular

The Sims begins its 25th anniversary celebration next week with a Behind The Sims episode of news and announcements

Path of Exile 2 numberlord spends 16 straight days killing rare monsters to prove that a stat that makes loot better makes better loot

Palestinian developer raises more than $200,000 to make Dreams on a Pillow, a game about the horrors of the 1948 Nakba

Piranha Games will lay off employees after Mechwarrior 5: Clans ‘performed below projections’

Marvel Rivals' latest update quietly killed the game’s burgeoning mod scene

Sorry everyone, Minecraft 2 is cancelled, and it might be our fault

Team Fortress 2 just had the single biggest trade in its history, with ‘the absolute best hat you can get’ going for around $40K

Is the new RTX 5070 really as fast as Nvidia’s previous flagship RTX 4090 GPU? Turns out the answer is yes. Kinda.

Elden Ring: Nightreign is getting a closed network test, and I’m totally not fuming because PC players aren’t invited to the party, or anything

If you’re a content creator and gamer looking for a CPU upgrade, this Ryzen 9 9900X deal will get you $90 off

AMD is now reportedly making all-American Ryzen 9000 CPU dies at TSMC’s Arizona fab

HARDWARE BUYING GUIDESLATEST GAME REVIEWS1Best Steam Deck accessories in Australia for 2025: Our favorite docks, powerbanks and gamepads2Best graphics card for laptops: the mobile GPUs I’d want in my next gaming laptop3Best mini PCs in 2025: The compact computers I love the most4Best 14-inch gaming laptop: The top compact gaming laptops I’ve held in these hands5Best Mini-ITX motherboards in 2025: My pick from all the mini mobo marvels I’ve tested1Thank Goodness You’re Here! review: An anarchic treasure trove of jokes and skits2Shiren the Wanderer: The Mystery Dungeon of Serpentcoil Island review—like juggling chainsaws on horseback3WD Black SN850X 8 TB NVMe SSD review4Ikea Utespelare desk review5Asus ROG Harpe Ace Mini wireless mouse review

HARDWARE BUYING GUIDESLATEST GAME REVIEWS

1Best Steam Deck accessories in Australia for 2025: Our favorite docks, powerbanks and gamepads

A bunch of the best Steam Deck accessories on a blue background.

1Best Steam Deck accessories in Australia for 2025: Our favorite docks, powerbanks and gamepads

Best Steam Deck accessories in Australia for 2025: Our favorite docks, powerbanks and gamepads

2Best graphics card for laptops: the mobile GPUs I’d want in my next gaming laptop

Best graphics card for laptops: the mobile GPUs I’d want in my next gaming laptop

3Best mini PCs in 2025: The compact computers I love the most

Best mini PCs in 2025: The compact computers I love the most

4Best 14-inch gaming laptop: The top compact gaming laptops I’ve held in these hands

Best 14-inch gaming laptop: The top compact gaming laptops I’ve held in these hands

5Best Mini-ITX motherboards in 2025: My pick from all the mini mobo marvels I’ve tested

Best Mini-ITX motherboards in 2025: My pick from all the mini mobo marvels I’ve tested

1Thank Goodness You’re Here! review: An anarchic treasure trove of jokes and skits

Thank Goodness You’re Here review

1Thank Goodness You’re Here! review: An anarchic treasure trove of jokes and skits

Thank Goodness You’re Here! review: An anarchic treasure trove of jokes and skits

2Shiren the Wanderer: The Mystery Dungeon of Serpentcoil Island review—like juggling chainsaws on horseback

Shiren the Wanderer: The Mystery Dungeon of Serpentcoil Island review—like juggling chainsaws on horseback

3WD Black SN850X 8 TB NVMe SSD review

WD Black SN850X 8 TB NVMe SSD review

4Ikea Utespelare desk review

Ikea Utespelare desk review

5Asus ROG Harpe Ace Mini wireless mouse review

Asus ROG Harpe Ace Mini wireless mouse review

The biggest gaming news, reviews and hardware deals#

The biggest gaming news, reviews and hardware deals