I created a highly personalised large language model with Nvidia's entertaining Chat with RTX app but at 60GB+ I'm now beginning to wonder if it's worth keeping around

HardwareGraphics CardsI created a highly personalised large language model with Nvidia’s entertaining Chat with RTX app but at 60GB+ I’m now beginning to wonder if it’s worth keeping aroundWhen you purchase through links on our site, we may earn an affiliate commission.Here’s how it works.

When you purchase through links on our site, we may earn an affiliate commission.Here’s how it works.

(Image credit: Nvidia)

Nvidia�s Chat with RTX software in use, which offers a personally trained LLM.

Owners of RTX 40- and 30-series graphics cards can now setup their own personalised large language model (LLM) on their own PC. It’s one that’s eminently capable of sifting through old documents or distilling down the essence of YouTube videos.

Chat with RTX is now available to download fromNvidia’s websitefor free from today, February 13. It works with any current or last generation graphics card with at least 8GB or more VRAM, which includes every desktop card bar the RTX 3050 6GB and excludes a few mid- to low-end laptop GPUs. It also requires 50—100GB of storage space on your PC, depending on the AI models downloaded.

There are two models to choose from: Mistral or Llama 2. The default is Mistral, and I’d recommend sticking with that.

The key parts of Chat with RTX are retrieval-augmented generation (RAG) and TensorRT-LLM. The former means you’re able to give the LLM information and it which it will use alongside its internal training to generate accurate responses to your queries. The latter builds TensorRT engines that can exploit the silicon in Nvidia’s GeForce GPUs to more efficiently run AI applications.

The result is an LLM you can feed your own data to (.txt, .pdf, and .doc filetypes) and which you can then query on that data.

For example, I’ve been playing around with the tool these past few days and since I create a lot of documents as a part of this job this feels like the prime dataset to stuff into its gaping maw. So I’ve set up Chat with RTX on myRTX 4080powered PC (install size of 61.7GB) and fed the Mistral model over 1,300 of wonderful prose (ahem, or rather my news article drafts). I then set about asking it some questions.

First, I asked ‘Could you name the articles where I mention Nvidia?’

The biggest gaming news, reviews and hardware deals

(Image credit: Nvidia)

Nvidia’s Chat with RTX software in use.

Out comes the above response listing three articles with their file path. Now, I’ve definitely spoken about Nvidia more than three times in 1,300 articles, so let’s give that another try.

I ask again, rewording the query a little, ‘Could you list every article in which I mention Nvidia?’

(Image credit: Nvidia)

Nvidia’s Chat with RTX software in use.

This time eight articles are listed, this time with the Google Doc titles listed. I’ve mentioned Nvidia many more times than that, but you get the general idea of how this all works. Each response does appear grounded in truth, with each response citing the data used to generate it, if not always the whole truth. Simply using the Windows search function within the article dataset brings up 128 drafts including the term ‘Nvidia’ in the title, let alone the body copy.

Another example is if I ask Chat with RTX to tell me how many times I’ve used the word cheese, it tells me I’ve never used the phrase, citing an untitled and unrelated document as the source for the information. Nevertheless, it’s probably right about the cheese thing. Until now, anyways.

Yet the tool is more exciting once you start asking it to summarise large quantities of information down into single bite-sized responses.

I asked Chat with RTX whether I should buy an Intel Core i9 14900K, and it came back to me with a trimmed version of my own14900K review, which succinctly summarised it to “Based on the review, it appears that the Intel Core i9 14900K may not be worth the extra cost compared to the Core i9 13900K”.

Couldn’t put it better myself.

(Image credit: Nvidia)

Nvidia’s Chat with RTX software in use, which offers a personally trained LLM.

I also asked Chat with RTX to summarise an article I wrote a while back on Alpine’s F1 esports team, which it explained succinctly, and then to tell me about Intel’s Meteor Lake processors, which I knew were covered a few times in the articles within the dataset.

Image1of3(Image credit: Nvidia)(Image credit: Nvidia)(Image credit: Nvidia)

Image1of3

(Image credit: Nvidia)(Image credit: Nvidia)(Image credit: Nvidia)

(Image credit: Nvidia)

Nvidia’s Chat with RTX software in use, which offers a personally trained LLM.

(Image credit: Nvidia)

Nvidia’s Chat with RTX software in use, which offers a personally trained LLM.

(Image credit: Nvidia)

Nvidia’s Chat with RTX software in use, which offers a personally trained LLM.

Oh, and I asked it who I was. This was more to make myself feel important as the LLM returned a description of me in near-enough the same words as I used to describe myself for my site bio. Theoretically, you could just feed Chat with RTX a thousands documents on how great you are and create a narcissist’s dream software.

Not that I’d do that, nope.

(Image credit: Nvidia)

Nvidia’s Chat with RTX software in use, which offers a personally trained LLM.

It’s the summarising of large datasets that I could see this tool being useful for. Though I doubt everyone has such a need for that. The average PC user might not fancy a 100GB app to tell them what they already know. But, say you’re working with a huge amount of responses to a survey and you want to quickly get an idea of the general thoughts and feelings of those who answered, this is one easy way to do it. But it is best used with caution and as only a guide to the inputted dataset, not as a way to accurately analyse it fully.

The other people it might appeal to are those that prefer to keep their content out of the cloud. The idea of asking an AI hosted God-knows-where to handle files that could contain sensitive information, or manuscripts for your big action movie idea, isn’t that appealing to many. We’ve already seen what this looks like when it goes wrong courtesy ofSamsung’s employees. That’s why a locally-run tool such as this might instead appeal.

I asked, ‘Is the Steam Deck easy to use day-to-day?’ and a response prints out that sums up Lauren and Mollie’s conversation with Tyler Colp on the matter.

(Image credit: Nvidia)

Nvidia’s Chat with RTX software in use, which offers a personally trained LLM.

I also then asked the obvious question, ‘Does the Steam Deck suit their PC gaming lifestyle so far?’ The response:

(Image credit: Nvidia)

Nvidia’s Chat with RTX software in use, which offers a personally trained LLM.

The console window that shows some of the inner workings of Chat with RTX.(Image credit: Nvidia)

Nvidia’s Chat with RTX software in use, which offers a personally trained LLM.

Anyways, the YouTube stuff takes a backseat for me with Chat with RTX. It’s the masslocaltext file digestion that feels the most important piece of the software. As an application, it’s pretty snappy. It generates responses swiftly once you hit send on a query. Though it does appear to gobble up around 85% of my VRAM—you need to be sure to close it properly with the off switch to release that back to the PC once you’re done with it.

Chat with RTX is a fun concept, and a good way for Nvidia to show what’s inference locally on its GeForce cards can do, but I’m not sure I’m going to keep it around on my PC. For one, it’s absolutely massive due to the huge model data, but more so because the actual practical uses are pretty limited for me, personally.

Perhaps some clever clogs will come up with new and exciting ways to put it into practice now that it’s available to the world. That could be you, providing you have the proper hardware. You candownload Chat with RTXto try for yourself today.

Best CPU for gaming: Top chips from Intel and AMD.Best gaming motherboard: The right boards.Best graphics card: Your perfect pixel-pusher awaits.Best SSD for gaming: Get into the game first.

Best CPU for gamingBest gaming motherboardBest graphics cardBest SSD for gaming

Best CPU for gaming: Top chips from Intel and AMD.Best gaming motherboard: The right boards.Best graphics card: Your perfect pixel-pusher awaits.Best SSD for gaming: Get into the game first.

More about graphics cardsAIB vendors getting the RTX 5080’s memory config wrong could mean 50-series Super cards have more and faster VRAMEarly Radeon RX 9070 benchmarks are positive, though don’t bank too much on themLatestMSI intros cheaper ‘back connect’ Project Zero Intel motherboards for super-clean PC builds you might actually be able to affordSee more latest►

More about graphics cards

AIB vendors getting the RTX 5080’s memory config wrong could mean 50-series Super cards have more and faster VRAMEarly Radeon RX 9070 benchmarks are positive, though don’t bank too much on them

Nvidia GeForce RTX 5090 render on green background

AIB vendors getting the RTX 5080’s memory config wrong could mean 50-series Super cards have more and faster VRAM

The AMD Radeon RX 9070 XT and RX 9070 RDNA 4 GPUs arranged in diagonal lines, taken from a CES 2025 presentation slide

Early Radeon RX 9070 benchmarks are positive, though don’t bank too much on them

LatestMSI intros cheaper ‘back connect’ Project Zero Intel motherboards for super-clean PC builds you might actually be able to affordSee more latest►

Latest

MSI intros cheaper ‘back connect’ Project Zero Intel motherboards for super-clean PC builds you might actually be able to afford

MSI MAG Z890 Tomahawk WIFI PZ

MSI intros cheaper ‘back connect’ Project Zero Intel motherboards for super-clean PC builds you might actually be able to afford

See more latest►

Most PopularWe may not have a roadmap or balance patch just yet, but at least Path of Exile 2’s bugfix patch notes are funnyDeus Ex: Mankind Divided writer didn’t like the ending either: ‘Nothing is resolved’It seems like Square’s finally consistently taking the PC ports of Final Fantasy seriouslyLike a Dragon: Pirate Yakuza in Hawaii looks more like a zany RGG take on Black Flag than I dared hopeThe most important tool in Remedy’s new co-op FPS is the shower: ‘Saving water is essential’If you missed it over the holidays, this noir Half-Life 2 mod turns you into a hardboiled detective in Prohibition-era ChicagoPath of Exile 2 players are calling foul on Elon Musk’s high-level hardcore character after he streams his struggles with core game mechanicsAlan Emrich, the game designer and writer who coined the term ‘4X,’ has diedTrack down a missing person in this ‘Geoguessr-inspired detective game’ set on an alien planetOld meets new as classic MMO players worship a famous Vtuber via gifts of bones so she can max out the Prayer skillHyper Light Breaker will cost $30 and for a limited time come with a free copy of Drifter

Most Popular

We may not have a roadmap or balance patch just yet, but at least Path of Exile 2’s bugfix patch notes are funny

Deus Ex: Mankind Divided writer didn’t like the ending either: ‘Nothing is resolved’

It seems like Square’s finally consistently taking the PC ports of Final Fantasy seriously

Like a Dragon: Pirate Yakuza in Hawaii looks more like a zany RGG take on Black Flag than I dared hope

The most important tool in Remedy’s new co-op FPS is the shower: ‘Saving water is essential’

If you missed it over the holidays, this noir Half-Life 2 mod turns you into a hardboiled detective in Prohibition-era Chicago

Path of Exile 2 players are calling foul on Elon Musk’s high-level hardcore character after he streams his struggles with core game mechanics

Alan Emrich, the game designer and writer who coined the term ‘4X,’ has died

Track down a missing person in this ‘Geoguessr-inspired detective game’ set on an alien planet

Old meets new as classic MMO players worship a famous Vtuber via gifts of bones so she can max out the Prayer skill

Hyper Light Breaker will cost $30 and for a limited time come with a free copy of Drifter

HARDWARE BUYING GUIDESLATEST GAME REVIEWS1Best Steam Deck accessories in Australia for 2025: Our favorite docks, powerbanks and gamepads2Best graphics card for laptops: the mobile GPUs I’d want in my next gaming laptop3Best mini PCs in 2025: The compact computers I love the most4Best 14-inch gaming laptop: The top compact gaming laptops I’ve held in these hands5Best Mini-ITX motherboards in 2025: My pick from all the mini mobo marvels I’ve tested1Thank Goodness You’re Here! review: An anarchic treasure trove of jokes and skits2Shiren the Wanderer: The Mystery Dungeon of Serpentcoil Island review—like juggling chainsaws on horseback3WD Black SN850X 8 TB NVMe SSD review4Ikea Utespelare desk review5Asus ROG Harpe Ace Mini wireless mouse review

HARDWARE BUYING GUIDESLATEST GAME REVIEWS

1Best Steam Deck accessories in Australia for 2025: Our favorite docks, powerbanks and gamepads

A bunch of the best Steam Deck accessories on a blue background.

1Best Steam Deck accessories in Australia for 2025: Our favorite docks, powerbanks and gamepads

Best Steam Deck accessories in Australia for 2025: Our favorite docks, powerbanks and gamepads

2Best graphics card for laptops: the mobile GPUs I’d want in my next gaming laptop

Best graphics card for laptops: the mobile GPUs I’d want in my next gaming laptop

3Best mini PCs in 2025: The compact computers I love the most

Best mini PCs in 2025: The compact computers I love the most

4Best 14-inch gaming laptop: The top compact gaming laptops I’ve held in these hands

Best 14-inch gaming laptop: The top compact gaming laptops I’ve held in these hands

5Best Mini-ITX motherboards in 2025: My pick from all the mini mobo marvels I’ve tested

Best Mini-ITX motherboards in 2025: My pick from all the mini mobo marvels I’ve tested

1Thank Goodness You’re Here! review: An anarchic treasure trove of jokes and skits

Thank Goodness You’re Here review

1Thank Goodness You’re Here! review: An anarchic treasure trove of jokes and skits

Thank Goodness You’re Here! review: An anarchic treasure trove of jokes and skits

2Shiren the Wanderer: The Mystery Dungeon of Serpentcoil Island review—like juggling chainsaws on horseback

Shiren the Wanderer: The Mystery Dungeon of Serpentcoil Island review—like juggling chainsaws on horseback

3WD Black SN850X 8 TB NVMe SSD review

WD Black SN850X 8 TB NVMe SSD review

4Ikea Utespelare desk review

Ikea Utespelare desk review

5Asus ROG Harpe Ace Mini wireless mouse review

Asus ROG Harpe Ace Mini wireless mouse review

The biggest gaming news, reviews and hardware deals#

The biggest gaming news, reviews and hardware deals