Chat With RTX brings custom local chatbots to Nvidia AI PCs

Key Takeaways:

– Nvidia is introducing Chat with RTX, a personalized local AI chatbot system.
– It allows users to harness the power of generative AI on their local devices.
– Chat with RTX showcases the potential of retrieval-augmented generation (RAG) and TensorRT-LLM software.
– It doesn’t require a lot of data center computing and prioritizes user privacy.
– Users can enjoy the benefits of generative AI on Nvidia GeForce RTX 30 Series GPUs or higher with a minimum of 8GB of VRAM.
– Chat with RTX is a personalized AI companion that users can customize with their own content.
– It leverages RAG, TensorRT-LLM software, and Nvidia RTX acceleration to provide quick, contextually relevant answers.
– Users can connect the application to local files on their PCs for a dataset.
– Chat with RTX supports various file formats and allows users to type natural language queries.
– It can include information from multimedia sources like YouTube videos and playlists.
– The application’s local processing capabilities ensure fast results and user data stays on the device.
– Users need a GeForce RTX 30 Series GPU or higher with a minimum of 8GB of VRAM and Windows 10 or 11 to use Chat with RTX.
– Developers can explore the potential of accelerating large language models with RTX GPUs.
– The Chat with RTX tech demo is available for free download.

VentureBeat:


Nvidia is introducing Chat with RTX to create personalized local AI chatbots on Windows AI PCs.

It’s the latest attempt by Nvidia to turn AI on its graphics processing units (GPUs) into a mainstream tool used by everyone.

The new offering, Chat with RTX, allows users to harness the power of personalized generative AI directly on their local devices, showcasing the potential of retrieval-augmented generation (RAG) and TensorRT-LLM software. At the same time, it doesn’t burn up a lot of data center computing and it helps with local privacy so that users don’t have to worry about their AI chats.

Chatbots have become an integral part of daily interactions for millions globally, typically relying on cloud servers with Nvidia GPUs. However, the Chat with RTX tech demo shifts this paradigm by enabling users to enjoy the benefits of generative AI locally, using the processing power of Nvidia GeForce RTX 30 Series GPUs or higher with a minimum of 8GB of video random access memory (VRAM).

GB Event

GamesBeat Summit Call for Speakers

We’re thrilled to open our call for speakers to our flagship event, GamesBeat Summit 2024 hosted in Los Angeles, where we will explore the theme of “Resilience and Adaption”.


Apply to speak here

Personalized AI experience

Nvidia said that Chat with RTX is more than a mere chatbot; it’s a personalized AI companion that users can customize with their own content. By leveraging the capabilities of local GeForce-powered Windows PCs, users can accelerate their experience and enjoy the benefits of generative AI with unprecedented speed and privacy, the company said.

The tool leverages RAG, TensorRT-LLM software, and Nvidia RTX acceleration to facilitate quick, contextually relevant answers based on local datasets. Users can connect the application to local files on their PCs, turning them into a dataset for open-source large language models like Mistral or Llama 2.

Rather than sifting through various files, users can type natural language queries, such as asking about a restaurant recommendation or any personalized information, and Chat with RTX will swiftly scan and provide the answer with context. The application supports a variety of file formats, including .txt, .pdf, .doc/.docx, and .xml, making it versatile and user-friendly.

Integration of multimedia content

Chat with RTX works on an Nvidia AI PC.

What sets Chat with RTX apart is its ability to include information from multimedia sources, particularly YouTube videos and playlists, Nvidia said.

Users can integrate knowledge from video content into their chatbot, enabling contextual queries. For instance, users can seek travel recommendations based on their favorite influencer’s videos or obtain quick tutorials and how-tos from educational resources.

The application’s local processing capabilities ensure fast results, and importantly, user data stays on the device. By eliminating the need for cloud-based services, Chat with RTX allows users to handle sensitive data without sharing it with third parties or requiring an internet connection.

System requirements and future possibilities

To experience Chat with RTX, users need a GeForce RTX 30 Series GPU or higher with a minimum of 8GB of VRAM, along with Windows 10 or 11 and the latest Nvidia GPU drivers.

Developers can explore the potential of accelerating large language models (LLMs) with RTX GPUs by referring to the TensorRT-LLM RAG developer reference project available on GitHub. Nvidia encourages developers to participate in the Generative AI on Nvidia RTX developer contest, running until February 23, offering opportunities to win prizes such as a GeForce RTX 4090 GPU and a full, in-person conference pass to Nvidia GTC.

The Chat with RTX tech demo is now available for free download.

VentureBeat’s mission is to be a digital town square for technical decision-makers to gain knowledge about transformative enterprise technology and transact. Discover our Briefings.

Source link

AI Eclipse TLDR:

Nvidia has introduced Chat with RTX, a new offering that allows users to create personalized local AI chatbots on Windows AI PCs. This move is part of Nvidia’s efforts to make AI on its GPUs a mainstream tool accessible to everyone. Chat with RTX enables users to utilize generative AI directly on their local devices, utilizing the power of Nvidia GeForce RTX 30 Series GPUs or higher. The tool leverages retrieval-augmented generation (RAG) and TensorRT-LLM software to provide quick and contextually relevant answers based on local datasets. It also supports various file formats and can integrate multimedia content, including YouTube videos. The application runs on Nvidia AI PCs and prioritizes user privacy by keeping data on the device and eliminating the need for cloud-based services. To experience Chat with RTX, users need a compatible GPU, Windows 10 or 11, and the latest Nvidia GPU drivers. Nvidia is encouraging developers to explore the potential of accelerating large language models with RTX GPUs through the TensorRT-LLM RAG developer reference project and a Generative AI developer contest. The Chat with RTX tech demo is available for free download.