On February 23, the editor shared some good news: NVIDIA released an application named “Chat with RTX” suitable for RTX 30 and RTX 40 series graphics cards.
After installing Chat with RTX, users can submit various local files to this application, which can automatically analyze and organize them. Users can then ask questions through text conversation, and the application can respond, indicating the basis and source of the answers.
In simple terms, Chat with RTX is akin to an AI chatbot running locally, similar to OpenAI’s ChatGPT and Baidu’s Wenxin Yiyan.
Following this news, AMD, feeling the pressure, also launched a similar feature, which is great news for fans of the company. However, AMD’s solution differs significantly from NVIDIA’s, with a slightly more complicated installation and configuration process.
Firstly, whether it’s NVIDIA or AMD, implementing such applications requires AI computing units. For NVIDIA, this is straightforward since it only develops GPUs, so Chat with RTX runs on graphics cards.
AMD is different as it produces both processors and graphics cards. Currently, some of its processors and Radeon 7000 series graphics cards have integrated AI computing units, so AMD offers two solutions: one for processors (7040 and 8040 series) and one for graphics cards (Radeon 7000 series).
Moreover, AMD doesn’t develop its solution but recommends using a third-party solution called LM Studio. This is a closed-source, cross-platform solution that supports Intel and Apple platforms.
There are two versions of the software. If using an AMD processor, download “LM Studio for Windows.” If using an AMD graphics card, download “LM Studio for AMD ROCm.”
After installing LM Studio, the next step is to choose the right model. Currently, two main models are Mistral 7b and LLAMA v2 7b.
To use Mistral 7b, enter “TheBloke/OpenHermes-2.5-Mistral-7B-GGUF” in the LM Studio search box and select it from the search results. To use LLAMA v2 7b, search for “TheBloke/Llama-2-7B-Chat-GGUF” and select it.
Next, scroll down in the right panel to find the Q4KM model file, click to download and install. Then switch to the chat tab, select the model from the dropdown menu at the top center, and wait for it to load.
If installing the AMD graphics card version of LM Studio, select “GPU unloading” on the right panel and move the slider to “maximum.” If LM Studio detects the GPU type as “AMD ROCm,” the setup is successful, and you can enjoy chatting.
If you’re using an AMD processor or graphics card and meet the requirements, consider visiting AMD’s and LM Studio’s websites for more details, as they offer extensive instructions.
The editor highly recommends trying this out and experiencing AI since the project is fully deployed and runs locally, ensuring no data is uploaded to the backend, so users can use it with confidence.
Related:
- Top AI Model Containers and How to Use Them Effectively
- Unleash AI Creativity: RX 7900 XT with ROCm Enhancement!
- Discover 8 Interesting and Surprising Computer Facts
- Folder Icon Hard Drive: Functional Storage with Style!
- OpenWrt One Router Launched with SSD Slot for $89
- Apple SSD Hacked: Affordable Alternatives at Half Price
Disclaimer: This article is created by the original author. The content of the article represents their personal opinions. Our reposting is for sharing and discussion purposes only and does not imply our endorsement or agreement. If you have any objections, please contact us through the provided channels.
