Get business pricing on tech for your team
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
Nvidia aims to dominate the inference side of generative AI
Nvidia, the leading provider of GPUs for training language models, is enhancing its AI-focused software development kit (SDK) to boost the efficiency of large language models (LLMs) and associated tools. The company has integrated support for its TensorRT-LLM SDK on Windows and models such as Stable Diffusion, allowing LLMs to function at a quicker speed. By enhancing the inference process, Nvidia is looking to make a greater impact on the progress and utilization of generative AI.
Products Worth Considering
Jackery Explorer 300 Portable Power Station with SolarSaga 100W Solar Panel
Bose QuietComfort Earbuds (2nd Gen) – True Wireless Bluetooth Earbuds with Active Noise Cancellation
Bose QuietComfort Headphones (2nd Gen), Wireless Over-Ear Noise-Cancelling Headphones, Black
As an affiliate, we earn on qualifying purchases.
TensorRT-LLM: Accelerating the LLM Experience
TensorRT-LLM, a component of Nvidia’s SDK, enables LLMs to run more efficiently on Nvidia’s H100 GPUs. This technology is compatible with popular LLMs like Meta’s Llama 2 and AI models such as Stability AI’s Stable Diffusion. By leveraging TensorRT-LLM, users can expect significant performance improvements, especially in the use of sophisticated LLM applications like writing and coding assistants.
Expanding Access and Reducing Reliance on Expensive GPUs
Nvidia plans to make TensorRT-LLM available to the public, allowing anyone to integrate and utilize the SDK for their projects. This move demonstrates Nvidia’s commitment to not only providing powerful GPUs for training and running LLMs, but also offering the necessary software to optimize their performance. The goal is to prevent users from seeking alternative cost-efficient solutions for generative AI.
Competition and the Future of Generative AI
Nvidia currently enjoys a near monopoly in the market for GPUs that train LLMs, resulting in skyrocketing demand and high prices. However, competitors like Microsoft and AMD have announced plans to develop their own chips, aiming to reduce reliance on Nvidia. Additionally, companies such as SambaNova are already offering services that facilitate the running of AI models. While Nvidia remains the hardware leader in generative AI, the company is positioning itself for a future where users are not solely dependent on purchasing large quantities of its GPUs.
