How Does Nvidia’s Blackwell Chip Make AI Development More Cost-Effective and Eco-Friendly?

Earlier this week at the annual Nvidia GTC conference in San Jose Convention Center in California, Nvidia announced the release of the Blackwell B200 GPU – “the world’s most powerful chip.”

Blackwell GPUs come with 208 billion transistors and enable organizations to train and run generative AI models with up to 25x less overall cost and energy consumption than its H100 series of chips.

This computationally-efficient chip is designed to support trillion-parameter large language models (LLMs), which are integral in allowing multimodal models to train against, process and respond to data in formats including text, image, audio, and video.

For context, arguable the most powerful LLM right on the market, GPT-4 is rumored to have 1.7 trillion parameters.

The release comes the same month that Nvidia achieved a $2 trillion valuation off the back of its dominance in the AI chip market, which has made it the third most valuable company in the world behind Microsoft and Apple, valued at $3.09 trillion and $2.77 trillion respectively.

Table of Contents Table of Contents

1. Key Takeaways
2. Winning the Economic Argument Over AI
3. A Move Toward Eco-Friendly AI?
4. The Bottom Line?

Show Full Guide

Table of Contents

1. Key Takeaways
2. Winning the Economic Argument Over AI
3. A Move Toward Eco-Friendly AI?
Show Full Guide
4. The Bottom Line?

Winning the Economic Argument Over AI

While generative AI has captured a wave of interest among enterprises since the release of ChatGPT in November 2022, its adoption has been restricted by the prohibitive cost of training and running AI Models.

For instance, according to some estimates, training GPT-3 required $5 million worth of GPUs, and the cost for more powerful LLMs is likely much higher. This means that traditional computing simply isn’t viable to meet the needs of AI-driven organizations.

“Accelerated computing has reached the tipping point – general purpose computing has run out of steam,” Nvidia CEO Jensen Huang said in his keynote speech at Nvidia GTC.

“We need another way of doing computing so that we can continue to scale; so that we can continue to drive down the cost of computing; so that we can continue to consume more and more computing while being sustainable.”

“Accelerated computing is a dramatic speedup over general purpose computing, in every single industry,” Huang said.

Under this accelerated computing approach, chips like the B200 have a critical role to play in lowering the overall cost of deploying AI, by increasing the amount of processing power available for inference tasks.

International best seller and generative AI expert Bernard Marr told Techopedia via email:

“Nvidia’s Blackwell chip is a monumental leap in AI’s hardware landscape. By delivering a quintuple increase in AI performance over its H100 predecessor, the Blackwell chip is not just a piece of hardware; it’s a gateway to previously unimaginable AI capabilities.”

“This launch is pivotal, not only for Nvidia’s market position but for the entire AI industry, enabling more complex, efficient, and innovative AI models,” Marr added.

The high processing power of chips like the B200 and the H100 series is a key reason why AI leaders such as OpenAI, Google, Microsoft, Amazon, and Meta are using the organization’s chips to power their AI applications.

Nvidia’s success in the market has largely been due to its decisions to “sell shovels in a gold rush,” situating itself as a key provider of powerful AI-optimized GPUs for other companies to use to enable their own AI journeys.

A Move Toward Eco-Friendly AI?

It’s also important to note that Nvidia’s Blackwell B200 GPU presents a notable step toward eco-friendly AI. Reducing energy consumption by 25x is a solid win for reducing the overall power consumption and carbon footprint generated by AI models during inference tasks.

Over the past couple of years, the environmental impact of AI workloads has come under scrutiny, due to their high energy consumption and carbon emissions.

According to some estimates, LLMs like GPT-3 use just under 1,300 megawatt hours – the equivalent annual power consumption of 130 US homes. The output of larger, more powerful LLMs like GPT-4 is likely to be much higher, and AI poses a very real threat to the world’s energy demands.

The reality is that the more powerful an LLM is, the more servers, GPUs, and cooling systems that need to be powered in the data center. Each of these components consume electricity and increase carbon emissions.

One study conducted by Alex de Vries, a data scientist at the Central Bank of the Netherlands, also a PH.D. candidate at Vrije University Amsterdam, calculates that if AI adoption and capacity continues at its current rate, then Nvidia is on track to ship 1.5 million AI server units per year by 2027.

According to Scientific American these servers would consume at least 85.4 terawatt-hours of electricity annually, which is more energy than some small countries use in a year.

When considering this, next-generation chips like the B200 GPU can be viewed as an attempt to make AI inference more environmentally-friendly.

While there’s still a very long way to go to offset the environmental impact of AI (which is still being understood), it is nonetheless a step in the right direction.

The Bottom Line?

If AI — and generative AI in particular — is to achieve greater adoption, then it needs to become a more cost-effective solution for enterprises. As of right now, it’s too expensive for most organizations to build their own models, which cuts down on the potential use cases available.

Accelerated computing, and chips like the B200 will make conducting AI inference tasks more accessible to researchers and enterprises, while also helping the industry to become more environmentally-friendly.