Nvidia wants to drive AI costs down as ‘reasoning’ models rise

Nvidia’s (NVDA) chips have been a driver of the current artificial intelligence boom — and the chipmaker only wants to make it move faster, chief executive Jensen Huang said. Read More...

Nvidia’s (NVDA) chips have been a driver of the current artificial intelligence boom — and the chipmaker only wants to make it move faster, chief executive Jensen Huang said.

During an appearance on the Tech Unheard podcast, Huang was asked by host Rene Haas, chief executive of British semiconductor company Arm (ARM), if the pace of AI innovation is moving “faster than you imagined.”

Huang replied, “No,” and said Nvidia is “trying to make it go faster.”

“We’ve gone to a one-year cycle,” Huang said about the company’s production of new chips. “And the reason for that is because the technology has the opportunity to move fast.”

Nvidia is designing “six or seven new chips per system,” Huang said, then using “co-design to reinvent the entire system” and inventing other technologies that allow it to improve performance by two or three times while using the same amount of energy and cost each year.

“That’s another way of, essentially, reducing the cost of AI by two or three times per year,” Huang said. “That is way faster than Moore’s Law.”

Over the next few years, Huang said, Nvidia wants to drive down the cost of AI amid the rise of even more complex models. This evolution is already underway. In September, OpenAI released a new series of “reasoning” AI models called o1, which are “designed to spend more time thinking before they respond,” the way humans do.

In the future, AI services such as OpenAI’s ChatGPT, which Huang said he uses every day, will “iteratively reason about the answer” and could go through hundreds or thousands of inferences before producing an output.

This level of complex processing would require significantly more computational power than current models. Despite the increased demands, Huang believes the trade-off is worthwhile.

“[T]he quality of the answer is so much better,” Huang said. “We want to drive the cost down so that we could deliver this new type of reasoning inference with the same level of cost and responsiveness as the past.”

On Wednesday, Nvidia’s stock was climbing back toward its record high $135 close in June. The chipmaker’s shares were down 0.27% during midday trading, but opened up almost 1% at around $134 per share.

For the latest news, Facebook, Twitter and Instagram.