Sign In or Create an Account.

By continuing, you agree to the Terms of Service and acknowledge our Privacy Policy

Technology

What Does OpenAI’s New Breakthrough Mean for Energy Consumption?

Why the new “reasoning” models might gobble up more electricity — at least in the short term

•
A robot with a smokestack coming out of its head.
Heatmap Illustration/Getty Images

What happens when artificial intelligence takes some time to think?

The newest set of models from OpenAI, o1-mini and o1-preview, exhibit more “reasoning” than existing large language models and associated interfaces, which spit out answers to prompts almost instantaneously.

Instead, the new model will sometimes “think” for as long as a minute or two. “Through training, they learn to refine their thinking process, try different strategies, and recognize their mistakes,” OpenAI announced in a blog post last week. The company said these models perform better than their existing ones on some tasks, especially related to math and science. “This is a significant advancement and represents a new level of AI capability,” the company said.

But is it also a significant advancement in energy usage?

In the short run at least, almost certainly, as spending more time “thinking” and generating more text will require more computing power. As Erik Johannes Husom, a researcher at SINTEF Digital, a Norwegian research organization, told me, “It looks like we’re going to get another acceleration of generative AI’s carbon footprint.”

Discussion of energy use and large language models has been dominated by the gargantuan requirements for “training,” essentially running a massive set of equations through a corpus of text from the internet. This requires hardware on the scale of tens of thousands of graphical processing units and an estimated 50 gigawatt-hours of electricity to run.

Training GPT-4 cost “more than” $100 million OpenAI chief executive Sam Altman has said; the next generation models will likely cost around $1 billion, according to Anthropic chief executive Dario Amodei, a figure that might balloon to $100 billion for further generation models, according to Oracle founder Larry Ellison.

While a huge portion of these costs are hardware, the energy consumption is considerable as well. (Meta reported that when training its Llama 3 models, power would sometimes fluctuate by “tens of megawatts,” enough to power thousands of homes). It’s no wonder that OpenAI’s chief executive Sam Altman has put hundreds of millions of dollars into a fusion company.

But the models are not simply trained, they're used out in the world, generating outputs (think of what ChatGPT spits back at you). This process tends to be comparable to other common activities like streaming Netflix or using a lightbulb. This can be done with different hardware and the process is more distributed and less energy intensive.

As large language models are being developed, most computational power — and therefore most electricity — is used on training, Charlie Snell, a PhD student at University of California at Berkeley who studies artificial intelligence, told me. “For a long time training was the dominant term in computing because people weren’t using models much.” But as these models become more popular, that balance could shift.

“There will be a tipping point depending on the user load, when the total energy consumed by the inference requests is larger than the training,” said Jovan Stojkovic, a graduate student at the University of Illinois who has written about optimizing inference in large language models.

And these new reasoning models could bring that tipping point forward because of how computationally intensive they are.

“The more output a model produces, the more computations it has performed. So, long chain-of-thoughts leads to more energy consumption,” Husom of SINTEF Digital told me.

OpenAI staffers have been downright enthusiastic about the possibilities of having more time to think, seeing it as another breakthrough in artificial intelligence that could lead to subsequent breakthroughs on a range of scientific and mathematical problems. “o1 thinks for seconds, but we aim for future versions to think for hours, days, even weeks. Inference costs will be higher, but what cost would you pay for a new cancer drug? For breakthrough batteries? For a proof of the Riemann Hypothesis? AI can be more than chatbots,” OpenAI researcher Noam Brown tweeted.

But those “hours, days, even weeks” will mean more computation and “there is no doubt that the increased performance requires a lot of computation,” Husom said, along with more carbon emissions.

But Snell told me that might not be the end of the story. It’s possible that over the long term, the overall computing demands for constructing and operating large language models will remain fixed or possibly even decline.

While “the default is that as capabilities increase, demand will increase and there will be more inference,” Snell told me, “maybe we can squeeze reasoning capability into a small model ... Maybe we spend more on inference but it’s a much smaller model.”

OpenAI hints at this possibility, describing their o1-mini as “a smaller model optimized for STEM reasoning,” in contrast to other, larger models that “are pre-trained on vast datasets” and “have broad world knowledge,” which can make them “expensive and slow for real-world applications.” OpenAI is suggesting that a model can know less but think more and deliver comparable or better results to larger models — which might mean more efficient and less energy hungry large language models.

In short, thinking might use less brain power than remembering, even if you think for a very long time.

Blue
Black Friday Sale

You’ve read your free article. Keep reading with a Heatmap membership

Take 25% off any plan with code THANKS.
Standard Plan
Member

Everything Heatmap publishes, plus a seat in the room.

$129
$97 /year
  • Unlimited access. Every story and newsletter, with no paywall.
  • Virtual briefings. Live sessions with our reporters.
  • Happy hours and events. Member gatherings throughout the year.
Premium Plan
Supporter

For readers who want to keep the journalism going

$349
$262 /year
  • Everything in Member, plus:
  • Exclusive polling access. Heatmap's climate comms polling.
  • VIP at Heatmap House. A reception ticket, plus a membership to gift.
To continue reading
Create a free account or sign in to unlock more free articles.
or
Please enter an email address
By continuing, you agree to the Terms of Service and acknowledge our Privacy Policy
AM Briefing

The Senate Permits

On the clean subsidy race, a deep-sea mining auction, and Canada’s carbon removal push

The Capitol.
Heatmap Illustration/Getty Images

Current conditions: A “rain bomb” just deluged parts of southeastern Florida, with as much as 15 inches pouring on suburbs west of Fort Lauderdale • As Pacific storms drench California, downtown San Francisco recorded more than 2.6 inches of rain on Wednesday alone, exceeding the 2.2-inch record set in 1918 • Indian authorities have upgraded Cyclone Akvan to a “severe cyclonic storm” as it makes landfall today in Andhra Pradesh.


THE TOP FIVE

1. The Senate passes permitting reform

The Senate passed the bipartisan permitting reform bill Thursday night on a 62-33 vote. Every Republican except Missouri’s Josh Hawley and South Dakota’s Mike Rounds, critics of the legislation’s provisions for transmission lines, joined more than a dozen Democrats to pass the Bipartisan American Affordability and Jobs Act. The bill passed without an 11th-hour amendment designed to address the concerns of some tribal groups, which was blocked by Oregon Senator Jeff Merkley, who objected that the body was not voting on his separate amendment to ban data centers on federal lands. “Tribes are being screwed by a last ditch environmental Hail Mary to make the bill as bad as possible before it gets to the House,” Julian Brave NoiseCat, a Native writer and filmmaker, wrote in a post on X. Earlier this week, NoiseCat explained to my colleague Robinson Meyer how tribal groups became power players in the negotiations over the legislation on an episode of our Shift Key podcast. Senator Lisa Murkowski, the Alaska Republican who had coauthored the tribal amendment with the Hawaii’s Democratic Senator Brian Schatz, told Punchbowl News’ Josh Siegel before the vote that it would be “short sighted” to pass BAAJA without the amendment, and said she would “find another way” to write the protections into law. “Not only is it significant from the tribal protections but there were issues in other parts of the country worried on the transmission side. So there was a good reason we worked so hard on this,” Murkowski said, according Siegel’s tweet. Renewable energy trade associations lauded the vote, even without the last-minute changes. “The Senate has cleared a path to enable the construction of hundreds of additional clean energy projects and strengthen rural communities,” Jason Grumet, the CEO of American Clean Power, said in a statement.

Keep reading...Show less
Blue
Daily Briefing

What GTA and the IEA Teach Us About the Future of Energy

The long-awaited video game Grand Theft Auto VI and the International Energy Agency’s voluminous annual report came out today. Here’s what they say about oil, gas, and the dawning age of electricity.

Fatih Birol in GTA 6.
Heatmap Illustration/Rockstar Games, Getty Images

Some people spend their whole lives dreaming about days like today.

A globally respected organization, headquartered in western Europe, has finally released its magnum opus — a vast interactive compendium representing the world in all its joy, violence, and money-grubbing craziness.

Keep reading...Show less
Blue
Climate Tech

Meta Joins the Clean Materials Race

It’s following Microsoft into deals with Fortera, Sublime, and Stegra.

The Meta logo.
Heatmap Illustration/Getty Images

Meta is reconsidering its biggest climate promise. In its 2026 sustainability report, released Thursday afternoon, the company said it is “re-evaluating” its goal of reaching net-zero by 2030, amidst a 45% year-over-year increase in its emissions. That tracks with the broader skepticism toward Meta’s climate bonafides. When Heatmap asked a group of our sharpest energy insiders late last year which hyperscaler they thought was doing the best on climate, Google and Microsoft emerged as clear frontrunners, while Meta narrowly beat out Amazon in the race to the bottom.

It was not surprising. After all, Google pioneered the concept of 24/7 energy matching, and made early bets on advanced geothermal and small modular reactors. Microsoft built the carbon removal market practically singlehandedly while making deals with low-carbon materials suppliers like steelmaker Stegra and clean cement startup Sublime Systems. Meta, meanwhile, was making headlines for its gigantic new data center campus in Louisiana, which it aims to power with 7.5 gigawatts of new natural gas plants.

Keep reading...Show less
Blue