Sign In or Create an Account.

By continuing, you agree to the Terms of Service and acknowledge our Privacy Policy

Technology

What Does OpenAI’s New Breakthrough Mean for Energy Consumption?

Why the new “reasoning” models might gobble up more electricity — at least in the short term

A robot with a smokestack coming out of its head.
Heatmap Illustration/Getty Images

What happens when artificial intelligence takes some time to think?

The newest set of models from OpenAI, o1-mini and o1-preview, exhibit more “reasoning” than existing large language models and associated interfaces, which spit out answers to prompts almost instantaneously.

Instead, the new model will sometimes “think” for as long as a minute or two. “Through training, they learn to refine their thinking process, try different strategies, and recognize their mistakes,” OpenAI announced in a blog post last week. The company said these models perform better than their existing ones on some tasks, especially related to math and science. “This is a significant advancement and represents a new level of AI capability,” the company said.

But is it also a significant advancement in energy usage?

In the short run at least, almost certainly, as spending more time “thinking” and generating more text will require more computing power. As Erik Johannes Husom, a researcher at SINTEF Digital, a Norwegian research organization, told me, “It looks like we’re going to get another acceleration of generative AI’s carbon footprint.”

Discussion of energy use and large language models has been dominated by the gargantuan requirements for “training,” essentially running a massive set of equations through a corpus of text from the internet. This requires hardware on the scale of tens of thousands of graphical processing units and an estimated 50 gigawatt-hours of electricity to run.

Training GPT-4 cost “more than” $100 million OpenAI chief executive Sam Altman has said; the next generation models will likely cost around $1 billion, according to Anthropic chief executive Dario Amodei, a figure that might balloon to $100 billion for further generation models, according to Oracle founder Larry Ellison.

While a huge portion of these costs are hardware, the energy consumption is considerable as well. (Meta reported that when training its Llama 3 models, power would sometimes fluctuate by “tens of megawatts,” enough to power thousands of homes). It’s no wonder that OpenAI’s chief executive Sam Altman has put hundreds of millions of dollars into a fusion company.

But the models are not simply trained, they're used out in the world, generating outputs (think of what ChatGPT spits back at you). This process tends to be comparable to other common activities like streaming Netflix or using a lightbulb. This can be done with different hardware and the process is more distributed and less energy intensive.

As large language models are being developed, most computational power — and therefore most electricity — is used on training, Charlie Snell, a PhD student at University of California at Berkeley who studies artificial intelligence, told me. “For a long time training was the dominant term in computing because people weren’t using models much.” But as these models become more popular, that balance could shift.

“There will be a tipping point depending on the user load, when the total energy consumed by the inference requests is larger than the training,” said Jovan Stojkovic, a graduate student at the University of Illinois who has written about optimizing inference in large language models.

And these new reasoning models could bring that tipping point forward because of how computationally intensive they are.

“The more output a model produces, the more computations it has performed. So, long chain-of-thoughts leads to more energy consumption,” Husom of SINTEF Digital told me.

OpenAI staffers have been downright enthusiastic about the possibilities of having more time to think, seeing it as another breakthrough in artificial intelligence that could lead to subsequent breakthroughs on a range of scientific and mathematical problems. “o1 thinks for seconds, but we aim for future versions to think for hours, days, even weeks. Inference costs will be higher, but what cost would you pay for a new cancer drug? For breakthrough batteries? For a proof of the Riemann Hypothesis? AI can be more than chatbots,” OpenAI researcher Noam Brown tweeted.

But those “hours, days, even weeks” will mean more computation and “there is no doubt that the increased performance requires a lot of computation,” Husom said, along with more carbon emissions.

But Snell told me that might not be the end of the story. It’s possible that over the long term, the overall computing demands for constructing and operating large language models will remain fixed or possibly even decline.

While “the default is that as capabilities increase, demand will increase and there will be more inference,” Snell told me, “maybe we can squeeze reasoning capability into a small model ... Maybe we spend more on inference but it’s a much smaller model.”

OpenAI hints at this possibility, describing their o1-mini as “a smaller model optimized for STEM reasoning,” in contrast to other, larger models that “are pre-trained on vast datasets” and “have broad world knowledge,” which can make them “expensive and slow for real-world applications.” OpenAI is suggesting that a model can know less but think more and deliver comparable or better results to larger models — which might mean more efficient and less energy hungry large language models.

In short, thinking might use less brain power than remembering, even if you think for a very long time.

Blue

You’re out of free articles.

Subscribe today to experience Heatmap’s expert analysis 
of climate change, clean energy, and sustainability.
To continue reading
Create a free account or sign in to unlock more free articles.
or
Please enter an email address
By continuing, you agree to the Terms of Service and acknowledge our Privacy Policy
Spotlight

How the Tax Bill Is Empowering Anti-Renewables Activists

A war of attrition is now turning in opponents’ favor.

Massachusetts and solar panels.
Heatmap Illustration/Library of Congress, Getty Images

A solar developer’s defeat in Massachusetts last week reveals just how much stronger project opponents are on the battlefield after the de facto repeal of the Inflation Reduction Act.

Last week, solar developer PureSky pulled five projects under development around the western Massachusetts town of Shutesbury. PureSky’s facilities had been in the works for years and would together represent what the developer has claimed would be one of the state’s largest solar projects thus far. In a statement, the company laid blame on “broader policy and regulatory headwinds,” including the state’s existing renewables incentives not keeping pace with rising costs and “federal policy updates,” which PureSky said were “making it harder to finance projects like those proposed near Shutesbury.”

Keep reading...Show less
Yellow
Hotspots

The Midwest Is Becoming Even Tougher for Solar Projects

And more on the week’s most important conflicts around renewables.

The United States.
Heatmap Illustration/Getty Images

1. Wells County, Indiana – One of the nation’s most at-risk solar projects may now be prompting a full on moratorium.

  • Late last week, this county was teed up to potentially advance a new restrictive solar ordinance that would’ve cut off zoning access for large-scale facilities. That’s obviously bad for developers. But it would’ve still allowed solar facilities up to 50 acres and grandfathered in projects that had previously signed agreements with local officials.
  • However, solar opponents swamped the county Area Planning Commission meeting to decide on the ordinance, turning it into an over four-hour display in which many requested in public comments to outright ban solar projects entirely without a grandfathering clause.
  • It’s clear part of the opposition is inflamed over the EDF Paddlefish Solar project, which we ranked last year as one of the nation’s top imperiled renewables facilities in progress. The project has already resulted in a moratorium in another county, Huntington.
  • Although the Paddlefish project is not unique in its risks, it is what we view as a bellwether for the future of solar development in farming communities, as the Fort Wayne-adjacent county is a picturesque display of many areas across the United States. Pro-renewables advocates have sought to tamp down opposition with tactics such as a direct text messaging campaign, which I previously scooped last week.
  • Yet despite the counter-communications, momentum is heading in the other direction. At the meeting, officials ultimately decided to punt a decision to next month so they could edit their draft ordinance to assuage aggrieved residents.
  • Also worth noting: anyone could see from Heatmap Pro data that this county would be an incredibly difficult fight for a solar developer. Despite a slim majority of local support for renewable energy, the county has a nearly 100% opposition risk rating, due in no small part to its large agricultural workforce and MAGA leanings.

2. Clark County, Ohio – Another Ohio county has significantly restricted renewable energy development, this time with big political implications.

Keep reading...Show less
Yellow
Q&A

How a Heatmap Reader Beat a Battery Storage Ban

A conversation with Jeff Seidman, a professor at Vassar College.

Jeffrey Seidman.
Heatmap Illustration

This week’s conversation is with Jeff Seidman, a professor at Vassar College and an avid Heatmap News reader. Last week Seidman claimed a personal victory: he successfully led an effort to overturn a moratorium on battery storage development in the town of Poughkeepsie in Hudson Valley, New York. After reading a thread about the effort he posted to BlueSky, I reached out to chat about what my readers might learn from his endeavors – and how they could replicate them, should they want to.

The following conversation was lightly edited for clarity.

Keep reading...Show less
Yellow