This transcript has been automatically generated.
Subscribe to “Shift Key” and find this episode on Apple Podcasts, Spotify, Amazon, or wherever you get your podcasts.
You can also add the show’s RSS feed to your podcast app to follow us directly.
Robinson Meyer:
Hello, it’s Tuesday, March 26, and the second unofficial day of summer here in the United States. yesterday was the first. And at least as of when markets closed last week, the chip maker NVIDIA was the world’s most valuable company. It currently has a market cap of around $5.3 trillion. The next biggest company, Alphabet or Google, is worth $4.6 trillion. Just last week, NVIDIA released its financial results for the first quarter, and it was another blowout. It was expected to generate just under $79 billion in revenue. Instead, it delivered $82 billion. That’s up 20% from the previous quarter and up 85% year over year. NVIDIA has now beaten Wall Street expectations for 14 quarters in a row.
Robinson Meyer:
I go into all of this, not because Shift Key is a technology business podcast, we are not, but to illustrate the centrality of NVIDIA to artificial intelligence and I think to the broader American economy right now.
Robinson Meyer:
NVIDIA produces the physical infrastructure behind the AI and data center boom. And since that boom is the biggest story in electricity, climate, and even energy, NVIDIA is probably the most important company to energy, electricity, and climate too. After all, America’s tech companies are building solar panels and batteries and gas turbines specifically to power NVIDIA chips. When we talk about data centers being built across American communities, we’re talking about warehouses holding NVIDIA chips. Utilities are tripping over themselves to power and have access to warehouses powering NVIDIA chips. NVIDIA chips are where America’s dominance of the global software and AI industries meets America’s physical economy. That is the actual electrons, copper wires, gas molecules, and infrastructure that runs through America’s towns and cities. So I’m excited to welcome to the Shift Key today, Josh Parker. He is NVIDIA’s Head of Sustainability, a role he’s held since 2023. Before that, he was Head of Sustainability and Assistant General Counsel at Western Digital. Josh and I had a good conversation last week. We talked about why he thinks AI is a net good for climate change, about whether AI and NVIDIA are already cutting emissions on the power grid, and about NVIDIA’s work with clean energy companies, as well as fossil fuel companies. It’s a very interesting conversation. I learned a lot from it. I’m Robinson Meyer, the founding executive editor of Heatmap News, and it’s all coming up on Shift Key.
Robinson Meyer:
Josh, welcome to Shift Key.
Josh Parker:
Thanks, I’m thrilled to be here.
Robinson Meyer:
So you joined NVIDIA in August 2023, which was right a few months after ChatGPT came out and completely changed the AI conversation. What did you walk into at the time? And where was the internal conversation around sustainability and climate at that moment in NVIDIA?
Josh Parker:
It was a really unique and wonderful time to join NVIDIA. You know, the company was just doing amazing things. the whole world was starting to wrap its head around the fact that AI was useful and was finally here in ways that would transform the world, transform the economy, and really our existence. And so the timing was fantastic for me, really thrilling just based on where the company was, what it was doing, and the whole conversation around it. The sustainability conversation was one of growing interest at NVIDIA. Jensen, our CEO, really has this vision of technology helping to solve the world’s biggest challenges. And sustainability is, of course, one aspect of that. Things like climate change and materials resources and water conservation. And he believed that AI had a very critical role to play in sustainability in the near future. And the company was looking to expand its sustainability program and efforts. And so I was very fortunate to come in at a time when the company was really trying to accelerate that program and find new ways to use tech for good and also to be a responsible organization ourselves.
Robinson Meyer:
How do you think about NVIDIA and sustainability today? What are the goals that you have? Because obviously at this point depends slightly on the day, but recently it’s the world’s most valuable company. It’s driving this enormous infrastructure boom. NVIDIA provides the physical infrastructure of the AI boom. And so to some degree, it’s an every sector of the economy story. And I wonder, given the company’s enormous importance right now, how do you think about its sustainability goals and what you focus on?
Josh Parker:
NVIDIA is a pretty unique company just across all the metrics. The culture here is very unique, very dynamic, and we could get into that and have a whole podcast on it. But the sustainability space follows that same pattern. We have a very unique approach to sustainability, I think, based on NVIDIA’s role in the ecosystem.
Josh Parker:
One of the first things that I did when I joined NVIDIA was to start some analyses.
Josh Parker:
Some incredible third-party validated products carbon footprints for some of our high-volume projects to figure out what does the data show us about where our lifecycle impacts are. So if you look in gaming or in AI or 3D modeling, pro-visualization, what are the kind of soup to nuts, cradle to grave hotspots for emissions in particular and then other impacts as well? And when you look at that, you very quickly realize that NVIDIA’s direct footprint, and this is something most people would understand just conceptually, NVIDIA’s direct footprint is a tiny, tiny fraction of the total lifecycle impacts of our products. So while traditional sustainability programs, especially tech companies that involve manufacturing and perhaps downstream use as well, really focus on their own footprint, if we focus myopically on our own footprint, we’re missing the forest for the trees. So very quickly realized that Jensen’s vision about sustainability and about AI’s potential to impact sustainability issues was much, much more significant than NVIDIA’s direct impacts through our operations. And so as a result of that, we’ve been focused from day one, really, on trying to unlock applications of AI for sustainability and to work with our value chain partners, both upstream and downstream.
Josh Parker:
To decarbonize, to manage impacts, et cetera, across the value chain. So it’s a lot more outward-focused sustainability program than most, which I think makes a lot of sense based on where NVIDIA sits in the ecosystem.
Robinson Meyer:
And can you talk a little bit maybe about why that is because i think what many listeners i expect will understand but just to be clear here nvidia designs its chips and it operates them as well but it doesn’t actually produce the chips the chips are usually produced by tsmc or another outside chip fab and so I guess from your standpoint, then, that makes these external projects especially important. But like, where did the emissions, I guess, in NVIDIA’s world come from? Do you focus on emissions as, you know, a key metric here?
Josh Parker:
Yes, our top issue for sustainability since I arrived at NVIDIA has been emissions and climate change. So that has been the top focus for us. And yeah, if you look at the value chain emissions and those product carbon footprints that I mentioned, we’ve published summaries of those that are cradle to gate. So they start from the very beginning of the value chain and end kind of when we ship our products to our customers, because we don’t have as good a visibility to how our customers are using our platform. But we are, as a company, historically, it’s accurate to say we were a chip design company. Nowadays, we’re more of kind of a platform infrastructure solutions company, but we are focused very much on the design. So on the AI side, we do very advanced networking. We have CPUs, GPUs, data center architecture. We co-design things like cooling solutions for data centers, and we publish reference designs for those. And then we work with manufacturing partners, contract manufacturers to actually build the systems and then to sell them. And then we do operate some data centers, but most of our business is really selling the tools, the infrastructure to the companies that go out and build great things with that infrastructure.
Robinson Meyer:
What’s the most important metric to focus? I mean, we were talking about emissions, but in terms of understanding kind of NVIDIA sustainability goals, what’s the most important metric to focus on?
Josh Parker:
I think at a moment when AI is growing rapidly, transforming the world, the most useful metric is one that takes into account both the footprint and the handprint. So it takes into account the impacts as well as the potential offsets, the benefits, the transformational impacts down the road. Now, consolidating that into a single metric is really difficult, but there are some studies that have tried to look at least the net impact on greenhouse gas emissions of AI broadly. So that’s, I think, the best indication of, is AI a hero or a villain or somewhere in between in terms of climate change and greenhouse gas emissions in particular? And the very rapidly growing consensus is that AI is most likely to lead to net emissions reductions, especially if it’s deployed broadly. So organizations like the International Energy Agency, World Economic Forum, Boston Consulting Group, Grantham Institute, have all come to that conclusion that AI, because of its transformational impacts on other sectors in particular around energy efficiency and so forth, is poised to drive net emissions reductions. So if I were to pick a metric, I would say, what’s the net impact on emissions that AI is creating? And it’s really a positive one if you look at those studies.
Robinson Meyer:
Can you... So... I think that this is like get set to some degree, the question that I want to talk about while we have your time, which is that there’s enormous focus on the on the energy use from AI, right? And of course, the energy use from chips. And we can talk about chip efficiency and what NVIDIA is doing there. And I think it’d be good to talk about it. But it does seem like to kind of step back that we are in this moment of massive infrastructure investment in AI. And that infrastructure investment is going to happen. And regardless, at this point, I think it’s just AI is too valuable. It’s too obviously useful for that infrastructure investment not to happen. And what we track at Heatmap and we look at data centers get built across the country and we become aware, for instance, that there’s a lot of off-site, you know, behind the meter gas being built to service these data centers. Obviously, there’s going to be a surge in electricity demand and there’s ways
Robinson Meyer:
in which electricity demand increases can be good. But just as we think through the next five years, given that at this point, the AI investment boom is happening and to some degree, you know, the AI story is a foregone conclusion. What needs to be true for AI to have been good for the climate or for NVIDIA’s efforts here to have been good for the climate?
Josh Parker:
The biggest variable in that analysis of what’s the net impact of AI is really, again, if you look at those studies that I mentioned, including the International Energy Agency, is how broadly we apply it in the near term. So yes, the infrastructure is getting built, it’s getting used, And contrary to what most of us consumers conceive of as AI, the vast majority of the really useful cases of AI is not the chatbots that you’re engaging with. It’s not the dogs surfing in Hawaii videos and photos that people create in their spare time. It’s the commercial applications where AI is saving energy. It’s saving material resources and so forth. And that infrastructure is being deployed for that purpose, in addition to the chatbots. And the real opportunity for us is to say, okay, we’ve got these amazing models. You’ve got Claude, you’ve got Gemini, ChatGPT, X.
Josh Parker:
They’re really, really powerful and obviously just growing in capabilities month over month. There’s so much potential there for those to transform manufacturing, for example, digital twins. And we see proof points of AI reducing energy in manufacturing by around 30% across the board if AI is deployed to optimize manufacturing for energy. That happened at one of our manufacturing partners in Guadalajara, Mexico, for example, a 30% reduction in energy. And so the opportunity is, and the risk is that if we build out all this infrastructure and we don’t use it effectively, if we don’t apply the AI to these big problems, then we may miss out on those significant emissions reductions. So what needs to be true, the biggest variable here is, are we taking advantage of what we’ve built? Because the infrastructure, like you said, is being built and it’s being used, but can we deploy it more broadly And can we bring in some of the sustainability-focused organizations to deploy it for good? How do we intentionally use AI for good in addition to the kind of regular efficiency, revenue, and cost-driven allocations that are happening very naturally and have very, very significant gains across sustainability? There are also very purpose-driven applications of AI that can have big impacts as well.
Robinson Meyer:
Do you think that AI by itself increases efficiency where it’s applied in that, you know, if you apply it to manufacturing, for instance, or another one of these industrial uses that it’s going to just increase the efficiency of that process by dint of its application and being very intelligent and finding, you know, ways to streamline processes or skip processes or augment processes that maybe wouldn’t have been considered otherwise? Or does it need to be applied in an intentional way where people say we need to look at this for efficiency or for emissions and that should be our main focus here.
Josh Parker:
So that’s the beauty of the concept of efficiency in free market is that the incentives to reduce costs are really well aligned with sustainability goals of reducing impacts, reducing consumption, and so forth. And so what we are seeing, and I think this will even grow more over time once we get out of this kind of Cambrian explosion of tech innovation that we’re in right now, which is a little chaotic, is that you’ll see optimization of, okay.
Josh Parker:
Using a huge LLM for this problem might be good, but it might not be the best tool for that particular task. Can we use a lighter weight model? And you see tons of innovation in this space. Mixture of experts has been around for a long time. We’re seeing a lot more innovation around how to use more efficient models and target them to specific applications. But the market and kind of customer demands and everything is really driving us. Plus supply constraints, compute constraints are really driving us towards efficiency and to optimize allocation of those resources. And if AI doesn’t end up being the right tool for every task, then it won’t be used there. And we can continue to use traditional techniques. But efficiency does happen to be one of AI’s kind of low-hanging fruits, one of its superpowers that is really easy to unlock and unlocks value immediately across the board. So it is very fundamentally true in general that AI does drive efficiency very, very rapidly in most areas.
Robinson Meyer:
I think what I hear you saying is that a lot of the good that will ultimately come from this build out there will be done from intentionally applying AI to intentional sustainability problems. Is that wrong? Or is it also just the diffusion? I mean, we were just talking about efficiency. So I guess that’s on the other side. But in your kind of first answer, I did hear a sense that a lot of the most important work on sustainability will come from NVIDIA intentionally applying its technology to sustainability problems.
Josh Parker:
I would say that’s important, mostly because it does require us to think about it and to do something. It’s not being driven necessarily automatically by existing incentives and market dynamics. So the market dynamics and the efficiencies that are being driven by that, like a 30% reduction in manufacturing efficiency, it’s really mind-boggling. When you think about we’re concerned about the energy that is being consumed by AI, AI still represents less than 1% of total electricity consumption worldwide. Now, it’s obviously higher in some regions, higher in the United States.
Robinson Meyer:
And it’s about to go up a lot too, is the other side.
Josh Parker:
No, it’s expected to double by 2030. So it’s growing very rapidly. But if you think about AI’s existing footprint, again, less than 1% of global electricity right now, even if it doubles, doubles again, doubles again, it’s still going to be a small share of global electricity. If, as we’re seeing the proof points for, it can reduce energy in much, much larger energy consuming sectors like transportation, like buildings, like industry, which are each in the 20 to 40% range of global electricity, then those savings dwarf AI’s footprint unambiguously. And that incentive is there because companies want to reduce costs. They want to reduce their energy consumption, especially when we’re in this environment of energy constraint, particularly in the United States, the incentives are there. So that is going to happen. I think that’s kind of inevitable because it’s an opportunity. There’s value and there’s sustainability. It’s good for everybody and the stars have aligned. The...
Josh Parker:
Additional piece is applications of AI intentionally for sustainability. And that’s where maybe it won’t happen unless we think about it, unless we try to apply it there. And the potential is just phenomenal. When you think about the way AI is already transforming drug discovery and healthcare and material science, there’s potential in nuclear fusion, advanced fission, geothermal, and carbon capture and storage just across the board. When you add intelligence to these sustainability challenges, you arrive at this wonderful inflection point where we might finally have a technology that can sufficiently complement policy to help us actually prevail on some of these sustainability challenges, help us to kind of reverse things and make progress that we otherwise wouldn’t have the opportunity to do.
Robinson Meyer:
There’s two types of AI that we’re talking about here, and I wonder if we can disambiguate them a little bit, in part just for my understanding. So there’s the large language models, which I feel like are the charismatic megafauna of AI. This is Claude, it’s ChatGPT, it’s Grok. Those are the models that I think people are most likely to have experienced when they think of AI. But there’s also this whole other set of AI applications, which I feel like you’ve alluded to, applying it to manufacturing, applying it to drug discovery, applying it to energy. And my sense is that type of ai it doesn’t look like Claude or it doesn’t look like ChatGPT it might have the same kind of organic structure where it was trained on a large data set and kind of allowed to self train itself on that data but it doesn’t have the same interface it’s much more kind of machine brains than maybe the LLMs of the world and to the extent you could share this data to what extent is ai demand and nvidia’s demand and energy use coming from the LLMs of the world like claude and Grok and ChatGPT versus these other AI applications.
Josh Parker:
It is true. There are very different applications of AI depending on the sector, and the consumer-facing chatbots that you see are one small use case and not where you see the biggest opportunities for advances in sustainability through AI, of course. Things like digital twins, for example, and that’s a really interesting marriage of NVIDIA’s expertise in 3D modeling and AI. And that is a very fundamentally valuable concept and technology for things like the manufacturing optimization that I was talking about.
Robinson Meyer:
You build a digital simulation of a real-life factory or physical space, right? Right.
Josh Parker:
That’s right. Yeah. And they become, it’s a lot more than what it sounds like at first blush, just a 3D rendering of a building. You actually can simulate robots going through this factory, simulate the airflow through the factory and the cooling system and all of the impacts of various factors on it. So it’s very complicated, and the emulations enabled by the AI really make the technology as valuable as it is today. That’s one example of something that is obviously not a chatbot that is fundamentally just extremely valuable when it comes to sustainability applications of AI. But there is actually substantial overlap. So when you see Anthropic training Claude Opus and devoting all of these resources to training that huge LLM, so many parameters, and same thing with ChatGPT and Gemini.
Josh Parker:
Those very large, large language models end up being really useful tools for helping us create more bespoke, lighter weight custom models as well that can do other things. So the multimodality functionality of modern day LLMs is just going through the roof. And the result of that is that these foundational models become even more valuable for lighter weight, more tailored applications of AI. So it’s true that the actual application of them in other areas probably won’t be the exact same model that was the huge foundational model that you started from, but through distillation and other techniques, you may end up using that as the basis for one of those other models.
Robinson Meyer:
There’s been a lot of excitement and i believe nvidia has invested in a number of companies or at least emerald ai companies that are look looking at whether data centers can be flexed up or down to meet the grid needs of the moment so instead of data centers simply being a huge energy suck on the grid they could modulate their usage and their they could modulate their compute and therefore their energy usage to kind of meet the grid’s needs i know nvidia is invested in this Can you give us a sense of where does that project stand right now in between, say, white paper and deployed scale?
Josh Parker:
So we are actively deploying this technology at our data centers. We’re building a data center right now in Virginia that will come online, I believe, later this year, that is, we think, the world’s first entirely flexible data center for AI. And we do see this as the future because it leads to a situation where we’re making better use of existing energy resources. And this is something that’s really, I think, underappreciated. And it might be a little nuanced for most people who don’t follow this to appreciate, but the concept of AI data centers becoming grid assets is really powerful because they’re being deployed rapidly. They’re using a lot of energy. And if they end up being good citizens of the electrical grid, then that can have actually a profound reductive impact on energy prices for retail consumers like you and me. The concept here is you have a grid that is built for peak load. So in the middle of the summer in Texas, when everybody’s running their AC units and you’re consuming the maximum energy that the system can deliver, that is what the system is designed for. So when you’re not at peak load, what does that mean? That means that all of those resources that you’ve built for the peak load are being underutilized.
Josh Parker:
This leads to the conversation about smart grids and virtual power plants, where I think everybody that looks into this closely wants to get where we’re saying, okay, how can we be more flexible, both primarily with our demand, but also on the variable generation side, how can we make better use of wind and solar that aren’t for power sources?
Josh Parker:
Data centers play a huge role in that, especially as they become a higher percentage of electricity consumption in the United States. If a data center can say, okay, I’m in Texas, I’m in the ERCOT region, and it’s a hot day in late July, everybody’s running their AC, I’m going to curtail my electricity draw slightly for a few hours until the system can get back to below peak load, and then I’ll ramp back up. That ends up becoming a net asset because you’re able to soak up the electrons when they’re more available and then reduce your load when they’re less available, which means we’re paying money for electricity that is otherwise being unused with existing grid infrastructure. So it’s fantastic for consumers. It’s fantastic for the energy sector. And it’s good for data centers because it means we can build them sooner and take advantage of existing resources. And one last comment on this, you may know that the concept of Emerald AI and this data center flexibility ties back to a study last year by Tyler Norris at Duke University, who said there’s 100 gigawatts.
Robinson Meyer:
And a Shift Key listener, I believe.
Josh Parker:
Yes, as am I. Yeah, I just want to get that in there as well.
Robinson Meyer:
Thank you.
Josh Parker:
Yeah, no, it’s fantastic work that you do, Shifky and heatmap. So 100 gigawatts, that is a ton of energy that could be accessed if we just ask data centers to be flexible for 1% of the year. And so that’s the concept here. It’s making the energy sector electrical generation more efficient, which leads to lower prices over time and better utilization.
Robinson Meyer:
I think when Tyler’s paper came out last year and when there was the initial wave of discussion about flexible data centers, the thought was that data centers would be flexing their compute, that they would change the operation, the programming, or the level of training that was happening in the data center at that moment to match real-life grid conditions. Since then, the focus has shifted more to data centers flexing how much energy they draw from the grid, but maybe the training itself or whatever compute is happening being more stable. It’s just the question is whether the facility is drawing from the grid or from battery storage that’s on site. When you talk about this data center in Virginia, or when you talk about flexible data centers going forward, are they flexing the compute mostly, or are they mostly flexing their grid use and where they draw electricity from? And sometimes they’re drawing electricity from the grid, and sometimes they’re drawing it from on-site batteries. But most of the flexibility per se is coming from where the electricity is coming from and not how much electricity is being used.
Josh Parker:
It’s really a mix. And where we end up will really depend on what customers the data center is serving, whether it’s a mix, whether they’re being served locally, whether it’s focused primarily on training versus inference. So what we’ll end up seeing is there will be a wide variety, I think, of data centers with different types of flexibility, perhaps, based on the needs of the data center. So if you have a data center that is running critical infrastructure and needs to be available even at peak load, then you may have more incentive to build out a large array of batteries so that you can continue to use that compute even when you’re at peak load on the grid and you can still be a good.
Josh Parker:
Citizen of the electrical grid by reducing your draw from the grid. But there are three different types of flexibility that we’re building into this framework. One of them is what you mentioned with batteries, where you can say, okay, grid’s at peak load. I’m going to use my batteries now temporarily instead. Good citizen. The second is also what we’ve been discussing, which is when you just ramp down your compute, you can say, some of the workloads that I have, I can pause on for a couple hours without deteriorating service or having any significant problems, it’s okay to pause right now. The third type of flexibility that doesn’t get spoken about as much, but that is rapidly developing is geographic flexibility. So if you have workloads that are really vital, but maybe you don’t have the battery storage on site to keep your compute running full steam all the time, you could actually transmit that workload to a different geography. Maybe somewhere in the Pacific Northwest, they’re not experiencing the same heat wave that they are in Texas. And the way a lot of interaction with AI works, that additional latency due to the different geography isn’t a huge factor because there’s already some delay built into the compute.
Robinson Meyer:
So latency is less of a... Is that training or inference that you would move geographically? Like, would you send the inference out to the Pacific Northwest? Or is this, you would actually send a training task out to the Pacific Northwest. And then it doesn’t matter in some ways because training doesn’t happen on a scale that the customer is always aware of.
Josh Parker:
Technically, either is possible. Training, because it’s kind of a large workload, chunking it up into discrete bits and then moving the data to the location where you need to continue the training, does have some additional complexities to it. Inferencing is a little easier to move because it’s smaller chunks, smaller amounts of data. And either one, again, because of the different latency requirements for AI compared to a traditional data center service, are feasible for a lot of workloads. Some inference workloads, the latency doesn’t matter if you’re doing real-time robotics and things like that. You do care about latency, so I don’t want to overstate this. But there’s a lot of inference that can happen where the latency is not a huge issue, and so those types of workloads could be shifted.
Robinson Meyer:
In some ways, the geographic flexing kind of addresses this. But when we talk about flexing compute or flexing grid use and turning data centers into grid assets, I do have to ask, I mean, are data centers getting built in the places where that capacity or that flexibility is useful? Because it often seems like, especially at this point, they’re getting built in places where there’s just energy that’s efficient or profitable to use because compute and energy are so constrained at this moment. And maybe not in the places where, say, that flexibility is useful. Do you see that changing or are we going to go in and maybe make existing data centers flexible in places like, say, the Mid-Atlantic or Texas where that flexibility could be actually useful to customers?
Josh Parker:
Again, I think we’ll end up with a mix. So right now, especially because of the challenges that we see in getting access to energy in the near term, as we’re rushing to build AI, because it’s so valuable and so important to us, you do see data centers being built just where they can get online, where there is electricity available.
Josh Parker:
And you do see increasingly some of these companies bringing their own energy, building new solar farms because they need it, sometimes bringing online new gas. But the good news is this flexibility is available in the future when we need it. And the companies that are bringing their own energy to their data centers, I haven’t heard of any that really want to be off grid. It makes a lot of sense economically and conceptually for data centers to be part of the grid so that they can be assets. They can take advantage of the shared resources, offer benefits to the grid through improved utilization, et cetera, especially with the flex technology. So I think where we end up will be a highly interconnected mesh of data centers that can flex and can transmit data. But we do have some hurdles that we need to cross to get there, especially in the United States. So permitting reform, transmission, of course, the things that we always talk about in the energy sector. This could be the golden moment where there is enough consensus around the importance of AI from an economic development, national security.
Josh Parker:
Scientific discovery, sustainability perspective, that we can find a way to make progress on these important issues and break through some of those backlogs. If we can do that, what we’ll end up with is a smarter grid, more robust economic development, more sustainable outcomes. It really will be good for society generally and help with energy affordability as well.
Robinson Meyer:
So the data center that we were discussing earlier, you said, is set to come on later this year. I think a lot of this conversation about data center flexibility is future focused, is looking at improvements that could happen in the future. Is there a substantive example of using AI on the grid right now to improve the supply side or the overall efficiency of the grid?
Josh Parker:
If you’re asking about kind of the data center flexibility piece, we have run several pilots. In conjunction with Emerald AI in Chicago, Virginia, and the UK to demonstrate that this is viable and it works. I’m not aware of it being implemented fully at a data center yet. I think this Virginia one that we’re building now is going to be the first one that is really built around that concept. But the pilots that we’ve run, the demonstrations have been really impressive. They’ve kind of hit all the metrics that we were hoping to achieve. So we think that it’s been demonstrated conceptually, and we’re excited to see it work in real life with this new Virginia facility.
Robinson Meyer:
So when I think about the AI electricity and AI energy use story, I’m thinking back almost to 2023. I think when AI was first forecast or projected to be a very large user of energy, frankly, from a lot of folks I talked to, including guests we had on very early episodes of this podcast, there was a lot of skepticism. Because if you go back 10 or especially 20, 25 years at the end of the dot-com boom and the beginning of the aughts, there was a lot of fears that electricity, that computers, personal computers in that case, and server farms to a lesser extent, as we called them then, were going to be a major user of electricity across the U.S. And they really weren’t. Those concerns really never panned out. And that’s because the actual chips, the computers themselves, got more efficient. Now, of course, it’s become a big user of electricity. it’s totally transforming the energy system. We’re compute constrained. We’re energy constrained. We’re in a very different moment. And...
Robinson Meyer:
That has put these efficiency gains that NVIDIA has made in its chips in a totally different light. And so NVIDIA has unlocked enormous efficiency gains in recent chips. The new AI chips are far more efficient, I think 95% more efficient than previous generations. But this seems to be contributing to a dynamic like a so-called Jevons paradox where we’re using them more. I wonder how you think about the Jevons paradox and AI and do you think we’re going to get to a point where the raw efficiency gains from AI ultimately do lead to a leveling off of energy or right now are just all those efficiency gains from NVIDIA going basically to just using AI more?
Josh Parker:
So I love Devin’s paradox in this context, because I think it says something really fascinating about the unique moment that we’re in. So absolutely, the efficiency gains that we’re seeing in AI are just astounding. And I’m not aware of any technology in history that has seen the type of efficiency gains, the magnitude of efficiency gains that we’ve seen in AI over the past decade or so. So we’re talking 100,000-time improvement in energy efficiency in the past decade. And the IEA, their estimate, which is actually a little lower than ours, is that on average, we see a 10x improvement in energy efficiency year over year with AI. And that improvement, which means, by the way, if you’re running an AI task now and you run the same AI task in five weeks, on average, it will use half the electricity in just five weeks. Again, aggregate and average if you’re doing the same task.
Josh Parker:
So that is a huge countervailing variable in terms of aggregate energy use by AI. But of course, the reason we’re building out more data centers and we need more energy for them is because AI is so incredibly valuable that even despite those energy efficiency gains, we need more of it. The scaling laws are holding so that more compute does translate into significantly more intelligence. And that intelligence is what is driving value across sectors in so many different areas. So to answer your question about where do we end up, I think it’s very clear based on what we’ve seen over the past couple of years, aggregate energy is growing, that it’s focused on AI. Still relatively low baseline globally again, but it’s growing and we expect it to continue to grow rapidly. Now, the question is, is that a problem? And I think if you look at it, there’s, again, this risk of losing the forest for the trees. On the sustainability front.
Josh Parker:
Do we care if AI uses more energy consumption if at the same time it’s reducing energy in other sectors at a much faster rate? So what we care about with emissions is net emissions. What we care about in energy, it’s actually less clear because sometimes energy growth is actually a good thing for sustainability through advancements in clean energy and so forth. But if you just look at the emissions side, what matters globally is the net. And even if AI grows, doubles, doubles, doubles, and doubles its emissions as well, which I don’t think is the case based on the data, you’ll end up in a world that has emissions reductions because of the huge impacts
Josh Parker:
that it’s having positively in other sectors.
Robinson Meyer:
Is there a current sector, though, where we can point and say emissions reductions are happening on a scale commensurate to the increase in data center electricity use?
Josh Parker:
In the near term, at the sectoral level, I don’t think that’s true. And that’s because we’re not deploying AI rapidly enough. Back to the earlier point about what is the key variable to capturing those emissions reductions. And again, going back to the manufacturing case, that kind of makes sense. Because for the economics of energy efficiency to convince you to tear down your existing manufacturing facility and build a new one that’s optimized, that’s a much harder case. But as everything gets naturally upgraded, as you’re ready to build a new factory, because the old one is ready to come offline, AI is undoubtedly going to be utilized in those circumstances. So over the course of the next decade, we will see entire sectors, I think, driving those net reduction that we’re already seeing the proof points for.
Robinson Meyer:
But it does sound, we are kind of in an interesting moment here where we are making a big infrastructure bet. And I understand why we’re making this infrastructure bet. And it’s kind to be reversible. And we think there’s a benefit on the other side, but we don’t fully know that yet, at least on the emissions front.
Josh Parker:
I would say that’s true, but I don’t think, I haven’t heard any arguments that suggest that the fundamentals don’t compel us in that direction. So again, sticking with manufacturing, but transportation and buildings are similar. If you’re building a new building and you have the option of using AI to manage the HVAC, manage the energy consumption, and you expect a 15 to 20% reduction in your builds, of course you’re going to use it and the economics just work out. So I don’t think it’s a question of if, it’s just a question of how rapidly the AI gets used for those purposes.
Robinson Meyer:
NVIDIA is working with a lot of companies and industries who I think have a very natural and mechanistic interest in improving their efficiency and who are very interested in improving their efficiency. NVIDIA is also working with SLB, which I think of still being called Schlumberger, putting together an AI factory for energy and for conventional energy and unlocking more fossil fuels. And it does seem to me that this is the place where AI could run against some of these sustainability goals, that instead of improving efficiency everywhere, it could cause, in the same way that we’re talking about Jevons Paradox, it could cause a general acceleration and unlock more fossil fuels and unlock more oil and gas and have those fuels be cheaper and have them crowd out the clean energy that I know NVIDIA is also working with clean energy companies too. Can you talk about how your work with SLB fits into the sustainability goals? And it does seem to me, doesn’t it kind of push against this idea that AI applied to every industry is going to make everyone more sustainable and reduce our emissions?
Josh Parker:
Yeah, so that’s a good question. And the truth is, AI really does, back to your original point, drive efficiency very easily across whatever purpose you’re trying to apply it for. So if you want to be more efficient at extracting fossil fuels, it can help with that. Now, where we end up, again, if the important thing is the net.
Josh Parker:
Then we need to look at, okay, is AI poised to accelerate fossil fuels more than it’s poised to accelerate clean energy adoption? And I think the data pretty clearly demonstrates that clean energy is likely to benefit at least as much as fossil fuels, not least because clean energy is already in many cases, if not most cases, the most economic and most secure form of energy that can be used. And then when you layer in things like this growth in energy demand that’s being driven by AI, the companies that build out those AI data centers, by and large, are looking for every clean electron they can find. Their commitments to clean energy are huge.
Josh Parker:
World-leading. And so the demand that AI is creating itself is very much focused on clean energy. That’s what Microsoft and Google and Meta, that’s the type of energy they want. And then you factor in the concepts of smart grids, VPPs, which AI can enable, and the demand flexibility of data centers themselves. That makes variable generation like solar and wind, at least incrementally more valuable relative to fossil fuels. So I think it only accelerates and improves the economics of clean energy relative to fossil fuels. So I think if, you know, agreed, AI can, I think, help fossil fuel companies be more efficient in their operations. But I think the overall demand picture is in the economics of clean energy are driving us unavoidably in that direction.
Josh Parker:
And the last thing I’ll say on this is AI is a fantastic complement to policy. It’s not a replacement. AI is technology agnostic. It helps you be more efficient at whatever you’re doing generally. But if we want policies that drive prioritization of clean energy and things like transmission and permitting reform and smart grids will lead us down that road naturally, then the policies, we should focus on the policies that unlock that feature.
Robinson Meyer:
I agree with that. The current set of companies that are using a lot of NVIDIA’s chips, most of NVIDIA’s chips and are applying AI, especially in the United States, are very focused on these clean energy goals. That’s not true of globally, right? I mean, that’s not true of China. It’s not true of the Gulf states, which I think are the next buyer of some of NVIDIA’s chips. Does this mean when we think about how to regulate AI, focus on keeping it at these American tech companies that have these clean energy goals? Yeah.
Josh Parker:
I’m not our political specialist, so I won’t be able to comment on the geopolitics of everything. But I will mention that I think the trend towards net emissions reductions enabled by AI, to me, looks almost unavoidable at this point, because the technology fundamentally helps us take better advantage of the resources that we have. So even if in the near term, we see an increase in emissions globally due to the build out of AI, I think in the medium and long term, we will end up with net reductions for all the reasons that are covered in those papers that I mentioned.
Robinson Meyer:
So Heatmap has been tracking what to us has been a very sudden and shocking rise of local pushback against AI data centers. And of course, this has become a larger meme over the past few months, as it’s gotten more attention. For instance, we think about 50 AI data centers or data centers broadly were canceled last year after facing local pushback. And we think more than 50 have already been canceled this year. Are you seeing that at all at NVIDIA? I mean, it doesn’t look your quarterly results came out yesterday and they were they absolutely blew out expectations. And so evidently it’s not affecting demand yet. But do you hear it from customers? Is this affecting NVIDIA’s business at all? And how do you think about it as a risk going forward?
Josh Parker:
So I’m aware of the sentiment, the paranoia around AI, mostly on a personal level, because I see it on social media like other people do as well. I’m not aware of any direct impact on our sales, so I can’t comment on that. But what I will say is I do think it’s particularly tragic because this technology has the potential to be the most beneficial, both for environmental goals and for social goals. So things like education and health care and kind of across the board, social issues benefit from AI as well. And the concerns about AI, a lot of them are based on either erroneous data or old data. and I worry that some people.
Josh Parker:
Don’t fully understand the net impacts, the positive as well as the negative of AI. Plus, we have the uphill battle of it’s really hard if the data center is being built a few miles down the road to tie that data center, which they don’t always look beautiful and things like that, to the benefits that the whole world is going to get from AI. So if, obviously not promising this, but AI could unlock cancer cures or cures to other diseases. And we’re seeing trends in the direction of cures and treatments and drug discovery and so forth. But it’s really hard for us as humans to draw a line between the infrastructure that we see down the street and especially the speculative, the moonshot benefits, but even the more fundamental ones, like the benefits and productivity that we’re seeing in potential for wage growth and education and so forth, even though it’s hard for us to draw the line between the infrastructure. So it’s understandable, but I do think it’s tragic. And I think it’s our responsibility in the tech industry to help people see the bigger picture and to address people’s concerns head on about environmental impacts and social impacts. Because the data really does demonstrate that, by and large, these data centers are pro-sustainability. They don’t have the impacts that most people are concerned about, and they’re manageable. And most data center operators are trying to operate them in a sustainable way.
Robinson Meyer:
Josh Parker, so much more to talk about, but we’re going to have to leave it there. Thank you so much for joining us here on Shift Key.
Josh Parker:
My pleasure. Thanks, Rob.
Robinson Meyer:
And that will do it for us on Shift Key today. We’ll be back soon with another episode. Until then, Shift Key is a production of Heatmap News. Our editors are Jillian Goodman and Nico Lauricella. Multimedia editing and audio engineering is by Jacob Lambert and by Nick Woodbury. Our music is by Adam Kromelow. Thanks so much for listening. See you next time.
