Reactions to MIT Technology Review's report on AI and the environment

Interesting useful facts, and some framing I think is super misleading

I was excited that MIT Technology Review just published “We did the math on AI’s energy footprint. Here’s the story you haven’t heard.”

The topic is sorely lacking in estimates from experts. Only a few people have actually taken a shot at guessing how much energy chatbots use per prompt. This article collected new useful estimates that include all the extra energy used in data centers beyond the AI chips for processes like cooling. This adds significantly to our short list of other guesses. It’s really useful! The report is also a great overview of just how wildly AI is expected to grow over the next few years, and how we’ll power it. You should read it!

I had some strong positive and negative reactions to different parts of the piece. Because this seems likely to be the big new post on chatbots and the environment for a while, I wanted to show how the data and framing compares to my last few posts.

In all these articles I try to add a lot of useful facts about AI and energy, so even if you’re not interested in the article I’m reviewing, it might still be interesting!

Some quick takes:

• This is all in line with everything I’ve written so far. I don’t need to adjust anything, except to say that new data backs my posts up even more.

• AI video does in fact seem environmentally wasteful. I didn’t address AI video at all in my posts. I might add a note that it’s very different from text and images. If the authors are right, AI uses something like 500 times as much energy as a chatbot prompt to produce just 5 seconds of video. It uses almost a 30th of the average American’s energy consumption for each 5 second video. That’s wild! If that’s true, I’m happy to say that people should avoid using AI video unless they’re pretty confident they’ll produce something of value. I’ve had a lot of people reach out to me since posting this saying they’re pretty skeptical that the number’s this large. When I have time I’ll write more summarizing their arguments.

• AI is going to be a gigantic part of our energy grid soon. It’s good to see new numbers on this. I’d like more people to wake up to how much energy demand it’s about to create and what we should do about it.

• Someone reading the article might take from it that their personal use of chatbots is bad for the environment. This is interesting, because the article makes all the same claims about chatbot energy use as mine. I disagree with the authors’ framing of the numbers and will go into detail about why below. I think a specific central part of the article is extremely misleading.

I’m not claiming to know more than the scientists who study this. I’m only disagreeing with the framing, not the facts. I have a physics degree. I know enough about energy to be able to debate and disagree with framings of how much energy society is using and where it’s causing the most harm.

I’ll write this as a reply to different important quotes from the piece, going in the order they appear. These will either be interesting takeaways, or criticisms. I’ll bold any parts of the quotes I think are especially important. All bolding is from me.

Tallying AI’s energy use is useful (I should know!)

Tallies of AI’s energy use often short-circuit the conversation—either by scolding individual behavior, or by triggering comparisons to bigger climate offenders. Both reactions dodge the point: AI is unavoidable, and even if a single query is low-impact, governments and companies are now shaping a much larger energy future around AI’s needs.

We’re taking a different approach with an accounting meant to inform the many decisions still ahead: where data centers go, what powers them, and how to make the growing toll of AI visible and accountable.

Obviously I don’t think MY tallies of AI’s energy use short-circuit the conversation, so I want to clarify some things here:

• All my posts have been about why your individual chatbot use is not harming the climate. I try to say over and over that I’m not making claims about the future of AI more broadly. I think AI’s going to be a gigantic part of our energy grid, it’s going to happen soon, and policymakers need to react. I also think AI labs should be regulated way way way more than they currently are. My posts aren’t dodging this point. We can walk and chew gum.

• A lot of authors underestimate just how beset with guilt people are about using chatbots, because those people think they’re raising their individual emissions a lot. That misconception is what I’m trying to fix. In almost every online community, I’ve seen people shamed for using chatbots and image generators because of the energy used in each prompt. Almost everyone new I talk to brings up how bad individually using AI is for the climate. We need more people making these kinds of simple tallies showing “Hey, given what we know, this specifically is a huge overreaction and chatbots aren’t meaningfully raising your carbon footprint at all.”

• Comparisons to bigger climate offenders do matter. The climate movement needs to prioritize where it’s going to spend its limited time and energy. Chatbots right now are not a serious climate issue, AI more broadly is bad for local environments but also doesn’t rank high on global emissions, future AI will be a problem. Getting this right matters a lot, so I do think it’s very important to make comparisons between chatbots as they exist now and the real climate enemies like fossil fuel power plants, driving, and the meat industry.

I fully support the broader push to make AI energy use more transparent and hold labs accountable, but I disagree that posts explaining why your individual AI use isn’t bad for the environment are “dodging the point.” I think it’s a huge problem that so many people misunderstand their individual AI environmental impacts and I want it fixed.

We do know some things about OpenAI’s energy use

“The closed AI model providers are serving up a total black box,” says Boris Gamazaychikov, head of AI sustainability at Salesforce, who has led efforts with researchers at Hugging Face, an AI platform provider of tools, models, and libraries for individuals and companies, to make AI’s energy demands more transparent. Without more disclosure from companies, it’s not just that we don’t have good estimates—we have little to go on at all.

I don’t understand this take. AI is complicated, but it’s not a complete mystery. We know a lot about the general systems and machines OpenAI is using. We know roughly how many advanced chips they own. If we just assume the chips are running all the time, we can make reasonable upper-bound estimates. We don’t need an exact down-to-the-watt-hour measurement of how much energy ChatGPT is using to make some reasonable claims about how bad it is for the environment relative to other things we do. Here’s roughly how many H100-equivalent chips each major AI company owns:

Source

Let’s assume Microsoft/OpenAI owns 800,000 H100-equivalent chips (the upper bound on the graph). Each H100 uses 700 W when at full capacity. Let’s also assume that every single chip is constantly answering ChatGPT prompts and isn’t being used for anything else. In that case, ChatGPT would be using 13,440,000,000 Wh per day. If ChatGPT gets 1 billion prompts per day, each individual prompt uses an average of 13.4 Wh. That’s the maximum upper bound of the average ChatGPT prompt.

This just doesn’t seem so fundamentally mysterious that we need to go around saying “We have little to go on at all.” That misleads the average reader about how much we actually know about AI chips and companies.

The report finds that AI text and image prompts use less energy than 3 Wh, which I used as an upper bound in my posts

The smallest model in our Llama cohort, Llama 3.1 8B, has 8 billion parameters—essentially the adjustable “knobs” in an AI model that allow it to make predictions. When tested on a variety of different text-generating prompts, like making a travel itinerary for Istanbul or explaining quantum computing, the model required about 57 joules per response, or an estimated 114 joules when accounting for cooling, other computations, and other demands. This is tiny—about what it takes to ride six feet on an e-bike, or run a microwave for one-tenth of a second.

The largest of our text-generation cohort, Llama 3.1 405B, has 50 times more parameters. More parameters generally means better answers but more energy required for each response. On average, this model needed 3,353 joules, or an estimated 6,706 joules total, for each response. That’s enough to carry a person about 400 feet on an e-bike or run the microwave for eight seconds.

A Watt-hour is 3,600 Joules, so if Llama 3.1 405B uses 6,706 Joules per response (accounting for cooling, other computations, and other demands) it’s using 1.9 Wh. So even including all the extra energy used in the data center outside of the AI chip, the average prompt still falls below the 3 Wh upper bound I set in my original posts. So far everything there is consistent with my estimates for chatbots. Nice! My previous posts seem to be validated by these numbers.

Should we assume ChatGPT uses more or less energy than LLaMA 3.1?

LLaMA 3.1 is a “dense” model, meaning all 405 billion of its parameters are active every time it generates text, which requires a lot of computation and energy. GPT-4o, by contrast, uses a technique called a Mixture of Experts, where only a subset of its total parameters (around 220 billion) are active at once, making it more efficient per prompt. That means GPT-4o probably uses less energy for a typical text-based interaction (despite being a larger model overall) because it spreads its compute across many specialized experts and only taps the relevant ones. While we don’t know for sure, I would personally be surprised if ChatGPT used more energy on an average prompt than LLaMA 3.1.

I would’ve liked this to be mentioned somewhere in the article. Instead we just get this point:

There is a significant caveat to this math. These numbers cannot serve as a proxy for how much energy is required to power something like ChatGPT 4o. We don’t know how many parameters are in OpenAI’s newest models, how many of those parameters are used for different model architectures, or which data centers are used and how OpenAI may distribute requests across all these systems. You can guess, as many have done, but those guesses are so approximate that they may be more distracting than helpful.

I think we do actually know enough about both models to say “We can’t be sure, but it’s likely that ChatGPT uses less energy on average than LLaMA 3.1.”

AI images use less energy than I expected

Generating a standard-quality image (1024 x 1024 pixels) with Stable Diffusion 3 Medium, the leading open-source image generator, with 2 billion parameters, requires about 1,141 joules of GPU energy. With diffusion models, unlike large language models, there are no estimates of how much GPUs are responsible for the total energy required, but experts suggested we stick with the “doubling” approach we’ve used thus far because the differences are likely subtle. That means an estimated 2,282 joules total. Improving the image quality by doubling the number diffusion steps to 50 just about doubles the energy required, to about 4,402 joules. That’s equivalent to about 250 feet on an e-bike, or around five and a half seconds running a microwave. That’s still less than the largest text model.

In my posts I assumed image models used 3 Wh based on old data, and clarified there were wide error bars on this because we don’t know. This new data makes it look like AI image models use less energy. 4,402 joules is 1.22 Wh.

The average emissions per prompt for the largest models are exactly what I assumed in my posts

Continuing from the last point chatbots use 2/3rds as much energy as I was assuming in my posts (and this includes all the extra energy costs of things like cooling, other computations, and other demands in the data center). The report also found that the average data center’s energy sources generate 48% more carbon emissions than typical American energy sources. If each prompt is 2/3rds as energy intense, but the energy is 48% more carbon intensive, we can multiply 2/3rds by 148% to get 98% to show how much our largest chatbots are actually emitting as a percentage of what I assumed. That basically means my numbers are exactly in line with our best guesses for our largest chatbots’ emissions. For now I won’t go back and adjust them. I’ll change them if new data comes along.

AI video uses so much more energy! It seems environmentally wasteful to the point that people shouldn’t use it a ton

An older version of the model, released in August, made videos at just eight frames per second at a grainy resolution—more like a GIF than a video. Each one required about 109,000 joules to produce. But three months later the company launched a larger, higher-quality model that produces five-second videos at 16 frames per second (this frame rate still isn’t high definition; it’s the one used in Hollywood’s silent era until the late 1920s). The new model uses more than 30 times more energy on each 5-second video: about 3.4 million joules, more than 700 times the energy required to generate a high-quality image. This is equivalent to riding 38 miles on an e-bike, or running a microwave for over an hour.

Whoa! That’s ridiculous! That’s 944 Wh, almost a kWh. The average American uses about 30 kWh per day in total. A single 5 second video increasing your daily energy budget by almost 1/30th does seem pretty wasteful to me! If you have a really goofy 5 second video that you’d feel justified spending 15 hours on a laptop making, then it makes sense to use AI to make it, but otherwise I’d personally pass. AI video does in fact seem bad for the environment.

I know a lot of people who use chatbots and image generators pretty regularly. I don’t know anyone who regularly makes AI videos. They look kind of ugly and confusing and seem to mostly just be useful for gimmicks (mainly brainrot¹ videos on TikTok and Instagram). It seems like it’d be really easy to just never use AI video and still get all the value you want out of current AI tools. Within one day of posting this this was invalidated by AI progress. AI video’s suddenly looking really good. I’d hold off on using it for now until we get more numbers on how much energy it’s using.

I’d really like to see more data on this. It’s big enough that I’m worried something might be off, but for now I’ll be adjusting to not making AI videos. I think I’ve made maybe 3 in total? They were never super interesting to me.

I’ve had a lot of people reach out to me since posting this saying they’re pretty skeptical that the number’s this large. When I have time I’ll write more summarizing their arguments.

My main criticism of this article: The big central attention-grabbing example of how much AI adds to your energy budget is drastically misleading

The next section gives an example of how using AI could make your daily energy use get huge quick. Do you notice anything strange?

So what might a day’s energy consumption look like for one person with an AI habit?

Let’s say you’re running a marathon as a charity runner and organizing a fundraiser to support your cause. You ask an AI model 15 questions about the best way to fundraise.

Then you make 10 attempts at an image for your flyer before you get one you are happy with, and three attempts at a five-second video to post on Instagram.

You’d use about 2.9 kilowatt-hours of electricity—enough to ride over 100 miles on an e-bike (or around 10 miles in the average electric vehicle) or run the microwave for over three and a half hours.

Reading this, you might think “That sounds crazy! I should really cut back on using AI!”

Let’s read this again, but this time adding the specific energy costs of each action, using the report’s estimates for each:

Let’s say you’re running a marathon as a charity runner and organizing a fundraiser to support your cause. You ask an AI model 15 questions about the best way to fundraise. (This uses 29 Wh)

Then you make 10 attempts at an image for your flyer before you get one you are happy with (This uses 12 Wh) and three attempts at a five-second video to post on Instagram (This uses 2832 Wh)

You’d use about 2.9 kilowatt-hours of electricity—enough to ride over 100 miles on an e-bike (or around 10 miles in the average electric vehicle) or run the microwave for over three and a half hours.

Wait a minute. One of these things is not like the other. Let’s see how these numbers look on a graph:

Of the 2.9 kilowatt-hours, 98% is from the video!

This seems like saying “You buy a pack of gum, and an energy drink, and then a 7 course meal at a Michelin Star restaurant. At the end, you’ve spend $315! You just spent so much on gum, an energy drink, and a seven course meal at a Michelin Star restaurant.” This is the wrong message to send readers. You should be saying “Look! Our numbers show that your spending on gum and energy drinks don’t add to much, but if you’re trying to save money, skip the restaurant.”

It’s confusing to me why they chose to present the data this way. Because they did the energy calculation to get 2.9 kWh, they clearly saw in the calculation itself that 98% of their reported energy cost was used by the videos. Why not add a note saying so?

What do they say next instead?

There is a significant caveat to this math. These numbers cannot serve as a proxy for how much energy is required to power something like ChatGPT 4o. We don’t know how many parameters are in OpenAI’s newest models, how many of those parameters are used for different model architectures, or which data centers are used and how OpenAI may distribute requests across all these systems. You can guess, as many have done, but those guesses are so approximate that they may be more distracting than helpful.

“We should stop trying to reverse-engineer numbers based on hearsay,” Luccioni says, “and put more pressure on these companies to actually share the real ones.” Luccioni has created the AI Energy Score, a way to rate models on their energy efficiency. But closed-source companies have to opt in. Few have, Luccioni says.

Part Three: Fuel and emissions

Now that we have an estimate of the total energy required to run an AI model to produce text, images, and videos, we can work out what that means in terms of emissions that cause climate change.

I’m really worried this will mislead readers. They should have included a note saying “This example shows that video is incredibly energy intensive, and everything else this person did with AI adds up to running a microwave for 3 minutes. Individually using ChatGPT is not bad for the environment, but skip the AI video.”

As a connoisseur of vegan chicken nuggies, I know 3 minutes in the microwave is inadequate to heat a single batch. If it’s okay for me to heat up some nuggies, it’s okay to use AI to make a ton of text and images for a marathon I’m organizing. That actually sounds way more useful than the nuggies! I’m really worried that readers will leave this example with the opposite conclusion.

I really don’t understand why these strange decisions about presenting simple numbers show up again and again in writing about AI and the environment. It should be really easy to just add nuance and say “Look, AI’s going to be a huge part of our energy grid! Here are the places it’s using a lot of energy and where it’s not.” but people often can’t seem to resist going for simplified examples that break apart when you apply high school-level math.

This piece in comparison is a fantastic criticism of how AI’s being used in science and the hype around it. It uses careful measured language and brings up the benefits and drawbacks of AI, and the good and bad incentives involved in AI and science. Why can’t all AI writing be like this?

I did appreciate that later on they said:

In reality, of course, average individual users aren’t responsible for all this power demand. Much of it is likely going toward startups and tech giants testing their models, power users exploring every new feature, and energy-heavy tasks like generating videos or avatars.

But I’m worried that the main example they use of how AI adds to your energy budget sends the opposite message. The section I quoted at the start has a bunch of cool graphics, and when it’s revealed that AI could add 2.9 kWh to your energy budget (enough to run a microwave all day) it includes an animation of a… I don’t actually know what… in a microwave exploding:

I think it’s a drastic problem that so many people incorrectly think individual chatbot use contributes to climate change. I’m really worried that the main memorable set-apart-from-the-rest part of this piece is reinforcing that exact idea.

To be clear there’s a lot of other great info, but this specifically left me pretty unhappy. This misconception is driving me crazy, so my heart sinks whenever I see an important publication reinforce it. I’m worried this is going to be one of the main effects this piece has on the discourse, even though the authors admirably and correctly note further down that individual chatbot use isn’t contributing to the problem and it’s not the place people should be focusing their attention. The exploding little dude in the microwave sends a different message!

This whole discourse is like a game of telephone. I’m worried that soon it will become common wisdom that “using AI” is like using a microwave for a whole day.

This is already happening!

I wish people felt more pressure to add error bars to show their uncertainty range

One can do some very rough math to estimate the energy impact. In February the AI research firm Epoch AI published an estimate of how much energy is used for a single ChatGPT query—an estimate that, as discussed, makes lots of assumptions that can’t be verified. Still, they calculated about 0.3 watt-hours, or 1,080 joules, per message. This falls in between our estimates for the smallest and largest Meta Llama models (and experts we consulted say that if anything, the real number is likely higher, not lower).

I don’t really like how often authors leave their audience hanging and say “the real number is likely higher” because a lot of readers 1) are really doomy about AI’s environmental impact, and 2) don’t know what a reasonable upper bound is. I think it’s very very likely that 4o uses below 3 Wh per prompt on average, but a doomy reader’s takeaway might be that a single prompt uses way more energy. I wish there were more pressure to add reasonable upper and lower bound guesses. It’d make the communication better, and I seriously doubt the authors of this paper would say, if they had to guess, that ChatGPT currently uses more than Llama 3.1.

It’s kind of crazy how little energy ChatGPT is using right now

One billion of these every day for a year would mean over 109 gigawatt-hours of electricity, enough to power 10,400 US homes for a year. If we add images and imagine that generating each one requires as much energy as it does with our high-quality image models, it’d mean an additional 35 gigawatt-hours, enough to power another 3,300 homes for a year. This is on top of the energy demands of OpenAI’s other products, like video generators, and that for all the other AI companies and startups.

13,700 homes per year is smaller than I guessed in my last posts. It’s incredibly small. In many people’s eyes, ChatGPT is the main climate villain in the AI story right now, and it’s using less energy than Lexington Massachusetts with 35,000 people.

If you don’t spent much time thinking about the climate impacts of Lexington, then ChatGPT as it exists right now shouldn’t be high up on your list either.

As the article notes:

ChatGPT is now estimated to be the fifth-most visited website in the world, just after Instagram and ahead of X.

The 5th most visited website on Earth using as much energy as Lexington seems like a massive win for energy efficiency.

The authors point out correctly that this is all about to change. AI agents are going to dramatically increase the energy cost of AI.

Being very clear about where we are now vs. where we’re headed

The precious few numbers that we have may shed a tiny sliver of light on where we stand right now, but all bets are off in the coming years,” says Luccioni.

I completely agree with this, except I think we have more than a tiny sliver of light. I want people to not worry at all about the current climate impacts of their individual AI use, and worry a lot about future AI systems. Communication about AI energy use should get both points across. Again, we can walk and chew gum.

Reinforcing that chatbots are a very small fraction of AI’s energy use

If we imagine the bulk of that was used for inference, it means enough electricity was used on AI in the US last year for every person on Earth to have exchanged more than 4,000 messages with chatbots.

That would be 32 trillion messages. There are about 1 billion ChatGPT messages per day. Assuming that was constant in the last year (it was actually lower, we’ll set an upper bound) this means that ChatGPT’s share of AI’s total energy use is 365 billion / 32 trillion = 0.01 = 1%. So at least 99% of AI’s energy use in the last year wasn’t from ChatGPT. I’ve previously argued that this is one of the main misconceptions about AI and energy use. People read about the real negative environmental impacts of AI data centers, look around, see everyone using ChatGPT, and assume there’s some connection. In fact chatbots are really small fractions of AI’s energy use. It was nice to see that reaffirmed.

Most of the rest of the article is great

Beyond my specific points of criticism, the rest of the article is great (and has pretty graphics). There are jaw dropping numbers here. I’m pretty concerned about how the next few years of AI are going to play out, and this added to that a lot. And again, I was so excited to see new useful estimates of the energy cost of text, image, and video generation. You should read it!

They were so close… but didn’t nail the ending

This appears toward the end:

When you ask an AI model to write you a joke or generate a video of a puppy, that query comes with a small but measurable energy toll and an associated amount of emissions spewed into the atmosphere. Given that each individual request often uses less energy than running a kitchen appliance for a few moments, it may seem insignificant.

But as more of us turn to AI tools, these impacts start to add up. And increasingly, you don’t need to go looking to use AI: It’s being integrated into every corner of our digital lives.

I think this is a very bad way to talk about climate impacts of individual choices. I explained why here in my original post. This used to be climate ethics 101 but I worry a lot of the movement kind of forgot about that.

An aside that the authors seem very nice

As an aside I posted a LinkedIn comment criticizing the article and got very welcoming specific replies from one of the authors. Kudos to them. I’m ultimately just some guy and always appreciate direct polite back-and-forth on this stuff.

1. Italian or otherwise