The Hidden Energy Consumer

The world is increasingly reliant on artificial intelligence (AI) to power various aspects of modern life. From email summaries to chatbots and even generating music and art, AI has become an integral part of our daily experiences. However, the true cost of this technological marvel is not as straightforward as one might think. In this article, we delve into the energy consumption of AI and the challenges of calculating its exact cost.

It is a common belief that machine learning consumes a significant amount of energy. The servers that power AI models are racking up a hefty bill, measured in megawatts per hour. But despite this knowledge, no one can provide an accurate estimate of the energy cost of AI.

Estimates do exist, but experts caution that these figures are only a partial glimpse of AI’s total energy usage. Machine learning models are incredibly variable, and their power consumption can be dramatically altered by their configuration. Moreover, the organizations best placed to provide a bill, such as Meta, Microsoft, and OpenAI, are not sharing the relevant information.

One important factor we can identify is the difference between training a model for the first time and deploying it to users. Training is extremely energy intensive, consuming much more electricity than traditional data center activities. For instance, training a large language model like GPT-3 is estimated to use just under 1,300 megawatt hours (MWh) of electricity, which is equivalent to the annual electricity consumption of 130 US homes. To put that into perspective, streaming an hour of Netflix requires around 0.8 kWh (0.0008 MWh) of electricity. That means you would have to watch 1,625,000 hours of Netflix to consume the same amount of power it takes to train GPT-3.

However, it is difficult to say how this figure applies to current state-of-the-art systems. The energy consumption could be higher due to the upward trend in model size, or it could be lower if companies are using proven methods to make these systems more energy efficient.

The challenge of making up-to-date estimates is that companies have become more secretive as AI has become profitable. In the past, firms like OpenAI would publish details of their training regimes, including the hardware and duration. However, this information no longer exists for the latest models like ChatGPT and GPT-4.

Sasha Luccioni, a researcher at French-American AI firm Hugging Face, suggests this secrecy is partly due to competition between companies but is also an attempt to divert criticism. Energy use statistics for AI, especially its most frivolous use cases, naturally invite comparisons to the wastefulness of cryptocurrency. “There’s a growing awareness that all this doesn’t come for free,” she says.

Last December, Luccioni and colleagues from Hugging Face and Carnegie Mellon University published a paper that contained the first estimates of inference energy usage of various AI models. They ran tests on 88 different models spanning a range of use cases, from answering questions to identifying objects and generating images. Most tasks they tested use a small amount of energy, like 0.002 kWh to classify written samples and 0.047 kWh to generate text. However, image-generation models use significantly more energy, with an average of 2.907 kWh per 1,000 inferences. This is equivalent to the energy consumed by charging a smartphone.

The study provides useful relative data, but it is not absolute. It shows that AI models require more power to generate output than they do when classifying input. It also shows that anything involving imagery is more energy intensive than text. Luccioni notes that the contingent nature of this data can be frustrating, but it tells a story in itself. “The generative AI revolution comes with a planetary cost that is completely unknown to us and the spread for me is particularly indicative,” she says. “The tl;dr is we just don’t know.”

So, trying to nail down the energy cost of generating a single Balenciaga pope is tricky due to the morass of variables. However, if we want to better understand the planetary cost, there are other approaches to take. One such approach is to zoom out and consider the sector’s global energy usage.

Alex de Vries, a PhD candidate at VU Amsterdam, has used Nvidia GPUs, the gold standard of AI hardware, to estimate the sector’s global energy usage. By combining Nvidia’s sales projections and energy specs for its hardware, de Vries calculates that by 2027, the AI sector could consume between 85 to 134 terawatt hours each year. That is roughly equivalent to the annual energy demand of the Netherlands.

A recent report by the International Energy Agency offered similar estimates, suggesting that electricity usage by data centers will increase significantly in the near future due to the demands of AI and cryptocurrency. The agency states that current data center energy usage stands at around 460 terawatt hours in 2022 and could increase to between 620 and 1,050 TWh in 2026, which is equivalent to the energy demands of Sweden or Germany, respectively.

However, it is essential to put these figures into context. Between 2010 and 2018, data center energy usage has been fairly stable, accounting for around 1 to 2 percent of global consumption. De Vries notes that demand went up during this period, but the hardware became more efficient, thus offsetting the increase. His concern is that things might be different for AI due to the trend for companies to simply throw bigger models and more data at any task. “That is a really deadly dynamic for efficiency,” says de Vries. “Because it creates a natural incentive for people to just keep adding more computational resources, and as soon as models or hardware becomes more efficient, people will make those models even bigger than before.”

In conclusion, the energy cost of AI is a complex issue with many variables. While estimates exist, they are only a partial glimpse of the true cost. The trend towards larger models and more data is a concern for efficiency, and the sector’s global energy usage is a significant concern. As AI continues to play an increasingly prominent role in our lives, it is essential to understand its energy consumption and find ways to make it more efficient.