If you saw Amperon CEO Sean Kelly’s recent op-ed in Renewable Energy World, you know we’re urging the utility industry to coalesce around a single metric for predictive error in solar forecasting. If you were interested enough to click over and download the white paper, then you know the common accuracy metrics used with demand forecasting don’t work well for solar.
But even so, you may have thought: “Meh, so what?”
After all, we’re talking about relatively minor errors, related to just a small percentage of the generation mix, and with only marginal practical value, right?
Not so fast.
If you haven’t read The Economist article titled “Sun Machines” from June of this year, it’s eye-opening regarding the explosive pace of solar growth — and by extension, why solar forecasting is rapidly becoming more important. The article turns to respected analyst Michael Liebreich to explain the scope of acceleration:
“In 2004, it took the world a whole year to install a gigawatt of solar-power capacity (1gw is a billion watts, or a thousandth of a terawatt); in 2010, it took a month; in 2016, a week. In 2023 there were single days which saw a gigawatt of installation worldwide. Over the course of 2024 analysts at BloombergNEF, a data outfit, expect to see 520-655gw of capacity installed: that’s up to two 2004s a day.”
Total installed capacity gives another shocking perspective. In 2009, solar advocates at Greenpeace predicted global solar capacity would total 921 gigawatts (GW) by the year 2030. The experts at the International Energy Agency (IEA) predicted a much more conservative amount of just 244 GW by 2030. But they were both way off.
Last year, still seven years short of the 2030 deadline, the world’s solar capacity reached 1,419 (GW). That still only represented about 6% of the electricity generated worldwide, but The Economist article explains that solar module production is growing equally fast, suggesting that exponential growth is likely to continue, meaning we could go from 6% to 12% and beyond very rapidly.
Regardless of the pace of growth, we know solar forecasting will become increasingly important, because even at just 6% globally, markets with significant solar penetration already struggled with the intermittency of solar availability in 2023. In September of 2023, ERCOT experienced a Level 2 Energy Emergency Alert when reserves fell below 1,750 megawatts (MW). ERCOT gets roughly 7.5% of its electricity from solar, and on this day, solar generation performed as predicted, but wind power underperformed.
Wind forecasts are another topic all together, but to operate with confidence when reserves are low, it’s important to understand whether your solar forecast has a large or small margin of error. Similarly, as solar begins to represent a larger and larger percentage of the generation mix, solar forecasting will be an essential tool for utilities and other energy traders participating in wholesale markets. (CAISO is already above 20%.)
Utilities with solar generating resources use solar forecasting to predict their net demand and make decisions about day-ahead energy purchases. Speculative traders, too, are becoming more reliant on solar forecasting as a tool to assess their short- and mid-term positions. But like any tool, if you don’t know how sharp it is (or isn’t) it can be dangerous to use. As the amount of solar generation grows exponentially, the margins of predictive error in solar forecasting will begin to represent serious amounts of money and real-world operating consequences.
The industry needs a single metric for predictive error that is accurate and useful — and allows for comparing different demand forecasts (and by extension, their providers).
Mean absolute percentage error (MAPE) is mathematically unstable (and therefore inaccurate) when applied to solar forecasting. (Here’s the white paper, if you want to read about the math.) Normalized mean absolute error (nMAE) can exaggerate or depreciate errors, because it doesn’t account for seasonal variations in solar output.
Solar Forecast Arbiter, under the stewardship of Electric Power Research Institute (EPRI), has put forward numerous alternative metrics. But at Amperon, we’ve come to believe that capacity normalized mean absolute error (cnMAE) is the best metric to replace the common error metrics used with demand forecasting.
cnMAE solves the accuracy problems associated with MAPE and nMAE. Plus, it allows for comparison between solar installations of different sizes, which Root Mean Squared Error (RMSE) does not. This is particularly useful for independent power producers (IPPs) or utilities that want to use the accuracy of their solar forecasts to investigate performance and prioritize maintenance across portfolios of solar installations with varying capacities.
Lastly, if the industry coalesces around cnMAE, it will allow for that important comparison between forecasting systems and vendors, and speed the maturity and acceptance of solar forecasting in the industry.