Price forecasting for wholesale electricity markets has gotten more difficult over the last ten years. Why is that?
When I began as an energy trader 20 years ago, pricing was pretty simple. You had your base stacks of coal and nuclear generation bidding in as a price-takers, which left the price to be determined by the efficiencies of combined-cycle and single-cycle natural gas peaker plants. New England, for example, simply traded multiples of the of the Algonquin gas pipeline price.
But now, because of renewables and flexible load resources, which include demand response (DR), virtual power plants (VPPs) and even data centers, the markets have much more flexibility. And that’s why accurate price forecasting today is more difficult — and more critical — than ever and requires a team of AI/ML data scientists and engineers.
How are inaccurate price forecasts affecting market participants? Is someone getting hit particularly hard?
The retail electricity providers in the 14 deregulated states are getting hit the hardest. They are mostly competing on price, but they don’t have good clarity on their own purchasing costs in the short or long term, because of the complexity of price-setting variables, which shifts from hour to hour. So, it’s very difficult for them to hedge their positions in the market. Ultimately though, as these retailers increase margins to protect against the price volatility and unreliable forecasts, it’s the retail customers who pick up the tab.
The situation is tough in public power, too. They aren’t at risk of going under like a retailer might be. But they have to pass the costs along to very unhappy citizens and co-op members. Investor-owned utilities are also affected in the short-term, but at the end of the day, they can generally re-coup costs through rate fees, which (again) is to say that the end-consumer is most-affected.
So, what’s needed to make price forecasting accurate again?
In short, you need a bunch of sophisticated machine-learning algorithms and lots of historical prices, fuel costs and generation, as well as accurate forecasts for net demand, which comes from having accurate forecasts of weather, demand, solar and wind. You also have to account for things like grid congestion, transmission limits, market rules, and how different participants bid. Ultimately, AI is up to the computational task, but the forecasts will only be as good as the models you create and data that goes into them.
So as AI and data scientists get price forecasting back on track, utilities and their customers obviously have a lot to gain. Who else?
Actually, I think independent power producers (IPPs) will be the biggest winners from improved price forecasting. They have to bid in every day on a curve that represents how much electricity they are willing to supply at varying price levels. With accurate price forecasts they can make strategic calculations to maximize their profitability. For thermal generators that know their fuel and operating costs, it’s relatively simple. For battery storage, price forecasts are really important, because they only have a limited capacity to dispatch and need to bid that capacity for the hours with the best price potential. For wind and solar it’s more complicated, because they also need accurate generation forecasts for their assets. If the generation forecast shows that they will likely fall short during an expensive hour, they can acquire from the market the additional supply they need to be safely hedged.
But it’s also about who has the most to gain from a human-resources standpoint. IOUs and hedge funds can afford full-time quant teams to grind out the figures and strategies in house. But IPPs, public power, and some trading shops don’t have that type of support, which means a comprehensive forecasting system is a big upgrade for them.
Can you explain more about how market participants use price forecasting strategically?
Yeah, so price forecasting isn’t really about: “Here’s the magic number of what tomorrow is going to clear.” It’s about knowing the shape and when price is going to peak — or bottom out. Going back to batteries again, because that’s the clearest example, they have a limited capacity to offer, so they need to get it right every day to maximize financial performance. It’s easier in California, because they can bid in four-hour blocks. But in ERCOT, they have to bid in one or two-hour increments. So they need to know precisely where the action is going to be.
How granular is Amperon’s price forecasting?
For the time being, we’re sticking to hubs and zones — what’s liquidly traded. PJM has approximately 11,000 nodes, so we’re not getting into that level of granularity. We will initially build out zones or hubs that have any liquidity in PJM. What that means is users will have to take responsibility for figuring their own basis risk, as that gets into power flow modeling, which isn’t currently on Amperon’s near-term road map.
So Amperon’s first day-ahead price forecasting is now available (for ERCOT). You mentioned PJM. Does that mean day-ahead forecasts for other markets are coming? And what about price forecasting for real-time markets?
Day-ahead price forecasts was the logical place to begin, but we do have plans for real-time price forecasting, as well. Historically, the people who needed day-ahead price forecasts weren’t the same people who used real-time forecasts. EMAs (energy management agreements) handled the bulk of intra-day trading when, for instance, a nuclear plant tripped offline and suddenly there was 2,000 MW less supply and a lot of price volatility. But with the increasing complexity of renewables and flexible load resources (which is increasing intra-day volatility and trading) we’re seeing more large generators and utilities bring real-time trading desks in-house. So ultimately, we will offer both forecasts on our platform.
###
Price Forecast Recap
A price spike in ERCOT in October gave us the opportunity to showcase the accuracy of our newly launched price forecast. The spikes were primarily driven by high net demand — due to weak wind during the solar ramp-down — and elevated generation outages typical of the shoulder season. Our forecasts provided customers with the insights needed to navigate these higher prices. If you'd like to see learn more about our price forecasts, contact us here.
.png)
.png)
.png)
