Aotearoa’s existing energy data system is stuck in the late 1990s, a creature of its time. But the power system it supports is evolving at rapid speed.
Right now, conditions across the power system are becoming a lot less predictable. Weather events are increasingly impacting the network. At the same time, we’re shifting away from highly controllable fossil fuels to more variable power sources such as wind and solar (that’s a good thing).
It’s not just where the power is coming from but who is using it. The number and types of people participating in the electricity market are increasing. We’re seeing the emergence of distribution system operators and flex coordinators, a steady scaling up of distributed generation such as rooftop solar panels and batteries, and people changing how and when they use power.
The trouble is our data system hasn’t caught up:
This is stifling innovation and competition as people are unable to get the data they need, when they need it, to make the efficient choices required for a smart, flexible power system.
Incremental changes to the data system will no longer cut it. We need to take a strategic approach to managing data and build a ‘digital nervous system’ that is inclusive, open, secure and adaptive.
In our latest FlexForum Insights we argue that a future-fit data system must be built on three core features:
1. Data users easily and routinely get the data they need
When the data system was built, it was for the electricity market participants of the time. Anyone not listed in the “dataset rules” ie, non-traditional participants, can not easily and routinely get the data they need for decision-making or to deliver a service.
Solutions have or will be implemented to help this issue, however, these are only tweaks and do not address the underlying problem that the data system does not easily accommodate new data users or uses.
To finally resolve the data access question, we must establish a data sharing infrastructure that is inclusive, open and adaptive. A handy model is being developed in the United Kingdom which we can look at as a starting point for upgrading the system.
2. The data system is adaptive and easily incorporates new datasets
Originally, datasets were established to operate the wholesale market, transmission and sub-transmission networks, to assign responsibility for connections, and to calculate monthly power bills.
The electricity sector currently struggles with parties wanting access to data and datasets which do not ‘routinely’ exist, and there is no consistent approach for adding a new dataset to the data system.
A future-fit data system would allow ‘new’ electricity datasets to be easily and routinely created and incorporated into the data system as they emerge or are needed by data users. A data sharing infrastructure can provide a catalogue of datasets currently in the data system and clear rules for how to request and include new datasets into the data system. This would avoid the current ad hoc dataset development that increases transaction costs.
3. Data is exchanged efficiently because standard formats and secure automated exchanges are the default
Data is currently exchanged in a range of formats, using a range of protocols. Some are mandated. Some are determined through bilateral agreements, sometimes with the option of adopting a standardised approach. Standardised data formats, even when regulated, are not consistently used, and data can arrive in spreadsheets, PDFs or needing cleaning to be machine-readable.
Going forward, standardised machine-readable data formats and secure automated exchange mechanisms should be thedefault for all datasets. Cybersecurity is critical to day-to-day data exchange and should be part and parcel of the data system.
These three features aren’t just a wish list, they are must haves for an enduring, adaptive data system.
Read the full Insights to delve deeper into these three features and explore the opportunities, challenges and roadmap for a smarter flexible power system.
Or if you’re in a hurry, download our one page summary here.
