Key areas of this case study include: University Research: Collaboration and education to identify data for opportunities. Data Authenticity: Ensuring the accuracy and reliability of data. Regulatory Compliance: Evaluating how well utilities comply with data access regulations and providing the ‘best available’ data. 

Case Study Project Goals 

Anomaly Detection: Developing methods to identify and resolve anomalies in energy consumption data using machine learning and other methods. 

Data Validation: Ensuring utilities provide accurate and up-to-date information. 

Educational Framework: Building a network to educate and take action on energy usage and emissions. 

Collaboration: Creating new business relationships between stakeholders. 

 

Insights and Research Findings 

Ease of Data Donation: UEnergyHub simplified the process for electric and natural gas utility customers to donate their energy data, facilitating the collection of consumption and billing patterns for research and educational initiatives. This ease of use also supported continuous reauthorizations needed to manage and validate utility issues.  

Costs: UEnergyHub avoids Utility setup (onboarding) overhead. It provides the University network a service that deals with Utility operational issues in an unbiased way. This allows the University to focus on data analysis while greatly reducing data acquisition costs and efforts on an ongoing basis to access real energy data.  The Hub also allows the account holder to share their data at no cost in a matter of minutes. 

Collaboration and Reporting: Although the University was not required to communicate the issues found, key problems were reported to the utilities and the government through Screaming Power and subject matter experts (SMEs). This added complexity to the research process, causing delays due to lengthy communication and collaboration with multiple stakeholders. However, it helped utilities improve their compliance with regulations and standard software testing practices. The EZGB platform, from which UEnergyHub was created, is now the main testing infrastructure in the Ontario market. 

Continuous Improvement: UEnergyHub had to continually adapt to accommodate for ease of use and log utility issues, requiring constant monitoring and updates to the system. Ensuring consistency and adherence to a standard format for the data provided by utilities was challenging but crucial for validating the “Source of Truth” data. Changes by some utilities improved their systems as the study continued. Timely utility support was needed as the research continued, but was often lacking as utilities had competing priorities.  

Utility Preparedness: Despite regulatory certifications required by Utilities and utilities confirming solutions were implemented by November 2023, many were not fully prepared to handle customer data requests. This gap between regulatory compliance and operational readiness posed significant challenges, delaying research activities and requiring adjustments to research goals. Screaming Power supported defining and resolving utility implementation gaps, with testing driven by UEnergyHub’s infrastructure using Screaming Power’s EZGB service which includes diagnostic tools. 

Data Quality Issues: All utilities providing data to UEnergyHub faced operational and data challenges, including inconsistencies and inaccuracies. The research identified these issues and the unpredictable loss of connection to the utility infrastructure from time to time. These issues complicated analysis and validation processes, related to this new utility digital sharing landscape. The research itself served as an evaluation tool for the utilities’ infrastructure, revealing that neither the utilities nor the regulator had independent means to appropriately test and validate this new data marketplace. This led to necessary changes in the data hub toolset to identify, report, and manage irregularities. 

Understanding Data Standards: The research emphasized the importance of understanding standardized processes and ensuring the validation of “Source of Truth”. The research team worked directly with the utility data source rather than the translated outputs available in UEnergyHub. This phase was crucial for accessing previously unavailable energy data. Although meter (usage) data comparison was key, analyzing the authenticity and operations of utility infrastructure was necessary to ensure the data and utility operations become trusted sources for future research and business practices. Researchers had access to both translated data for easy reuse and raw outputs within the Hub for comprehensive evaluation. 

Sources of Truth: The research team used two different sources of truth from utilities: UEnergyHub, providing direct access to data defined as “Best Available” by the regulator, and data obtained through a legacy business-to-business process, which required a more manual authorization process and offline delivery to the researchers. Managing these different data sources required additional cost, effort, external support, and coordination. The UEnergyHub demonstrated a concrete benefit over the legacy route of data acquisition managed by the Ontario Energy Board (OEB) Electronic Business Transactions (EBT) infrastructure.  

Support for University Research: UEnergyHub supported the University of New Brunswick (UNB) in gathering data for phase 1 of their research. This research was financially and in-kind supported by Mission:data Coalition, Screaming Power, and multiple industry SMEs from May to October 2024.  

Enhanced Features for Analysis: Due to research requirements, UEnergyHub expanded its feature set to include tools for comparing uploaded PDF bills with data exports from utility sources. Manual importing of source data was also created when automated retrieval processes were unavailable. Continuous improvements to the platform enhanced ease of use and logging of utility communications and data issues, benefiting all EZGB service customers. 

Impact on Costs and Emissions: By sharing energy data and understanding usage patterns, the education community now has a data hub to identify opportunities to reduce costs and greenhouse gas emissions, contributing to a sustainable future. 

 

Positive Surprises and Unexpected Benefits 

Strong Engagement from Utility Customers: There was a high level of engagement from utility customers willing to donate their energy data for research purposes. Despite needing to reauthorize several times due to utility technical issues, this enthusiasm was maintained and helped gather data, crucial for the research. The research engagement also improved the evaluation of utility systems, as the UEnergyHub (EZGB) infrastructure assisted with utility communications and data testing. 

Effective Collaboration: The collaboration between the University of New Brunswick (UNB), not-for-profit organizations, residential and institutional utility customers, and private SMEs was highly effective. Although utilities were not fully aware of the research itself, they were very interested in fixing their systems. This network continues to work together, enhancing the quality and scope of the research while improving the marketplace. 

Government and Utility Support: Despite initial challenges and limited support from the OEB and the Ministry of Energy, utilities were receptive to resolving issues once the issues were communicated and proven. UEnergyHub’s tools provided the proof which facilitated improvements in data quality, communication, and regulation compliance by the Utilities. 

Machine Learning Applications: Researchers successfully employed machine learning and data mining techniques to identify and resolve anomalies in energy consumption data. UEnergyHub also developed machine learning techniques to manage the non-standard datasets and the different operational processes of the utilities. This technological advancement added significant value to the research while working with different datasets. 

Improved Data Standards: Two years of data history was provided through UEnergyHub, usually within a few minutes when the utility system was fully performing. Research highlighted the importance of data standards and oversight, leading to improvements in the quality and consistency of data provided by utilities. This improvement process is ongoing and expected to have long-term benefits for future research and energy/GHG management initiatives. 

Increased Awareness: The project raised awareness about the importance of accurate and reliable energy data among utilities, regulators, and the public. This increased awareness is likely to drive further improvements in data quality and regulatory compliance. A research white paper created by the University provides detailed results for public review. 

Enhanced Utility Operations: Feedback and data analysis from the research team, Screaming Power and SMEs helped utilities identify and address operational inefficiencies, leading to improvements in data management practices and overall service delivery.  

Regulatory Compliance: The project highlighted areas where utilities needed to improve regulatory compliance and issue turnaround times. The research also emphasized the need for the government to focus on more effective and timely compliance measurement and action.   A several month turnaround time is not adequate for this type of technical rollout.  

Innovation and Development: Collaboration between researchers, utilities, and innovators led to the development of new tools and solutions for data management within UEnergyHub. Innovations and improvements have been scaled and implemented more broadly, benefiting the UEnergyHub and Screaming Power’s EZGB third-party hub platform. 

Economic Benefits: By identifying inefficiencies and opportunities for improvement, UEnergyHub helped Universities and consumers reduce costs and effort. This economic benefit extends beyond the immediate participants to the broader energy community. 

Community Engagement: The project raised public awareness about a new way to gather data from Utilities, which will assist everyone to manage energy consumption and sustainability, leading to more informed and proactive energy use behavior. 

Policy Development: Insights from the research provided valuable data for policymakers, and regulators. Accurate and comprehensive information with unbiased assessment is the cornerstone of an effective and inclusive marketplace. 

Strengthened Partnerships: The project fostered stronger relationships between universities, utilities, government agencies, and private sector innovators. These partnerships are expanding, leading to further collaborative projects and initiatives. The UEnergyHub pilot demonstrated the infrastructure’s reuse by multiple universities globally and provided a nexus for collaboration with the US and the Caribbean. 

Technological Advancements: The need to handle and analyze large energy datasets led to advancements in data infrastructure and analytical capabilities. These technological improvements were applied to UEnergyHub.  

 

Privacy Measures 

User Authorization: Explicit Consent: Utility customers and their delegates had to explicitly authorize the use of their data. Utility customer consent was obtained through a clear and straightforward process within UEnergyHub that evolves as new use cases appear. User authorization ensures that customers are aware of what data was being shared and for what purpose(s). 

Transparency: Clear Communication: UEnergyHub provided information to utility customers about how their data would be used. This included details on the types of research being conducted and the benefits of participating in the data sharing initiative. Privacy Policies: Privacy policies were made available within the UEnergyHub solution to outline the measures taken to protect data and rights as data providers and users of the UEnergyHub infrastructure. 

 

User Control: Data Management Tools: Utility customers were provided with tools to manage and share their data.  Revoking access to their data at any time is a feature that allows the user to go to the utility anytime to revoke access. UEnergyHub provided the ability for the Utility customer to view and download their data; giving them full control over their own information and allowed them to reuse the data themselves.