Ryn Hamilton has an advanced degree in econometrics and spearheaded some of the earliest econometric-based impact evaluation studies in the 1980s.
The past dec
ade has produced advances in the art of evaluation that have significantly elevated the practice.
The optimal evaluation approach integrates the guidance in industry protocols with the unique attributes of each program, with consideration of forces impacting its performance and expectations of regulators.
As program savings evolved into a tangible resource in utility portfolios and became a basis for system planning decisions, impact evaluation became more essential.
With AMI roll-out, interval meter data is being collected at a far more granular level to support impact evaluation. Evaluators should theoretically be able to track the timing of changes in energy use and rely less on monthly billing data. The industry has been slow, however, in connecting the smart grid and flow of digital data to impact evaluation.
Pressure for improved impact evaluation is mounting. Twenty-two states have adopted Energy Efficiency Resources Standards and require rigorous EM&V to demonstrate the reliability of this resource. Regulators are looking to energy efficiency as a compliance strategy for the Environmental Protection Agency’s various rules.
Many of the remaining impact evaluation challenges are the result of entrenched practices. For example, it is not uncommon in states in which demand-side management programs were cancelled back in the 1990s to be getting started again.
As a result there are many relatively new programs being evaluated. It can be difficult to predict which ones will warrant the greatest impact evaluation resources, and utilities do not always have the ability to shift funding mid-cycle. This can result in negligible evaluation of programs that turn out to be very successful because there is no mechanism to shift resources away from other programs such as new construction that may have low activity in a down economy.
Energy efficiency is now a capacity resource in ISO New England and PJM Interconnection wholesale markets, and demand response is permitted in additional ISOs/RTOs. Many utilities are now expected to bid demand side resource savings into forward capacity markets and comply with wholesale rules for M&V.
Impact evaluation must be performed on a regular schedule and a reasonable percent of program expenditures should be devoted to its practice. The risk of under-investment is a drift in program focus that can result in non-attainment of goals and erosion of program quality.
