ML-11-001 -- The Nearest Neighborhood Method to Improve Uncertainty Estimates in Statistical Building Energy Models ✓ Most Recent

Accurate estimation of uncertainty in energy use predictions from statistical models finds applications in a number of diverse areas of interest to building energy professionals. Examples include the determination of measured energy savings in monitoring and verification (M&V) projects; continuous commissioning; and automated fault detection, wherein improper building or equipment performance is to be detected. All these applications generally involve identifying a baseline statistical model representative of energy use prior to the retrofit (or to energy use under fault-free operation) and then ascertaining the energy savings (or the penalty for faulty operation) as the difference between the measured post-retrofit energy use and the corresponding model-predicted value. Unfortunately, the model residual outliers are ill-behaved, and estimates of the uncertainty in the energy savings tend to be unrealistic. Developing a general methodology for determining more realistic, robust, and credible estimates of the uncertainty in energy savings would be of great value and is the objective of this paper.
The proposed approach is to determine the uncertainty from "local" system behavior rather than from global statistical indices of the model fit, such as root-mean-square error and other measures, as is the current practice. This is done using the nonparametric nearest-neighborhood-points approach, which is well known in traditional statistics. The methodology is applicable to any type of statistical model approach, such as regression, time series, and neural networks, and could be coded into a computer package that can be appended to existing M&V analysis programs. Two case study examples using daily building energy-use data serve to illustrate the proposed methodology. The ultimate benefit of such a reliable and statistically defensible method is to lend more credibility to the determination of risk associated with energy savings from energy efficiency projects and thereby induce financial agencies to become more involved in "white tag" and allied certification programs.