Program Effectiveness & Impact

Strategic planning; strengthen reputation value of EERE portfolio by executing communication plans for each deliverable; Special Project: Recovery Act evaluation, monitoring, and verification.

Primary Contact:

Previous and Ongoing Analyses :

Program Evaluation

  • Brown, M.A., V. Kreitler, and A.K. Wolfe, 1996. "The Persistence of Program Impacts: Methods, Applications, and Selected Findings." Energy Services Journal 2(3):167-192 (Amy Wolfe)
  • Wolfe, A.K., M.A. Brown, and D. Trumble, 1995. Measuring Persistence: A Literature Review Focusing on Methodological Issues. ORNL/CON-401. Oak Ridge National Laboratory, Oak Ridge, Tennessee (Amy Wolfe)
  • Weatherization Assistance for Low-Income Households: An Evaluation of Local Program Performance, February 1987. Prepared with M. Schweitzer, R. Cantor, T. Mason, B. Ragins, and S. Rayner. Draft Final Report, ORNL-6337, Oak Ridge National Laboratory, Oak Ridge, TN (Marty Schweitzer)
  • Retrospective evaluation of DOE's Weatherization Assistance Program (Bruce Tonn)
  • Weatherization Assistance Program ARRA evaluation (Bruce Tonn)
  • Evaluation of DOE's State Energy Program (Bruce Tonn)
  • Evaluation of DOE Automotive Lightweighting program projects (Bruce Tonn)
  • Program evaluations of both the Weatherization Assistance Program and the State Energy Program using competitively selected contractors under an EERE and OMB approved process for selection independent evaluators (Joel Eisenberg)

Prospective Analysis

  • Greene, D.L. and P.N. Leiby. (2006). The Oil Security Metrics Model: A Tool for Evaluating the Prospective Oil Security Benefits of DOE's Energy Efficiency and Renewable Energy R&D Programs, ORNL/TM-2006/505, Oak Ridge National Laboratory, Oak Ridge, Tennessee, May. (David Greene, Paul Leiby)
  • Assisted EERE in developing methodological framework for prospectively estimating the benefits of its R&D programs (http://www.esd.ornl.gov/benefits_conference/) (Russell Lee)
  • Lee, R., Jordan, G., Leiby, P.N., Owens, B., Wolf, J.W. (2003) Estimating the benefits of government-sponsored energy R&D," Evaluation Research, Vol. 12, no. 3, pp. 183-195 (Russell Lee, Paul Leiby)
  • Lee, R., "OE Benefits Analysis for GPRA, Annual Budget Submission, and Addressing R&D Investment Criteria, and other R&D Planning," prepared for the Office of Energy Reliability and Electricity, Oak Ridge, TN: Oak Ridge National Laboratory, May 2007
  • Lee, R., "Option Benefits of Government Investment in Energy Technology RD3: Concepts and Approach," prepared for the Office of Energy Efficiency and Renewable Energy, U.S. Department of Energy, Oak Ridge, TN: Oak Ridge National Laboratory, August 2007 (original draft written July 2005)

Metrics and Strategic Planning

  • Synergies among NYSERA energy efficiency programs (Bruce Tonn)
  • Building upon team's strengths in ITP's data systems and metrics development, analysis and reporting, the team also supports ITP in strategic planning activities; develops planning models based on the empirical from our metrics system and evaluation results; and is responsible for impacts reporting for ITP activities with industry. The team also has had some success gaining OMB approval on survey instruments (Michaela Martin and M. Schweitzer, S.U. Nimbalkar, B. Tonn, M. Lapsa, A. Wolfe, L. Truett, R. Schmoyer, R. Goeltz)

Evaluating Different Types of Benefits

  • Non-energy impacts of energy-oriented water and wastewater treatment projects (for NYSERDA) (Amy Wolfe)
  • Environmental impact assessment of disposition of uranium tailings (Shih-Miao Chin and Ho-Ling Hwang)
  • Traffic impact and noise analysis environmental assessment study (Shih-Miao Chin and Ho-Ling Hwang)
  • Temporarily lost highway capacity study (Shih-Miao Chin and Ho-Ling Hwang)
  • Disruption of rail services (Shih-Miao Chin and Ho-Ling Hwang)
  • Non-energy benefits of WAP (Bruce Tonn)

Relevant Areas of Expertise and Capabilities:

Program Evaluation Methods

  • Cost-benefit analysis (Russell Lee)
  • PHEV usage behavior modeling (David Greene, Zhenhong Lin)
  • National commodity flows and analysis (Shih-Miao Chin and Ho-Ling Hwang)
  • Expertise in passenger and freight characteristics and their movements in the U.S. (Shih-Miao Chin and Ho-Ling Hwang)
  • Knowledge of data and movement involving foreign trades (Shih-Miao Chin and Ho-Ling Hwang)
  • Intermodal and multimodal transportation analysis (Shih-Miao Chin and Ho-Ling Hwang)
  • Project/technology/program effectiveness and impacts assessment - evaluate technology effectiveness, program effectiveness, infrastructure changes, program changes, etc. (Rekha Pillai, et al.)
  • Readiness or resiliency assessments - performance metrics at technology, asset, systems level (Rekha Pillai, et al.)
  • Pilots planning and assessment - operational testing and evaluation (Rekha Pillai, et al.)
  • Energy program evaluation (Bruce Tonn)
  • Methods development (Bruce Tonn)
  • Evaluation of public and private sector energy efficiency programs, assessment of socioeconomic impacts related to construction and operations of large-scale energy-related facilities (Martin Schweitzer)
  • ORNL has extensive experience in the analysis of DOE's grants programs and the statistical and engineering expertise to conduct these projects.  We have also had extensive experience with both the EE&RE and OMB procedural requirements (Joel Eisenberg)
  • Well-versed in design and use of ITP industrial databases, EIA-MECS, industrial customers. Team comprised of engineers, evaluators and a statistician (Michaela Martin and M. Schweitzer, S.U. Nimbalkar, B. Tonn, M. Lapsa, A. Wolfe, L. Truett, R. Schmoyer, R. Goeltz)

Preliminary Ideas on Possible Studies

  • Use PHEV consumer choice model as a quick analysis tool to analyze the impact of policies on PHEV development. The model currently can respond to purchase subsidy, tax credit (e.g. ARRA PHEV tax credit), battery warrantee extension, free parking, free HOV access. It can be adjusted to allow more types of policies. (David Greene, Zhenhong Lin)
  • Compare effectiveness of alternative policies, such as PHEV purchase tax credit vs. Cash for Clunkers, in reducing GHG (David Greene, Zhenhong Lin)
  • Continue and expand the oil security metrics project (David Greene, Zhenhong Lin)
  • Develop, implement and apply a methodological framework to estimate the benefits of investments made by the Office of Energy Efficiency and Renewable Energy, under the American Recovery and Reinvestment Act (Russell Lee)
  • Estimate economic impacts and energy supply shortages due to interruptions in transportation services (Shih-Miao Chin and Ho-Ling Hwang)
  • Systematically identify synergies among EERE Technology R&D Programs and Projects (Bruce Tonn)
  • Develop methodology and systems for program performance tracking, reporting, feedback and planning. (Michaela Martin and M. Schweitzer, S.U. Nimbalkar, B. Tonn, M. Lapsa, A. Wolfe, L. Truett, R. Schmoyer, R. Goeltz)

Highlights

Transportation Energy Data Book: Edition 31
On July 31, 2012, ORNL release Edition 31 of the Transportation Energy Data Book (TEDB). The TEDB is a compendium of data on transportation with an emphasis on energy. Designed for use as a desktop reference, the TEDB was first published in 1976 and has continued to Edition 31. The TEDB is produced by Oak Ridge National Laboratory for the U.S. Department of Energy's Office of Energy Efficiency and Renewable Energy.

ORNL Sustainability Summit
On May 2-3, ORNL held a Sustainability Summit to advance the deployment of sustainable technologies in the southeast. Diane Davidson developed and moderated at the Sustainable Transportation session, with Ron Littlefield, Mayor of Chattanooga, along with ORNL's Stacy Davis and David Greene.

U.S. DOE's Annual Merit Review and Peer Evaluation Meeting
David Greene, Zhenhong Lin and Jing Dong attended the U.S. DOE's Annual Merit Review (AMR) and Peer Evaluation Meeting in Washington, DC, May 14-17, 2012. David's presentation, "Fuel Cell Vehicle Penetration Relative to Vehicle Portfolio," and Zhenhong's presentation, "Impact of Program Targets on Vehicle Penetration and Benefits," were reported on by Slate and the Green Car Congress.

White House and U.S. DOE sponsored "Energy Data Jam"
David Greene participated in the White House and U.S. DOE sponsored "Energy Data Jam" at Stanford University On May 21, 2012. The invitation only workshop brought together "40 of the most innovative entrepreneurs, software developers, chief executive officers (CEOs), energy experts and policy makers to discuss how we can unleash the power of data, technology, and innovation to help with our pressing energy goals."

Vehicle Technologies Program Fact of the Week
Stacy Davis, Bob Boundy, and Susan Diegel develop Facts of the Week that are posted on the DOE Vehicle Technologies Homepage. For the month of March, half of the top 10 landing pages for the DOE Vehicle Technologies Homepage were Facts that they have written. (A landing page is where one enters the website for the first time.) "The FOTW pages are our biggest traffic-getters" says Suzanne Williams who compiles the Vehicle Technology website statistics for DOE.

 

Finding . . .