This first appeared as a PATImes column in December 2005.

 “It’s the Outcomes, Stupid!” 
by John M. Kamensky

A variation on a Presidential campaign slogan from days past – “It’s the outcomes, studpid!” -- seems to reverberate in the stories of the aftermath of the Katrina hurricane catastrophe.  One striking example of the difference between “outputs” and “outcomes” was a media report on how the Federal Emergency Management Agency (FEMA) provided cots for homeless victims.  That’s what they were supposed to do.  But there were no mattresses, blankets, etc.  Only cots.  That’s the output FEMA thought meant success.  But the victims wanted a comfortable place to sleep at night!

Several recent reports show that the “outcomes” mantra is going global, and is having an impact in the US federal government.  For example, a recent report, “Governments That Deliver,” summarizes the results of a December 2004 roundtable comprised of international practitioners with lessons on how governments are strategically moving from outputs to results.  This roundtable was co-sponsored by the IBM Center for The Business of Government and the World Bank.  In addition, the U.S. Government Accountability Office (GAO) assessed how the Office of Management and Budget (OMB) is increasing agencies’ attention to focus on program results.  In addition, the Key National Indicators Initiative is making progress in framing national attention on progress toward broader societal results.  All three provide useful lessons for public managers.

IBM-World Bank Report on Government-wide Outcomes.  In  the IBM-World Bank report, “Governments That Deliver:  Moving from Outputs to Outcomes,” author Burt Perrin describes state-of-the-art practices and practical lessons coming from a roundtable comprised of officials from six developed and six developing countries that are considered to be leaders in managing for results.  While these countries differed dramatically, and there was no single “best approach,” there were some amazing commonalities.  The roundtable discussion evolved around four themes:

First, in getting started on an outcome initiative, the roundtable agreed that to successfully focus on outcomes, the end results have to be central to something that matters to citizens.  For example, in Uganda and Tanzania, it is poverty reduction; in Mexico, it is presidential reform; and in Canada and the United Kingdom, it is the value taxpayers are receiving for their money.   If not, the effort will fail.  The effort can provide an overarching framework for coordinating different policy and program areas, and if successful, this can increase trust in government as well as the ruling party.

Second, when implementing such an initiative, the adoption of an outcome orientation requires a “fundamental shift in the nature of thinking, acting, and managing within the public sector,” away from a focus on activities and administrative processes and more of a focus on benefits to the citizen.   For example, while citizens want a customer-friendly process for issuing passports, they really want the border crossing functions to work effectively and efficiently. The roundtable also agreed that both a top-down and bottom-up approach was needed to be effective.

Third, when reporting on outcomes, countries found that documenting what actually happens and reporting it publicly in understandable terms is “absolutely critical” to creating an outcome orientation in the public service, preferably by an independent body so the public can trust the reporting.  The outcome focus also has to “take into account the entire results chain” because intermediate outcomes are sometimes just as important as final outcomes.

And finally, when using outcome information, the roundtable noted that while this information can “provide for more rationality to the resource allocation process,” the “most fundamental use and benefit. . . .is the shifting in thinking and the orientation of public services from a primary preoccupation on inputs and activities to a focus on the benefits and results of these activities.”   The purpose of outcome information is for it to be used in making decisions.  In Columbia, for example, budgeters find that they can stretch resources further when they created a results-based budget.

GAO Reports on Program-Level Outcomes.  Two recent GAO reports on OMB’s implementation of the Program Assessment Review Tool (PART) are a microcosm of the lessons in the IBM-World Bank report.  The PART is a set of assessment questions that OMB is systematically applying over a five-year period to all major federal programs.  This initiative is now in its fourth year.  Each of over 1,000 programs are being assessed and scored as “effective” or “ineffective” and the results of these scores are increasingly being used to influence program funding levels.  The PART assessments also contain OMB recommendations to agencies on how to improve their focus on, or reporting of, results.

The first GAO report looks at the PART’s focus on budgeting; the second GAO report looks at its effect on getting agencies to improve the evaluation of program results.

In these reports, GAO followed up on earlier work it had done on the implementation of the PART in four agencies, which served as case studies to better understand the effects of PART within an agency.  GAO concluded that “The PART has helped to structure and discipline OMB’s use of performance information for internal program analysis and budget review, and made its use more transparent.”  And that, because of the PART: “Federal agencies are increasingly expected to demonstrate effectiveness in achieving agency or governmentwide goals.” 

GAO looked at the recommendations OMB made back to agencies and found that they were largely aimed at improving performance measurement, and there was “limited evidence” that OMB recommendations were improving program results.  GAO also concluded that “because of limited agreement between OMB and Congress about the PART,” most congressional staff do not use the PART information in making their program or budget decisions.  Nevertheless, GAO concluded that “. . . the PART helped to create or strengthen an evaluation culture within agencies by providing external motivation for program review and focused attention on performance measurement.”

Key National Indicators Initiative – Societal-Level Outcomes.  While the OMB initiative focuses on specific programs, a completely separate effort -- the Key National Indicators Initiative – is being run by the National Academy of Sciences.  This effort creates a different focus on national outcomes, from the perspective of how well the nation is doing, in areas such as the economy, the environment, and society.  This effort reflects some of the lessons in the IBM-World Bank report on the need for cross-cutting indicators of progress that are developed by an independent third party.

Launched nearly three years ago, with a long-term horizon, this initiative is on the cusp of a more public launch in January 2006.  In recent months, experts from a wide range of policy arenas have developed scores of questions that might be asked about how well our nation is doing:  “What environmental and social impacts result from natural resource extraction and use?;” “ How affordable is US housing?;” “How have prices paid by US consumers changed over time?”

In addition to questions, hundreds of existing indicators of progress have been identified, such as housing prices, and others that need to be collected have been identified as well.  The next step is to engage stakeholders, such as policymakers, community leaders, and interest groups, in a year-long dialog over what the most important questions and indicators.  Ultimately, the goal is to have relevant data that provides useful information of interest to policy- and opinion-makers from the national to the neighborhood level. 

The initial effort, titled “The State of the USA Indicator Set,” will be coming to a website near you in January.  Here’s where you could get involved in the next phase of the global movement from outputs to outcomes!