This first appeared as PATImes column in June 2008.

Sense-Making in the Age of Information Overload

by John M. Kamensky

Technology makes it almost too easy to collect performance information. In fact, a major complaint in many government agencies is information overload. The real challenge is: how do you make sense out of it all?

The 911 emergency response service in Washington, DC has been collecting data on its calls for years. In fact it was probably experiencing information overload. It knew when, where, and the time of day for each of 127,000 emergency calls it receives annually. In fact, it even knew that responding to these calls averaged $700 per visit. However, only recently did city officials try to make sense out of the data. By analyzing the data, DC officials found that 20 residents accounted for ten percent of the calls. That averaged out to 635 calls by each of these residents -- more than one a day! At a cost of $700 per visit, the city realized that sending medical workers in vans to regularly visit the top 20 offenders who habitually call 911 in non-emergency situations would be far more cost effective. In addition, the city is now able to be more responsive to true emergency calls. But it would not have taken this course of action if it had not analyzed its data.

This approach to sense-making using performance data is called "analytics" by some and "business intelligence" by others. Whatever it is called, it refers to "the extensive use of data, statistical, and quantitative analysis, explanatory and predictive models, and fact-based management to drive decisions and actions," according to Professor Thomas Davenport, a well-known business scholar. He and a colleague, Sirkka Jarvenpaa, co-authored a recent report for the IBM Center for The Business of Government, called "Strategic Use of Analytics in Government." They examined a series of government program areas to determine how far along government is in applying analytic approaches to its work. In the process, they developed a set of attributes to assess agencies' capacity to effectively use performance information.

Attributes for Assessing Your Analytic Capabilities

Successful program executives develop and use analytic programs with the following characteristics, according to Davenport and Jarvenpaa:

Accessible, high-quality data. Government often has access to a great volume of data, but it needs to not only collect and "warehouse," it, it must be of high enough quality to be used to make decisions, it needs to be current, and it needs to be separate from agencies' transaction systems. Many state revenue agencies, for example, are using commercial data management software to detect tax cheaters.and users of agencies' own internal data, such as finance or personnel. Davenport and Jarvenpaa believe the fragmented nature of many government programs is probably the greatest difference between public and private use of analytics.

An enterprise orientation. "Enterprise" means agency- or department-wide. Oftentimes data are segregated into specific programs and cannot be compared or analyzed across programs. Agencies need to be able to provide a unified face to citizens

Analytic leadership. Leaders who recognize and understand the value of analytics is key. A good example is the former undersecretary for health at the Department of Veterans Affairs, Kenneth Kaiser. He understood the value of identifying key health outcomes and using analytics to drive improvements that resulted in veterans receiving higher quality of care than most private sector hospitals.

A long term strategic target. Closely tied to leadership is having a clear strategic intent is critical. Setting a long-term goal with intermediate targets begins to develop an enterprise-wide common understanding of priorities and innovations to achieve those goals. Employees see that it isn't measurement for measurement's sake.

A cadre of analysts. Having a cadre of trained analysts is important. In cities using a Citi-Stat approach, there is always a small core of analysts. In the federal government, there are federally funded research and development centers such as RAND and MITRE that provide analytic support, especially for the military. State governments sometimes create partnerships with local universities to prove such capacities, as well.

Taken together, these characteristics are a useful checklist for understanding what you may want to ensure is in place for your agency.

Use of Analytics in Government

Private sector companies have developed sustained competitive advantages by using data-based analytics. For example, Harrah's gambling casinos regularly use analytics to identify and reward loyal customers and Amazon.com extensively uses analytics to predict which products will be successful.

Government agencies are beginning to exploit the use of analytics to meet their strategic goals, as well. Some approaches are well-known, such as Activity-Based Costing, Lean Six Sigma, and Citi-Stat. While these uses have been successful, they tend to be limited. For example:

Health Care: Using Analytics to Prevent Fraud. Nassau County in New York State saw claims for Medicaid reimbursement decrease by $1 million after developing an analytical fraud prevention initiative. Common types of fraud are duplicate payments, overpayments to health care providers, non-billing to Medicare, and miscoding of diseases and payments. There are obvious opportunities to expand this approach across a range of federal state government health care programs.

Supply Chain: Using Analytics to Plan Capacity. The military has a highly-developed use of analytics in managing its "supply chain" of materiel in wartime. Analytics have long been used to forecast demand, optimize supply routes, and to manage naval and air force operations. These methods are being employed in the private sector as well - WalMart being one of the most well-known. Agencies can use supply chain analytics in managing human resources as well, to predict workforce needs.

Tax Collection: Using Analytics to Forecast Revenue. The U.S. Congressional Budget Office developed an income tax model to project individual and aggregate tax liabilities for future tax years. Dynamic revenue forecasting provided new and useful insights and opened new lines of discussion in Congress regarding the ramifications of tax policy changes. These models are being developed and used at the state levels as well.

Intelligence: Using Analytics to Model Operational Forecasts. The Center for Army Analysis created the "FORECITE Monitor" which collects data for indices of the "character and intensity of interactions between individuals, organizations, and states." This data is used to monitor, assess, and forecast trends in behavioral interactions between people, organizations, and targeted countries, and predict changes at the event level. However, the intelligence community still tends to be fragmented and problems continue in sharing data across the intelligence enterprise.

While there are many examples of the successful use of analytics in government, Davenport and Jarvenpaa conclude that government today still lacks the needed elements of leadership, an enterprise-wide orientation, and long-term strategic targeting, all of which are important factors in successful analytic programs. For example, only 20 percent of states use analytics in their tax collection efforts, even though data shows increases of 10-15 percent in compliance in the states that do use these approaches.

In order to create effective analytical capabilities, government leaders must not only invest in their technology and data, but also in managerial innovations to transform their organizational cultures, business processes, and the day-to-day behaviors of their employees. Private sector companies that have committed to these approaches find that it takes at least five years to make the transition but that once they do they have increased their capabilities significantly.

* * * *