RMmetrics

For the Record, Metrics Matter ... But Does Uncle Sam Care About Quality Management?

Owen Ambur, University of Maryland University College, November 11, 1999

Quality Records and Metrics

In support of his vision of Total Improvement Management (TIM), Harrington devotes a chapter of his book to the ISO 9000 standard for Quality Management Systems (QMS). When implementing a QMS, he suggests that it is helpful to start with procedures that are closest to the customer. However, he emphatically asserts:

absolutely must

Harrington notes that documentation tends to be "event-driven." (p. 182) In other words, records should result as a byproduct of the routine work processes, rather than as a separate, make-work, after-the-fact reporting activity. Indeed, as Harrington observes:

The major problem with most processes is that performance is measured only at the end. In most cases, this provides little relative feedback about individual activities within the process or, when it does, it is too late... (p. 421) Metrics as well as the records in which they are documented should both be generated routinely, preferably automatically, in real or near-real time, as an inherent part of the business process. Heskett, Sasser, and Schlesinger (1997) reinforce the point by asserting:

Efforts to attain total customer satisfaction have little prospect of success without proper measurement - measurement that not only tells an organization where it's been but also where it's going. (p. 197) Statutory Requirements and Results

In laws enacted to govern the activities of Federal agencies, Congress has endeavored to implement many of the principles of TIM and service quality as espoused by Harrington and Heskett. For example, the Federal Records Act requires agencies to make and preserve adequate documentation of their activities as well as the important information they possess. With limited exceptions, the Freedom of Information Act (FOIA) requires agencies to supply records upon request, and the Electronic Freedom of Information Act Amendments (E-FOIA) require that they be made available by electronic means. The Government Performance and Results Act (GPRA) requires agencies to document their mission, goals, and objectives; consult with their stakeholders; link their inputs to their outputs; compare the actual outcomes to their objectives; and report the results.

In carrying out its role to provide guidance to other Executive Branch agencies, the Office of Management and Budget has made the requirements of these laws fairly explicit. For example, in a section entitled "Recording and Documentation," OMB Circular A-123 specifies:

Transactions should be promptly recorded, properly classified and accounted for in order to prepare timely accounts and reliable financial and other reports. The documentation for transactions, management controls, and other significant events must be clear and readily available for examination. However, most observers believe agency performance leaves much to be desired not only in terms of their unique statutory missions but also their more generic "good government" obligations under GPRA, FOIA/E-FOIA, and the Federal Records Act. For example, in an article entitled "OMB Nominee Gets Earful At Confirmation Hearing," Barr (1999) reports that Senate Government Affairs Committee chairman Thompson "criticized the administration's efforts to implement [GPRA] and said agencies have not addressed half of the 'high risk' programs identified by congressional auditors as vulnerable to waste, fraud, and abuse." The nominee, Sally Katzen, acknowledged that the government faces major management problems but suggested that GPRA "seems to be taking hold" in agencies.

The auditors at the General Accounting Office, to whom Thompson referred, have reviewed all of the agencies' GPRA plans. Typical of their findings are their comments on the plans of the Department of the Interior and its subordinate bureaus. GAO cited two "key weaknesses":

2) Most of the plans do not identify or recognize issues that would significantly affect data limitations and their implications for assessing whether goals are being achieved.

Both of these weaknesses relate directly to the management of electronic records. GAO also noted the following problems:

...most of the ... plans ... still do not discuss the procedures to control data quality, such as those for automated information systems... only one plan ... improved its discussion on the weaknesses that are inherent in the form of data collection and that could result in significant error in the data used for measuring performance... (pp. 7 & 8)

Interior's only strategic information technology goal is ensuring that the department's critical information systems have been remediated and are operating correctly for the year 2000 date change.

The later is noteworthy because GPRA requires agencies to describe the technology required to meet their performance objectives. Thus, information management technology requirements should be addressed in the agencies' long-term strategic plans as well as in their annual objectives statements, both of which should encompass the means to enable agencies to meet their obligations under FOIA/E-FOIA and the Federal Records Act.

Reporting on the Microsoft antitrust case, Ignatius (1999) presented a mini case study on the tension between information technology and records management. He observed "... lawyers are always looking in the rear view mirror, while technologists are always looking over the next hill..." That is, the wheels of justice grind on the record of the past - as best it can be documented - while technology is driven by the potentials of the future, as best they can be foreseen. He noted that the government's attorney "showed grainy videotape of Bill Gates's deposition" and displayed "Microsoft e-mails from the same year" that contradicted what Gates said on the video. The implied message was "you can't trust these folks on anything." Ignatius concluded, "The legal process will grind on, perhaps for several more years. The technological process will race ahead at warp speed."

Miller (1999) chronicled an example that is emblematic of the problem of mistrust fomented by the failure of the Federal Government to properly steward the public record. The Clinton administration has devoted a great deal of time and the taxpayers' money defending itself against a class action lawsuit alleging mismanagement of billions of dollars in Indian trust funds. Recently, a judge ordered the Government to spend $625,000 more - to pay legal bills accrued by the Indians thus far in the litigation. In issuing the court order, Miller reports:

... [Judge] Lamberth said he regretted that he could not hold the officials and lawyers personally responsible for the costs, adding, "The court is aware of the unfortunate consequences of today's ruling on American taxpayers. In this judge's view, the American taxpayers should not continue to be forced to bear the burden of these types of misdeeds. Instead, as is the case in the private sector, these attorneys and officials themselves should bear individual responsibility for their actions..." Measures and Incentives for Improvement

A basic precept of change management is that proper incentives must be present. As the colloquialism states, "You get what you pay for." Regardless of what the law requires, Lamberth's pointed remarks suggest that government employees may have little incentive to effectively create and efficiently maintain, much less provide quick and ready public access to the records of their activities. Indeed, they may be personally insulated from the outcome of record-keeping activities, good or bad. Moreover, little or no effort may be made to measure their performance within such a mundane administrative process as records management. Lacking personal accountability measures, the organizational failure of agencies to maintain adequate records should not be surprising, for as Harrington notes:

Without measurement, there is no sense of accomplishment and little or no reason to try to improve or even to perform at the present level. Performance is directly related to how well the activity is measured and the interest other people exhibit in the measurement. (p. 420) Referencing change management theory, Heskett notes that the feasibility of achieving change is thought to vary according to the degree that three factors - the level of dissatisfaction, the existence of a model or vision for change, and the availability of a process for change - compound each other so as to overcome the cost of change to those involved in the process. (p. 216)

Virchow, Krause & Company suggest, "Research has shown that the difficulty for many governments is not the task of developing appropriate measures, but in integrating them into the management and operational decision-making systems of the organization." There can be little doubt of the difficulty of the latter. Those who have climbed the bureaucratic ladder to positions of authority under the old rules may have little incentive to relinquish their long-coveted personal prerogatives to the dictates of metrics in a newly customer-focused, process-oriented regime. Moreover, the ability of government agencies even to develop appropriate measures, much less to effectively apply them, is also in doubt. As Judge Lamberth witnesses, the proper incentives simply may be lacking.

Thus, to establish a baseline for improvement, the following question occurs:

By what means and how effectively do government agencies measure the performance of their employees in documenting, managing, and providing access to their records as required by law, including the Federal Records Act, Electronic Freedom of Information Act, and the Government Performance and Results Act?

References

Barr, Stephen. (1999, Sept. 16) "OMB Nominee Gets Earful At Confirmation Hearing." The Washington Post. p. A11

General Accounting Office. (1999, July 20). Report on the Department of the Interior's FY 2000 Performance Plan. Available at: http://www.gao.gov/new.items/gpra/rc99207r.pdf

Harrington, H.J. (1995) Total Improvement Management: The Next Generation in Performance Improvement. Washington, DC: McGraw-Hill, Inc.

Heskett, J.L., Sasser, W.E., Jr., and Schlesinger, L.A. (1997) The Service Profit Chain: How Leading Companies Link Profit and Growth to Loyalty, Satisfaction, and Value. New York, NY: The Free Press.

Ignatius, D. (1999, September 22) "Microsoft Show." The Washington Post. p. A33

Miller, B. (1999, August 11) "Judge Orders U.S. to Pay Indians; $625,000 Fine Assessed over Trust Funds." The Washington Post. p. A17

Office of Management and Budget. (1995, June 21) OMB Circular A-123, Management Accountability and Control. Originally available at: http://www.whitehouse.gov/WH/EOP/OMB/html/circulars/a123/a123.html Archived copy available at: http://web.archive.org/web/19991009181700/http://www.whitehouse.gov/WH/EOP/OMB/html/circulars/a123/a123.html

Virchow, Krause & Company, LLP. (1999) "Performance Measurement: A Means to Better Ends." Originally available at: http://www.virchowkrause.com/benchmk1.htm Archived copy available at: http://web.archive.org/web/19990501184958/http://www.virchowkrause.com/benchmk1.htm