Owen D. Ambur
Metadata or Malfeasance: Which Will It Be?
Division of Information Resources Management
U.S. Fish and Wildlife Service
4401 N. Fairfax Drive - MS 340
Arlington, VA, USA 22203
phone: (703) 358-2138
fax: (703) 358-2251
Copyright 1997 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.
Owen D. Ambur
Division of Information Resources Management
U.S. Fish and Wildlife Service
4401 N. Fairfax Drive - MS 340
Arlington, VA, USA 22203
phone: (703) 358-2138
fax: (703) 358-2251
The right of the public to access to public information held by the Federal
Government is set forth in the Freedom of Information Act, and the Electronic
Freedom of Information Act Amendments of 1996 require such information be made
available in electronic form. The Privacy Act restricts personal information
from inappropriate release. The Paperwork Reduction Act contains various
provisions to facilitate access to information, including authorization of the
Government Information Locator Service. Under the Federal Records Act, proper
documentation of agency activities must be created and maintained. The
Information Technology Management Reform Act mandates interagency planning and
the establishment of an on-line index of public information in electronic form.
The Government Performance and Results Act requires agencies to document program
outcomes and link them to budgetary inputs. If Federal agencies are to meet the
requirements of these laws, not only must the information itself be made
available but also the metadata required to make it useful to the public. Should
agencies fail to do so, there is no question how their actions should be
characterized under the law.
malfeasance n. Law. the performance by a public official of an act that is legally unjustified, harmful, or contrary to law.
-- The Random House Collegiate Dictionary, 1980
The right of the public to obtain information held by the Federal Government is summarized in a report published by the U.S. House of Representatives, entitled "A Citizens Guide on Using the Freedom of Information Act and the Privacy Act of 1974 to Request Government Records" (H.Rpt. 102-146), as follows:
The Freedom of Information Act (FOIA) establishes a presumption that records in the possession of agencies and departments of the Executive Branch of the United States government are accessible to the people. This was not always the approach to federal information disclosure policy. Before enactment of the FOIA in 1966, the burden was on the individual to establish a right to examine these government records. There were no statutory guidelines or procedures to help a person seeking information. There were no judicial remedies for those denied access. With the passage of the FOIA, the burden of proof shifted from the individual to the government. Those seeking information are no longer required to show a need for information. Instead, the "need to know" standard has been replaced by a "right to know" doctrine. The government now has to justify the need for secrecy.
More than secrecy, what has hampered public access to public information has been the sheer volume of it and thus the logistical difficulties of managing, sorting, cataloging, and retrieving it when and where needed. As further explained in the House report:
The FOIA sets standards for determining which records must be disclosed and which records can be withheld. The law also provides administrative and judicial remedies for those denied access to records. Above all, the statute requires federal agencies to provide the fullest possible disclosure of information to the public.
While logistical difficulties should not be taken as an excuse for agencies to refuse or fail to make information readily available, a legitimate concern is the right of citizens for privacy. The report continues:
The Privacy Act of 1974 is a companion to the FOIA. The Privacy Act regulates federal government agency record keeping and disclosure practices. The Act allows most individuals to seek access to federal agency records about themselves. The Act requires that personal information in agency files be accurate, complete, relevant, and timely... [It] also restricts the disclosure of personally identifiable information by federal agencies. Together with the FOIA, the Privacy Act permits disclosure of most personal files to the individual who is the subject of the files. The two laws restrict disclosure of personal information to others when disclosure would violate privacy interests.
The House report sums up the effect of the laws as follows:
The essential feature of both laws is that they make federal agencies accountable for information disclosure policies and practices. While neither law grants an absolute right to examine government documents, both laws establish the right to request records and to receive a response to the request. If a record cannot be released, the requester is entitled to be told the reason for the denial. The requester also has a right to appeal the denial and, if necessary, to challenge it in court. These procedural rights granted by the FOIA and the Privacy Act make the laws valuable and workable. As a result, the disclosure of federal government information cannot be controlled by arbitrary or unreviewable actions.
In the interest of full disclosure of public information and in recognition that information delayed is information denied, Congress passed and the President signed into law the Electronic Freedom of Information Act Amendments of 1996 (E-FOIA, P.L. 104-231). When E-FOIA was passed by the House, Representative Maloney (Congressional Record, September 17, 1996, page H10451) asserted:
... the bill ... forces agencies to exercise foresight when installing computer systems which must help expedite agency FOIA requests and operations, rather than impeding them... it would encourage agencies to offer online access to Government information, effectively transforming an individual's home computer into a Government agency's reading room.
Among the findings set forth in the E-FOIA is the assertion that "Government agencies should use new technology to enhance public access to agency records and information." Included among the purposes set forth in the bill are:
The E-FOIA mandates that agencies make certain records available by electronic means no later than November 1, 1997. Records that must be made available by that date include those created on or after November 1, 1996, that are not "promptly published" and "offered for sale"(1) and which fall into the following categories:
With respect to the latter category, by the end of the century,
December 31, 1999, E-FOIA also requires agencies to make available an on-line
index -- a mandate that relates closely to the Government Information Locator
Service (GILS) and the Green Pages of the X.500 directory.(2)
Statutory authority for the establishment of GILS is provided in section 3511 of the Paperwork Reduction Act of 1995 (PRA, P.L. 104-13). Initially, the goal was to establish a "...distributed agency-based electronic [service to] identify the major information systems, holdings, and dissemination products of each agency." However, the House committee report (H.Rpt. 104-37, p. 28) accompanying the bill asserted: "Ultimately, this system should become a path to the holdings themselves." The Senate committee report (S.Rpt. 104-8, pp. 25-26) contained identical wording and went on to say:
As better standards for organizing and accessing databases are developed, agencies need to work toward common protocols that will make direct public access a practical reality. The goal of creating a means for agencies and the public to obtain, not merely locate, government-held information should guide the development of GILS. (emphasis added)
The committee's reference to the ability of agencies to "obtain" government-held information is insightful, for if they cannot retrieve it efficiently themselves, they cannot possibly make it readily available to the public! The wording of the law itself [Sec. 3506(b)(1)(C)] is equally explicit in requiring agencies to:
... manage information resources to ... improve the integrity, quality, and utility of information to all users within and outside the agency ... (emphasis added)
By this wording, Congress has debunked the myth that agencies can somehow uphold their obligation to the public without first effectively managing information internally. Nor can information be managed effectively if agencies cannot even determine without great effort what information resources they hold. Under paragraph 3506(b)(4), agencies are required:
... in consultation with [GSA and NARA to] maintain a current and complete inventory of the agency's information resources, including [GILS] directories ... (emphases added)
Although many have chosen to ignore the guidance in the committee reports and interpret section 3511 merely to require that an inventory of information systems be made available via GILS, the intent of paragraph 3506(b)(4) is clear on it's face. It is not just an inventory of systems that is required under the law, but a complete inventory of "information resources". Lest there be any doubt about the meaning of that term, it is defined in the law [Sec. 3502(6)] as follows: "the term 'information resources' means information and related resources, such as personnel, equipment, funds, and information technology..." (emphasis added) Clearly, it is the information itself that is the focal point, rather than the systems by which it is processed and maintained. Moreover, subparagraph 3506(d)(1)(B) explicitly requires agencies to:
... ensure that the public has timely and equitable access to the agency's public information [including] information maintained in electronic format ...
It is noteworthy that enactment of the E-FOIA came little more than a year after the PRA became law. Indeed, both were passed by the 104th Congress. In the committee report (S.Rpt. 104-272) accompanying the bill, Senator Leahy explained why Congress saw the need for further action:
The efficient operation of the Freedom of Information Act has been hindered by 5 years of foot-dragging by the Federal bureaucracy... Curiously, it was often argued that the FOIA was not a primary program of the departments and agencies, a contention that sadly ignored the importance of Government information accessibility for the citizens of a democracy.
In remarks on the Senate floor (Congressional Record, September 17, 1996, p. S10715), Leahy added:
... failure to comply with the statutory time limits [for FOIA responses] ... breeds contempt by citizens who expect government officials to abide by, not routinely break, the law.
On the other hand, in the Senate report, Leahy approvingly noted the Committee's reference to GILS, calling it a "helpful tool" and noting its relationship to the Internet:
Significantly, many Federal agencies are also establishing sites on the World Wide Web to educate the public about their mission and facilitate access to information about the agency. Agencies should be encouraged to establish a FOIA requester section on the Web site homepage to facilitate on-line access ...
Agency Web sites are just one of many potential sources of Government information. In a memorandum dated September 29, 1995, Director Rivlin of the Office of Management and Budget set forth guidance for implementation of the information dissemination provisions of the PRA, including the following:
... the PRA makes agencies responsible for carrying out sound information dissemination practices ... One of the major goals of the Act is to encourage a diversity of sources for information based on government public information. It recognizes that State and local governmental entities, the information industry, libraries and educational institutions, and other entities are partners in promoting the use of government information for the maximum benefit of society.
Indeed, paragraph 3506(d)(4) of the PRA prohibits agencies from establishing any "... exclusive, restricted, or other distribution arrangement that interferes with the timely and equitable availability of public information."(1)(8) It is ironic, though perhaps not coincidental, that this provision is included in the Paperwork Reduction Act, for while paper remains the most universal medium of distribution of information, it is also highly "restricted". Moreover, it is "exclusive" in the sense that it lacks time and place utility and is thus useful only to the elite group of those who are privileged to be in intimate proximity to it at precisely the appropriate time. To the extent that paper is used to provide access to information originated by electronic means, it even more needlessly and exclusively restricted than it might otherwise be. In recognition of that fact, Congress explicitly prohibited the use of paper as the only medium of distribution, as follows [subparagraph 3506(d)(1)(B)]:
... in cases in which the agency provides public information maintained in electronic format, [it shall provide] timely and equitable access to the underlying data... (1)
Nor was Congress content to leave a loophole based upon the potentially transitory nature of electronic information.(3) To eliminate that threat, subsection 3506(f) specifies:
With respect to records management, each agency shall implement ... procedures ... for archiving information maintained in electronic format ...
And to capitalize on the positive potentials offered by electronic representations of information, paragraph 3506(h)(3) of the law requires each agency to:
... promote the use of information technology ... to improve the productivity, efficiency, and effectiveness of agency programs ...
In each case, it should be noted that these provisions do not
merely suggest discretionary actions; they are legal mandates. As Senator Leahy
noted, the issue is whether agencies will uphold their obligations to the public
or whether they will continue to drag their feet and flaunt the law.
The Federal Records Act (FRA) governs the creation, management and disposal of Federal agency records. Among other things, the FRA requires each agency to:
make and preserve records containing adequate and proper documentation of the organization, functions, policies, decisions, procedures and essential transactions of the agency and designed to furnish the information necessary to protect the legal and financial rights of the Government and of persons directly affected by the agency's activities.
The FRA also prescribes the only manner in which Federal records can be disposed, including "machine readable" records:
made or received by an agency of the United States Government under Federal law or in connection with the transaction of public business and preserved or appropriate for preservation by that agency ... as evidence of the organization, functions, policies, decisions, procedures, operations, or other activities of the Government or because of the informational value of data in them.
Records which fall under the province of this statute may only be
removed from the agency's computers with the approval of the National Archivist.
Several provisions of the Information Technology Management Reform
Act (ITMRA, P.L. 104-106, Titles 50-57) are pertinent to the storage and
accessibility of public information. For example, ITMRA requires (emphases
For any system from which information is disseminated to the
public, ITMRA [Sec. 5403] states that an index should be included in the on-line
directory maintained by the Government Printing Office (GPO) pursuant to 44 USC
4101. That section of the "public printing and documents" title of the U.S. Code
of Statutes requires GPO to:
To the extent practicable, GPO is required by 44 USC 4101 to
accommodate any request by heads of departments and agencies to include
information in the system. Building upon that mandate, the effect of ITMRA is to
require each Federal agency to make available in GPO's system at least an index
to information that it disseminates to the public, if not necessarily the
information itself. Bearing in mind that E-FOIA now requires agencies to make
available by electronic means any document that is released to anyone and is
likely to be of interest to others, the scope of this requirement becomes
virtually all-encompassing ... unless of course the assumption is made that
Federal employees are producing documentation of interest to no one but
themselves. And Congress has addressed that issue too -- through passage of the
Government Performance and Results Act (GPRA). No longer is activity for its own
sake sufficient to warrant the expectation of continued funding and support.
As summarized by Alliance for Redesigning Government (ARG), the intent of GPRA is to shift the focus of Government officials and managers from program inputs toward program execution. Officials are expected to attend to the results (outcomes and outputs) being achieved, and to assess how well programs are meeting intended objectives. To bring about this shift, GPRA sets out requirements for: 1) defining long-term general goals, 2) setting specific annual performance goals (targets) that are derived from the general goals, and 3) annual reporting of actual performance compared to the targets.(4) As Federal managers are held more accountable for achieving results, they are also given more flexibility and discretion in how they manage programs. Finally, the legislation provides for tests of various performance budgeting concepts.
Pilot projects are required over the next several years to test and demonstrate annual performance plans, program performance reports, and managerial accountability and flexibility. Additional pilot projects on performance budgeting are to be conducted during fiscal years 1998 and 1999. Full-scale, government-wide implementation of strategic planning, annual program goal-setting, and annual program performance reporting of expenditures in the Federal budget begins in 1997.
ARG suggests that implementation of GPRA will be characterized,
among other things, by:
To use the phrase so oft repeated by Vice President Gore, each of these actions should contribute toward realization of a Government that "costs less and works better." However, it should be noted that all of them depend upon the efficient and effective documentation and management of information.(6) Indeed, it is no exaggeration to suggest that documenting linkages between inputs and outcomes is the very essence of GPRA. A commonly understood truism is "you can't manage it if you can't measure it." With respect to documentation, a corollary is that "you can't manage it if you don't manage it." Explicit documentation of Government inputs and outputs clearly constitutes the kind of records to which the public is entitled. Providing quick and easy access to such records in readily comprehensible form is key to demonstrating what the taxpayers are getting for their money.(7) As Senator Leahy says (S.Rpt. 104-272):
The American taxpayer has paid for the collection and maintenance of ... records and should get prompt access ... upon request. That is what the law requires and that is the standard of service Government agencies should meet. Long delays in access can mean no access at all.
In the information age, a search on the Internet should be
considered a "request" just as much as one transmitted orally or in writing, by
phone or by subpoena. A delay of more than a few seconds may be considered
unacceptable and that is the standard to which Federal employees should
Metadata or malfeasance? That is the question. If Federal
employees choose not to engage in malfeasance,
the question is how best to gather and make public information available to the
public, not just the raw information itself but also the metadata required to
make it readily available and useful. Neither the
technical challenges nor the costs should be minimized. However, the cost of
inaction is far greater. Thus, the real issue is whether Federal agency
decision-makers recognize their duty to the public and whether they have the
will to carry it out in a responsible fashion. Should they fail to do so, there
is no question how their actions should be characterized under the law. Metadata
specifies what "freedom of information" really means.
Without it, these laws are mere slogans.
Alliance for Redesigning Government (ARG), Summary
of Implementation of the Government Performance and Results Act of 1993 (GPRA),
National Academy of Public Administration. Available at: http://www.clearlake.ibm.com/Alliance/clusters/op/gprabref.html
Lawrence Livermore Lab, Guidelines for
Performance Measurements, Appendices. Available at: http://nssc.llnl.gov/PBMSig/Review/PMG/DRAFT5CN_1c.html
H.Rpt. 102-146, A Citizens Guide on Using the Freedom of
Information Act and the Privacy Act of 1974 to Request Government Records, U.S.
House of Representatives, July 10, 1991. Available at: http://hi-tec.twc.state.tx.us/general/foia.htm
Loundy, David J., E-Law 3.0.1: Computer Information Systems Law
and System Operator Liability in 1995. Information on the Federal Records Act.
Available at: http://www.leepfrog.com/E-Law/E-Law/Part_VIII.html
S.Rpt. 104-272, Electronic Freedom of Information Improvement Act
of 1995, Committee on the Judiciary, United States Senate, May 15, 1996. [Back]
1. The exemption of documents that are promptly offered for sale creates an interesting anomaly with respect to the prohibition against making any exclusive distribution arrangements for public information, much less the prohibition on the use of paper as the only means of distribution. The exemption seems to imply that there is no need to make public information available by electronic means under the E-FOIA if it is printed and offered for sale. [Back]
2. GSA is responsible for Federal coordination of the X.500 directory. The directory is to be implemented on a distributed basis by all Federal agencies. In addition to the Green Pages for document metadata, the directory will include White Pages for employee metadata, Yellow Pages for services metadata, and Blue Pages for office metadata. [Back]
3. The "foot-draggers" of whom Sen. Leahy spoke may argue that just because a "record" is created or processed in electronic form does not mean that it is thusly "maintained". Therefore, there is no need either to provide access to the underlying data or to preserve it for any period of time in archival. However, if the information is in existence in electronic form anywhere in the organization at anytime, by definition it has been "maintained" in that form, albeit perhaps inappropriately and inefficiently (e.g. with little or no metadata on a PC hard drive). Deleting the file from electronic media does not absolve individuals or organizations of their responsibilities anymore than shredding a paper record does. Both are inappropriate, if not illegal, especially in light of the new requirements of the E-FOIA to make records available to the public in electronic form. Indeed, deleting a file from a PC and/or file server's active storage may only complicate the process of recovering it from electronic storage archives -- the exact opposite of the mandate to make public information readily available.
With respect to access, the only issue is whether the information documents official business or not. If it does, it should be maintained in an electronic document management system, and if the information is public, it should made available in electronic form. If it does not document official business, the question is why public funding has been spent on it. E-mail systems are not appropriate vehicles for documenting official business and, thus, should not be subject to record-keeping requirements. However, if agencies fail to maintain records in an electronic document management system, E-mail systems will be inappropriately used as inadequate substitutes. The National Archives and Records Administration (NARA) noted as much when issuing the final rule on E-mail record-keeping [60 FR 44636, August 28, 1995].
It is appropriate to spend some amount of the taxpayers' money for purposes like E-mail, to facilitate official business. However, as directed by GPRA, the focus should be on outputs (e.g., documents) and officially documented outcomes, rather than on inputs (e.g., meetings and informal messages). The tools should be efficiently and appropriately applied to the purposes for which they are suited. [Back]
4. The Government Management Reform Act (GRMA) of 1994 requires each agency to: 1) submit to OMB a single, audited financial statement for the preceding fiscal year, which includes performance measures of outputs and outcomes; 2) give a clear and concise description of accomplishments, financial results, and conditions; and 3) disclose whether and how the mission of the agency is being accomplished and what, if anything, needs to be done to improve either program or financial performance. (Lawrence Livermore Lab) [Back]
5. The Chief Financial Officers (CFO) Act of 1990 requires each agency to: "... develop and maintain an integrated agency accounting and financial management system including financial control, which ... provides for the ... systematic measurement of performance." [Back]
6. In the June 5, 1997, edition of The Washington Post, OMB Deputy Director Koskinen is quoted as saying that the goal of GPRA is "not to produce perfect documents" but to provide information useful to Congress and the agencies. Thus, while perfection is not the aim, documentation clearly is, and any self-respecting agency will naturally make it's documentation as good and useful as reasonably possible. [Back]
7. Xerox, The Document Company, defines "document" as "information structured for human consumption." While much information is exchanged among computers, if a system provides no output "structured for human consumption," the wisdom of spending the taxpayers' money on it may be questionable. In terms of GPRA, if the outcome is not greater human understanding, what is the point? And if human understanding is the desired outcome, the information obviously must be structured, managed, and made available for human consumption. [Back]
8. Failure to make records available on the Internet could be considered a violation of the spirit, if not the letter of paragraph 3506(d)(4) of the PRA, which prohibits "exclusive, restricted ... distribution arrangements." Indeed, if implemented as a metadata or perhaps even a full-text search form, Sen. Leahy's suggestion that agencies should establish FOIA-requester pages on their Web sites could be interpreted to require them to treat each query as a formal request under FOIA, in which case the statutory requirements would apply. [Back]