In the centuries since Robert Beverly, Jr., wrote and published his History and Present State of Virginia (1705) in London, historians and genealogists have often consulted or copied foreign documents about Virginia subjects and residents. The Virginia Colonial Records Project was novel only in the magnitude of its ambition: it aimed at reconstituting on microfilm the documentary record of Virginia's colonial history that had been decimated by war, fire, and accident during three centuries.

The destruction of records began early. In 1676, Nathaniel Bacon and his rebellious followers burned the capitol at Jamestown, and many early records went up in smoke. Some Virginians recognized that without accurate records, government and the legal system, dependent upon precedents, would be hamstrung. As a law student at the College of William and Mary, Thomas Jefferson copied the colony's handwritten early laws from aging manuscripts in Williamsburg and later, as a young lawyer riding the court circuits, made a point of recording every copy of a Virginia statute that he found in his travels. His labors, and those of others like the indefatigable William Hening, helped make those statutes available in printed form as Hening's Statutes at Large during the 1820s.

Records less essential to the day-to-day operation of the courts and legislature did not fare so well. Virginia's archive survived the American Revolution and the War of 1812 without severe damage, but the Civil War brought disaster. As the Confederates evacuated Richmond in April 1865, the General Court building, where the records were stored, went up in flames. The fire proved doubly disastrous because many county clerks in counties with courthouses exposed to warfare had sent their most valuable records to the General Court in Richmond for what they thought would be safekeeping. To that disaster, one can add the effects over the years of fires, floods, leaky roofs, and neglect. The sad result was that Virginians could rightly boast of a glorious colonial heritage, but the state had few of the official records necessary for scholars to document and study that heritage.

The problem of reconstituting the lost archive of colonial Virginia had a solution. The British are a record-keeping nation, and it was recognized that copies of the papers of royal governors and other documents could be secured there and brought to America. In fact, not long after the Civil War, the Virginia State Library employed an agent in London to make transcripts of a few of the most important records lost in the General Court fire. The state's poverty and a climate of indifference about the missing records made that experiment unique in the nineteenth century. Scholars are indebted to those individuals who obtained copies of records and published transcripts in the early volumes of the Virginia Magazine of History and Biography, the William and Mary Quarterly, Tyler's Quarterly, and other journals, but those documents, often selected for their value to genealogists, are frustratingly incomplete and scattered. Early in this century, the Virginia State Library did obtain copies of the journals of the House of Burgesses and of the Governor's Council for publication here, and other records became available through the work of dedicated editors— notably Susan Myra Kingsbury's Records of the Virginia Company of London (1906– 1935)— but by far the greater part of Virginia's archive remained on the other side of the Atlantic Ocean.

In 1939, William J. Van Schreeven, the State Archivist of Virginia, began proposing in his annual reports that the state secure copies of its colonial records for posterity. World War II soon made a profound argument in favor of his proposal. Historians and archivists here and in Great Britain watched the destruction by bombing of London and other cities and feared for the survival of irreplaceable records and documents. Recognition that, with copies of records secure in the United States, the British archive could be reconstituted in the event of disaster helps to explain the support that Van Schreeven won for his proposal after the war, and it also helps to explain the cooperation of British librarians and archivists with the Virginia Colonial Records Project.

After the war, Van Schreeven gained a valuable ally in the person of Francis L. Berkeley, Jr., the curator of manuscripts at Alderman Library of the University of Virginia, who obtained a Fulbright grant in 1952– 53 to make a quick survey of manuscripts bearing upon Virginia in England and Scotland. He and Van Schreeven used his survey to demonstrate that a Virginia Colonial Records Project was feasible. For a year, Berkeley sent back to scholars across the United States mimeographed reports on the holdings of British repositories, and enthusiasm for the project grew.

Then, opportunity knocked. As the 350th anniversary of the founding of Jamestown approached, the state and federal governments created commissions to oversee the celebration. The commissions chose a committee of historians and archivists as consultants and asked them to propose projects of enduring value relating to the celebration. The committee unanimously recommended surveying the records of colonial Virginia in Great Britain and arranging for microfilmed copies of them. The Jamestown commissions agreed to begin reconstituting the archive of colonial Virginia for the benefit of posterity.

Van Schreeven and Berkeley were ready to act. In March 1955, the federal government appropriated money to carry the VCRP through 1957. One month later, in the reading room of the British Library, agent George H. Reese examined manuscript records of a 1771 suit brought in the General Court of Virginia between John Hite and Thomas Fairfax, rival claimants to lands in Virginia. His one-page description of these records became Survey Report No. 1 of the VCRP.

The committee in charge of the project— representatives of the Virginia State Library, Alderman Library of the University of Virginia, the Virginia Historical Society, and the Colonial Williamsburg Foundation— determined the project's scope and methods and hired the overseas agents. It decided that the chronological span of the VCRP was roughly from the founding of the colony in 1607 to the end of the American Revolution in 1783 but trusted to the wisdom of the agents in England for inclusion of relevant documents that fell outside these limits. The committee also decided to trust the judgment of the agents rather than to try to devise a geographical definition of Virginia to fit all periods.

The most important decisions concerned the methods of the project. The committee decided that the agents must base their reports on an examination of the actual documents rather than repeating the information in existing guides. The agents were able to give fresh, eyewitness reports of documents and do so in more detail than a cataloger might. Most important, because the agents were on the lookout for Virginia-related material, they spotted many relevant documents not specifically reported in catalogs or finding aids.

The committee also drew up the form for the Survey Reports, the means by which the agents would report their findings. Each Survey Report has a space for the agent to record the following:

1. The name of the foreign repository,
2. The class, or the call number, of the documents used at the repository,
3. The title of the documents,
4. The date(s) or date range of the documents,
5. Bibliographical references to printed guides and catalogs where the documents are described,
6. The date the agent examined the documents,
7. Pages to be microfilmed, and
8. The number of microfilm exposures required.

The rest of the Survey Report is the agent's description, which could run from a single paragraph to more than one hundred pages.

Each Survey Report served three functions. First, and most important, the Survey Report facilitated ordering the microfilmed copies of documents. Second, agents were to include enough information about the original documents to help a researcher determine whether it was worthwhile to examine the original document on microfilm, enabling the Survey Reports to function as a finding aid. The third function of the Survey Report was to organize and control the immense number of documents that the VCRP would generate. Agents were to include on each Survey Report— and there are now 14,704 of them— enough information about the microfilmed materials to enable VCRP staff in Virginia to link each Survey Report with the proper reel of microfilm (and there are now 963 reels).

The first targets of the agents were the major repositories in London. At the British Public Record Office, they began with the Colonial Office papers, in particular the class known to historians as C.O. 5: the acts of the assembly, the journals and correspondence of the royal governors, and other administrative records. This class alone produced forty-eight reels of microfilm, and the Colonial Office papers as a whole generated more than a thousand Survey Reports.

The agents surveyed relevant private papers in addition to official documents and also surveyed some local and provincial archives, including ones in Scotland, Wales, Ireland, and France. Naturally, research at the Public Record Office generated more than half of all the Survey Reports, but agents managed to visit more than a hundred other foreign libraries and archives during this phase of the VCRP.

After 1957, support from Virginia institutions kept the VCRP alive, although often on a shoestring budget. Interest in and money for the project waned in the mid-1960s, and surveying and microfilming temporarily came to a halt. Then, in 1970, the approaching bicentennial of the American Revolution revived the VCRP. The project continued to accumulate Survey Reports into the 1980s (although microfilming had been halted during the 1970s when the cost of silver film skyrocketed).

As a preservation project, with more than 28,000 pages of Survey Reports and nearly one million images on microfilm, the VCRP could claim success; as a tool for scholarship, the VCRP's potential remained unrealized. Because the numbering system for the Survey Reports and microfilm reflected the order in which they arrived in Virginia, scholars aware of relevant materials in a British archive faced the daunting prospect of wading through thousands of pages of Survey Reports before locating the documents they sought. While copies of the microfilm gathered dust at the participating Virginia institutions, the VCRP drifted toward antiquarianism.

Disappointed at this prospect, the legislature gave State Librarian Donald Haynes consolidated authority over the VCRP and a one-time appropriation to support computerization. Then, in 1985, Haynes transferred the VCRP to the editorially-experienced historians of the State Library's Publications Division, then headed by Jon Kukla. I joined the project in February 1986.

The project's new directors concurred with a comment by one of the original committee members that "there is no real way of knowing when a mammoth project such as this is over," but did not regard this reality as an excuse for inaction. We knew the necessary first step to providing researchers better access to the riches of the VCRP was to gain control over the mass of material already on hand. We suspended redundant survey work in Great Britain and concentrated on the pressing problem: restoring the link between the Survey Reports and the microfilm that the committee had intended as a guide for scholars.

The first visible product of the renewed VCRP appeared in August 1990, when the Publications Division of the Virginia State Library and Archives published A Key to Survey Reports and Microfilm of the Virginia Colonial Records Project, edited by John T. Kneebone and Jon Kukla (with assistance from Daphne Gentry and Donald W. Gunter). The Key is composed of two volumes, one a guide to Survey Reports and microfilm from the British Public Record Office and the other to Survey Reports and microfilm from all the other repositories that have been surveyed. The Key enables researchers to link commonly known archival classification references from foreign repositories to the proper Survey Report and microfilm reel.

Next, we turned our attention to salvaging an index to the Survey Reports. As early as 1955, the VCRP committee had discussed the necessity of indexing the Survey Reports, but it was not until the mid-1960s, with the VCRP moribund, that State Archivist Van Schreeven began developing an indexing system, creating files of several thousand 3x5 cards on subjects of special interest to him.

The revived VCRP in the 1970s hired indexers who soon discovered that Van Schreeven's system was too detailed for efficient creation of an index. They simplified that system, recording far less information on each 3x5 card. In doing so, they shifted from a controlled vocabulary of terms to a natural-language system of indexing. That is, indexers recorded terms exactly as the terms appeared in the texts of the Survey Reports. This method had unfortunate consequences because different Survey Reports used different terms for the same subject, such as "mariner," "sailor," and "seaman." Filed alphabetically, these index entries would be separated. A user interested in seamen would have to check every possible synonym for that term to be sure of finding all relevant entries. A natural-language system also requires conscientious attention to cross references because the classification vocabulary is uncontrolled. Instead, the indexers deferred editorial decisions and attempted to index everything. They produced vast numbers of index cards but the result was chaos.

When the Publications Division took over the VCRP, it inherited 600,000 3x5 cards, which represented an index to fewer than two-thirds of the Survey Reports. Except in scale, of course, that situation is hardly unique to the VCRP. In libraries and archives across the United States, one can find aging wooden cardfiles filled with the fruits of long-ago indexing projects, perhaps to local newspapers or a manuscript collection, often incomplete and the work of several hands over a period of years. The information that these indexes contain may be invaluable to researchers, but the problems of inconsistency and incompleteness make these indexes frustrating to use.

Unlike the situation six years ago, computer software packages exist today which promise to enable libraries and other institutions to create specialized index files from machine-readable bibliographical records, but even now none of them is yet able to solve editorial problems on the scale of those found in the VCRP index. Only the computer could handle such a quantity of information efficiently, but we would have to devise a unique system to solve our problems. With an appropriation from the Virginia General Assembly and a generous agreement from VTLS, Inc., to allow the project to use and modify the VTLS format for library cataloging, we began computerizing the index.

In consultation with VTLS, Inc., we modified the VTLS format for MARC bibliographical records so that the computer treats each Survey Report as if it were a book. A MARC bibliographical record has been created for each of the 14,704 Survey Reports. In the modified format, the Survey Report Number appears in the call-number field (090), which means that each report can be retrieved from the database through a call-number search. The name of the repository holding the original manuscript is entered in the author field (110) and the title of the manuscript in the title field (245). In addition, the microfilm reel number appears in the 049 field, the date(s) of the document(s) in the 260 field, and bibliographical references to the document(s) in the 500 field.

From the beginning, we intended to employ the searchable subject-heading fields (6xx) in the MARC records for the index entries, but a careful analysis of the existing index revealed the necessity of additional modifications. Not only did the index lack a controlled vocabulary, but the indexers' method of recording verbatim what appeared in the text of a Survey Report resulted in numerous ambiguous references. Did the term "China" refer to the country or to porcelain? Did the important term "Tobacco" refer to the plant or to imports and exports? And there were forty thousand index cards referring to "Tobacco"! The index contained references to "Chairs" but also to "Leather chairs" and "Cane chairs." Faced with limited funds and staff, we reluctantly set aside the subject index.

Fortunately, personal-name entries did not present the same order of editorial problems that subject entries did. Genealogists, of course, would welcome a name index to Survey Reports of the Virginia Colonial Records Project, but scholars, familiar with the identities of historical actors, could use a name index to track down relevant documents, too. While data technicians created the MARC bibliographical records, project staff prepared the personal names in the remaining unindexed Survey Reports for data entry.

The main task was to develop a controlled vocabulary for the modifiers— those terms such as "sailor," "lawsuit involving," "merchant," "indentured servant"— which help the researcher to identify a particular person. After the analysis of the subject index, we compiled a thirty five-page list of possible modifiers and then boiled them down to a controlled vocabulary of some forty standard descriptive terms.

This controlled vocabulary of modifiers organizes data entry of personal names into the MARC records. We first divided the controlled vocabulary of modifiers into related groups and assigned each group to a different 6xx field: nautical modifiers (e.g., shipbuilder, pirate, privateer) to the 610 field; legal modifiers (e.g., debts involving, lawyer, petition by) to 611; status modifiers (e.g., clergyman, loyalist, indentured servant) to 630; occupational modifiers (e.g., farmer, planter, tradesman) to 650; and governmental modifiers (e.g., governor, military officer, diplomat) to 651. Personal-name entries without modifiers would appear in the 600 field.

To simplify data entry, we then created a code of alphabetical characters to represent the controlled vocabulary of modifiers. The data entry technician uses a code book to determine the correct formulation for translation of natural-language modifiers on the 3x5 cards into the controlled vocabulary and to identify the proper 6xx field in which to enter that personal name entry. For example, an entry with the natural-language modifier "clockmaker" changes to "craftsman" in the controlled vocabulary. The code book directs the data-entry technician to enter all personal-name entries with the modifier "clockmaker" into the 650 field using the alphabetical character "n" to signify the modifier "craftsman."

Within each 6xx field, the personal-name entry is further organized into subfields encoded by delimiters and alphabetical characters. Each entry can include the following subfields: \a, the personal name itself; \c, a title such as Bishop or Mrs.; \x, a modifier; \s, a ship's name (where appropriate); \z, a geographical location; \y, the data of the reference; \k, the Survey Report Number; and \e, the page number(s) on which the personal name appears.

As an example of how this code works, consider the names John Hite and Thomas Fairfax, which appear in Survey Report No. 1. The modifier associated with both those names in that Survey Report is "lawsuit involving." The code for "lawsuit involving" is the letter "l" that appears after \x in the 611 field (legal modifiers). In the MARC record for Survey Report No. 1, the entries appear as follows: 611 00 Hite, John, \x 1 \y 1771, \k \e 1. 611 00 Fairfax, Thomas Lord, \x 1 \y 1771, \k \e 1. If the modifier associated with the personal names were, say, "testimony by," the code book would direct the data entry technician to enter the names in the 611 field with the alphabetical character "t" following \x.

In the same MARC record, the name of John Murray, fourth earl of Dunmore, the last royal governor of colonial Virginia, is entered into the 651 field (governmental modifiers) with the letter "g" following \x to signify "governor." If the personal name were entered into the 610 field (nautical modifiers), the letter "g" following \x would signify "sailor."

By means of this code, the editorial problems presented by the existing cardfile index could be solved. Most important, the shift from natural language to a controlled vocabulary takes place at the time of data entry. Although the code includes a complex mixture of electronic library cataloging signifiers, most elements can be entered with a single stored keystroke, thereby speeding up data entry while at the same time reducing the incidence of typographical errors.

The full code also provides the signals necessary to operate the programming routines that will transform the coded database into a publishable (and readable) index. For example, the most important data element in each entry, other than the personal name itself, is the Survey Report Number. Rather than keying the Survey Report Number into each entry and risking typographical errors, the data-entry technician inserts \k at the place where that number should appear. Programming routines will subsequently replace the \k in each index entry with the Survey Report Number previously entered into the 090 field of the proper MARC record.

The full code embeds typographical signals, too. For example, in a personal-name entry for a "master of a ship," the \s precedes the name of the ship and signifies that the name should be italicized. Similarly, the \y preceding a date and the \k also represent typographical signals for changes in typefaces and font sizes so that the dates and the Survey Report Numbers might be distinguishable in the final index.

Thus a personal-name entry in a MARC record that looks like this:

610 00 Davidson, James, \x m \s Nancy, \y 1798, \k \e 1 will, after database manipulation, look like this in the final index:

Davidson, James

Master of ship: Nancy, 1798, SR1860: 1

In effect, the data-entry system employs capabilities of VTLS bibliographical formats to create a language of textual and typographical signals specific to the personal-name index of the Virginia Colonial Records Project. The code solves editorial problems and makes data entry efficient. With programming designed to extract and manipulate the coded data, the VCRP database of MARC records in VTLS format will become a personal-name index, ready for final editing. With typographical codes already embedded in the index, the edited text will be ready for production of camera-ready copy and ultimate publication.

The code system has proven efficient. Project staff indexed and entered the remaining five thousand Survey Reports in less than two years and completed entering the hundreds of thousands of entries created earlier on cards nearly a year ahead of schedule. In fact, progress has been so swift that we also entered the ship names from the Survey Reports for a supplement to the personal-name index, employing the same codes and formats. Project staff are now editing and proofreading the MARC records— many of them containing hundreds of entries— in preparation for the programming routines to manipulate the database.

As with any other special collection of documentary materials, in the final analysis the value of the Virginia Colonial Records Project depends upon the uses that researchers make of it. That use depends upon access. If researchers cannot get to relevant materials— indeed, if researchers do not even realize that relevant materials exist in the VCRP— the project has been almost pointless. The VCRP is on schedule for publication of the personal-name index in four volumes before 1996. In addition, they have completed preliminary planning for the eventual publication of the Survey Reports themselves— all 28,000 pages of them— in ten volumes. Our hope is that when researchers across the country have access through the Key to Survey Reports and Microfilm of the Virginia Colonial Records Project and the indexes, the virtues of the VCRP will speak for themselves and new light will be cast into the dark recesses of Virginia's colonial history.

—John T. Kneebone is an editor at the Virginia Colonial Records Project.