Printer Friendly Version

The saying "garbage in, garbage out" is often familiar to computer users. It means that computer systems are only as good as the information provided to them. The saying is a reminder of how important proper document selection is for any document retrieval system an organization chooses to deploy.

Document retrieval solutions provide the most worth to their users when they contain the majority of an organization's documents with usable knowledge and few low-value documents. This allows users to quickly find documents that are truly useful and not waste their time. Because the document vetting process is so crucial to the eventual success of document retrieval system integration, some vendors actually assign content consultants to help the firm oversee the process and help them create a strategy and process for selecting documents.

This article has two primary sections. The first section provides a discussion about document selection approaches, covering suggestions and guidelines for identifying documents for inclusion in a document retrieval system. The second section presents models for an ongoing document vetting process.

Document Selection

The Big Picture
The general objective of selecting documents for a document retrieval system is to identify those documents that contain "usable knowledge". Usable knowledge means that people in the firm could find the content of a document valuable in the future. Let's use the example of a law firm. Briefs, motions, precedent documents, and reports are examples of documents that could contain usable knowledge. Time sheets, fax cover sheets, and billing lists are not considered usable knowledge. Only a fraction of the documents in a firm's document management system (DMS), or file structure, are likely to contain usable knowledge. A good rule of thumb is to expect 5 to 10 percent of the total number of documents in a DMS to be those that meet the firm's definition of usable knowledge.

An iterative process of identifying documents with usable knowledge is frequently followed. The process, depicted in the figure below, can be thought of as a funnel. At the top of the funnel are all of the documents in a DMS. An initial document selection criterion is applied to identify a smaller number of documents likely to meet the organization's definition of usable knowledge. A refinement of the selection criteria may be applied to eliminate less useful documents, resulting in an even smaller number of documents selected. Although this process can continue as many times as necessary until the final set of documents contains a maximum of usable knowledge documents and a minimum of unusable documents, one or two iterations is often sufficient.

After the initial loading of documents, very little typically is required to maintain a useful document collection.

Major steps to be taken by content consultants in selecting documents

1. Create initial criteria for "usable knowledge."
   Users should be asked to describe the characteristics of documents they find most useful. This is often based on the profile of the document, such as titles, practice groups, document type, keywords or author.

2. Conduct document analysis.
   The selection criteria are translated to database queries used by the document retrieval system to select documents. The reports are created analyzing the documents matching the criteria. The number of documents matching the selection criteria should be approximately 10 percent of the total number of documents available.

3. If necessary, refine document selection criteria.
   Look for obviously useless documents that could be removed from the candidate document collection. Examples include fax cover sheets, document templates with no content, files titled "junk, temp, delete...," documents by certain authors or practice groups, etc.

4. Configure manual vetting.
   Once the system is in use, documents may be discovered that should be removed or documents may need to be added that did not meet the selection criteria. A means to manually add or remove documents from the system is helpful. This can be accomplished by modifying the document profile(s) provided by the DMS.

Page 2: Document Selection Approaches