I was reading over Cliff Shnier’s June 2008 column on InsideCounsel.com entitled "A time to reap, a time to cull." It is an excellent, concise overview of the workflow required for a successful e-discovery project, sprinkled with several doses of common sense on collecting and processing ESI.
Toward the end, Cliff discusses the perils of too-hastily concocting a list of keywords used for filtering and culling electronic data. He points readers to In Re Seroquel Products Liability Litigation, 244 F.R.D. 650 (D. Fla. 2007) (generously hosted on the Electronic Discovery Blog; good cases summaries located here and here).
I dug a little deeper into the case, and discovered a fascinating litany of delinquent behavior surrounding basic principles of e-discovery. The Court was not happy.
First, the Court lists the shortcomings of defendant’s production:
"… although [Defendant] has a system to deliver voicemail, faxes, and video into Outlook inboxes, none has been produced; there are few emails from some custodians, and email boxes are missing from alternative email boxes. Plaintiffs also contend that many relevant emails and documents were not identified and produced because [Defendant] performed an unreasonable key word search."
And in a footnote, the Court provides some examples of words and phrases that were omitted from the "unreasonable key word search:"
"Seroquel’s generic name [(The case involves a slew of cases over the anti-psychotic drug Seroquel)], acronyms for diabetes, hyperglycaemia spelled the British way; and endocrine. The search method apparently failed to include common misspellings or the singular forms of words and failed to make allowance for spaces or dashes."
The Court doesn’t let the issue rest:
"The key word search was plainly inadequate. … While key word searching is a recognized method to winnow relevant documents from large repositories, use of this technique must be a cooperative and informed process. rather than working with Plaintiffs from the outset to reach agreement on appropriate and comprehensive search terms and methods, [Defendant] undertook the task in secret."
As you can imagine, the Court would have rather the parties agreed on a set of key words together. I especially like how the court ever-so-gingerly emphasized the importance of having technical expertise at these meetings:
"Many of the other technical problems identified by Mr. Martin and Mr. Jaffe [e-discovery specialists for the plaintiff] likely could have been resolved far sooner and less expensively had [Defendant] cooperated by fostering consultation between the technical staffs responsible for production. Instead, [Defendant] shielded its third party technical contractor from all contact with Plaintiffs. This approach is antithetical to the Sedona Principles and is not an indicium of good faith."
After dismissing other arguments from the Defendant, and recognizing that many of the disagreements could have easily been worked out in a more efficient manner, the Court provides a strong conclusion:
"The Court finds that [Defendant] has been ‘purposely sluggish’ in making effective production to Plaintiffs. … It is undisputed that the [completed production] … had load file, metadata, page break and key word search problems, making the 10 million pages of documents unaccessible, unsearchable, and unusable as contemplated under the Rules."
In a fascinating February 18, 2008 opinion, Magistrate Judge John M. Facciola references the Federal Rules of Civil Procedure regarding electronically stored information (ESI) in a seemingly mundane federal criminal matter.
United States v. O’Keefe Case No. Cr. 06-249 (D.D.C. February 18, 2008) charges the defendant (formerly employed with the U.S. Department of State stationed in Canada) with accepting "gifts and other benefits" from a co-defendant in exchange for expediting visa requests for employees of the co-defendant’s company.
Judge Facciola recognizes:
"In criminal cases, there is unfortunately no rule to which the courts can look for guidance in determining whether the production of documents by the government has been in a form or format that is appropriate. This may be because the ‘big paper’ case is the exception rather than the rule in criminal cases."
and continues:
"It [would be] foolish to disregard [the Federal Rules of Civil Procedure] merely because this is a criminal case, particularly where, as is the case here, it is far better to use these rules than to reinvent the wheel when the production of documents in criminal and civil cases raise the same problems."
This is an intriguing but embraceable statement. Criminal cases can experience the same issues of collection and preservation of ESI that a civil matter would see. And since the issues have been (and continue to be) thoroughly vetted in the civil litigation world, there’s no reason that they can’t be equally applied in a criminal matter.
Judge Facciola details the search activities of the government which are disturbing at some points. I share Ralph Losey’s concerns that there was no mention of an attorney being involved when the Visa Unit Chief of the United States Consulate General in Toronto, Canada (presumably the defendant’s former supervisor) conducted the search for paper records and "responsive emails." Ralph does an excellent job of referring to the duties directed upon legal counsel to communicate with the key players to better understand how potentially relevant information is stored and located.
The more noteworthy issue in the opinion that others have commented upon is in the second to last paragraph where Judge Facciola writes that developing a set of "search terms or ‘keywords’ … is a complicated question involving the interplay … of the sciences of computer technology, statistic and linguistic."
Judge Facciola continues:
"Given this complexity, for lawyers and judges to dare opine that a certain search term or terms would be more likely to produce information than the terms that were used is truly to go where angels fear to tread. This topic is clearly beyond the ken of a layman and requires that any such conclusion be based on evidence that, for example, meets the criteria of Rule 702 of the Federal Rules of Evidence."
Mark Foley in his article "Expert Testimony May Be Needed for E-Discovery Keyword Searches" suggests that taking Judge Facciola’s comments literally would mean that any objection to a list of search terms would need to be supported by the testimony of an expert in the field of computer science, statistics, and/or linguistics.
I don’t believe Judge Facciola intends for us to take his comments to that extreme, but it does highlight the absolute necessity to seek agreement on search terms early and quickly. It’s also advisable to employ an expert to review the search terms, or at least earnestly discuss the issue with your e-discovery vendor. Some attorneys who may be more familiar with Westlaw or Lexis or Google type searches insist on developing a lengthy keyword list to circumvent the fear of "missing something." While I understand the concern, a foolhardy and over-broad keyword list will end up costing a party much more time and money in the long run.
