A Game of Chicken? Setting Forth a Detailed TAR Review Protocol

What Is TAR?

Technology assisted review, or TAR, “uses software that can be trained by a human being to distinguish between relevant and non-relevant documents.” Progressive Cas. Ins. Co. v. Delaney, No. 2:11-cv-00678-LRH-PAL, 2014 WL 3563467, at *8 (D. Nev. July 18, 2014). It “allows parties to prioritize and/or categorize documents for purposes of document review.” Winfield v. City of New York, No. 15-CV-05236 (LTS) (KHP), 2017 WL 5664852, at *4 (S.D.N.Y. Nov. 27, 2017). TAR works by having an experienced attorney or subject matter expert involved in the litigation prepare a “seed set” of responsive and non-responsive documents in order to “train the computer system how to distinguish between them.” Id.; Progressive, 2014 WL 3563467, at *8.

TAR is taking on growing importance as “cheaper, more efficient and superior to keyword searching.” Hyles v. New York City, No. 10 Civ. 3119 (AT)(AJP), 2016 WL 4077114, at *2 (S.D.N.Y. Aug. 1, 2016); see also Duffy v. Lawrence Mem’l Hosp., No. 2:14-cv-2256-SAC-TJJ, 2017 WL 1277808, at *3 (D. Kan. Mar. 31, 2017) (“Technology-assisted review can (and does) yield more accurate results than exhaustive manual review, with much lower effort.” (citation omitted)). Indeed, as one court has recognized:

With the advent of TAR and improvements made to that process, there appears to be a growing chorus that key word searching is not best practices, and that TAR is the way to go. Many trusted ESI experts—and courts—understandably sing the praises of TAR, especially its ability to scientifically quantify the successfulness of gathering and producing relevant documents.

City of Rockford v. Mallinckrodt ARD Inc., 326 F.R.D. 489, 493 (N.D. Ill. 2018).

My colleagues, Russell Beets, Amy Catton and Clara Skorstad, recently wrote about our own team’s experience with TAR in a complex case involving millions of documents (you can find Part One of their blog here and Part Two here). In addition to the valuable information provided in those blogs, last year, the United States District Court for the Northern District of Illinois laid out a very detailed protocol for conducting TAR that can serve as a go-by for parties wishing to conduct TAR in their own litigation. See In re Broiler Chicken Antitrust Litig., No. 1:16-cv-08637, 2018 WL 1146371 (N.D. Ill. Jan. 3, 2018).

Background on the Broiler Chicken Litigation

The Broiler Chicken case involved a class action brought by purchasers of broiler chickens against nearly 30 defendants. Maplevale Farms, Inc. v. Koch Foods, Inc., No. 1:16-cv-08637, 2016 WL 4607834 (N.D. Ill. Sept. 2, 2016) (Class Action Complaint). Plaintiffs alleged that “in order to maintain price stability and increase profitability, beginning at least as early as January 2008 Defendants conspired and combined to fix, raise, maintain, and stabilize the price of” broiler chickens “by coordinating their output and limiting production with the intent and expected result of increasing prices” of broiler chickens in the United States. Id. ¶ 1. (For those who are unfamiliar, “broiler chickens” are chickens that are bred and raised specifically for meat production.) On January 3, 2018, Magistrate Judge Gilbert, in conjunction with a special master, issued an Order Regarding Search Methodology for Electronically Stored Information in the litigation.

The Court Establishes a Detailed TAR Review Protocol

According to the court, the goal of the protocol was to provide “an appropriate level of transparency into a producing Party’s electronic search process, without micromanaging how the producing Party meets its discovery obligations and without requiring the disclosure of attorney work product or other privileged information.”  2018 WL 1146371, at *1.

1.  Document Culling

The order then set forth the procedures for using culling technologies prior to searching. The protocol addressed de-duplication, email threading, targeted collection, and exception reporting. Id. If a requesting party wanted certain email domains excluded from the data set, it had to provide a list of such domains prior to the testing of search terms. Id. If a party elected to use any additional culling techniques beyond those discussed in the protocol, such parameters had to be disclosed. Id. at *2.

2.  TAR Searching

The protocol next discussed the search methods that would be used, dividing them into a TAR search process and keyword search process. As to the TAR search process, the parties were allowed to propose a limited number of custodians as to whom the culling techniques outlined in the protocol would not be applied. Id. The culling parameters would apply to all other custodians. Id. A producing party using TAR was required to disclose the name of its software and vendor, a general description of its TAR process, a general description of the categories or sources of documents included and excluded, and the quality control methods it was undertaking. Id. The requesting party then had seven days to raise any concerns it had with the proposed TAR process. Id. The requesting party could also propose exemplars to train the TAR tool or narrow keyword search strings to be used to generate exemplars to train the TAR tool. Id. The producing party maintained the right to reject and oppose any such requests, subject to resolution by the special master or court. Id.

3.  Keyword Searching

As to keyword searches, the protocol required the parties to disclose what search software they had decided to use and provide certain information about that software. Id. at *2-3. The producing party would propose a list of search terms and the requesting party would have twelve days to propose revisions to those search terms. Id. at *3. If the producing party objected to the suggested revisions, it had eight days to provide information in support of its objections. Id. The protocol also allowed plaintiffs collectively and defendants collectively to propose additional search terms to a producing party on a one-shot basis. Id. at *3-4.

4.  Validation Protocol

Finally, the court provided a validation protocol. This validation protocol established a procedure for quality control sampling on the part of a producing party, regardless of whether it had used TAR or manual review. Id. at *4-6. An appendix to the protocol provided additional information. Id. at *6-7.

Conclusion

As set forth above, there is a growing belief among both courts and ESI experts “that TAR is the way to go.” That is certainly the experience of our own KT LitSmart team over the past year. The Broiler Chicken litigation provides a good example of a thoughtful and detailed TAR review protocol, which may become a useful framework for the ever-increasing number of cases where the parties will seek to use TAR in the years to come.

DISCLAIMER: The information contained in this blog is not intended as legal advice or as an opinion on specific facts. For more information about these issues, please contact the author(s) of this blog or your existing LitSmart contact. The invitation to contact the author is not to be construed as a solicitation for legal work. Any new attorney/client relationship will be confirmed in writing.