Lawson v. Spirit Aerosystems, Inc.
Filing
335
MEMORANDUM AND ORDER denying 226 Motion to Compel the Production of Responsive Documents. Signed by Magistrate Judge Angel D. Mitchell on 4/9/2020. (ctv)
IN THE UNITED STATES DISTRICT COURT
FOR THE DISTRICT OF KANSAS
LARRY A. LAWSON,
Plaintiff,
v.
SPIRIT AEROSYSTEMS, INC.,
Defendant.
)
)
)
)
)
)
)
)
)
Case No. 18-1100-EFM-ADM
MEMORANDUM AND ORDER
This matter comes before the court on plaintiff Larry A. Lawson’s (“Lawson”) Motion to
Compel the Production of Responsive Documents. (ECF No. 226.) In April 2019, the court first
directed Lawson and defendant Spirit AeroSystems, Inc. (“Spirit”) to work together on an
electronically stored information (“ESI”) search protocol. After the parties spent months trying to
engage in what proved to be an unworkable, costly, and largely fruitless effort to search Spirit’s
ESI, in September 2019 Spirit agreed to conduct a technology-assisted review (“TAR”) using
Lawson’s proposed search terms and custodians. Spirit ceased producing responsive documents
through the TAR process after reaching an 85% recall rate, meaning that the TAR algorithm had
correctly identified 85% of the responsive documents in the data set. At that point, approximately
1,850 potentially responsive documents remained in the TAR set (the “residual TAR documents”).
Spirit’s first-level review team had identified these documents as potentially responsive but,
because the TAR process reached an 85% recall rate, they were not produced.1 Lawson now moves
to compel Spirit to produce these residual TAR documents.
1
Spirit clarified in its response brief that only 800 documents were identified as responsive,
and the remainder are associated family-member documents that are non-responsive. (ECF No.
240, at 5.) Lawson is no longer seeking these non-responsive documents. (See ECF No. 247, at 2
For the reasons discussed below, the court is persuaded that Spirit conducted a reasonable
and diligent search for documents responsive to Lawson’s ESI demands. The court is satisfied
that Lawson has had a full and fair opportunity to obtain ESI that is relevant and proportional to
the needs of the case via the TAR process, as well as through other targeted document productions
outside of that process. The court will therefore deny Lawson’s motion to compel Spirit to produce
the residual TAR documents.
I.
BACKGROUND
The background of this lawsuit is more thoroughly set forth in this court’s prior orders,
familiarity with which is presumed. Highly summarized, Lawson is Spirit’s former chief executive
officer who retired on July 31, 2016. His Retirement Agreement contained non-compete
obligations for two years, until July 31, 2018. In early 2017, Lawson engaged in business dealings
with non-party investment firms Elliott Associates, L.P. and Elliott International, L.P.
(collectively, “Elliott”) to provide consulting services in connection with a proxy contest Elliott
launched to replace five board members of Arconic, Inc. (“Arconic”). When Spirit learned about
this, Spirit notified Lawson that his involvement with Arconic constituted a breach of his noncompete. Spirit stopped paying Lawson and demanded that he repay what the company had already
paid him under the Retirement Agreement. Lawson disputes that he breached the non-compete.
He filed this lawsuit seeking to recover what he believes Spirit owes him.
A.
The Parties’ Initial Discussions Regarding Spirit’s ESI
This case was reassigned to the undersigned on March 26, 2019. By that time, Lawson had
already filed a motion to compel Spirit to produce ESI directed to the issue of whether Spirit and
(stating that Lawson would agree to Spirit producing slipsheets for these non-responsive
documents).)
2
Arconic are in the same “Business,” as that term is defined in the non-compete in Lawson’s
Retirement Agreement. (See ECF No. 57, at 23-24.) Lawson’s motion to compel was based on
its Requests for Production (“RFPs”) seeking documents related to Spirit’s relationship with
Arconic and the overlap between their businesses. (See id. at 12.) Those RFPs largely sought
“all” such documents and communications. (ECF No. 58-2, at 9-14, RFP Nos. 19, 25-30, 34-38,
40.) Likewise, Lawson’s motion to compel sought “all” such documents. (ECF No. 57, at 5, 2124.) The parties could not agree on ESI custodians or search terms, and had difficulty meeting and
conferring productively. (Id. at 7-8, 11, 13-15; ECF No. 72, at 3-4.) So Lawson filed a motion to
compel Spirit to produce documents responsive to Lawson’s list of search terms and custodians.
(ECF No. 57, at 24-30.)
Spirit responded, arguing that Lawson’s ESI demands were “nothing short of a fishing
expedition,” disproportionate to the needs of the case, and “abusive,” and that Lawson was “using
discovery for the sake of creating obvious burden.” (ECF No. 72, at 2-3.) Lawson’s motion asked
the court to compel Spirit to search nearly 70 custodians’ ESI using about 90 search terms.2 (Id.
at 9.) Spirit argued that Lawson had made “no effort . . . to identify and remove duplicative or
tangential custodians.” (Id. at 24.) Spirt also contended that Lawson’s proposed search terms
were “equally overbroad and unworkable.” (Id. at 25.) Many of those terms were common terms
in the aviation industry, such as “fuselage,” “bulkhead,” and “pylon,” without any limiting
modifiers. (See id.; ECF No. 61-1, at 3-4.)
After receiving Lawson’s list of proposed custodians and search terms, Spirit identified
eight individuals it believed would be appropriate custodians (seven of whom were included in
2
Lawson initially demanded that Spirit search each custodian’s assistant’s ESI also, but his
motion did not seek to compel these searches. (ECF No. 72, at 9.)
3
Lawson’s list). (ECF No. 72, at 25; ECF No. 72-10.) Spirit ran searches on four of those
custodians’ ESI using Lawson’s proposed search terms. (ECF No. 72-13 ¶ 11, at 2-3.) After
running the searches, Spirit informed Lawson that they returned more than 320,000 documents.
(ECF No. 72, at 26.) Spirit reviewed approximately 400 of these documents and determined that
85% were irrelevant. (ECF No. 72-13 ¶ 13, at 3.) Because Spirit viewed Lawson’s proposed
search terms as ineffective, Spirit stated that it would craft its own search terms. (ECF No. 58-13,
at 4.) Spirit also proposed limiting ESI searches to ten custodians it believed most likely to have
information relevant to the parties’ dispute. (See id. at 5, 7.)
B.
The Initial ESI Protocol
On April 23, 2019, the court convened a hearing on Lawson’s motion to compel. (ECF
Nos. 76, 81.) After consultation with the parties, the court granted Lawson’s motion in part and
denied it in part. In relevant part, the court ordered Spirit to produce documents responsive to the
Arconic-related requests to the extent that they would be captured by the ESI protocol set forth in
the court’s Memorandum and Order. See Lawson v. Spirit AeroSystems, Inc., No. 18-1100-EFMADM, 2019 WL 1877159, at *2 (D. Kan. Apr. 26, 2019). That ESI protocol directed the parties
to proceed as follows:
Lawson would first identify up to seven categories of documents for which he was
seeking ESI;
For each category, Spirit would then serve a list of the top three custodians most
likely to have relevant ESI, from the most likely to the least likely, along with a
brief explanation as to why Spirit believed the custodian would have relevant
information;
Lawson would then serve an initial list of five custodians with proposed search
terms for each, and a second list of five custodians with proposed search terms for
each a week later; and
Spirit would conduct searches of the custodians’ ESI using Lawson’s proposed
search terms, engage in a sampling exercise to determine responsiveness rates, and
4
suggest modified search terms if the terms proposed by Lawson “produced an
unreasonably large number of non-responsive or irrelevant results.”
Id. at *2-*3. Ultimately, the court envisioned an iterative process in which the parties would work
together to formulate more effective search terms. The court ordered the parties to “meet and
confer about search terms and try to achieve an estimated responsive hit rate of at least 85%.” Id.
at *3.
The parties initially proceeded according to this protocol.
Lawson identified seven
categories of documents for which he sought ESI and, for each category, Spirit identified the top
three custodians. (See ECF No. 136-4.) But when Lawson chose ten custodians for Spirit to
search, he chose only three of the custodians that Spirit had identified as most likely to have
relevant information. Lawson provided Spirit with 803 search terms (counting terms with “OR”
as multiples) and asked Spirit to run those terms on all ten custodians’ ESI. (ECF No. 136-5; ECF
No. 136-6; ECF No. 135 ¶ 7, at 2.)
Spirit searched the first five custodians’ ESI with Lawson’s 803 terms, which returned
nearly 196,000 documents. (ECF No. 152-1, at 27.) After reviewing a 384-document sample,
Spirit determined that only 16.7% were responsive to Lawson’s discovery requests. (Id.) Spirit
provided this information to Lawson, along with hit reports for Lawson’s search terms. (See id. at
29-42.) In an effort to achieve the 85% hit rate goal set by the court, Spirit also proposed its own
set of search terms with corresponding hit reports for the first five custodians. (See id. at 43-56.)
Spirit also searched the second five custodians’ ESI with Lawson’s 803 terms. (See ECF
No. 136-7.) In total, the searches across all ten custodians returned approximately 304,000
documents. (Id.; ECF No. 135 ¶¶ 7-8, at 2.) Spirit reviewed a 384-document sample of these
results and determined that only 7.8% were responsive. (ECF No. 136-7; ECF No. 135 ¶ 9, at 2.)
5
C.
The Modified ESI Protocol
On June 6, 2019, the court convened a discovery conference at the parties’ request to
discuss their progress on ESI. (ECF Nos. 84, 87.) After consultation with the parties, the court
limited Lawson to 25 search terms for each individual custodian. (ECF No. 88, at 1.) The court
again told the parties to work together “in a back-and-forth, iterative process to agree on search
terms that will achieve an estimated responsive hit rate of at least 85%.” (Id.)
On June 28, Lawson sent Spirit revised proposed search terms in accordance with the
modified ESI protocol. (See ECF No. 136-8.) Many of Lawson’s revised terms were again
common aviation-related terms, such as “nacelle” or “nut,” as well as verbs commonly used in
many industries, such as “fasten*” and “procure*.” (See id. at 4.) Spirit conducted new searches
of the ten custodians’ ESI with Lawson’s revised terms in July 2019, which returned approximately
322,000 documents. (ECF No. 135 ¶ 12, at 3.) After conducting a sampling exercise, Spirit told
Lawson the responsiveness rates for each custodian. (See ECF No. 136-9.) Those rates ranged
from 0.5% to 13.5%, with an average across all custodians of 5.1%. (Id.; ECF No. 135 ¶ 13, at 3.)
The parties met and conferred about search terms on August 9. (See ECF No. 147-2, at
133.) At that time, Spirit stated that it believed discussing individual search terms and custodians
would not be productive. (Id.) Spirit agreed to produce to Lawson responsive documents from its
sampling exercises, as well as some non-responsive documents, to assist in the parties’ efforts to
refine search terms. (Id.) Lawson agreed to provide additional information on what documents
Lawson was expecting to see from particular custodians. (Id.)
D.
Technology Assisted Review
By September 2019, the parties had abandoned all efforts to work together on refining
search terms to meet the 85% hit-rate goal. They instead discussed conducting a TAR of the
6
322,000-document set identified in July 2019. (See ECF No. 227-1, at 5.) Spirit’s ESI vendor
Legility offers a TAR tool called “Predict.” On September 12, Spirit provided Lawson with
information on this tool. (See 147-2 ¶ 15, at 3.) After an initial set of documents is coded for
responsiveness, the Predict tool—which utilizes “continuous active learning”—uses those
standards to code additional documents. (See id. at 155.) Predict ranks coded documents from the
most likely responsive to the least; top-ranked documents are then reviewed by humans. (See id.)
When Predict determines that the pool of responsive documents is depleted such that the effort of
continued review is disproportionately outweighed by the possibility of additional gain, review
ceases. (See id. at 156.) Legility then conducts a statistical validation of the TAR’s results. (See
id.)
The parties met and conferred regarding TAR logistics on September 26. (ECF No. 2271, at 5.) They agreed that the initial review of TAR documents would be conducted by Legility’s
contract attorneys, and a second-level review would be conducted by attorneys at Arcadi Jackson,
LLP, Spirit’s law firm. (Id.) The parties did not at that time agree to a target recall rate, i.e., the
percentage of responsive documents out of the entire set they aimed to locate and produce through
the TAR process.
On November 8, the court convened a discovery conference to discuss the current status of
discovery, including the completion of document production. (ECF No. 169.) Spirit stated that it
was producing documents identified through TAR on a rolling basis and planned to end the TAR
process after achieving a 65% recall rate. (Id. at 2.) Because the parties had not previously
discussed their respective expectations regarding a recall rate, the court ordered them to meet and
confer in an attempt to reach an agreement. (Id.) The parties conferred through November and
December 2019. Spirit believed that a 65% recall rate was proportional to the needs of the case,
7
taking into the account the number of responsive documents located and the cost and time of
review. (ECF No. 227-1, at 15.) Lawson, however, was concerned that 65% would be inadequate,
and that a 75-85% recall rate was more typical. (Id.) Spirit explained to Lawson that the expected
additional cost to reach an 85% recall rate could be up to an additional $35,000, on top of the
thousands of dollars already spent on TAR and non-TAR productions. (Id. at 11.) Ultimately,
however, Spirit agreed to expand its review to reach an 80% recall rate in an effort to avoid further
motion practice, and in light of the depositions that were, at that time, set to begin in January 2020.3
(Id.)
E.
Lawson’s Motion
On January 10, 2020, the court convened a discovery conference during which the parties
explained a dispute they were having over the residual TAR documents. (Id. at 37; ECF No. 128.)
Spirit stated that it would be producing the additional TAR documents it had agreed to review to
reach the 80% recall rate within the next few days. (ECF No. 227-1, at 67.) Lawson stated that
the residual TAR documents had already been identified as responsive and should be produced.
(See id. at 58-60.) Spirit explained that these documents had been through the first-level of review,
but Spirit believed that engaging in the second level of review and preparing the residual TAR
documents for production was not proportional to the needs of the case. (Id. at 63-64.) Spirit cited
the relatively small number of responsive documents found in the 322,000-document set and
described the costs incurred in producing those documents. (See id. at 62-65.) On top of the
approximately $380,000 Spirit had already spent in costs relating to the TAR process, engaging in
3
The court later allowed the parties to postpone the timeline for conducting depositions. (ECF
No. 221, at 1.)
8
a second-level review of the residual TAR documents and producing them would cost an estimated
$40,000. (Id. at 62-63.)
With the hope of efficiently resolving this dispute, the court asked if Lawson would be
willing to bear the costs of the second-level review and production of the residual TAR documents.
(Id. at 65.) The Agreed Order Establishing Protocol for ESI and Paper Documents in this case,
which the parties agreed to and submitted to the court, provides that the identification and
production of responsive ESI is “subject to . . . the development of reasonable and appropriate cost
allocation agreements.” (ECF No. 40, at 8.) Lawson immediately refused to bear these costs.
(ECF No. 227-1, at 65.) After considering the parties’ arguments, the court explained that it would
not be inclined to order Spirit to proceed with a second-level review at that time, but, because the
parties remained at an impasse, the court granted Lawson leave to file the instant motion to compel.
(ECF No. 227-1, at 66; ECF No. 221, at 2-3.)
Lawson’s motion now argues that the residual TAR documents are “critical to [his] ability
to rebut Spirit’s unsupported assertion that it and Arconic are in the same Business” and their
production is proportional to the needs of the case. (ECF No. 227, at 5-6, 8.) Lawson contends
that Spirit is obligated to produce documents it has identified as responsive, even though Spirit’s
production already reached its target recall rate; allowing otherwise “would open the door to
discovery gamesmanship.” (Id. at 6-7.) Lawson further argues that a second-level review is
unnecessary, and Spirit’s claims regarding costs are inflated. (Id. at 6-8.)
Spirit argues that second-level review and production of the residual TAR documents is
not proportional to the needs of the case. Spirit states that it has already produced (or withheld as
privileged) 85% of the responsive documents in the TAR set, which has cost approximately
$400,000 in fees and costs to Legility and $200,000 in fees incurred by Spirit’s law firms. (ECF
9
No. 240, at 3 n.7, 4.) Spirit contends that the residual TAR documents are unlikely to be important
in resolving the issues in this case, as there are only about 800 that have been preliminarily marked
responsive, along with approximately 1000 associated documents (i.e., family members that are
non-responsive and irrelevant. (Id. at 4-5.) Spirit estimates that review and production of all of
these documents will cost over $30,000.4 (Id. at 4.) Spirit also argues that its cost estimates are
not inflated, and a second-level review by outside counsel is necessary to confirm responsiveness
and to review documents for confidentiality, privilege, and compliance with International Traffic
in Arms Regulations. (Id. at 6.)
II.
LEGAL STANDARD
“Parties may obtain discovery regarding any nonprivileged matter that is relevant to any
party’s claim or defense and proportional to the needs of the case.” FED. R. CIV. P. 26(b)(1). In
other words, considerations of both relevance and proportionality now expressly govern the scope
of discovery. FED. R. CIV. P. 26(b)(1) advisory committee’s note to the 2015 amendment. When
evaluating proportionality, the court considers “the importance of the issues at stake in the action,
the amount in controversy, the parties’ relative access to relevant information, the parties’
resources, the importance of the discovery in resolving the issues, and whether the burden or
expense of the proposed discovery outweighs its likely benefit.” FED. R. CIV. P. 26(b)(1). The
party resisting discovery on proportionality grounds still bears the burden to support its objections.
FED. R. CIV. P. 26(b)(1) advisory committee’s note to the 2015 amendment (“Restoring the
proportionality calculation to Rule 26(b)(1) does not change the existing responsibilities of the
court and the parties . . . .”); Nat’l R.R. Passenger Corp. v. Cimarron Crossing Feeders, LLC, No.
4
As noted above, Lawson has since clarified that he does not seek production of the nonresponsive, irrelevant family members. (See ECF No. 247, at 2.)
10
16-CV-1094-JTM-TJJ, 2017 WL 4770702, at *4 (D. Kan. Oct. 19, 2017) (“The party resisting
discovery bears the burden to support its objections based upon proportionality[.]”).
The practical effect of the rule is that both parties must typically provide information
pertinent to the proportionality analysis. See In re Bard IVC Filters Prod. Liab. Litig., 317 F.R.D.
562, 565 (D. Ariz. 2016); FED. R. CIV. P. 26(b)(1) advisory committee’s note to the 2015
amendment (“The parties and the court have a collective responsibility to consider the
proportionality of all discovery and consider it in resolving discovery disputes.”). This is because
[a] party claiming undue burden or expense ordinarily has far better
information — perhaps the only information — with respect to that
part of the determination. A party claiming that a request is
important to resolve the issues should be able to explain the ways in
which the underlying information bears on the issues as that party
understands them. The court’s responsibility, using all the
information provided by the parties, is to consider these and all the
other factors in reaching a case-specific determination of the
appropriate scope of discovery.
FED. R. CIV. P. 26(b)(1) advisory committee’s note to the 2015 amendment. “No single factor is
designed to outweigh the other factors in determining whether the discovery sought is proportional,
and all proportionality determinations must be made on a case-by-case basis.” Oxbow Carbon &
Minerals LLC v. Union Pac. R.R. Co., 322 F.R.D. 1, 6 (D.D.C. 2017). The court retains an
independent ongoing obligation to assess proportionality.
See FED. R. CIV. P. 26(b)(2)(C)
(providing that, on a motion or on its own, the court must limit the frequency and extent of
disproportional discovery); see also FED. R. CIV. P. 26 advisory committee’s notes to the 2015
amendment (stating the court has a “responsibility to consider the proportionality of all discovery
and consider it in resolving discovery disputes”).
III.
ANALYSIS
It is well settled that TAR “is an acceptable way to search for relevant ESI in appropriate
cases.” Da Silva Moore v. Publicis Groupe, 287 F.R.D. 182, 183 (S.D.N.Y. 2012); see also, e.g.,
11
Youngevity Int’l, Corp. v. Smith, No. 16CV00704BTMJLB, 2019 WL 1542300, at *11 (S.D. Cal.
Apr. 9, 2019) (“Predictive coding or TAR has emerged as a far more accurate means of producing
responsive ESI in discovery than manual human review of keyword searches.”); Entrata, Inc. v.
Yardi Sys., Inc., No. 2:15-CV-00102, 2018 WL 5470454, at *7 (D. Utah Oct. 29, 2018) (stating
that “it is ‘black letter law’ that courts will permit a producing party to utilize TAR”); Aurora
Coop. Elevator Co. v. Aventine Renewable Energy-Aurora W., LLC, No. 4:12CV230, 2015 WL
10550240, at *2 (D. Neb. Jan. 6, 2015) (encouraging the parties to “work cooperatively . . . in
developing and implement[ing] computer-assisted review”). TAR is defined as “[a] process for
prioritizing or coding a collection of [ESI] using a computerized system that harnesses human
judgments of subject matter expert(s) on a smaller set of documents and then extrapolates those
judgments to the remaining documents in the collection.” The Sedona Conference Glossary: EDiscovery & Digital Information Management (Fourth Edition), 15 SEDONA CONF. J. 305, 357
(2014); see also Maura R. Grossman & Gordon V. Cormack, The Grossman-Cormack Glossary
of Technology-Assisted Review, 7 FED. CTS. L. REV. 1, 32 (2013). In other words, human reviewers
“code a ‘seed set’ of documents. The computer [then] identifies properties of those documents that
it uses to code other documents.” Da Silva Moore, 287 F.R.D. at 184. With TAR processes like
the Predict tool used in this case, which use continuous active learning (sometimes referred to as
“TAR 2.0”),
the software continuously analyzes the entire document collection
and ranks the population based on relevancy. Human coding
decisions are submitted to the software, the software re-ranks the
documents, and then presents back to the human additional
documents for review that it predicts as most likely relevant. This
process continues until the TAR team determines that the predictive
model is reasonably accurate in identifying relevant and nonrelevant
documents, and that the team has identified a reasonable number of
relevant documents for production.
12
BOLCH JUDICIAL INST. & DUKE LAW, TECHNOLOGY ASSISTED REVIEW (TAR) GUIDELINES 4 (Jan.
2019) [hereinafter TAR GUIDELINES], available at https://judicialstudies.duke.edu/wpcontent/uploads/2019/02/TAR-Guidelines-Final-1.pdf.
Measures to determine whether TAR is effective include “recall” and “precision.” “Recall
is the fraction of relevant documents identified during a review; precision is the fraction of
identified documents that are relevant. Thus, recall is a measure of completeness, while precision
is a measure of accuracy or correctness.” Da Silva Moore, 287 F.R.D. at 189-90. When a recall
rate increases, a search is generally less precise. See The Sedona Conference Best Practices
Commentary on the Use of Search & Information Retrieval Methods in E-Discovery, 15 SEDONA
CONF. J. 217, 238 (2014). “Effectively, one can cast either a narrow net and retrieve fewer relevant
documents, along with fewer irrelevant documents, or cast a broader net and retrieve more relevant
documents, but at the expense of retrieving more irrelevant documents.” Id.
The proportionality standard in Rule 26(b)(1) applies to ESI discovery. Indeed, in cases
involving a considerable amount of ESI, “proportionality considerations [are] particularly
significant.” In re EpiPen (Epinephrine Injection, USP) Mktg., Sales Practices & Antitrust Litig.,
No. 17-MD-2785-DDC-TJJ, 2018 WL 1440923, at *1 (D. Kan. Mar. 15, 2018) (citing The Sedona
Principles, Third Edition: Best Practices, Recommendations & Principles for Addressing
Electronic Document Production, 19 SEDONA CONF. J. 1, Principle 2, 51 (2018)). The goal of
TAR is “to result in higher recall and higher precision than another review method, at a cost
proportionate to the ‘value’ of the case.” Da Silva Moore, 287 F.R.D. at 190.
The court has not found, and Lawson tellingly has not identified, any instance in which a
court has required a party engaging in TAR to reach a 100% recall rate. “[W]hile parties must
impose a reasonable construction on discovery requests and conduct a reasonable search when
13
responding to the requests, the Federal Rules do not demand perfection.” Reinsdorf v. Skechers
U.S.A., Inc., 296 F.R.D. 604, 615 (C.D. Cal. 2013); see also Da Silva Moore, 287 F.R.D. at 191
(“[C]omputer-assisted review is not perfect, [but] the Federal Rules of Civil Procedure do not
require perfection.”); Treppel v. Biovail Corp., 233 F.R.D. 363, 374 (S.D.N.Y. 2006) (“[T]here is
no obligation on the part of a responding party to examine every scrap of paper in its potentially
voluminous files in order to comply with its discovery obligations.”); Radiologix, Inc. v. Radiology
& Nuclear Med., LLC, No. 15-4927-DDC-KGS, 2019 WL 354972, at *11 (D. Kan. Jan. 29, 2019)
(explaining that courts do not require perfection, rather a party must conduct a reasonable search
for responsive information pursuant to a reasonably comprehensive search strategy). “[I]t is
inappropriate to hold TAR to a higher standard than keywords or manual review.” Rio Tinto PLC
v. Vale S.A., 306 F.R.D. 125, 129 (S.D.N.Y. 2015); see also Winfield v. City of New York, No.
15CV05236LTSKHP, 2017 WL 5664852, at *9 (S.D.N.Y. Nov. 27, 2017) (“[P]erfection in ESI
discovery is not required . . . .”).
In contrast, authority supports the reasonableness of Spirit’s recall rate. Courts have found
TAR processes achieving a 75% recall rate to be appropriate. See The Sedona Conference Tar
Case Law Primer, 18 SEDONA CONF. J. 1, 37 (2017). Parties have mutually agreed on similar
recall rates to include in court-approved ESI protocols.5 See, e.g., In re Bair Hugger Forced Air
Warming Prod. Liab. Litig., No. MDL 15-2666(JNE/FLN), 2016 WL 3702959, at *2 (D. Minn.
July 8, 2016) (setting an 80% target recall rate). A court should take into account the facts and
circumstances of each case to determine whether a particular recall rate is reasonable, but recent
guidance suggests that rates of 75-85% are appropriate in many cases. TAR GUIDELINES, supra,
5
The court notes that the parties’ Agreed Order Establishing Protocol for ESI and Paper
Documents (ECF No. 40) does not address TAR or recall rates.
14
at 23; see also TIMOTHY T. LAU & EMERY G. LEE III, FED. JUDICIAL CTR., TECHNOLOGY-ASSISTED
REVIEW FOR DISCOVERY REQUEST: A POCKET GUIDE FOR JUDGES 12 (Mar. 28, 2017), available at
https://www.fjc.gov/content/321579/technology-assisted-review-discovery-requests (“[A] recall
or precision of 80% may be appropriate for one particular review, this does not mean that 80% is
a benchmark for all other reviews.”).
In this case, the court finds that Spirit’s TAR was reasonable and production of the residual
TAR documents is not proportional to the needs of the case. As explained above, the parties have
a long and tortured history with ESI discovery. After the court’s ESI Protocol proved unworkable
because Lawson demanded searches (both of custodians and search terms) that yielded such
exceptionally low responsiveness rates, the parties resorted to Spirit conducting a TAR of the
322,000-document set identified in July 2019 (using Lawson’s revised proposed search terms and
his selected custodians). The parties agreed to TAR parameters in September 2019 that included
two levels of review. As Spirit points out, Lawson had no objection to a second-level review at
that time, and even requested that Spirit’s second-level reviewers look at representative samples
of documents marked non-responsive to ensure that first-level reviewers were not undercoding
responsive documents. (ECF No. 227-1, at 5.) The parties agreed to this second-level review
previously, for sound reasons. As Spirit explains, this second-level review is necessary for Spirit’s
law firm to review the documents for confidentiality, privilege, and compliance with TAR. Courts
have found a second-level manual review following a TAR to be reasonable. See TAR Case Law
Primer, supra, at 41. The court will not now allow Lawson to renege on this agreed second-level
review.
The parties never reached a mutual agreement as to the appropriate recall rate. Spirit
believed a 65% recall rate was adequate. During the parties’ meet and confer in November 2019,
15
Lawson informed Spirit that a 75-85% recall rate was typical. (ECF No. 227-1, at 15.) Spirit
eventually agreed to reach a recall rate of 80% and “produced (or withheld as privileged) 23,951
documents, consisting of 170,083 pages (of these, 9,128 are responsive documents, and the
remainder are non-responsive, irrelevant family members).” (ECF No. 240, at 3.) Spirit states
that the 322,000 document set had a 3.3% responsive rate and, ultimately, it produced 85% of
those responsive documents, i.e. an 85% recall rate. (Id.) The court finds this rate reasonable
under the circumstances of this case, and falls within the range that commentators—and Lawson
himself—indicate is typical for most cases.
Requiring Spirit to engage in a second-level review and produce the residual TAR
documents is not proportional to the needs of the case. Under the proportionality analysis, the
court must consider the importance of the discovery in conjunction with whether its burden or
expense outweighs its likely benefit. Lawson argues generally that the residual TAR documents
are “critical to [his] ability to rebut Spirit’s unsupported assertion that it and Arconic are in the
same Business.” (ECF No. 227, at 8.) But Lawson admits that Spirit has already produced
documents bearing on this issue through the TAR process. (See id. at 7.) Spirit has also produced
documents relating to overlap outside of the TAR process. (See ECF No. 240, at 2 (discussing
Spirit’s proposal to continue reviewing and producing documents “regarding the ‘Business’ and
overlap based on Spirit’s identification of custodians most likely to have relevant and responsive
information”).)
Spirit has spent approximately $600,000 in vendor fees and attorneys’ fees relating to the
TAR process, all to produce less than 10,000 responsive documents. The court will not require
Spirit to spend thousands more to review and ready for production documents that would appear
to have marginal or duplicative benefit to the parties, if any benefit at all. Perhaps most telling to
16
the court is the fact that—although Lawson refers to the residual TAR documents as “critical,” he
does not apparently believe them to be critical enough to cover the costs of their production. And,
there is no guarantee that the 800 documents that have been marked preliminarily responsive by
first-level reviewers are actually responsive. As Spirit notes, second-level reviewers have made
coding changes previously. (ECF No. 240, at 6.) Further, this is not a case where Lawson requires
an elusive “smoking gun” document to establish his claims. Spirit and Arconic either provided,
marketed, or sold overlapping products or services, or they did not. Lawson has already likely
received far more documents bearing on the issue than he would have had Spirit used any other
form of review. See Da Silva Moore, 287 F.R.D. at 190 (“[T]he myth that exhaustive manual
review is the most effective—and therefore the most defensible—approach to document review is
strongly refuted. Technology-assisted review can (and does) yield more accurate results than
exhaustive manual review, with much lower effort.” (alteration in original).)
The court has also considered the remaining proportionality factors, including the amount
in controversy and the parties’ resources. Lawson cursorily points out that approximately $50
million is at stake in this case, and that Spirit is a company worth $8 billion in annual revenues.
(ECF No. 227, at 5-6.) As the court has recognized before, this is relevant to the analysis, but not
determinative.
If these factors were determinative, “they would eradicate proportionality
considerations in every case against high-profile litigation targets with substantial resources.”
Lawson v. Spirit AeroSystems, Inc., No. 18-1100-EFM-ADM, 2020 WL 1285359, at *8 (D. Kan.
Mar. 18, 2020).
In this case, the court is particularly troubled by Lawson’s apparent disincentive to
meaningfully tailor his ESI demands to further the “just, speedy, and inexpensive” determination
of this action. FED. R. CIV. P. 1. Lawson’s original motion to compel demanded that Spirit search
17
more than 70 custodians using about 90 search terms. After the court imposed an ESI Protocol,
Spirit identified the custodians most likely to have responsive ESI, and Lawson was allowed to
select ten ESI custodians. Lawson apparently largely ignored Spirit’s list, selecting only three ESI
custodians from Spirit’s list and selecting seven other ESI custodians who had not been listed as
custodians most likely to have the categories of ESI Lawson was seeking. Compounding this
unwieldy list of custodians, Lawson provided Spirit with 803 search terms to run on those
custodians’ ESI, with no effort to tailor those searches to particular ESI custodians. Given
Lawson’s scattershot approach, it is no wonder that the ESI process yielded such exceptionally
low responsive rates. Meanwhile, Spirit has incurred hundreds of thousands of dollars trying to
appease Lawson. And, now, Lawson refuses to pay the approximate $40,000 for the second-level
review and production of the residual TAR documents that he claims are “critical” but yet Lawson
has not cited a single case or other authority to support what he effectively demands, which is a
100% target recall rate. This lack of reasonableness throughout the pendency of this case borders
on the abusive.
Ultimately, the court finds that the second-level review and production of the residual TAR
documents is not proportional to the needs of the case under Rule 26(b)(1). The court therefore
denies Lawson’s motion.
IT IS THEREFORE ORDERED that Larry A. Lawson’s Motion to Compel the
Production of Responsive Documents (ECF No. 226) is denied.
IT IS SO ORDERED.
Dated April 9, 2020, at Topeka, Kansas.
s/ Angel D. Mitchell
Angel D. Mitchell
U.S. Magistrate Judge
18
Disclaimer: Justia Dockets & Filings provides public litigation records from the federal appellate and district courts. These filings and docket sheets should not be considered findings of fact or liability, nor do they necessarily reflect the view of Justia.
Why Is My Information Online?