Hyles v. City of New York et al

Filing 97

OPINION & ORDER: The key issue is whether, at plaintiff Hyles' request, the defendant City (i.e., the responding party) can be forced to use TAR (technology assisted review, aka predictive coding) when the City prefers to use keyword searching . The short answer is a decisive "NO." (As further set forth in this Order.) To be clear, the Court believes that for most cases today, TAR is the best and most efficient search tool. That is particularly so, according to research studies (cited in Rio Tinto), where the TAR methodology uses continuous active learning ("CAL"), which eliminates issues about the seed set and stabilizing the TAR tool. The Court would have liked the City to use TAR in this case. But the Court cannot, and will not, force the City to do so. There may come a time when TAR is so widely used that it might be unreasonable for a party to decline to use TAR. We are not there yet. Thus, despite what the Court might want a responding party to do, Sedona Principle 6 controls. Hyles' application to force the City to use TAR is DENIED. (Signed by Magistrate Judge Andrew J. Peck on 8/1/2016) (cf)

UNITED STATES DISTRICT COURT
SOUTHERN DISTRICT OF NEW YORK
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -x
PAULINE HYLES,

:
Plaintiff,

-againstNEW YORK CITY, et al.,
Defendants.

10 Civ. 3119 (AT)(AJP)

:

OPINION & ORDER

:
:
:

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -x
ANDREW J. PECK, United States Magistrate Judge:
This case was referred to me to resolve a discovery dispute as to the proposed scope of
discovery of ESI requested by plaintiff Hyles from defendant City of New York. (Dkt. Nos. 86, 90 &
92.) The key issue is whether, at plaintiff Hyles' request, the defendant City (i.e., the responding party)
can be forced to use TAR (technology assisted review, aka predictive coding) when the City prefers to
use keyword searching. The short answer is a decisive "NO."
BACKGROUND
Plaintiff Pauline Hyles, a black female of West Indian/Guyanese descent employed by
the City Finance Department, alleges that in September 2008 she was demoted and her salary reduced,
and she was replaced by a white male. (Dkt. No. 24: 2d Am. Compl. ¶¶ 10, 32-34, 42.) Hyles asserts
discrimination and hostile environment claims under, inter alia, Title VII, the ADEA, the NYSHRL, and
the NYCHRL. (See generally 2d Am. Compl.) Discovery was delayed because of a motion to dismiss,
a long stay to allow the parties to engage in mediation, and an apparent lack of effort by counsel to move
the case forward until faced with an already extended discovery deadline; at that point, the parties had
a major discovery dispute, the case was referred to me, and I held a discovery conference on July 27,
2016. (See Dkt. Nos. 89, 91.)

2
The Discovery Dispute on Scope: Date Range and Custodians
By joint letter dated July 18, 2016 (but re-filed on ECF on July 25, 2016), counsel sought
rulings as to the proper scope of ESI discovery (mostly issues as to custodians and date range) and search
methodology, that is, keywords or TAR. (See Dkt. No. 90.) The Court began the discovery conference
by addressing the scope issues, since that would affect the collection of documents and ESI which then
would have to be reviewed using either keywords or TAR.
As to date range, the parties agreed on a start date of September 1, 2005 but disagreed
on the end date. (Dkt. No. 90: 7/18/16 Ltr. at 5, 7.) Hyles sought an end date of 2013, when defendant
Patricoff left the City Finance Department. (Dkt. No. 90-3: Attached 6/3/16 Ltr. at 1-2.) The City
proposed April 30, 2009, when defendant Stark left the Finance Department. (Dkt. No. 90-4: Attached
City 6/23/16 Ltr. at 1-2.) After hearing the parties' arguments at the conference, the Court ruled that the
end date would be April 30, 2010 (when defendant Patricoff was reassigned from her First Deputy
Commissioner position), without prejudice to Hyles seeking documents or ESI from a later period, if
justified, on a more targeted inquiry basis.
As to custodians, the City agreed to search the files of nine custodians (including Hyles),
but not six additional custodians that Hyles requested. (7/18/16 Ltr. at 5, 7.) The Court ruled that
discovery should be staged, by starting with the agreed upon nine custodians (Hyles, Stark, Patricoff and
six others). After reviewing the production from the nine custodians, if Hyles could demonstrate that
other custodians had relevant, unique and proportional ESI, the Court would consider targeted searches
from such other custodians.1/

1/

For example, Hyles wanted to add Annie Long, a City HR/EEO employee, because Hyles
allegedly emailed a discrimination complaint to her. The City responded that there was no
such complaint. The Court held that if such a complaint was produced from Hyles' emails,
it would allow a targeted search of Long's emails. (See also page 5 n.4 below.)

3
Discovery Methodology: Keywords or TAR
After the parties had initial discussions about the City using keywords, Hyles' counsel
consulted an ediscovery vendor and proposed that the City should use TAR as a "more cost-effective and
efficient method of obtaining ESI from Defendants." (Dkt. No. 90: 7/18/16 Ltr. at 2; see Dkt. No. 90-4:
Attached 6/13/16 Ltr.) The City declined, both because of cost and concerns that the parties, based on
their history of scope negotiations, would not be able to collaborate to develop the seed set for a TAR
process. (7/18/16 Ltr. at 6.)2/ At the conference, even after the Court's ruling on the custodians and date
range largely accepted the City's scope parameters, the City still declined to agree to use TAR.
Hyles absolutely is correct that in general, TAR is cheaper, more efficient and superior
to keyword searching. (See 7/18/16 Ltr. at 2, 4, 5.) In March 2009, the "dark ages" in terms of
ediscovery advances, this Court described problems with keywords and the need for "careful thought,
quality control, testing, and cooperation with opposing counsel in designing search terms or 'keywords.'"
William A. Gross Constr. Assocs., Inc. v. Am. Mutual Mfrs. Ins. Co., 256 F.R.D. 134, 134 (S.D.N.Y.
2009) (Peck, M.J.). Further elaborating on the deficiencies of keyword searching, my seminal Da Silva
Moore decision in 2012 approved the use of predictive coding, aka TAR, in appropriate cases. Da Silva
Moore v. Publicis Groupe, 287 F.R.D. 182, 190-91, 193 (S.D.N.Y. 2012) (Peck, M.J.). In again
approving the use of TAR in 2015, I wrote that "the case law has developed to the point that it is now
black letter law that where the producing party wants to utilize TAR for document review, courts will
permit it." Rio Tinto PLC v. Vale S.A., 306 F.R.D. 125, 127 (S.D.N.Y. 2015) (Peck, M.J.).3/ Dicta in

2/

The Court acknowledges that some vendor pricing models charge more for TAR than for
keywords. Usually any such extra cost is more than offset by cost savings in review time.
(See 7/18/16 Ltr. at 4.) Here, however, the City indicated that document review will be done
in house by salaried staff of Corporation Counsel's Office.

3/

Since Rio Tinto, judicial approval of TAR has gone international. See Irish Bank Resolution
Corp. v. Quinn, [2015] EWHC 175 (H.Ct. Ir.), upheld by Irish Court of Appeals; Brown v.
(continued...)

4
a footnote in Rio Tinto stated that "[i]n contrast, where the requesting party has sought to force the
producing party to use TAR, the courts have refused." Rio Tinto PLC v. Vale S.A., 306 F.R.D. at 127
n.l. "The Court note[d], however, that in [the cited] cases the producing party had spent over $1 million
using keyword search (in Kleen) or keyword culling followed by TAR (in Biomet), so it is not clear what
a court might do if the issue were raised before the producing party had spent any money on document
review." Rio Tinto PLC v. Vale S.A., 306 F.R.D. at 127 n.l. Since the search methodology issue arose
in this case before the City spent much, if any, money on searching for responsive ESI, this case squarely
raises the issue of whether the requesting party can have the Court force the responding party to use
TAR.
Hyles' counsel is correct that parties should cooperate in discovery. I am a signatory to
and strong supporter of the Sedona Conference Cooperation Proclamation, and I believe that parties
should cooperate in discovery. See William A. Gross Constr. Assocs., Inc. v. Am. Mfrs. Mut. Ins. Co.,
256 F.R.D. at 136; Rio Tinto PLC v. Vale S.A., 306 F.R.D. at 129 n.6. The December 1, 2015 Advisory
Committee Notes to amended Fed. R. Civ. P. 1 emphasized the need for cooperation. Cooperation
principles, however, do not give the requesting party, or the Court, the power to force cooperation or to
force the responding party to use TAR.
It certainly is fair to say that I am a judicial advocate for the use of TAR in appropriate
cases. I also am a firm believer in the Sedona Principles, particularly Principle 6, which clearly provides
that:
Responding parties are best situated to evaluate the procedures, methodologies,
and technologies appropriate for preserving and producing their own
electronically stored information.

3/

(...continued)
BCA Trading Ltd., [2016] EWHC (Ch.) 1464 (Eng.); Pyrrho Inv. Ltd. v. MWB Prop. Ltd.,
[2016] EWHC (Ch.) 256 (Eng.).

5
The Sedona Principles: Second Edition, Best Practices Recommendations & Principles for Addressing
Electronic Document Production, Principle 6 (available at www.TheSedonaConference.org).
Under Sedona Principle 6, the City as the responding party is best situated to decide how
to search for and produce ESI responsive to Hyles' document requests. Hyles' counsel candidly admitted
at the conference that they have no authority to support their request to force the City to use TAR. The
City can use the search method of its choice. If Hyles later demonstrates deficiencies in the City's
production, the City may have to re-do its search.4/ But that is not a basis for Court intervention at this
stage of the case.
The Court feels like the Tax Court Judge in Dynamo Holdings, who stated:
[T]he Court is not normally in the business of dictating to parties the process that
they should use when responding to discovery. If our focus were on paper
discovery, we would not (for example) be dictating to a party the manner in
which it should review documents for responsiveness or privilege, such as
whether that review should be done by a paralegal, a junior attorney, or a senior
attorney. Yet that is, in essence, what the parties are asking the Court to consider
– whether document review should be done by humans or with the assistance of
computers.
Dynamo Holdings Ltd. P'ship v. Comm'r of Internal Revenue 143 T.C. 9, 2014 WL 4636526 at *3
(2014). The Dynamo Holdings court went on to approve the responding party's request that it be allowed
to use TAR. Id. at *5.
Here, too, it is not up to the Court, or the requesting party (Hyles), to force the City as
the responding party to use TAR when it prefers to use keyword searching. While Hyles may well be
correct that production using keywords may not be as complete as it would be if TAR were used
(7/18/16 Ltr. at 4-5), the standard is not perfection, or using the "best" tool (see 7/18/16 Ltr. at 4), but
whether the search results are reasonable and proportional. Cf. Fed. R. Civ. P. 26(g)(1)(B).

4/

See The Sedona Principles: Second Edition, Principle 7: "The requesting party has the
burden on a motion to compel to show that the responding party's steps to preserve and
produce relevant electronically stored information were inadequate."

6
To be clear, the Court believes that for most cases today, TAR is the best and most
efficient search tool. That is particularly so, according to research studies (cited in Rio Tinto), where the
TAR methodology uses continuous active learning ("CAL"), which eliminates issues about the seed set
and stabilizing the TAR tool. The Court would have liked the City to use TAR in this case. But the
Court cannot, and will not, force the City to do so. There may come a time when TAR is so widely used
that it might be unreasonable for a party to decline to use TAR. We are not there yet. Thus, despite what
the Court might want a responding party to do, Sedona Principle 6 controls. Hyles' application to force
the City to use TAR is DENIED.

SO ORDERED.

Dated:

New York, New York
August 1, 2016

________________________________
Andrew J. Peck
United States Magistrate Judge
Copies by ECF to:

All Counsel
Judge Torres