Massachusetts Mutual Life Insurance Company v. Residential Funding Company, et al

Filing 182

Chief Judge Patti B. Saris: MEMORANDUM AND ORDER entered. "...ORDER: The Motion to Exclude the Opinions Expressed in the April 12, 2013 Report of Plaintiff's Expert, Charles D. Cowan, Ph.D., (11-cv-30039, Dkt No. 143 ) is DENIED without prejudice." Associated Cases: 3:11-cv-30035-PBS et al.(LaFlamme, Jennifer)

UNITED STATES DISTRICT COURT
DISTRICT OF MASSACHUSETTS

MASSACHUSETTS MUTUAL LIFE
INSURANCE COMPANY
Plaintiff,
v.
RESIDENTIAL FUNDING COMPANY,
LLC, et al.
Defendants.

)
)
)
)
)
)
)
)
)
)
)

Civil Action No. 11-30035-PBS

MEMORANDUM AND ORDER
December 9, 2013
Saris, C.J.
I. INTRODUCTION
Plaintiff Massachusetts Mutual Life Insurance Company
(“MassMutual”) has filed eleven actions1 against eleven corporate
and thirty-three individual defendants2 alleging violations of

1

Mass. Mut. Life Ins. Co. v. DB Structured Prods., Inc., et
al., 11-cv-30039; Mass. Mut. Life Ins. Co. v. RBS Fin. Prods.,
Inc., et al., 11-cv-30044; Mass. Mut. Life Ins. Co. v. DLJ Mortg.
Capital, Inc., et al., 11-cv-30047; Mass. Mut. Life Ins. Co. v.
Credit Suisse First Boston Mortg. Sec. Corp., et al., 11-cv30048; Mass. Mut. Life Ins. Co. v.JPMorgan Chase Bank, N.A., et
al., 11-cv-30094; Mass. Mut. Life Ins. Co. v. Goldman Sachs
Mortg. Co., et al., 11-cv-30126; Mass. Mut. Life Ins. Co. v.
Impac Funding Corp., et al., 11-cv-30127; Mass. Mut. Life Ins.
Co. v. HSBC Bank USA, Nat’l Ass’n, et al., 11-cv-30141; Mass.
Mut. Life Ins. Co. v. Countrywide Fin. Corp., et al., 11-cv30215; Mass. Mut. Life Ins. Co. v. Merrill Lynch, Pierce, Finner
& Smith, et al., 11-cv-30285.
2

MassMutual asserts claims against individual defendants officers and directors of certain corporate defendants - in seven
1

the Massachusetts Uniform Securities Act, Mass. Gen. Laws ch.
110, § 410, based on its purchases of residential mortgage-backed
securities (“RMBSs”).
Plaintiff purchased 121 securitized certificates, totaling
approximately $2 billion, from the corporate defendants in these
actions. The 121 certificates represent 95 securitizations,
collateralized by 99 unique Supporting Loan Groups (“SLGs”),
commonly referred to as “loan pools.” The 99 SLGs represent
278,609 individual residential loans. Plaintiff alleges that the
certificates concerning each loan pool contained material
misrepresentations. In order to determine whether a single loan’s
riskiness was misrepresented, MassMutual intends to
“reunderwrite” the loan, scrutinizing the original loan file to
determine whether it was originated in accordance with applicable
standards. According to the parties, the process of
reunderwriting each loan will take approximately two to three
hours and cost hundreds of dollars. In order to avoid the costly

of the eleven actions. These claims remain pending after Judge
Ponsor granted in part and denied in part motions to dismiss
filed by defendants in nine of the eleven MassMutual cases. Mem.
and Order on Defs. Mot. to Dismiss (Feb. 14, 2012) (11-cv-30035,
Dkt. No. 98); Mass. Mut. Life Ins. Co. v. Residential Funding
Co., 843 F. Supp. 2d 191, 198 (D. Mass. 2012). By stipulation of
the parties, the ruling was applied to the two remaining cases.
In one of the cases, 11-cv-30035, claims against two of the three
defendants have been stayed pursuant to a joint stipulation and
order entered on June 10, 2013 in the bankruptcy action In re
Residential Capital, LLC, et al. (S.D.N.Y. Bankr. Case No. 1212020). Pl.’s Notice of Bankr. Filing (11-cv-30035, Dkt. No.
145).
2

and time-consuming process of reunderwriting all 278,609
individual loan files, MassMutual intends to analyze and present
information about a 100-loan sample from each of the 99 SLGs.
This approach will require reunderwriting 9,900 loan files.
On April 12, 2013, MassMutual filed a report from its expert
witness Dr. Charles D. Cowan (“Report”). The Report describes the
statistical sampling methodology Dr. Cowan will use to select the
100 sample loans from each loan pool and analyze the rate of
misrepresentation in the sample. Dr. Cowan plans to determine the
probable rate of misrepresentation in the full SLG by
extrapolating from the misrepresentation rate in the sample.
Defendants filed a joint motion to exclude the opinion
expressed in the Report, based on Federal Rule of Evidence 702
and Daubert v. Merrell Dow Pharms., Inc., 509 U.S. 579 (1993). An
evidentiary hearing was held on October 18, 2013 at which
Plaintiff's expert Dr. Charles D. Cowan and Defendants' expert
Dr. Arnold Barnett testified. After the evidentiary hearing and a
review of the record, Defendants’ Motion to Exclude the Opinions
Expressed in the April 12, 2013 Report of Plaintiff’s Expert,
Charles D. Cowan, Ph.D., (11-cv-30039, Dkt. No. 143) is DENIED.
II. BACKGROUND
The amended complaints allege that material
misrepresentations were made in the sale of securities in
violation of the Massachusetts Uniform Securities Act (MUSA),
3

Mass. Gen. Laws ch. 110, § 410. Section 410(a) provides:
Any person who . . . (2) offers or sells a security by means
of any untrue statement of a material fact or any omission
to state a material fact necessary in order to make the
statements made, in the light of the circumstances under
which they are made, not misleading, the buyer not knowing
the truth or omission . . . [or] in the exercise of
reasonable care could not have known, of the untruth or
omission, is liable to the person buying the security from
him . . . .
A plaintiff does not need to show negligence, scienter, reliance,
or causation of loss to prove a MUSA violation, and the buyer’s
level of sophistication is irrelevant. Marram v. Kobrick Offshore
Fund, Ltd., 809 N.E.2d 1017, 1026-27 (Mass. 2004).
In seven of the eleven pending cases, MassMutual also
asserts claims against individual defendants for “control person”
liability under Section 410(b), which imposes joint and several
liability on “every person who directly or indirectly controls a
seller liable under [410(a)].” Mass. Gen. Laws ch. 110, § 410(b).
Generally speaking, MassMutual alleges that the defendants
marketed the certificates with representations that the loans
backing the securities were underwritten in accordance with
prudent underwriting standards and the underlying properties were
appraised in accordance with sound appraisal standards, in order
to ensure that the borrower could repay the loan and to decrease
the risk of default. Plaintiff asserts that the loans underlying
each SLG were, in reality, far riskier than represented.
Plaintiff also alleges that the defendants knowingly reported

4

false loan-to-value (“LTV”) ratios, and in the case of defendant
HSBC, inaccurate owner-occupancy rates for underlying properties.
The defendants deny that they made any material
misrepresentations in the marketing and sale of the certificates.
III. EXPERTS
A. Dr. Cowan
Plaintiff’s expert, Charles D. Cowan, Ph.D., earned a
Bachelor of Arts and a Master of Arts in Economics from the
University of Michigan. He holds a doctorate in Mathematical
Statistics from George Washington University. Currently, Dr.
Cowan is the Managing Partner of Analytic Focus LLC, a consulting
group focusing on the design, implementation, and evaluation of
statistical and sampling techniques for research. He is also an
adjunct professor of biostatistics at the University of Alabama.
Among other positions, Dr. Cowan has served as the Chief
Statistician of the Federal Deposit Insurance Corporation,
director of quantitative methods at PricewaterhouseCoopers LLP,
Chief Statistician of the U.S. Department of Education’s National
Center for Education Statistics, and Chief of the Survey Design
Branch of the U.S. Bureau of the Census. Dr. Cowan has taught
undergraduate and graduate coursework at various academic
institutions and held positions within multiple professional
organizations. He has authored numerous books and articles on

5

statistical design methods. Defendants do not challenge his
qualifications.
Dr. Cowan’s expert report, together with his testimony,
describes his plan to analyze the loans underlying each SLG using
statistical sampling, a common technique used to analyze
representative samples of large populations. Report ¶¶ 41-45. In
his analysis of each securitization, Dr. Cowan will first select
one 100-loan sample from the loan pool underlying that
securitization. He asserts that the size of the sample will
provide scientifically valid conclusions about the full
population of loans in each SLG. Dr. Cowan states that a 96-loan
sample would achieve a 95% confidence level with a maximum margin
of error of ± 10 percentage points, but he “rounded up to 100 out
of caution. This ‘oversampling’ creates a cushion for [the]
calculations.” Report ¶ 53 n.9. While a larger sample size would
decrease the margin of error to ± 5 percentage points, according
to Dr. Cowan, the sample size would need to quadruple from 100 to
400. Therefore, he concludes that a 100-loan sample and
accompanying ± 10 percentage points margin of error “strikes the
correct balance between cost and accuracy.” Id. at ¶ 55.
Dr. Cowan's methodology for the selection of the 100 sample
loans from each SLG involves the stratification of the entire
loan pool in order to “improve the representativeness and
reliability” of the sample. Stratification is a process by which

6

a population of data is divided into mutually exclusive subgroups
by a variable known for the entire population. This occurs before
the sample loans are pulled from the full population.
Stratification cannot increase the margin of error, but can
“reduce the maximum margin of error below ± 10 percent[age
points].” Id. at ¶ 5. The maximum margin of error occurs if the
sample misrepresentation rate is at 50% of the loans; as the
sample rate increases or decreases from 50%, the margin of error
decreases from ± 10 percentage points. Id. at ¶ 58.
Dr. Cowan plans to use each borrower’s Fair Isaac
Corporation credit score (“FICO score”),3 which measures the
creditworthiness of the borrower, as the stratification variable.
He will divide the entire population of loans in a SLG into four
equal groups by FICO score: each loan will be grouped into a
quartile measuring high, somewhat high, somewhat low, and low
FICO scores, as compared to the full population of loans in that
SLG. He will then generate a random number for each loan in each
of the four strata, and reorder the loans in each stratum from

3

A credit score is a number representing the
creditworthiness of an individual. JOHN DOWNES & JORDAN ELLIOT
GOODMAN, DICTIONARY OF FINANCE AND INVESTMENT TERMS 159 (8th ed. 2010). It
is used by lenders to predict the likelihood that a borrower will
repay his or her debt. A FICO score can range from 300 to 850,
with a higher score indicating less risk of borrower default. Dr.
Cowan states, “In my experience, lenders, including mortgage loan
originators, use credit scores to determine who qualifies for a
loan, at what interest rate, and to what credit limits.” Report ¶
59.
7

lowest to highest random number. Numbers 1 to 25 will serve as
the initial sample, and numbers 26 to 50 will serve as a
supplemental, “back-up” sample from that stratum. Thus, the full
sample from each SLG will include 25 loans from each of the four
strata, for 100 loans total in the sample, with 100 back-up
sample loans (again, 25 from each stratum). If a loan file from
one of the primary sample loans cannot be located, a back-up
sample loan from the same stratum will be available to replace
the missing loan in the primary sample. Dr. Cowan will test the
primary and back-up samples from each SLG against the full
population on eleven variables to ensure that the sample selected
is representative of the full population. Id. at ¶ 64.
Once selected, the sample loans will be re-underwritten to
determine whether material misrepresentations were made in the
certificates about (1) origination in compliance with applicable
underwriting guidelines; (2) appraisals of the underlying
properties in accordance with sound appraisal standards; (3)
number/percentage of loans with LTV ratios above specified
values; and in the HSBC case, 11-cv-30141, (4) the
number/percentage of loans collaterized by owner-occupied
properties. Dr. Cowan will not be involved in the process of
reunderwriting loans; third-party servicers will re-underwrite
the loan files for purposes of this litigation.

8

Once the loans are reunderwritten and determinations made
about the above attributes, Dr. Cowan will extrapolate the
results to the full SLG population. Extrapolation is a term used
to describe the process of using the results of a sample to draw
conclusions about the full population. The Report does not commit
itself to using a particular extrapolation method. Dr. Cowan
offers two examples of possible extrapolation techniques, and
states that he will select the method that “minimizes the margin
of error.” Id. at ¶ 68. For example, if 50% of the loans
collateralizing a SLG contained material misrepresentations, one
might simply extrapolate that ratio to the full loan population
to conclude with a 95% confidence level that between 40% and 60%
of the loans supporting the SLG contained material
misrepresentations.
B. Dr. Barnett
Defendants’ expert Arnold Barnett, Ph.D., is a professor at
the Massachusetts Institute of Technology’s Sloan School of
Management. Dr. Barnett received his Bachelor of Arts in Physics
from Columbia University, and earned a doctorate in Applied
Mathematics from M.I.T. Dr. Barnett’s research specializes in
applied statistical analysis. He has taught probability and
statistics at M.I.T. for over thirty-five years. In addition to
dozens of articles on applied statistics, he has written a
textbook on Probability and Statistics. Plaintiff does not

9

challenge Dr. Barnett’s qualifications. In support of their
Daubert motion, Defendants submitted a declaration by Dr. Barnett
supporting the challenges to Dr. Cowan’s methodology. He also
testified at the Daubert hearing. Those challenges are described
in full below.
IV. DISCUSSION
A. The Court’s Gatekeeping Role
The admission of expert evidence is governed by Federal Rule
of Evidence 702, which codified the Supreme Court’s holding in
Daubert v. Merrell Dow Pharms., Inc., 509 U.S. 579 (1993), and
its progeny. See United States v. Diaz, 300 F.3d 66, 73 (1st Cir.
2002). Rule 702 states:
If scientific, technical, or other specialized knowledge
will assist the trier of fact to understand the evidence
or to determine a fact in issue, a witness qualified as
an expert by knowledge, skill, experience, training, or
education, may testify thereto in the form of an opinion
or otherwise, if (1) the testimony is based upon
sufficient facts or data, (2) the testimony is the
product of reliable principles and methods, and (3) the
witness has applied the principles and methods reliably
to the facts of the case.
Fed. R. Evid. 702.
The trial court must determine whether the expert’s
testimony “both rests on a reliable foundation and is relevant to
the task at hand” and whether the expert is qualified. Daubert,
509 U.S. at 597; Diaz, 300 F.3d at 73. An expert’s methodology is
the “central focus of a Daubert inquiry.” Ruiz-Troche v. Pepsi
Cola of P.R. Bottling Co., 161 F.3d 77, 81 (1st Cir. 1998).
10

Daubert itself listed four factors which should guide judges
in this determination: (1) whether the theory or technique can be
and has been tested; (2) whether the technique has been subject
to peer review and publication; (3) the technique’s known or
potential rate of error; and(4) the level of the theory’s or
technique’s acceptance within the relevant discipline. United
States v. Mooney, 315 F.3d 54, 62 (1st Cir. 2002) (citing
Daubert, 509 U.S. at 593-94). “These factors, however, are not
definitive or exhaustive, and the trial judge enjoys broad
latitude to use other factors to evaluate reliability.” Mooney,
315 F.3d at 62 (citing Kumho Tire Co. v. Carmichael, 526 U.S.
137, 153 (1999)).
The Court must, however, keep in mind the Supreme Court’s
admonition that “[v]igorous cross-examination, presentation of
contrary evidence, and careful instruction on the burden of proof
are the traditional and appropriate means of attacking shaky but
admissible evidence.” Daubert, 509 U.S. at 596. If an expert’s
testimony is within “the range where experts might reasonably
differ,” the jury, not the trial court, should be the one to
“decide among the conflicting views of different experts.” Kumho
Tire, 526 U.S. at 153. As the First Circuit has stated:
In short, Daubert neither requires nor empowers trial
courts to determine which of several competing scientific
theories has the best provenance. It demands only that
the proponent of the evidence show that the expert’s
conclusion has been arrived at in a scientifically sound
and methodologically reliable fashion.
11

Ruiz-Troche, 161 F.3d at 85. It is with these principles in mind
that the Court assesses the defendants’ motion.
B. A Sneak Preview
As a preliminary matter, plaintiff asked for an early
determination of the reliability of its sampling methodology.
Defendants press the argument that it is premature to determine
the admissibility of Dr. Cowan’s sampling methodology set forth
in his April 12, 2013 report before he has applied it to the
reunderwriting results. A magistrate judge granted MassMutual’s
motion for an early determination (11-cv-30039, Dkt. No. 117) and
the matter was set for hearing before the judge originally
assigned to the case.
Early resolution of the viability of the sampling
methodology makes sense as a case management matter. See David H.
Kaye & David A. Freedman, Reference Guide On Statistics, in
FEDERAL JUDICIAL CENTER, REFERENCE MANUAL

ON

SCIENTIFIC EVIDENCE 211, 216

(3d ed. 2011) (“To minimize debates at trial over the accuracy of
data and the choice of analytical techniques, pretrial discovery
procedures should be used, particularly with respect to the
quality of the data and the method of analysis.”). The plaintiff
would have to incur significant expense and the litigation would
be unnecessarily delayed, if the sampling methodology does not
survive a Daubert challenge late in the litigation. For example,

12

an additional 300 loan files, or more, could have to be
reunderwritten for each securitization at issue. While defendants
argue some of the sampling issues might have to be reexamined in
light of the extrapolation methodology chosen, defendants have
not persuasively shown this is likely.
C. The Challenge
In challenging the Report, the defendants do not challenge
Dr. Cowan’s expertise. Rather, Defendants identify six
methodological errors that they claim render the Report’s
sampling protocol unreliable. Similar challenges have failed in
other actions involving RMBSs. See In re Countrywide Fin. Mortg.
Backed Sec. Litig., 2013 WL 6231713, – F. Supp. 2d – (C.D. Cal.
Dec. 2, 2013); Fed. Hous. Fin. Agency v. JPMorgan Chase & Co.,
2012 WL 6000885 (S.D.N.Y. Dec. 3, 2012); In re Washington Mut.
Mortg. Backed Sec. Litig., 2012 WL 2995046 (W.D. Wash. July 23,
2012); MBIA Ins. Corp. v. Countrywide Home Loans, Inc., 958
N.Y.S.2d 647, 2010 WL 5186702 (Sup. Ct. Dec. 22, 2010).
1. Extrapolation Method
Defendants contend that Dr. Cowan failed to provide a
specific extrapolation method. Although Dr. Cowan describes two
possible ways to extrapolate data to a full SLG population, the
Report does not commit itself to a certain extrapolation method.
Instead, he asserts that it is prudent to select a method after
the sample design is confirmed and test results determined.

13

Defendants highlight a statement made by Dr. Cowan at his
deposition that determining the extrapolation method is
"integral" to "planning for and acceptance of sampling as a
viable scientific method." Cowan Dep., Tr. 226. Dr. Cowan
clarified his view at the Daubert hearing that it is not
necessary to choose a method of extrapolation before
reunderwriting in order to have a valid sample design. Hr’g Tr.
94.
Defendants’ expert Dr. Barnett states that Dr. Cowan cannot
later choose an extrapolation method that minimizes the margin of
error because Dr. Cowan stratified the population before pulling
sample loans and thus committed himself to one of the
extrapolation formulae pertinent to proportional stratified
sampling. Hr’g Tr. 193-94; Barnett Report ¶ 37-38; Barnett Dep.,
Tr. 170-72. Even if Dr. Cowan is unable to reduce the margin of
error below ± 10 percentage points through the use of a marginreducing extrapolation method, that does not render his
methodology unreliable. Dr. Cowan testified that he will test
multiple extrapolation methods once he gets the reunderwriting
results, utilizing perhaps as many as twenty techniques. Hr’g Tr.
92-93. So long as Dr. Cowan ultimately employs an extrapolation
technique which is itself reliable, the failure to specify the
specific method in his Report does not make his sampling
methodology excludable in this preliminary review.

14

2. Binary Nature of Inquiries
Defendants primary challenge hinges on the assertion that
Dr. Cowan’s methodology erroneously depends on binary answers to
questions of misrepresentation. Dr. Barnett testified that the
formulae used by Dr. Cowan “are predicated on a binary analysis
where you have a series of data points; there’s a yes/no question
for every single one of them." If the question is not binary,
they argue, then Dr. Cowan’s statistical analysis, including
maximum margin of error, is incorrect. Hr’g Tr. 162-63.
Defendants argue that the four major inquiries (compliance with
underwriting guidelines, compliance with appraisal standards,
understatement of LTV ratios, and overstatement of owneroccupancy rates) depend on complex and subjective analyses and
cannot be reduced to simple “yes-no” formulations. Specifically,
they emphasize that the originators' underwriting guidelines were
not strict, inflexible rules and expressly permitted exceptions.
Plaintiff counters that the questions, as framed, are binary,
since the loans were either valid under relevant criteria or not.
At the Daubert hearing, Dr. Barnett acknowledged that some
of these questions are or may be binary. Hr’g Tr. 181:10-22
(compliance with underwriting guidelines); 173:19-174:12
(compliance with appraisal standards, provided set benchmarks are
accepted); 172:15-20 (owner occupancy status). Defendants press
the issue concerning the accuracy of the LTV and CLTV ratios

15

stated in the offering documents. A LTV ratio is the ratio of the
mortgage loan's original principal balance to the appraised value
or sales price of the mortgaged property. A CLTV ratio is a
similar calculation for properties with two or more loans.
Defendants continue to emphasize that the weighted average LTV
and CLTV are continuous variables, meaning that each observation
may have many possible values.
Plaintiff concedes that a benchmark for materiality must be
set, and that a determination of the average weighted LTV/CLTV
involves a review of individual loan files that will ask a series
of non-binary questions. Still, as Plaintiff points out, once a
weighted average LTV/CLTV for the 100 loans is calculated, it is
a binary question whether it is materially different from the
percentage stated in the offering materials. See Assured Guar.
Mun. Corp. v. Flagstar Bank, 920 F. Supp. 2d 475, 503 (S.D.N.Y.
2013). While some of the underlying steps or formulae may involve
non-binary questions, defendants may later challenge the
methodology used by the reunderwriters for calculating the
average as unreliable or deserving of less weight.
3. Multi-Originator Scenario
Defendants' strongest argument is that the methodology
cannot distinguish among originators. In 57 of the 99 SLGs at
issue in these cases, the SLG is backed by loans originated from
multiple lenders. As the defendants rightly point out, different

16

originators may have followed different underwriting guidelines
and adhered to different practices in issuing loans.
Defendants offer an example from the action against Deutsche
Bank. The prospectus supplement for the ACE 2006-SL1
securitization lists three originators, but only discloses
underwriting criteria and appraisal standards for the two
originators that originated 20% or more of the asset pool,
American Home Mortgage Corporation and Residential Funding
Corporation. In other words, Deutsche Bank never made any
representations as to the appraisal standards of the third named
originator, Chapel Funding Corporation, and therefore cannot be
said to have made a misrepresentation about Chapel Funding. The
defendants argue that only loans from American Home and
Residential Funding would be relevant for sampling purposes.
Perhaps in response to this challenge, plaintiff dropped this
securitization as a basis for liability.
Plaintiff correctly responds that the question is not
whether each originator is liable for material
misrepresentations, but whether the defendants themselves made
misrepresentations in the certificates regarding the underwriting
standards applied to, and LTV and appraisal information for, all
loans backing the certificates. See Fed. Hous. Fin. Agency, 2012
WL 6000885, at *10. However, to the extent the offering documents
differentiate among the originators, defendants may well be right

17

that the sampling methodology, as proposed, is inadequate. Dr.
Cowan acknowledged at hearing that sampling from the entire loan
pool where representations were only made in say, 20% of the
loans in the pool, could widen the margin of error. Hr’g Tr. 13435.
Although Defendants assert that certain alleged
misrepresentations in the certificates are specific to certain
originators, the record is unclear as to the specific
certificates where the sampling methodology may need to be
adjusted. I will reserve this issue until the expert evaluates a
sample for the specific representations in each action.
4. Margin of Error
Defendants argue that Dr. Cowan’s ± 10 percentage point
margin of error is twice as wide as the typical margin of error
in the litigation context, noting that Dr. Cowan’s reports in
other mortgage-backed securities actions have employed a ± 5
percentage point margin of error. E.g., MBIA Ins. Corp., 2010 WL
5186702 at *5. Defendants bolster this contention with examples
outside litigation, noting that the U.S. Department of Housing
and Urban Development (“HUD”) and the Federal Home Loan Mortgage
Corporation (“Freddie Mac”) each employ a ± 2 percentage point
margin of error in their respective quality control guidelines.
In other words, Defendants assert that the sample size of 100
loans is too small, and a larger sample size will reduce the

18

margin of error. See REFERENCE MANUAL

ON

SCIENTIFIC EVIDENCE, supra at

246 (“Generally, increasing the size of the sample will reduce
the level of random error (‘sampling error’).”).
As Dr. Barnett concedes, though, using a confidence interval
of 20 percentage points does not make a statistical methodology
inherently unreliable. Hr’g Tr. 185-87; Barnett Dep., Tr. 100102. Plaintiff takes the risk that using the ± 10 percentage
point margin of error will result in a lower estimated rate of
defective loans backing the certificates. As other courts have
concluded, the ± 10 percentage point margin of error does not
render Dr. Cowan’s methodology unreliable. The margin of error
speaks to the “persuasive power of the sample, not its
admissibility.” Fed. Hous. Fin. Agency, 2012 WL 6000885, at *10;
see also In re Countrywide, 2013 WL 6231713, at *8-9.
5. Stratification of Loan Population by FICO Score
Defendants object to Dr. Cowan’s assertion that a borrower’s
FICO score is an appropriate stratification variable. However, as
Dr. Cowan testified, and defendants’ expert concedes, even if
stratification does not diminish the margin of error, it cannot
increase the margin of error. Hr’g Tr. 84-85; Barnett Dep., Tr.
127-29, 134. Stratification increases the precision. See GEORGE W.
SNEDECOR & WILLIAM G. COCHRAN, STATISTICAL METHODS 441-442 (8th ed. 1989)
(hereinafter “COCHRAN”) ("If we can form strata so that a
heterogeneous population is divided into parts, each of which is

19

fairly homogeneous, we may expect a substantial gain in precision
over simple random sampling.”). Moreover, defendants don't
provide a persuasive reason why the borrower's FICO score is not
a useful criterion. Cf. In re Countrywide, 2013 WL 6231713, at
*11 (“The use of FICO scores as the selected stratification
variable comports with common sense . . . since higher FICO
scores indicate a positive borrower credit history and a lower
risk borrower profile.”).
Defendants also object to Dr. Cowan’s proposed practice of
automatically assigning a loan missing a FICO score to the lowest
quartile. Dr. Cowan explains that a missing FICO score is, in his
experience, indicative of a loan underwriting breach. Hr’g Tr.
81-82; Cowan Decl. in Response to Barnett Decl. ¶ 25. Dr. Cowan
did not make the factual basis for this assumption clear.
However, it is premature to challenge the methodology on this
ground because the record is unclear as to how many files are
missing FICO scores.
6. Possibility of Missing Loan Files
Defendants assert that the probable absence of some loan
files renders the full sample non-random and statistically
unreliable. Plaintiff responds that Dr. Cowan’s proposal to
create a randomized 100-loan “back-up” sample is a statistically
acceptable method to anticipate and address potential snafus in
collecting loan files. Defendants' arguments have been rejected

20

by other district courts. See In re Countrywide, 2013 WL 6231713,
at *7-8; Fed. Hous. Fin. Agency, 2012 WL 6000885, at *10; In re
Washington Mut., 2012 WL 2995046, at *22. Missing data are common
in statistical sampling surveys. COCHRAN, supra, at 454; David W.
Chapman, Substitution for Missing Units, in PROCEEDINGS

OF THE

SURVEY

RESEARCH METHODS SECTION, AMERICAN STATISTICAL ASSOCIATION 76, 76 (1982).
To the extent that, as the defendants suggest, an unexpected
event such as the absence of all files from a single originator
skews the full sample of the SLG,4 Dr. Cowan will address the
impact of the missing files in his analysis and presentation of
the data. Hr’g Tr. 85-86; Cowan Dep., Tr. 207-13; Barnett Dep.,
Tr. 149-54; see also COCHRAN, supra, at 273-75, 454-55; Tom W.
Smith, Notes on the Use of Substitution in Surveys (Apr. 2007),
available at www.issp.org/member/documents/Substitution
_MC_Review.doc (unpublished manuscript distributed to
International Social Survey Programme members). The defendants
can challenge Dr. Cowan’s methodology for replacing missing files
later in the litigation.

4

For example, Dr. Cowan mentioned that all the records of
one originator were destroyed by Hurricane Sandy. Hr’g Tr. 12829.
21

V. ORDER
The Motion to Exclude the Opinions Expressed in the April
12, 2013 Report of Plaintiff’s Expert, Charles D. Cowan, Ph.D.,
(11-cv-30039, Dkt. No. 143) is DENIED without prejudice.

/s/ PATTI B. SARIS
Patti B. Saris
Chief United States District Judge

22