Oracle Corporation et al v. SAP AG et al

Filing 773

MOTION No. 6: To Exclude Testimony of Defendants' Expert Bruce Spencer filed by Oracle EMEA Limited, Oracle International Corporation, Oracle USA Inc., Siebel Systems, Inc.. Motion Hearing set for 9/30/2010 09:00 AM in Courtroom 3, 3rd Floor, Oakland. (Attachments: # 1 Proposed Order)(Howard, Geoffrey) (Filed on 8/19/2010)

Oracle Corporation et al v. SAP AG et al

Doc. 773

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

BINGHAM MCCUTCHEN LLP DONN P. PICKETT (SBN 72257) GEOFFREY M. HOWARD (SBN 157468) HOLLY A. HOUSE (SBN 136045) ZACHARY J. ALINDER (SBN 215695) BREE HANN (SBN 215695) Three Embarcadero Center San Francisco, CA 94111-4067 Telephone: [phone] Facsimile: 415.393.2286 [e-mail] [e-mail] [e-mail] [e-mail] [e-mail] BOIES, SCHILLER & FLEXNER LLP DAVID BOIES (Admitted Pro Hac Vice) 333 Main Street Armonk, NY 10504 Telephone: [phone] [e-mail] STEVEN C. HOLTZMAN (SBN 144177) 1999 Harrison St., Suite 900 Oakland, CA 94612 Telephone: [phone] [e-mail] DORIAN DALEY (SBN 129049) JENNIFER GLOSS (SBN 154227) 500 Oracle Parkway, M/S 5op7 Redwood City, CA 94070 Telephone: [phone] Facsimile: 650.506.7114 [e-mail] [e-mail] Attorneys for Plaintiffs Oracle USA, Inc., et al. UNITED STATES DISTRICT COURT NORTHERN DISTRICT OF CALIFORNIA OAKLAND DIVISION ORACLE USA, INC., et al., No. 07-CV-01658 PJH (EDL) NOTICE OF MOTION AND MOTION Plaintiffs, NO. 6: TO EXCLUDE TESTIMONY v. OF DEFENDANTS' EXPERT BRUCE SPENCER SAP AG, et al, Date: September 30, 2010 Defendants. Time: 9 a.m. Place: Courtroom 3 Judge: Hon. Phyllis J. Hamilton
Case No. 07-CV-01658 PJH (EDL) NOTICE AND MOT. NO. 6: TO EXCLUDE TESTIMONY OF DEFENDANTS' EXPERT BRUCE SPENCER

Dockets.Justia.com

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 II. III. I.

TABLE OF CONTENTS Page SUMMARY OF INADMISSIBLE OPINIONS AND RELIEF REQUESTED ................ 1 A. Levy Report............................................................................................................. 1 B. Spencer Report ........................................................................................................ 2 WHAT DEFENDANTS MUST SHOW TO JUSTIFY SPENCER'S OPINIONS AND WHAT THE COURT MUST DO TO TEST THEM................................................ 3 ARGUMENT ...................................................................................................................... 5 A. Spencer's Testimony Should Be Excluded Because He Has No Understanding Of The Facts Of This Case, The Purpose Of His Report Or The Purpose Of Levy's Report ............................................................................... 5 B. Spencer's Testimony Should Be Excluded Because His Analysis Is Predicated On An Erroneous Conclusion That Levy's Opinions Relate To Damages And That The Burden Of Proof Is Higher In This Case Due To The Magnitude Of Oracle's Damages Claims ........................................................ 7 1. Spencer's Testimony Should Be Excluded As Based On An Erroneous Conclusion That Levy's Analysis Is Related To Damage Calculations, Not Liability .......................................................................... 7 2. Spencer's Assertion That Oracle's Burden Of Proof Must Increase As Its Claim For Damages Increases Is An Improper And Unsupportable Legal Argument.................................................................. 9 3. Spencer's Testimony About The Precision, Sampling Technique And Sample Sizes Used By Levy Misapplies Statistics, And Its Probative Value Is Outweighed By Unfair Prejudice And Confusion To The Jury ............................................................................. 11 a. Precision........................................................................................ 11 b. Sampling "With" Versus "Without" Replacement ....................... 13 c. Sample Sizes ................................................................................. 14 C. Spencer Fails To Employ Reliable Principles And Methods In His Critique Of Levy's Use Of Microsoft Excel To Create Random Samples ......................... 16 1. Spencer's Suggestions That Levy Misused Or Even Lied About Using Excel To Generate Sample Order Are Wholly Improper ............... 16 2. Spencer's Calculations Regarding The Randomness Of Levy's Samples Are Erroneous, And Thus Are Not The Result Of The Application Of Reliable Principles And Methods .................................... 18 D. Spencer Failed To Reliably Apply Appropriate Principles And Methods When Analyzing Levy's Methods And, When Tested, Spencer's Theoretical "Corrections" Have No Effect On Levy's Results ............................ 19 1. Portions Of Spencer's Report Are Based On "Implicit Calculations" Which Are Not Verifiable Or Replicable And Are Therefore Unreliable ................................................................................. 19 2. Spencer's View That Levy Should Have Used The Student's t Distribution Instead Of The Normal Distribution Is Wrong And Would Have Made No Difference To Levy's Analysis............................ 20 i Case No. 07-CV-01658 PJH (EDL)
NOTICE AND MOT. NO. 6: TO EXCLUDE TESTIMONY OF DEFENDANTS' EXPERT BRUCE SPENCER

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 IV. 3.

TABLE OF CONTENTS (continued)

Page

Spencer's Measurement Of Skewness As To Population Measures Has No Bearing On The Sample Measures............................................... 21 4. Spencer Makes Repeated Speculative Assertions About Potential Measurement Error By Mandia, But Assumes It Away In His Own Analyses .................................................................................................... 22 E. Spencer's Report Is Riddled With Misleading And Prejudicial Statements Based Upon Uncorrected And Untimely Errors That Are Objectionable Under Fed. R. Evid. 403 And Fed. R. Civ. P. 26(E)............................................. 22 1. Spencer's Comments About A Calculation Error That Levy Identified And Corrected More Than A Month Before Spencer's Report Issued Should Be Excluded........................................................... 22 2. Variance .................................................................................................... 23 CONCLUSION ................................................................................................................. 23

ii

Case No. 07-CV-01658 PJH (EDL)

NOTICE AND MOT. NO. 6: TO EXCLUDE TESTIMONY OF DEFENDANTS' EXPERT BRUCE SPENCER

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 CASES

TABLE OF AUTHORITIES Page

Abuan v. General Elec. Co., 3 F.3d 329 (9th Cir. 1993)................................................................................................ 10, 11 Claar v. Burlington R.R., 29 F.3d 499 (9th Cir. 1994)...................................................................................................... 3 Concrete Pipe and Prod. of Cal., Inc. v. Constr. Laborers Pension Trust for S. Cal., 508 U.S. 602 (1993) ............................................................................................................... 11 Daubert v. Merrell Dow Pharm. Inc., 509 U.S. 579 (1993) ....................................................................................................... 3, 4, 20 Domingo ex rel. Domingo v. T.K., 289 F.3d 600 (9th Cir. 2002).................................................................................................. 15 Fisher v. Dees, 794 F.2d 432 (9th Cir. 1986).................................................................................................... 2 General Electric Co. v. Joiner, 522 U.S. 136 (1997) ........................................................................................................... 4, 20 Hathaway v. Bazany, 507 F.3d 312 (5th Cir. 2007).............................................................................................. 4, 10 Heller v. Shaw, 167 F.3d 146 (3d Cir. 1999)..................................................................................................... 4 In re Agent Orange Product Liability Litigation, 611 F. Supp. 1223 (D.C.N.Y., 1985) ....................................................................................... 4 In Re Brand Name Drugs, 186 F.3d 781 (7th Cir. 1999)................................................................................................ 4, 9 Mukhtar v. California State University, 299 F.3d 1053 (9th Cir. 2002), as amended, 319 F.3rd (9th Cir. 2003) .................................................................................... 3 Nationwide Transport Finance v. Cass Information Systems, Inc., 523 F.3d 1051 (9th Cir. 2008).......................................................................................... 10, 11 Nebraska Plastics, Inc. v. Holland Colors Americas, Inc., 408 F.3d 410 (8th Cir. 2005).................................................................................................... 5 iii
Case No. 07-CV-01658 PJH (EDL)

NOTICE AND MOT. NO. 6: TO EXCLUDE TESTIMONY OF DEFENDANTS' EXPERT BRUCE SPENCER

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

TABLE OF AUTHORITIES (continued)

Page

Perry v. Schwarzenegger, 2010 WL 3025614 (N.D. Cal.)................................................................................................. 3 QR Spex, Inc. v. Motorola, Inc., 2004 WL 5642907 (C.D. Cal.)................................................................................................. 9 Robinson v. G.D. Searle & Co., 286 F. Supp. 2d 1216 (N.D. Cal. 2003) ................................................................................... 9 Salinas v. Amteck of Kentucky, Inc., 682 F. Supp. 2d 1022 (N.D. Cal. 2010) ................................................................................... 3 Sunstar, Inc. v. Alberto-Culver Co., Inc., 2004 WL 1899927 (N.D. Ill.)................................................................................................... 5 Trout v. Milton S. Hershey Medical Center, 576 F. Supp. 2d 673 (M.D. Pa. 2008) ...................................................................................... 5 U.S. v. Ravel, 930 F.2d 721 (9th Cir. 1991).............................................................................................. 4, 13 United States v. Barnard, 490 F.2d (9th Cir. 1973)......................................................................................................... 17 United States v. Jawara, 474 F.3d 565 (9th Cir. 2007).................................................................................................... 3 RULES Fed. R. Evid. 104 ....................................................................................................................... 3, 8 Fed. R. Evid. 403 .................................................................................................................. passim Fed. R. Evid. 702 .................................................................................................................. passim

iv

Case No. 07-CV-01658 PJH (EDL)

NOTICE AND MOT. NO. 6: TO EXCLUDE TESTIMONY OF DEFENDANTS' EXPERT BRUCE SPENCER

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

PLEASE TAKE NOTICE that on September 30, 2010, at 9:00 a.m., in the courtroom of the Honorable Phyllis J. Hamilton, of the above-entitled Court, Plaintiffs Oracle USA, Inc., Oracle International Corporation, Oracle EMEA Limited, and Siebel Systems, Inc. (collectively, "Oracle") shall and hereby do move for an order excluding opinions and testimony of Dr. Bruce D. Spencer ("Spencer") designated by Defendants SAP AG, SAP America, Inc., and TomorrowNow, Inc. ("SAP TN") (collectively "Defendants") as an expert witness in this matter, on the grounds that Spencer's proposed expert opinion testimony is inadmissible on the basis of the authorities and evidence set forth herein and in the accompanying declarations. I. SUMMARY OF INADMISSIBLE OPINIONS AND RELIEF REQUESTED Each side has designated an expert statistician -- Plaintiffs' is Dr. Daniel S. Levy and Defendants named Dr. Bruce Spencer in rebuttal. Ordinarily these two witnesses would present their competing opinions to the jury. However, Spencer's testimony is so untethered to the facts of the case and the science of statistics that it must be excluded. Among other things, Spencer: · · Admitted he does not know the purpose of either his testimony or that of Levy, Testified he "doesn't know" if any of his criticisms "make a difference in the case," · · Erroneously assumed his testimony related to damages instead of liability, Contended contrary to law that Plaintiffs should be put to a higher burden of proof in this case (and a higher level of statistical precision) due to the large amount of money at stake, and · Repeatedly suggested criticisms of Levy's work that make no difference to the results (and without checking if they do). These fundamental errors riddle his opinions and require exclusion of all of his testimony. A. Levy Report

Plaintiffs retained Levy, National Managing Director of Advanced Analytical Consulting Group, Inc., to design a statistically valid sample of SAP TN's fixes and updates for PeopleSoft HRMS, and to calculate population and sample statistics based on data about those fixes and updates gathered by Plaintiffs' computer forensics expert, Mr. Kevin Mandia. Sampling was 1 Case No. 07-CV-01658 PJH (EDL)
NOTICE AND MOT. NO. 6: TO EXCLUDE TESTIMONY OF DEFENDANTS' EXPERT BRUCE SPENCER

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

determined to be an appropriate manner in which to deal with some of the voluminous data produced by SAP TN after Mandia determined that exhaustive collection and review of individual data would require several thousands of hours. See Sherrod Decl. Ex. C (Expert Report of Dr. Daniel S. Levy, November 16, 2009, as amended, February 12, 2009) ("Levy Report"), at 9 n. 29. Levy has a Ph.D. in Economics from the University of Chicago and has previously testified on statistical sampling issues in numerous matters. Id. at 44-45. Levy began his analysis by providing two random sets of numbers to Mandia for the two different categories of SAP TN support (retrofit and critical support). Those random sets were used to determine which fixes Mandia analyzed from his data on a sample basis in order to estimate the frequency of SAP TN's cross-use of environments when it generated fixes for PeopleSoft HRMS product and to estimate the number of objects copied by SAP TN in generating those fixes. Id. at 2. After Levy received those sampling results and other data from Mandia, he counted overall statistics for the aspects of the two categories where data was individually collected for every fix (fully enumerated measures which required no sampling) and calculated and extrapolated sample statistics for other aspects where data was collected only for the fixes in the random samples (sample measures). As a result, Levy concluded that over 83% of retrofit fixes and over 99% of critical support fixes that SAP TN delivered to customers contained objects that resulted from cross-use, with a 90% confidence interval within a 50% precision range. See Sherrod Decl. Ex. C, at 5. Levy's report disproves an element of Defendants' alleged license defenses to liability, because cross-use of customer environments violates Oracle's software licenses. It also proves an element of Oracle's copyright liability case, because each copy of one or more objects containing more than de minimis protected expression may constitute a copyright violation. Fisher v. Dees, 794 F.2d 432, 434 n.2 (9th Cir. 1986). B. Spencer Report

Defendants designated Spencer as a rebuttal expert to both Levy and Mandia. However, his report and testimony only address Levy, and in his only analysis of material provided by Mandia, Spencer assumed that the Mandia data was correct. See Sherrod Decl., Ex A, Expert 2 Case No. 07-CV-01658 PJH (EDL)
NOTICE AND MOT. NO. 6: TO EXCLUDE TESTIMONY OF DEFENDANTS' EXPERT BRUCE SPENCER

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

Report of Dr. Bruce D. Spencer, March 17, 2010 ("Spencer Report") at 25 n. 45. Spencer criticized Levy with respect to Levy's selection of statistical methodologies (particularly his level of precision) and his implementation of those methodologies. See Sherrod Decl., Ex A, (Spencer Report) at 12-13. But he did not correct any perceived errors, determine their magnitude or reach any counter opinions as to a more precise level of cross-used environments or number of copied contaminated objects. Furthermore, he did not perform any original analysis based on his own sampling of SAP data which he had full access to. II. WHAT DEFENDANTS MUST SHOW TO JUSTIFY SPENCER'S OPINIONS AND WHAT THE COURT MUST DO TO TEST THEM Fed. R. Evid. 702 requires exclusion of expert testimony unless: (1) the testimony is based upon sufficient facts or data, (2) the testimony is the product of reliable principles and methods, and (3) the witness has applied the principles and method reliably to the facts of the case. The party proffering an expert opinion must demonstrate it meets the Rule 702 admissibility standards by a "preponderance of proof." Daubert v. Merrell Dow Pharm. Inc., 509 U.S. 579, 593 (1993); Salinas v. Amteck of Kentucky, Inc., 682 F. Supp. 2d 1022, 1029 (N.D. Cal. 2010) (Hamilton, J.,); Perry v. Schwarzenegger, 2010 WL 3025614 at *21 (N.D. Cal.) ("The party proffering the evidence `must explain the expert's methodology and demonstrate in some objectively verifiable way that the expert has both chosen a reliable . . .method and followed it faithfully.'") (quoting Daubert v. Merrell Dow Pharm., 43 F3d 1311, 1319 n11 (9th Cir 1995) ("Daubert II")). Absent an explicit finding by the court of admissibility of a challenged opinion, the opinion may not properly be offered at trial. Fed. R. Evid. 104(a) ("Preliminary questions concerning the qualifications of a person to be a witness. . .shall be determined by the court."); United States v. Jawara, 474 F.3d 565, 583 (9th Cir. 2007) ("failure to make explicit reliability finding was an error"); Mukhtar v. California State University, 299 F.3d 1053, 1066-68 (9th Cir. 2002) (district court prejudicially erred by admitting expert testimony without explicit reliability determination), as amended, 319 F.3rd 1073 (9th Cir. 2003); Claar v. Burlington R.R., 29 F.3d 499, 501 (9th Cir. 1994) (courts are both "authorized and obligated to scrutinize carefully the 3 Case No. 07-CV-01658 PJH (EDL)
NOTICE AND MOT. NO. 6: TO EXCLUDE TESTIMONY OF DEFENDANTS' EXPERT BRUCE SPENCER

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

reasoning and methodology underlying" expert testimony). As the Supreme Court has made clear, "the trial judge must ensure that any and all [expert] testimony . . .is not only relevant but reliable." Daubert, 509 U.S. at 589, 595. While the Daubert reliability analysis focuses on an expert's methodology, the Supreme Court has also noted that "conclusions and methodology are not entirely different from one another." General Electric Co. v. Joiner, 522 U.S. 136, 146 (1997). Trained experts commonly extrapolate from existing data. But nothing in either Daubert or the Federal Rules of Evidence requires a district court to admit opinion evidence that is connected to existing data only by the ipse dixit of the expert. A court may conclude that there is simply too great an analytical gap between the data and the opinion offered. Id.; accord Heller v. Shaw, 167 F.3d 146, 153 (3d Cir. 1999) ("district court must examine the expert's conclusions in order to determine whether they could reliably follow from the facts known to the expert and the methodology used"). Expert opinions must be based on sufficient facts or data. An expert's testimony is inadmissible if it is based on suppositions rather than facts. Hathaway v. Bazany, 507 F.3d 312, 318 (5th Cir. 2007) (trial court properly excluded proffered expert testimony because it relied on insufficient factual support and "a host of unsupported conjectures that falls far short of a methodology."). Expert opinions may be excluded if the expert does not have sufficient familiarity with the relevant facts of the case or the purpose of the expert's report. See e.g. In Re Brand Name Drugs, 186 F.3d 781, 788 (7th Cir. 1999) (affirming exclusion of expert testimony where expert did not know facts of the case). Rule 702 further requires that the evidence or testimony "assist the trier of fact to understand the evidence or to determine a fact in issue." Quoted in Daubert, at 590-591; see also In re Agent Orange Product Liability Litigation, 611 F. Supp. 1223, 1242 (D.C.N.Y., 1985). Testimony which would be of no appreciable help to the jury, but would serve instead only to confuse it, should be excluded. U.S. v. Ravel, 930 F.2d 721, 726 (9th Cir. 1991); see also Daubert, 509 U.S. at 595 (because it "can be both powerful and quite misleading because of the difficulty in evaluating it," courts exclude otherwise relevant expert testimony "if its probative value is substantially outweighed by the danger of unfair prejudice, confusion of the issues, or 4 Case No. 07-CV-01658 PJH (EDL)
NOTICE AND MOT. NO. 6: TO EXCLUDE TESTIMONY OF DEFENDANTS' EXPERT BRUCE SPENCER

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

misleading the jury.") (quoting Jack Weinstein, Rule 702 of the Federal Rules of Evidence is Sound; It Should Not Be Amended, 138 F.R.D. 631, 632 (1991)). III. ARGUMENT A. Spencer's Testimony Should Be Excluded Because He Has No Understanding Of The Facts Of This Case, The Purpose Of His Report Or The Purpose Of Levy's Report.

Spencer's opinions should be excluded because he lacks a fundamental understanding of the facts of this case or even the purpose of his own analysis. Nor does he know what consequences would result from application of his opinions. As a result, he does not have the requisite knowledge to form any valid, reliable opinions in this matter. See Nebraska Plastics, Inc. v. Holland Colors Americas, Inc., 408 F.3d 410, 416 (8th Cir. 2005) (affirming trial court's preclusion of expert's opinions because they did not fit the facts of the case and addressed none of the relevant facts, noting that "if the expert's opinion is so fundamentally unsupported that it can offer no assistance to the jury, it must be excluded (citations omitted) and that "an expert opinion that fails to consider the relevant facts of the case is fundamentally unsupported" and "should not be admitted if it does not apply to the specific facts of the case."); accord Sunstar, Inc. v. Alberto-Culver Co., Inc., 2004 WL 1899927, at *26-27 (N.D. Ill.); Trout v. Milton S. Hershey Medical Center, 576 F. Supp. 2d 673, 678 (M.D. Pa. 2008) (expert's testimony based on generalizations excluded because opinions not applied to specific case facts). Remarkably, Spencer admitted that he knows virtually nothing about this case: "[w]hat I know about this case is, I was asked to review and evaluate Levy's report and his work, and that's what I know about. And I'm pretty myopic about the rest of the case." Sherrod Decl. Ex. B at 94:22-25. Exacerbating Spencer's "myopic" misunderstanding of this case is his total lack of knowledge regarding how his own numbers and/or Levy's numbers will be used, "Q. You don't know how the numbers tie into the case. Is that your testimony? A. That's my testimony, yes." Id. at 76:2-4; see also id. at 281:18, 282:10-24 ("Q. Is it too low? A. I don't know what purposes we're talking about. For some purposes it would be too low, and for other purposes it might not matter. It's context-dependent. Can't say that it's too low for all purposes. Q. Well, is it too low for purposes of this case? Do you know? A. I don't know all the purposes to which 5 Case No. 07-CV-01658 PJH (EDL)
NOTICE AND MOT. NO. 6: TO EXCLUDE TESTIMONY OF DEFENDANTS' EXPERT BRUCE SPENCER

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

this is going ton [sic] used in this case. Q. Do you know any of the purposes? A. Not as I sit here, no.") (emphasis supplied). Even more troubling is the fact that Spencer does not know the consequences of his own analysis: "Q. Therefore, you don't know whether any of your criticisms of Levy make a difference in the case. Right? A. I'm uncertain. My opinion is that they do have some consequences for the case. Q. What? A. I don't know." [OBJECTIONS OMITTED]. Sherrod Decl., Ex. B at 76:5-15 (emphasis supplied). Not surprisingly, given his extraordinary unfamiliarity with this case, his role, the consequences of his own opinions and how the numbers tie into the case, Spencer admitted time and time again that he is equally clueless about the purpose of Levy's report. See e.g. id. at 81:19-25 ("Q. Do you have any information at all with respect to how the summary statistics from Levy's report translate into damages calculations or other kinds of arguments in the case? A. I have nothing explicit or concrete in the way of understanding."); 85:18-25 ("A. It depends on how the -- Levy's numbers get used. Q. Which you don't know? A. Which I don't know."); 122:15-22 ("Q. But are you of the opinion that sampling with replacement is not sufficiently precise for the purpose at hand? A. I don't know how the statistics are going to be used, and so that's why I can't answer that question. I don't have an opinion on that."); 253:14-254:2, ("Q. Based on what you know, are you able to say whether or not his precision ranges are precise enough? . . . A. Without knowing the purposes of the case or having seen how the numbers are going to be used, I can't answer your question and say I know, no. So the answer is, I don't know. At this point. If I had more information about the use, I might be able to answer.") [OBJECTIONS OMITTED] (emphasis supplied). But the time for "hav[ing] more information" was when he was drafting his report (or at least testifying at deposition). That time has long since passed. It should come as no surprise that because he has no understanding of virtually any aspect of this case or his and Levy's role in it, Spencer consistently applies the wrong standards of law and principles of statistics to formulate inaccurate and unreliable conclusions regarding Levy's report. Due to his lack of knowledge about this case and misapplication of both law and statistical theory, Spencer's report is riddled with irrelevant criticisms and incorrect conclusions 6 Case No. 07-CV-01658 PJH (EDL)
NOTICE AND MOT. NO. 6: TO EXCLUDE TESTIMONY OF DEFENDANTS' EXPERT BRUCE SPENCER

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

which render his entire report unreliable, irrelevant and confusing to the trier of fact. Accordingly, Spencer's testimony and exhibits should be excluded. B. Spencer's Testimony Should Be Excluded Because His Analysis Is Predicated On An Erroneous Conclusion That Levy's Opinions Relate To Damages And That The Burden Of Proof Is Higher In This Case Due To The Magnitude Of Oracle's Damages Claims.

Much of Spencer's report is premised on two related erroneous conclusions about damages. First, he somehow believes that Levy's analysis (and his rebuttal of it) are relevant to and will have some unknown impact on Oracle's damages claim. In fact, the statistics are presented to support liability and have nothing to do with damage calculations. Second, he believes the veracity of Levy's study and the burden it addresses must be linked to the magnitude of Oracle's damages. He concludes that both the burden of proof at trial and the requisite statistical precision must be higher than usual due to the size of Oracle's claim. That of course is not true, and since he assumes an improper level of requisite proof, his analysis is unfounded. These errors so permeate Spencer's report that it must be excluded in its entirety. See Fed. R. Evid. 403 and 702. 1. Spencer's Testimony Should Be Excluded As Based On An Erroneous Conclusion That Levy's Analysis Is Related To Damage Calculations, Not Liability.

Spencer's criticisms of Levy are wrong because they are based on Spencer's imagined (and provably false) link between Levy's opinions and Oracle's damages claims. Comparative evaluation of statistical precision requires an understanding of the purposes for which the statistics to be calculated will be used. See Sherrod Decl. Ex. J, Cochran, W.G., Sampling Techniques. Third Edition. New York, New York: John Wiley & Sons. 1977. ("Cochran") p. 8 (stating that sampling theory "attempt[s] to develop methods of sample selection and of estimation that provide, at the lowest possible cost, estimates that are precise enough for our purpose.")1 (emphasis supplied). Spencer admits that conclusions in his rebuttal are based on an
1

As Levy stated in his deposition, one of the issues he considered when selecting his sampling method was the simplicity of certain sample design and exposition. Levy specifically mentioned (Footnote Continued on Next Page.) 7 Case No. 07-CV-01658 PJH (EDL)
NOTICE AND MOT. NO. 6: TO EXCLUDE TESTIMONY OF DEFENDANTS' EXPERT BRUCE SPENCER

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

"implicit assumption" that there was a direct connection between the results of Levy's analysis and the amount of Oracle's damages claim. Sherrod Decl. Ex. B at 93:18-25; see also id. at 26;8-22 ("A. Well, the way it affects -- the way this point that we've been discussing affects my conclusions is that . . . it's plausible to me that the statistics will be used in some way that -- or another to make inferences about the levels of damages. And in that case, the lower bounds could affect money.") Spencer also admitted that he did not know whether his assumption was correct: "Q. You don't know one way or the other [whether `a 25 percent error' would `translate to a lot of money']. A. I don't know what the links are between Levy's statistics and any damages claims." Id. at 94:6-9. Spencer's critique of Levy's methodology is therefore based on insufficient and erroneous facts. See Fed. R. Evid. 702(1) (requiring that expert analysis be "based upon sufficient facts or data"); Fed. R. Evid. 104(b). As Spencer should have known from his reading the reports of Oracle's experts, his conclusion about the purpose of Levy's report was false. Levy's analysis, as noted above, was about liability, not damages. In it he estimated both the frequency with which SAP TN generated fixes through unauthorized cross-use of environments and the number of objects, on average, copied improperly by SAP TN per fix. Sherrod Decl. Ex. C, (Levy Report) at 2 ("Oracle is interested in capturing SAP TN's activity as it related to the use of Objects and Environments.") (footnotes omitted). Levy's report supports liability because cross-use of environments was improper under the relevant software licenses and copying of objects was a copyright violation. Defendants cannot plausibly contend that it makes no difference that Spencer concluded Levy's statistical analysis is part of Oracle's damages proof and was not aware of its role in the

(Footnote Continued from Previous Page.) that he did not use stratified random sampling, as well as other sampling methods, in this case because of "the complexity of explaining it, the potential for more debate about it. I thought a classic, random sampling with replacement would be straightforward, effective to implement, and, as I found ­ as I thought, and as I found later, it has a level of precision that's far greater than needed in this matter." Sherrod Decl., Ex. D, Deposition of Daniel S. Levy ("Levy Depo.") at 128:9-16. 8
Case No. 07-CV-01658 PJH (EDL)

NOTICE AND MOT. NO. 6: TO EXCLUDE TESTIMONY OF DEFENDANTS' EXPERT BRUCE SPENCER

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

liability case. No criticisms in his rebuttal make any sense unless he placed them in the context of the liability proof and the actual purpose of the sampling and extrapolations. Spencer's report and any related testimony and exhibits should be excluded, because the analysis contained in it was predicated upon an erroneous conclusion about the purpose of Levy's analysis. See Fed. R. Evid. 702(1); QR Spex, Inc. v. Motorola, Inc., 2004 WL 5642907 at *9 (C.D. Cal.) (excluding expert report and opinion where expert didn't review relevant underlying evidence); Robinson v. G.D. Searle & Co., 286 F. Supp. 2d 1216, 1221 (N.D. Cal. 2003) (expert testimony inadmissible when based on factual premise directly contradicted by evidence on the record); In Re Brand Name Drugs, 186 F.3d at 788. 2. Spencer's Assertion That Oracle's Burden Of Proof Must Increase As Its Claim For Damages Increases Is An Improper And Unsupportable Legal Argument.

Spencer also seeks to improperly raise Oracle's burden of proof based upon an incorrect legal argument that the requisite burden of proof and statistical precision level in this case depends on the size of Oracle's damages claim. That opinion is dead wrong and would only serve to confuse the jury. See Fed. R. Evid. 403 and 702. Moreover, it would result in a perverse rule that the more damage an infringer wreaks, the higher the burden of proof and level of precision required of the victim. Spencer concludes that Levy's use of any sampling at all was improper because Levy's report contains no cost-benefit analysis to support his decision to analyze a sample of some of the measures rather than forego sampling and count all of the measures for the entire population. Sherrod Decl. Ex. A, Spencer Report at 10. Spencer supports this novel allegation with conjecture: even if individually counting all measures without any sampling would be expensive, "when compared with the significant damages Plaintiffs seek in this case, the cost savings from sampling may be modest by comparison." Id. (emphasis supplied). Based on this conjecture, he concludes that "[i]n the context of damages Plaintiffs seek in this case, I believe that a cost-benefit analysis should have been done." Id. The underpinning to Spencer's sampling conclusion is his belief that the burden of proof is higher in this case due to the amount of money at issue: "Q. So the higher the dollars at stake 9 Case No. 07-CV-01658 PJH (EDL)
NOTICE AND MOT. NO. 6: TO EXCLUDE TESTIMONY OF DEFENDANTS' EXPERT BRUCE SPENCER

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

in the lawsuit, the higher the level of precision in the statistical analysis should be? A. In general, yes." Sherrod Decl. Ex. B (Spencer Depo.) at 84:19-22. This ill-conceived notion that Levy's precision range should rise or fall based on the amount of damages at stake simply does not comport with the law. Obviously, regardless of the size of damage claims (due of course to the harm inflicted by the infringer), "[p]laintiff has the burden of proving damages by a preponderance of the evidence." Ninth Circuit Manual of Model Jury Instructions (Civil), Instructions 5.1 and 5.2. Spencer's misapplication of the law is all the more troubling given his admission that he has no understanding of Oracle's actual burden of proof at trial: "Q. Do you understand that there is a burden of proof in the trial? A. I would expect that there is, yes. Q. Do you know what that burden of proof is? A. Not exactly." Sherrod Decl. Ex. B (Spencer Depo.) at 95:6-14. Because Spencer does not know the burden of proof in a civil matter or that it does not depend on the magnitude of damages, he has misapplied it to both the law and to Levy's report. Based on his misunderstanding, Spencer mistakenly draws the conclusion that Levy's level of statistical precision is too low because it is "quite a large range of uncertainty . . . in a case where the Plaintiffs' allegations place large sums of money at stake." Sherrod Decl. Ex. A (Spencer Report) at 12. Spencer's ignorance as to Oracle's burden of proof renders unreliable his opinions regarding the necessary level of Levy's precision because, by his own admission, he has no basis to form these opinions. Hathaway, 507 F.3d at 318, Abuan v. General Elec. Co., 3 F.3d 329, 332 (9th Cir. 1993) (upholding rejection of expert reports which "failed to comply with relevant legal standards"); Nationwide Transport Finance v. Cass Information Systems, Inc., 523 F.3d 1051, 1055-64 (9th Cir. 2008) (affirming preclusion of expert testimony based on "erroneous or inapplicable legal theories"). More importantly, Levy's 90% confidence interval (or even somewhat wider confidence interval if one assumes all of Spencer's criticisms could be right) is far above the more-likely-than-not 50.1% required by the "preponderance of the evidence" 10 Case No. 07-CV-01658 PJH (EDL)
NOTICE AND MOT. NO. 6: TO EXCLUDE TESTIMONY OF DEFENDANTS' EXPERT BRUCE SPENCER

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

standard. Concrete Pipe and Prod. of Cal., Inc. v. Constr. Laborers Pension Trust for S. Cal., 508 U.S. 602, 622 (1993) ("The burden of showing something by a preponderance of the evidence, the most common standard in the civil law, simply requires the trier of fact to believe that the existence of a fact is more probable than its nonexistence before [he] may find in favor of the party who has the burden to persuade the [judge] of the fact's existence.") (citations omitted). Therefore, Spencer's opinion that "more precision is always better" and thus required in Levy's analysis is speculative, based on incorrect law and if offered would only serve to confuse the trier of fact. See Sherrod Decl. Ex. A (Spencer Report) at 12 ("A small precision range is desirable, and a large precision range is not desirable."). His report should be excluded. See Abuan, 3 F.3d at 332; Nationwide Transport Finance, 523 F.3d at 1055-64 (9th Cir. 2008). 3. Spencer's Testimony About The Precision, Sampling Technique And Sample Sizes Used By Levy Misapplies Statistics, And Its Probative Value Is Outweighed By Unfair Prejudice And Confusion To The Jury.

Spencer compounds his fundamental misunderstandings about his and Levy's opinions by applying those general errors in his specific criticisms of the level of Levy's confidence intervals or precision ranges and Levy's sampling techniques. Spencer's misbelief that the level of statistical precision must be higher than normal in this case corrupts his specific opinions about the requisite precision calculations, the sampling methodology used by Levy and Levy's sample sizes. a. Precision

In statistics, the "precision" of an estimate can be measured by the standard error of the estimate which reflects the probability with which a variable of interest falls within a given range.2 Whenever an expert measures a sample set of data instead of counting the entire population set of data, the expert should report not only the value of the measurement, but also

For a general description of how the measure of standard error relates to the precision of an estimate see Sherrod Decl. ¶ Ex. J, Cochran, W.G., Sampling Techniques. Third Edition. New York, New York: John Wiley & Sons. 1977. ("Cochran") at pp. 25-26. 11
Case No. 07-CV-01658 PJH (EDL)

2

NOTICE AND MOT. NO. 6: TO EXCLUDE TESTIMONY OF DEFENDANTS' EXPERT BRUCE SPENCER

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

the possibility of sample error, usually reported as a standard error or another measure of precision such as the confidence interval associated with results of the calculations, as well as the results themselves. See Sherrod Decl., Ex. K Federal Judicial Center, Reference Manual on Scientific Evidence ("FJC Manual"), 116 (2d ed. 2000).3 Levy complied with these best practices. Whenever he calculated and reported sample measures (or measures that combined both population and sample measures), he also calculated and reported the associated standard errors and/or confidence intervals. Sherrod Decl., Ex. C (Levy Report) at 3-6, and tables 23-34. As discussed above, Spencer mistakenly thought Levy's statistics analyses were being used for damage calculations and then substituted his erroneous hypothesis about the need for greater than normal precision given the magnitude of Oracle's damages claim. Those two general errors spawned numerous specific opinions that Levy's level of precision (i.e. that over 83% of retrofit fixes and over 99% of critical support fixes resulted from cross-use of environments, with a 90% confidence interval and a 50% precision range) was not precise enough. For example, Spencer repeatedly asserted in his expert report that, because the potential cost to Defendants (in the form of damages caused by them) related to any measurement error was so high, Levy should have made extraordinary efforts to be as precise as possible. See, e.g., Sherrod Decl. Ex. A, (Spencer Report) at 13 (asserting that the chosen precision and confidence interval "is quite a large range of uncertainty to deliberately aimed for in a case where Plaintiffs [sic] allegations placed large sums of money at stake . . . ."); id. (asserting that there are "financial consequences [] of sampling error"); id. at p. 44 ("[T]he smaller the sample size the greater the chance for a large error unfavorable to Defendants"); see also id. at p. 12 ("A small precision range is desirable, and a large precision range is not desirable."); id. at p. 39 (same). Every one of these assertions must be excluded because each one is predicated on the false assumptions that Levy's analysis affected damages and Levy was required to increase his precision range due to the amount at stake. Indeed, at deposition, Spencer admitted that he could
3

In statistics, the term "error" refers to the uncertainty with which the data are reported, rather than to a mistake made by the practitioner. 12
Case No. 07-CV-01658 PJH (EDL)

NOTICE AND MOT. NO. 6: TO EXCLUDE TESTIMONY OF DEFENDANTS' EXPERT BRUCE SPENCER

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

not support his criticisms regarding the precision of Levy's measures. See, e.g., Sherrod Decl. Ex. B, (Spencer Depo.) at 253:14-25 (admitting that "[w]ithout knowing the purposes of the case or having seen how the numbers are going to be used, I can't answer" whether or not Levy's "precision ranges are precise enough"). Spencer's testimony and exhibits should be excluded under Fed. R. Evid. 702 and 403, because testimony or evidence based on false assumptions cannot be the result of reliable application of principles and methods, has no probative value, would be unfairly prejudicial and would be likely to confuse the jury. Ravel, 930 F.2d at 726. b. Sampling "With" Versus "Without" Replacement

In a similar vein, Spencer categorically rejected Levy's use of the sampling with replacement methodology,4 again based on Spencer's erroneous belief about the requisite degree of precision. See Sherrod Decl. (Spencer Report) at 18-19 & nn.29-30 (characterizing sampling with replacement as "inferior" and "inadmissible" because the precision of sampling without replacement is never less than the precision of sampling with replacement for a fixed sample size). Again, Spencer admitted at deposition that he could not support his criticism of Levy's use of sampling with replacement. See Spencer Depo. at 122:15-21 (stating, when asked whether sampling with replacement was not precise enough for Levy's purposes, "I don't know how the statistics are going to be used, and so that's why I can't answer that question."). The statistics text championed by Spencer, Sampling Techniques by W.G. Cochran, also states that sampling with replacement methodology may be appropriate, depending on the purpose of a particular study.5 Spencer's absolutist rejection of sampling with replacement is unsupported in the

In contrast to sampling without replacement, sampling with replacement means that "at any draw, all N members of the population are given an equal chance of being drawn, no matter how often they have already been drawn." Sherrod Decl. Ex. J, (Cochran) at 18. 5 "Random sampling with replacement is entirely feasible: at any draw, all N members of the population are given an equal chance of being drawn, no matter how often they have already been drawn. The formulas for the variances and estimated variances of estimates made from the sample are often simpler when sampling is with replacement than when it is without replacement. For this reason sampling with replacement is sometimes used in the more complex sampling plans, although at first sight there seems little point in having the same unit two or more times in the sample." Sherrod Decl. Ex. J , (Cochran) at 18. 13
Case No. 07-CV-01658 PJH (EDL)

4

NOTICE AND MOT. NO. 6: TO EXCLUDE TESTIMONY OF DEFENDANTS' EXPERT BRUCE SPENCER

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

statistical literature,6 and was belied by Spencer's concession at deposition. Sherrod Decl., Ex. B (Spencer Depo.) 122:15-25, 123:1-3. It should be excluded. c. Sample Sizes

Spencer also criticizes the sample sizes used by Levy. His assertions that precision is the only important consideration in determining appropriate sample size is similarly objectionable, because the only proffered support is, again, a provably false hypothesis. Increasing sample size tends to increase both precision and increase cost. See Sherrod Decl. Ex. K, FJC Manual at 118 n.116 ("standard error goes down as sample size goes up"); Sherrod Decl. Ex. J (Cochran), at 1 ("If data are secured from only a small fraction of the aggregate, expenditures are smaller than if a complete census is attempted.") As discussed above, Spencer relied upon his two erroneous damages assumptions to imagine a cost-benefit framework where any decrease in precision would unfairly shift the burden of proof to Defendants. See Sherrod Decl., Ex. A (Spencer Report) at 44 (accusing Levy of "shift[ing] some of the burden from sampling error onto Defendants"); id. at 10 (suggesting that Levy should have performed a cost-benefit analysis, and further suggesting that such an analysis "might have led to much-increased sample sizes, [and possibly] dispensing with sampling altogether").7 Spencer's speculation that a cost-benefit analysis "might have led" to large sample sizes or eliminated the need for any sampling is improper. Even if a cost-benefit analysis were appropriate (and it is not), Spencer himself did not perform any such analysis (nor could he because he did not know the purpose for which Levy's report would be used)8. Thus, his testimony should be excluded under Fed. R. Evid. 702

Furthermore, because sampling with replacement is simpler in some aspects, it may be easier to explain to a jury. See Sherrod Decl. Ex. K, FJC Manual at 117 n.113 (recognizing a "trade-off between accuracy and simplicity" in study design). It is also telling that, without disclosing it in his supplemental rebuttal materials, Spencer reduced his own bootstrap sample size from 10,000 to 5,000 due to time and resource constraints. See Spencer Depo. at 289:24-290:7. 8 Spencer also ignores the fact that for all fully enumerated measures the precision of the calculation is exact; there is no possibility of any sampling error. In addition, for many measures estimated by Levy the level of precision achieved is substantially greater than originally targeted. Levy Report, p. 30 14
Case No. 07-CV-01658 PJH (EDL)
7

6

NOTICE AND MOT. NO. 6: TO EXCLUDE TESTIMONY OF DEFENDANTS' EXPERT BRUCE SPENCER

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

and 403. Domingo ex rel. Domingo v. T.K., 289 F.3d 600, 607 (9th Cir. 2002) ("The reasoning between steps in a theory must be based on objective, verifiable evidence and scientific methodology.") (emphasis supplied). Further, Spencer's position on sample sizes is unreasonable, given the state of the discovery record and the complexities associated with actual counts of the massive data produced by SAP TN. Both parties have independently concluded that mere collection of the data underlying Levy's analysis for the over 1600 fixes created by SAP TN would require many thousands of hours. See Sherrod Decl., Ex. C (Levy Report) at 13 n.33; Sherrod Decl., Ex. E, TomorrowNow, Inc.'s Third Amended and Supplemental Response to Plaintiff Oracle USA, Inc.'s Second Set of Interrogatories, Interrogatory No. 14(A) (stating that collection of comparable data for five PeopleSoft and five JD Edwards fixes took in excess of 500 hours). Judge Laporte repeatedly urged the parties to stipulate to the use of statistical sampling due to the volume of data at issue in this matter. See e.g. Transcript of Proceedings Held on 07/01/08 before Elizabeth D. Laporte, Dkt. 105 (urging the parties to "limit discovery, limit the expense, come up with a fair extrapolation, whether it's perfect or not."); (Tr. of Discovery Hearing before Magistrate Judge Laporte, February 13, 2009, at 17:22-18:8) (noting that stipulation to an extrapolation proposal was "a suggestion that I've made for eight months"). On this record, Spencer's criticism of Levy's use of samples that would likely achieve a 90% confidence interval with a 50% precision range is based on "flaws of logic and statistical analysis" that are objectionable under Fed. R. Evid. 403. In sum, Spencer's criticisms about sample sizes amount to an attack on Oracle's choice to use any sampling at all, and the unfair prejudice resulting from that attack substantially outweighs any probative value. See Sherrod Decl. Ex. A at 11; see also id. at 13 (characterizing 90% confidence interval with a 50% precision as a "meager standard").9

9

Spencer's suggestion that Oracle requested 50% precision as to every measure is incorrect, and will further confuse the jury. See Sherrod Decl. Ex. A at 12-13 (stating that Levy did not achieve his desired precision because measures other than measure 116 had precision of greater than 50%; see also id. at 39-40 (discussing same). 15
Case No. 07-CV-01658 PJH (EDL)

NOTICE AND MOT. NO. 6: TO EXCLUDE TESTIMONY OF DEFENDANTS' EXPERT BRUCE SPENCER

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

C.

Spencer Fails To Employ Reliable Principles And Methods In His Critique Of Levy's Use Of Microsoft Excel To Create Random Samples. 1. Spencer's Suggestions That Levy Misused Or Even Lied About Using Excel To Generate Sample Order Are Wholly Improper.

Spencer also attacks Levy's use of a Microsoft Excel random number generator to create sample sets of random numbers. He asserts, without citation and without checking to see if Levy's use of Excel produced random samples, that Levy was required to use some different randomizing program that produces a "random seed" that would record the selections. See, e.g., Sherrod Decl., Ex. A (Spencer Report) at 8 (accusing Levy of "failing to adequately document the process by which he selected his samples"); id. At 14 ("[I]t is standard practice for the statistician to select the sample in such a way that the sample selection can be reproduced."). Spencer is wrong for at least two reasons. First, Levy did document his sample by providing the list of items that were selected, and second, publications in respected journals often do not provide the level of detail, such as programs and original random seeds used in sample selection. See Declaration of Daniel Levy ("Levy Decl.") at ¶¶ 9-15, Exs. C, D, E, and G. In fact, Levy's documentation sample selection process goes beyond what is often found in scholarly texts and journals.10 See e.g. Sherrod Decl., Ex. I, Macfie B.P. and M.N. Nufrio, Levy's description in his report for how the samples were selected is as follows: "1. There were 223 Fixes in the Retrofit population and 1,386 Fixes in the Critical Support population. Within each population, each Fix was assigned an item number sequentially. 2. Using a random number generator, I selected numbers ranging from 1 to 223 for Retrofit and from 1 to 1,386 for Critical Support, and I generated the random sample for each population... 3. The sampling numbers were then given to Mr. Mandia to collect the relevant data for each Fix in the sample." A book that is used in the Statistics department at Northwestern, Statistical Methods for the Social Sciences, describes the sampling process as follows: "The most common method for selecting a random sample is to (1) number the subjects in the sampling frame, (2) generate a set of these numbers randomly, and (3) sample the subjects whose numbers were generated." Sherrod Decl. Ex. H, Agresti A. and B. Finlay, Statistical Methods for the Social Sciences, Upper Saddle river, New Jersey: Prentice Hall, 2009 (Agresti) at 15; Levy's processes are described in CASRO Code of Standards and Ethics for Survey Research Industry Groups lists this as the process in the section "Responsibilities in Reporting to Clients and the Public," "2. The purpose of the study, including the specific objectives. 3. The dates on or between which the data collection was done. 4. A definition of the universe that the survey is intended to represent and a description of the population frame(s) that was actually sampled. 5. A description of the sample design, including the method of selecting sample elements, method of interview, cluster size, number of callbacks, Respondent eligibility or screening criteria, and (Footnote Continued on Next Page.) 16 Case No. 07-CV-01658 PJH (EDL)
NOTICE AND MOT. NO. 6: TO EXCLUDE TESTIMONY OF DEFENDANTS' EXPERT BRUCE SPENCER
10

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

Applied Statistics for Public Policy, Armonk, New York: M.E. Sharpe, 2006, ("Macfie") at 490, also cited in Expert Report of Mr. Clarke, discusses random sampling methods such as the Vietnam era random selection for military service, where numbers were drawn out of a bowl. It does not discuss any need to use a random seed. Levy Decl. at ¶ 13. This testimony and any related exhibits reflects unsupported assertions rather than reliable principles and methods. Fed. R. Evid. 702(2). In contrast, regarding his colleague Dr. Vandaele's sample selection process on which he relies, Spencer does not know how Vandaele chose the random seed for random sampling: "Q: do you know how Dr. Vandaele chose his seed? A: No, I do not." Sherrod Decl., Ex. B, (Spencer Depo.) 138:1-3. Given the importance that Spencer purports to give to the use of (and documentation resulting from) a random seed, it is telling that he has no idea how Vandaele developed his random samples or whether he chose multiple random seeds prior to settling on the one he finally used. Finally, Spencer goes one step further in evaluating Levy's report, and takes the curious position that Levy's use of Excel to generate random numbers somehow suggests that Levy lied in his report, and did not use random numbers at all to generate the sample order. See Sherrod Decl. Ex. B (Spencer Report) at 2 (stating that use of Excel, which does not use a recordable random seed, "makes it difficult to confirm that the samples were randomly selected as Levy claims.") (emphasis supplied); id. at 14 ("There are serious questions about whether the samples put forth by Levy are proper generated random samples of obtained from Excel."). Any such testimony is objectionable on two grounds. First, questions of witness credibility are reserved to the finder of fact, and are not permissible subjects for expert testimony. United States v. Barnard, 490 F.2d, 907, 912 (9th Cir. 1973) (credibility "is for the jury"). Second, as discussed above, Spencer's assumptions about standard industry practice are incorrect, and as such are not the product of reliable principles and methods. Fed. R. Evid. 702(2). Spencer's aspersions must (Footnote Continued from Previous Page.) other pertinent information." 17
Case No. 07-CV-01658 PJH (EDL)

NOTICE AND MOT. NO. 6: TO EXCLUDE TESTIMONY OF DEFENDANTS' EXPERT BRUCE SPENCER

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

be excluded because they lack probative value and are extremely prejudicial. Fed. R. Evid. 403. 2. Spencer's Calculations Regarding The Randomness Of Levy's Samples Are Erroneous, And Thus Are Not The Result Of The Application Of Reliable Principles And Methods.

In fact, as to the issue of the randomness of Levy's samples, well-defined statistical tests called "hypothesis tests" exist that allow one to determine whether a set of numbers is random and therefore likely to have occurred by chance. A typical scientific standard declares an occurrence statistically significant and therefore probably not random because it would only have been likely to occur by random chance 1 or 5 out of 100 times (statisticians call these cut offs alpha-levels or -levels). Sherrod Decl., Ex. H , Agresti A. and B. Finlay, Statistical Methods for the Social Sciences, ("Agretsi") Upper Saddle River, New Jersey: Prentice Hall, 2009, p. 154 ("In practice, the most common -levels are 0.05 and 0.01"). Based on the characteristics that Spencer used as indicators of randomness, any pair of randomly drawn samples has just under a one out of five chance (0.19) to have occurred by chance and therefore is well above the test for randomness discussed above. Samples that would occur 19% of the time in repeated random samples are not considered rare based on the .01 or .05 -levels. Accordingly, if Spencer had based his conclusion about randomness on a standard hypothesis test using generally accepted scientific methods, he would have had to accept the conclusion that Levy generated his samples through a random process. Indeed, when presented with a hypothetical of that type of test at deposition, Spencer confirmed the calculation. Sherrod Decl., Ex. B (Spencer Depo.) 240:4-14 ("Q. Okay. Let me ask you this more simply: What is the probability of A or B occurring if the probability of A is 10 percent and the probability of B is 10 percent? A: Well, it's the sum of the probabilities minus the probability of the joint event. So it would be -- if my mental arithmetic is right, it would be 19 percent.) [Objections omitted]. Instead of running these standard scientific hypothesis tests, Spencer ran an ad hoc test based on some calculations combined with his individual impressions. In combination, they are not replicable tests that would allow all or even any other scientist to reach the same conclusions as Spencer. Sherrod Decl. Ex. B , (Spencer Depo.) 152:18-25 and 153:1-6; Sherrod Decl., Ex. A (Spencer Report) at 15 & Appx. 4 (describing the ad hoc test); Levy Decl., ¶¶ 4-8. Spencer's 18 Case No. 07-CV-01658 PJH (EDL)
NOTICE AND MOT. NO. 6: TO EXCLUDE TESTIMONY OF DEFENDANTS' EXPERT BRUCE SPENCER

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

failure to use reliable principles and methods in testing the randomness of the sample is yet another reason his testimony and exhibits must be excluded. Finally, in his deposition, Spencer admitted that his test did not, in fact, prove the claim he made in his report: "Q. Based on the discrepancy analysis, you don't know whether Levy's random set of numbers are random or not? A: The discrepancy analysis by itself does not prove with absolute certainty that Levy's sample was selected in a manner other than how he claims;" "Q. On the basis of the discrepancy of 8 you find for Mr. Levy's sample, you cannot say on that basis alone that his sample is not random? A: On the basis of that alone, I do not say that his sample is not random. [Objections omitted]. Sherrod Decl., Ex. A (Spencer Depo.) 150:2-10, 151:12-18. D. Spencer Failed To Reliably Apply Appropriate Principles And Methods When Analyzing Levy's Methods And, When Tested, Spencer's Theoretical "Corrections" Have No Effect On Levy's Results.

In a number of instances, Spencer's criticisms either are unfounded on general principles of statistics or make no difference in Levy's results or both. Unfounded and/or irrelevant opinions must be excluded. 1. Portions Of Spencer's Report Are Based On "Implicit Calculations" Which Are Not Verifiable Or Replicable And Are Therefore Unreliable.

Spencer admitted at deposition that some of his criticisms were simply based on eyeballing numbers rather than any legitimate analysis. For example, his evaluation of the degree of lower variance11 using stratification was based on an "implicit calculation" which is not verifiable and has no basis in scientific methodology: Q. Have you calculated what the lower variance would be for using a stratified sample? A. I did not do an explicit calculation. Q. Did you do an implicit calculation? Well first of all, what is an implicit The variance Spencer is referring to is the variance of the estimated means of the sampled measures. This variance is a reflection of the precision with which the means are reported. More precisely, a definition of the variance of the mean can be found in Sherrod Decl. Ex. J (Cochran) at 23-24. 19
Case No. 07-CV-01658 PJH (EDL)
11

NOTICE AND MOT. NO. 6: TO EXCLUDE TESTIMONY OF DEFENDANTS' EXPERT BRUCE SPENCER

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

calculation? A. An implicit calculation is, I look at some numbers and get a sense and do a calculation in my head and say it's about such-and-such. Sherrod Decl., Ex. B (Spencer Depo.) at 223:5-8 (emphasis supplied). Similarly, Spencer's evaluation of Levy's alleged "underestimation" and the application to sampled measures, which are measures for which data was collected only for the fixes in the random samples, is based entirely on an unwritten, unknown analysis: "The analysis I just described is one that I did not even write down. It was one that I have looked at the tables, saw the underestimation, and thought that it could also apply to the sampled measures." Id. at 215:20-25. It is impossible for Oracle to know just how many of Spencer's criticisms and conclusions are based on his "implicit calculations" about "such-and-such." Spencer's testimony and report is mere ipse dixit, and it should be excluded because it is unreliable under Fed. R. Evid. 702 . Daubert, 509 U.S. at 589, 595; Joiner, 522 U.S. at 146. 2. Spencer's View That Levy Should Have Used The Student's t Distribution Instead Of The Normal Distribution Is Wrong And Would Have Made No Difference To Levy's Analysis.

Spencer's theoretical discussion of the advantages of the student's t distribution and his claims that Levy should have used that distribution methodology are also objectionable. Spencer asserts that Levy's use of an alternative methodology, the normal distribution, is improper, and does so by claiming that Levy has made a mistake that high school students or college freshmen would not have made. See Sherrod Decl., Ex. A (Spencer Report) at 26-27 & nn. 49-50. However, a statistical text used in courses listed in Spencer's own statistics department at Northwestern teaches that there is no expected difference between the student's t distribution and the normal distribution for more than 30 observations, as is true for Levy's samples. Sherrod Decl. Ex. H, (Agresti), 119 ("When df is about 30 or more, the two distributions are nearly identical.") Moreover, Levy provided calculations that show his results using the student's t distribution are in fact immaterially different from those produced using the normal distribution. Sherrod Decl., Ex. F (ORCLX-AACG-000020). Spencer's own work essentially confirms that 20
Case No. 07-CV-01658 PJH (EDL) NOTICE AND MOT. NO. 6: TO EXCLUDE TESTIMONY OF DEFENDANTS' EXPERT BRUCE SPENCER

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

conclusion. In attempting to show the importance of the use of the t- distribution, Spencer stated: "[f]or example, when the confidence intervals were based on the student's t distribution instead of the normal distribution as Levy did, their average coverage probability increased from 83.8% to 84.4% for the Retrofit and increased from 88.1% to 88.2% for the Critical Support." Sherrod Decl., Ex. A, (Spencer Report) at 29. Spencer appears to believe that these extremely small changes in probability would make some difference in this case. They clearly do not. All of the related confidence intervals are well above the legal standards and have no practical impact on Levy's results or conclusions. Again Spencer's de minimus ipse dixit is not proof to the contrary. See, e.g., Sherrod Decl. Ex. B, Spencer Depo, at 251:23-252:3 ("Q. So your inclination has no basis? Right? A. My inclination is borne of my experience as a practicing statistician."). They are therefore simply irrelevant to this case. Because Spencer's theoretical critique was neither supported by the literature nor true in practice, it is not the result of application of reliable principles and methods under Fed. R. Evid. 702(2)-(3). 3. Spencer's Measurement Of Skewness As To Population Measures Has No Bearing On The Sample Measures.

In analyzing Levy's samples, Spencer performed some measurements of the skewness of certain fully enumerated measures.12 Spencer asserted in his report that analysis of these