"The Apple iPod iTunes Anti-Trust Litigation"

Filing 768

RESPONSE (re 763 Administrative Motion to File Under Seal [Plaintiffs' Reply Memorandum in Support of Daubert Motion to Exclude Certain Opinion Testimony of Kevin M. Murphy and Robert H. Topel and Exhibit 1] ) filed byApple Inc.. (Attachments: # 1 Proposed Order Granting Plaintiffs' Administrative Motion to Seal, # 2 Declaration of David C. Kiernan ISO Apple's Response, # 3 Exhibit 1-3 to Kiernan Declaration, # 4 Proposed Redactions to Plaintiffs' Reply Memorandum)(Kiernan, David) (Filed on 2/4/2014)

1 ROBBINS GELLER RUDMAN
& DOWD LLP
2 BONNY E. SWEENEY (176174)
THOMAS R. MERRICK (177987)
3 ALEXANDRA S. BERNAY (211068)
CARMEN A. MEDICI (248417)
4 JENNIFER N. CARINGAL (286197)
655 West Broadway, Suite 1900
5 San Diego, CA 92101
Telephone: [phone]
6 619/231-7423 (fax)
[e-mail]
7 [e-mail]
[e-mail]
8 [e-mail]
[e-mail]
9
Class Counsel for Plaintiffs
10
[Additional counsel appear on signature page.]
11
UNITED STATES DISTRICT COURT
12
NORTHERN DISTRICT OF CALIFORNIA
13
OAKLAND DIVISION
14
THE APPLE IPOD ITUNES ANTI-TRUST ) Lead Case No. C-05-00037-YGR
15 LITIGATION
)
) CLASS ACTION
16
)
) PLAINTIFFS’ REPLY MEMORANDUM IN
This Document Relates To:
17
) SUPPORT OF DAUBERT MOTION TO
) EXCLUDE CERTAIN OPINION
ALL ACTIONS.
18
) TESTIMONY OF KEVIN M. MURPHY
AND ROBERT H. TOPEL
19
DATE:
TBD
20
TIME:
TBD
CTRM:
5, 2nd Floor
21
JUDGE:
Hon. Yvonne Gonzalez Rogers
22
UNREDACTED VERSION OF DOCUMENT SOUGHT TO BE SEALED IN ENTIRETY
23
24

[APPLE'S PROPOSED REDACTIONS]

25
26
27
28

912519_1

1

TABLE OF CONTENTS

2

Page

3 I.

INTRODUCTION ...............................................................................................................1

4 II.

CLUSTERING DOES NOT FIT THE FACTS OF THE CASE.........................................2

5

A.

There Is No Clustering Problem Among iPod Transactions ...................................2

6

B.

Apple’s Pricing Practices Do Not Give Rise to Correlation of the
Residuals ..................................................................................................................3

C.

Apple’s “Test” for Correlation of the Residuals Is Flawed .....................................4

D.

Apple’s “Omitted Variables” Argument Does Not Support Clustering ..................5

E.

Through the Construction of Arbitrary Clusters, Apple’s Experts
Artificially Introduce Correlation into the Error Residuals .....................................7

11

F.

Clustering Where Unnecessary Is Harmful .............................................................9

12

G.

The Number of Observations Per Cluster Distorts the Standard Error..................10

13 III.

APPLE’S ATTACKS ON PROFESSOR WOOLDRIDGE ARE UNFOUNDED ...........11

14

A.

Professor Wooldridge’s Opinions Are Supported by Generally Accepted
Econometrics and Wooldridge’s Own Peer-Reviewed Publications and
Research and Were Not Manufactured for This Litigation ...................................11

16

B.

Plaintiffs Timely Disclosed Wooldridge ...............................................................12

17 IV.

CONCLUSION ..................................................................................................................14

7
8
9
10

15

18
19
20
21
22
23
24
25
26
27
28
912519_1

PLAINTIFFS’ REPLY MEMO IN SUPPORT OF DAUBERT MOTION TO EXCLUDE CERTAIN
OPINION TESTIMONY OF KEVIN M. MURPHY AND ROBERT H. TOPEL - C-05-00037-YGR

-i-

1

TABLE OF AUTHORITIES

2

Page

3 CASES
4 Baxter Healthcare Corp. v. Fresenius Med. Care Holding, Inc.,
No. C 07-1359, 2009 U.S. Dist. LEXIS 32380
5
(N.D. Cal. Apr. 2, 2009) ..........................................................................................................13
6

Cabreara v. Cordis Corp.,
134 F.3d 1418 (9th Cir. 1998) .................................................................................................11
7
8 Daubert v. Merrell Dow Pharms.,
509 U.S. 579, 113 S. Ct. 2786, 125 L. Ed. 2d 469 (1993) ............................................... passim
9
FTC v. Wellness Support Network, Inc.,
10
No. 10-cv-04879-JCS, 2013 U.S. Dist. LEXIS 144140
(N.D. Cal. Oct. 4, 2013) .........................................................................................................1, 2
11
12 In re Enron Corp. Sec., Derivative & ERISA Litig.,
MDL No. 1446, 2007 U.S. Dist. LEXIS 98619
(S.D. Tex. Feb. 1, 2007)...........................................................................................................14
13
14 In re Paoli R.R. Yard PCB Litig.,
35 F.3d 717 (3d Cir. 1994).......................................................................................................13
15
Jeffries v. Centre Life Ins. Co.,
16
No. 1:02-cv-351, 2004 U.S. Dist. LEXIS 30769
(S.D. Ohio Jan. 28, 2004) ........................................................................................................12
17
18 Lust by & Through Lust v. Merrell Dow Pharms., Inc.,
89 F.3d 594 (9th Cir. 1996) .....................................................................................................11
19
Moore v. Napolitano,
20
926 F. Supp. 2d 8 (D. D.C. 2013) ............................................................................................12
21 Nightlight Sys. v. Nitelites Franchise Sys.,
No. 1:04-CV-2112, 2007 U.S. Dist. LEXIS 95538
22
(N.D. Ga. May 11, 2007) .........................................................................................................13
23
Reed v. Smith & Nephew, Inc.,
24
527 F. Supp. 2d 1336 (W.D. Okla. 2007) ................................................................................12
25 Wendt v. Host Int’l,
125 F.3d 806 (9th Cir. 1997) ...................................................................................................13
26
27
28
912519_1

PLAINTIFFS’ REPLY MEMO IN SUPPORT OF DAUBERT MOTION TO EXCLUDE CERTAIN
OPINION TESTIMONY OF KEVIN M. MURPHY AND ROBERT H. TOPEL - C-05-00037-YGR

- ii -

1
2

Page

3
STATUTES, RULES AND REGULATIONS
4
5
6

Federal Rules of Civil Procedure
Rule 26(a)(2) ............................................................................................................................12
Rule 37(c)(1) ............................................................................................................................13

7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
912519_1

PLAINTIFFS’ REPLY MEMO IN SUPPORT OF DAUBERT MOTION TO EXCLUDE CERTAIN
OPINION TESTIMONY OF KEVIN M. MURPHY AND ROBERT H. TOPEL - C-05-00037-YGR

- iii -

1 I.

INTRODUCTION

2

In their Daubert Motion to Exclude Certain Opinion Testimony of Kevin M. Murphy and

3 Robert H. Topel (“Plaintiffs’ Daubert Motion” or “Pltfs’ Mem.”), Plaintiffs demonstrated that
4 Apple’s experts’ clustering opinions should be excluded because they are not “sufficiently tied to the
5 facts of the case that will assist the jury in resolving a factual dispute.” Daubert v. Merrell Dow
6 Pharms., 509 U.S. 579, 591, 113 S. Ct. 2786, 125 L. Ed. 2d 469 (1993). Relying on the expert
7 opinions of Professors Noll and Wooldridge,1 Plaintiffs showed that the “clustering adjustment” that
8 Murphy and Topel apply to Professor Noll’s regressions is both inappropriate and harmful.
9 Clustering grossly exaggerates the standard errors, leading to the incorrect conclusion that Professor
10 Noll’s regression results are not statistically significant.2 Pltfs’ Mem. at 12; Sweeney Daubert Decl.,
11 Ex. 1 (Wooldridge Decl.) at 5; Sweeney Reply Daubert Decl., Ex. 1 (Wooldridge Supp. Decl.) at 1
12 26, 32-33.
13

In response, Apple merely repeats its experts’ discredited theories and attacks Professor

14 Wooldridge by distorting the record. But Apple cannot escape the fact that its experts’ litigation15 driven clustering opinions have no place in this case. As explained by Professors Noll and
16 Wooldridge in their supplemental reports and confirmed through theoretical calculations, statistical
17 tests and simulations, Murphy and Topel’s clustering adjustment is arbitrary, improperly distorts the
18 standard errors, and should be excluded. See FTC v. Wellness Support Network, Inc., No. 10-cv19

1

Despite Apple’s contention to the contrary, Plaintiffs do not rely solely on Professor
Wooldridge. They also rely on Professor Noll’s opinions (see, e.g., Pltfs’ Mem. at 2 n.3 (citing Noll
20 Rebuttal at 8); id. at 4 n.7 (citing Noll Rebuttal at 5-6); id. at 9 (citing Noll Rebuttal at 10); id. (citing
21 Noll Rebuttal at 9, 36-39); id. at 12 (citing Noll Rebuttal at 43-44, 47)) and undisputed factual
evidence about the data sets used in the regressions. See Pltfs’ Mem. at 8 n.11 (quoting Topel:
22 “The data came out pretty neat the way Professor Noll had it, so we’re going to go with that.”).
2
references to (“Sweeney Daubert Decl.”) are to the Declaration of
Sweeney in
23 SupportAll Plaintiffs’ Daubert Motion to Exclude Certain Opinion TestimonyBonny E. M. Murphy
of
of Kevin
dated
(“Kiernan
24 and Robert H. Topel,David December 20, 2013; all references toOpposition Daubert Decl.”) are to
the Declaration of
C. Kiernan in Support of Apple’s
to Plaintiffs’ Daubert

Motion, dated
13, 2014; all
to (“Sweeney Reply Daubert Decl.”) are to
25 Declaration of January E. Sweeney inreferencesof Plaintiffs’ Reply Memorandum in Supportthe
Bonny
Support
of
Daubert Motion to Exclude Certain Opinion Testimony of Kevin M. Murphy and Robert H. Topel,
26 filed concurrently herewith; and all references to (“Sweeney MSJ Opp. Decl.”) are to the
Sweeney
of Plaintiffs’ Memorandum of Law in Opposition to
27 Declaration of Bonny E. Summary in Support and to Exclude Expert Testimony of Roger G. Noll,
Defendant’s Motion for
Judgment
28 dated January 13, 2014.
912519_1

PLAINTIFFS’ REPLY MEMO IN SUPPORT OF DAUBERT MOTION TO EXCLUDE CERTAIN
OPINION TESTIMONY OF KEVIN M. MURPHY AND ROBERT H. TOPEL - C-05-00037-YGR

-1-

1 04879-JCS, 2013 U.S. Dist. LEXIS 144140, at *32 (N.D. Cal. Oct. 4, 2013) (excluding expert
2 testimony for lack of “fit”).
3 II.

CLUSTERING DOES NOT FIT THE FACTS OF THE CASE

4

Despite Apple’s protestations, ample “real-world facts” and “real-world evidence” support

5 Plaintiffs’ claim that Apple’s disabling of Harmony through its 7.0 updates enhanced and maintained
6 Apple’s monopoly power and enabled Apple to sell iPods at supracompetitive prices. See Dkt. No.
7 751-3 (Plaintiffs’ Memorandum of Law in Opposition to Defendant’s Motion for Summary
8 Judgment and to Exclude Expert Testimony of Roger G. Noll) at 8-9.
9
10
11
12

Recognizing that Professor Noll’s credentials and regression methodologies have been

13 widely accepted, Apple aims its principal attack at the statistical significance of Professor Noll’s
14 results, claiming that he should have clustered the standard errors. According to Apple, after
15 “correcting” for clustering, the statistical significance of Noll’s regressions disappears, and damages
16 – conveniently – are reduced to zero. But this argument has no basis in economics or the facts of
17 this case.
18

A.

19

As an initial matter, Apple’s claim that the data observations used in Professor Noll’s

There Is No Clustering Problem Among iPod Transactions

20 regressions are not independent lacks merit. As Apple’s own expert concedes, if the transactional
21 data “were truly independent, then you wouldn’t have to cluster.” Sweeny MSJ Opp. Decl., Ex. 52
22 (1/8/14 Topel Dep.) at 242:5-7. That is exactly the situation here. Each transaction involved a
23 different buyer making an independent decision about whether to make a purchase and, if so, how
24 many units to buy. As a result, each is properly regarded as an independent event. Sweeney
25 Daubert Decl., Ex. 4 (Noll Rebuttal) at 9, 36; id., Ex. 1 (Wooldridge Decl.) at 8 (“If one properly
26 views a transaction as a unit of observation, as is done by Professor Noll, the data need not be treated
27 as a cluster sample. In particular, the clustering procedure suggested by Professors Murphy and
28 Topel . . . is inappropriate.”)
912519_1

PLAINTIFFS’ REPLY MEMO IN SUPPORT OF DAUBERT MOTION TO EXCLUDE CERTAIN
OPINION TESTIMONY OF KEVIN M. MURPHY AND ROBERT H. TOPEL - C-05-00037-YGR

-2-

1

Equally important, clustering is not appropriate here because Professor Noll analyzed the

2 entire population of Apple’s transactional data, not just a sample. Sweeney MSJ Opp. Decl., Ex. 3
3 (Noll Supp. Rebuttal) at 3. Clustering problems arise when dealing with data drawn from clustered
4 samples, not when dealing with a randomly-drawn sample or the whole population.3 Id.; Sweeney
5 Daubert Decl., Ex. 4 (Noll Rebuttal) at 10; Sweeney MSJ Opp. Decl., Ex. 3 (Noll Supp. Rebuttal) at
6 3-7; Sweeney Daubert Decl., Ex. 1 (Wooldridge Decl.) at 10; Sweeney Reply Daubert Decl., Ex 1
7 (Wooldridge Supp. Decl.) at 11-27, 32-33.
8

B.

Apple’s Pricing Practices Do Not Give Rise to Correlation of the
Residuals

9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
3
As described in Thompson (2012), a cluster sample is one in which a simple random sample
26 of primary units is taken from the population of primary units. Sweeney Reply Daubert Decl., Ex. 1
(Wooldridge Supp. Decl.) at 11. Without cluster samples there can be no clustering problems.
27
4
See Def.’s Mem. at 9 n.12.
28
912519_1

PLAINTIFFS’ REPLY MEMO IN SUPPORT OF DAUBERT MOTION TO EXCLUDE CERTAIN
OPINION TESTIMONY OF KEVIN M. MURPHY AND ROBERT H. TOPEL - C-05-00037-YGR

-3-

1

C.

2

Lacking any economic or factual support for its clustering adjustment, Apple claims Murphy

Apple’s “Test” for Correlation of the Residuals Is Flawed

3 and Topel have demonstrated, using “standard tests,” that the residuals are “highly correlated” at the
4 family and quarter level. Def.’s Mem. at 10. But as Professor Noll concluded in his Supplemental
5 Report, those “tests” are “unreliable because the groups into which [Murphy and Topel] divide the
6 observations are artificially created.” Sweeney MSJ Opp. Decl., Ex. 3 (Noll Supp. Rebuttal) at 5.
7 To demonstrate this artifice, Professor Noll used Murphy and Topel’s testing method, but divided
8 the data by different time periods (family/month and family/week instead of family/quarter). The
9 results were “qualitatively identical,” demonstrating that the correlations that Murphy and Topel
10 allegedly identified are “not due to the standard problem of cluster samples . . . but are the expected
11 result of dividing the residual errors into a large number of arbitrarily selected groups.” Sweeney
12 MSJ Opp. Decl., Ex. 3 (Noll Supp. Report) at 6 & Exhibits 1a-b & 2a-b.5
13

Professor Wooldridge’s simulations confirm that Apple’s experts’ “exercise is entirely

14 predictable, and tells us nothing about whether clustering is needed.” Sweeney Reply Daubert
15 Decl., Ex. 1 (Wooldridge Supp. Decl.) at 25. Indeed, their test demonstrates why clustering is
16 inappropriate in this case and leads to artificial inflation of the standard errors:
17

Not only do Professors Murphy and Topel provide no useful evidence in
favor of clustering, their exercise actually supports the simulation evidence above
that clustering badly overestimates the true precision in the estimates. Ironically, by
extension, the Murphy-Topel analysis actually explains why their cluster-robust
standard errors in the iPod transactions are much too large.

18
19
20

. . . Given that the theoretical derivation in Section 4 and the simulation
evidence presented here completely support my earlier intuition, I am more strongly
convinced than ever that the “tests” used by Professors Murphy and Topel show
nothing of value when it comes to deciding whether standard errors need to be
clustered.

21
22

23 Id. at 26.
24
25
26

5

In its Opposition, Apple tries to minimize its previous reliance on clustering by family and
quarter, stating that “even if the standard errors are clustered only by family and not by time period,
27 Noll’s results still are not statistically significant.” Def.’s Mem. at 18. But the same problems apply
28 to clustering by family. See infra at 8.
912519_1

PLAINTIFFS’ REPLY MEMO IN SUPPORT OF DAUBERT MOTION TO EXCLUDE CERTAIN
OPINION TESTIMONY OF KEVIN M. MURPHY AND ROBERT H. TOPEL - C-05-00037-YGR

-4-

1

D.

2

As Plaintiffs’ experts have explained, Apple’s “omitted variables” argument does not support

Apple’s “Omitted Variables” Argument Does Not Support Clustering

3 clustering. First, clustering is not required if the observation data are a random sample or an entire
4 population of data (as here), rather than a clustered sample.6 Sweeney Reply Daubert Decl., Ex. 1
5 (Wooldridge Supp. Decl.) at 1, 11-27, 32-33; Sweeney Daubert Decl., Ex. 1 (Wooldridge Decl.) at
6 10; Sweeny Daubert Decl., Ex. 4 (Noll Rebuttal) at 10; Sweeney MSJ Opp. Decl., Ex. 3 (Noll Supp.
7 Rebuttal) at 3-7. In his Supplemental Declaration, Professor Wooldridge cites several examples of
8 influential empirical studies that used large random samples but did not correct for clustering,
9 despite clear evidence of omitted variables. Sweeney Reply Daubert Decl., Ex. 1 (Wooldridge
10 Supp. Decl.) at 12-13. In those cases, “no clustering is needed to obtain the appropriate standard
11 errors. The usual heteroskedasticity-robust standard errors [which Noll used] are appropriate.” Id.
12

Indeed, Murphy’s own research has used large random samples with obvious omitted

13 variables, and yet Murphy made no clustering correction. Id. at 13-14. For example, in Juhn,
14 Murphy, and Pierce (1993), Murphy utilized 27 years worth of data from the Current Population
15 Survey (“CPS”) to study wage inequality, but did not cluster standard errors. Id. As Wooldridge
16 puts it: “It seems pretty clear that unobserved factors affecting wage can vary by state and year. Yet
17 there is no discussion of how omitted factors that vary by state and time cause the underlying errors
18 in their models to be correlated.” Id.
19

This same Murphy study also refutes Apple’s (unsupported) argument that Noll’s

20 standard errors are “unbelievably small” and the corresponding t-statistics7 unbelievably large.
21 Def.’s Mem. at 8. In his 1993 study, Murphy reports t-statistics well above 300 with only
22 approximately 50,000 observations. Here, with more than 600 times that number of
23 observations, Noll reports a t-statistic on the key variable of approximately 448. This is well
24
6
Wooldridge explained
calculations that
25 drawn about clustering randomthat in his First Declaration that the same Sweeney Daubertcan be
samples also apply to a whole population.
Decl.,
Ex. 1 (Wooldridge Decl.) at 5, 10-12; see also Sweeney Reply Daubert Decl., Ex. 1 (Wooldridge
26 Supp. Decl.) at 9-15.

27

7

A t-statistic is a ratio of the departure of an estimated parameter from its notional value and
its standard error. It is typically used in hypothesis testing.
28
912519_1

PLAINTIFFS’ REPLY MEMO IN SUPPORT OF DAUBERT MOTION TO EXCLUDE CERTAIN
OPINION TESTIMONY OF KEVIN M. MURPHY AND ROBERT H. TOPEL - C-05-00037-YGR

-5-

1 within acceptable limits, as Professor Murphy’s own work attests. Sweeney Reply Daubert
2 Decl., Ex. 1 (Wooldridge Supp. Decl.) at 14.
3

Second, the “omitted variables” that Apple asserts should have been included in Noll’s

4 regressions cause multicollinearity. Given that Professor Noll’s regressions include a large number
5 of product characteristics that affect iPod prices, multicollinearity should be tested for and avoided –
6 even if the omitted variables purportedly add to the explanatory power of the regression. Sweeney
7 MSJ Opp. Decl., Ex. 3 (Noll Supp. Rebuttal) at 8-9. Multicollinearity causes a reduction in the
8 precision of the estimated coefficients in the regression, making them less reliable. Sweeney
9 Daubert Decl., Ex. 4 (Noll Rebuttal) at 31; Sweeney MSJ Opp. Decl., Ex. 3 (Noll Supp. Rebuttal) at
10 7.
11

Here, neither of Apple’s experts conducted any of the standard tests for multicollinearity or

12 provided support for their opinion. Sweeny MSJ Opp. Decl., Ex. 3 (Noll Supp. Rebuttal) at 7-10;
13 id., Ex. 53 (1/8/14 Murphy Dep.) at 294:25-298:14. Had they done so, they would have discovered
14 that all but one of the omitted attributes generated a multicollinearity problem. Sweeney MSJ Opp.
15 Decl., Ex. 3 (Noll Supp. Rebuttal) at 10. Professor Noll utilized three separate tests (the adjusted R16 squared, condition number, and VIF) and each one unequivocally demonstrated multicollinearity
17 problems when adding the omitted variables to the regressions. Id. at 10-11.
18

Nor did Murphy and Topel provide any logical reason – based on economic theory or

19 econometrics – that would justify adding any of the omitted variables to the regression.8 Id. at 9-10;
20 see, e.g., id., Ex. 53 (1/8/14 Murphy Dep.) at 285:17-22 (“I would say it’s basically based on
21 economics and the actual evidence on pricing. I – I don’t recall a specific document that talks about
22 the pricing of those. But it’s something economics would lead us to believe should be included.”).
23 Topel’s rationalization for adding the omitted variables is even more rudimentary: “my recollection
24 is that these were variables that were in the data that Professor Noll had, and he chose to omit them.
25
8
Contrary to
should include
26 variables where theApple’s misrepresentations, Noll did not admit that his regression10 (citing Noll
“‘prices plausibly could be affected by it.’” Def.’s Mem. at
that if an indicator
turned on, it means that that
27 Supp. Rebuttal at 9). Instead, Noll statedthe price of an iPodvariable is is has to be turned on. Dkt.
particular variable should have affected
– not that

28 No. 685 (Second Supplemental Declaration of Roger G. Noll on Class Certification) at 9.
912519_1

PLAINTIFFS’ REPLY MEMO IN SUPPORT OF DAUBERT MOTION TO EXCLUDE CERTAIN
OPINION TESTIMONY OF KEVIN M. MURPHY AND ROBERT H. TOPEL - C-05-00037-YGR

-6-

1 So we simply said let’s not omit them. Let’s see what happens. They surely seem to be variable
2 characteristics.” Sweeney MSJ Opp. Decl., Ex. 52 (1/8/14 Topel Dep.) at 216:17-22; see also id.,
3 Ex. 53 (1/8/14 Murphy Dep.) at 288:4-11 (haphazardly guessing that a “vast bulk of” the iPods
4 would be affected by one particular omitted variable). As Murphy admits, “if you thought there was
5 no economic reason to include those kinds of variables in a regression, then I don’t think you’d want
6 to put them in.” Sweeney MSJ Opp. Decl., Ex. 53 (1/8/14 Murphy Dep.) at 289:16-19. Because
7 neither Murphy nor Topel could provide any economic or econometric reason for including the
8 omitted variables, these variables rightly should be excluded from Professor Noll’s regressions.
9

E.

10

Through the Construction of Arbitrary Clusters, Apple’s Experts
Artificially Introduce Correlation into the Error Residuals

Through the use of ex post clustering,9 Murphy and Topel effectively collapse the

11
transactions data for each class/generation/family of an iPod into a calendar quarter and thus into a
12
single observation, which works to reduce the number of transaction observations and the number of
13
degrees of freedom in the regression. Sweeney Daubert Decl., Ex. 4 (Noll Rebuttal) at 47. Murphy
14
and Topel’s clustering analysis is unreliable because the groups into which they divide their
15

observations are artificially created.10 Sweeney MSJ Opp. Decl., Ex. 3 (Noll Supp. Rebuttal) at 5.

16
Murphy and Topel first divide the observations into families, using the premise that Apple’s pricing
17
committee used common features within a family to set “uniform” prices. Id. As discussed above,
18
this assertion is contradicted by Apple’s own admissions and the transactions data.
19
Next, Murphy and Topel further group these families of iPods by quarter, but provide no
20
logical basis for doing so. Sweeney Daubert Decl., Ex. 4 (Noll Rebuttal) at 44; Sweeney Reply
21
9
Both Professor Noll and Professor
“ex post clustering.” This
22 phrase is in reference to clustering where aWooldridge refer to the idea of a large population with the
random sample is drawn from
observations subsequently grouped on
basis
23 Decl., Ex. 1 (Wooldridge Supp. Decl.) the11-12.of an observed variable. Sweeney Reply Daubert
at

24
25
26
27
28
912519_1

10

Apple mischaracterizes Noll’s testimony. Noll did not state that the “independence
assumption applies when using the entire population of transactions and that residuals could be
correlated within groups.” Def.’s Mem. at 13. Noll argued that where the R-square is extremely
high (indicating a high explanatory power in the regression), you would not need to test the
residuals, particularly if you already have group identifiers. Here, it is unnecessary to perform a test
of the independence assumption because “there are no groups with outlying residual errors in the Rsquared spot. And by definition, the mean residual errors by group are going to be zero.” Sweeney
MSJ Opp. Decl., Ex. 50 (12/18/13 Noll Dep.) at 46:6-9.

PLAINTIFFS’ REPLY MEMO IN SUPPORT OF DAUBERT MOTION TO EXCLUDE CERTAIN
OPINION TESTIMONY OF KEVIN M. MURPHY AND ROBERT H. TOPEL - C-05-00037-YGR

-7-

1 Daubert Decl., Ex. 1 (Wooldridge Supp. Decl.) at 4-5. Grouping the residual errors into families by
2 quarter only makes sense if there is an unobserved variable that affects iPod families differently in
3 different quarters. Sweeney MSJ Opp. Decl., Ex. 3 (Noll Supp. Rebuttal) at 5. Murphy and Topel
4 could have just as easily grouped the residuals by day, week, or year – the arbitrary nature of their
5 clustering by quarter is entirely ad hoc. When pressed, neither Murphy nor Topel could provide any
6 rationale for grouping the residuals by quarter. Murphy admitted that there is no evidence that Apple
7 evaluates its iPod pricing on a quarterly basis and that grouping by quarter was done as a
8 “compromise,” conceding that aggregating at other levels would also be “reasonable.” Sweeney
9 MSJ Opp. Decl., Ex. 53 (1/8/14 Murphy Dep.) at 264:11-265:15 (“I don’t think [Apple does]
10 anything that lines up precisely on quarter boundaries.”). Likewise, Topel acknowledged that Apple
11 “hold[s] prices constant across a couple of quarters.” Id., Ex. 52 (1/8/14 Topel Dep.) at 206:1312 207:23.
13

The arbitrariness of Murphy and Topel’s clustering adjustment is underscored by the fact that

14 neither did any additional testing to determine the effect of aggregating the data for different time
15 periods. Id., Ex. 53 (1/8/14 Murphy Dep.) at 265:17-266:5; id., Ex. 52 (1/8/14 Topel Dep.) at 209:116 15. If they had done so, they would have seen that the same results occur whether the data are
17 clustered by family/quarter, family/month, or family/week. Sweeney MSJ Opp. Decl., Ex. 3 (Noll
18 Supp. Rebuttal) at 6 & Exs. 1a-b, 2a-b. These results demonstrate that Murphy and Topel had no
19 basis for clustering the residuals into quarters; the supposed correlations of the residual errors are the
20 expected result of dividing the residual errors into a large number of arbitrarily selected groups.
21 Sweeney MSJ Opp. Decl., Ex. 3 (Noll Supp. Rebuttal) at 6; Sweeney Reply Daubert Decl., Ex. 1
22 (Wooldridge Supp. Decl.) at 25-26 (Apple’s exercise is “predictable” and “tells us nothing about
23 whether clustering is needed.”)
24

Recognizing that clustering by quarter is inappropriate, Apple attempts to salvage Murphy

25 and Topel’s analysis by arguing that clustering the standard errors at the family level alone
26 demonstrates that Noll’s results are still not statistically significant. Def.’s Mem. at 18. But
27 clustering at the family level (without regard to time period) is still ex post clustering, so the
28 correlation of residuals that is observed has been introduced artificially (irrespective of the particular
912519_1

PLAINTIFFS’ REPLY MEMO IN SUPPORT OF DAUBERT MOTION TO EXCLUDE CERTAIN
OPINION TESTIMONY OF KEVIN M. MURPHY AND ROBERT H. TOPEL - C-05-00037-YGR

-8-

1 ex post cluster chosen), and does not stem from a true cluster sampling problem that needs to be
2 “properly corrected.” Sweeney Reply Daubert Decl., Ex. 1 (Wooldridge Supp. Decl.) at 17-18.
3

F.

4

Both Murphy and Topel concede that making a clustering adjustment when it is not required

Clustering Where Unnecessary Is Harmful

5 can bias the standard errors. Sweeney MSJ Opp. Decl., Ex. 52 (1/8/14 Topel Dep.) at 244:4-19; id.,
6 Ex. 53 (1/8/14 Murphy Dep.) at 279:3-7. In fact, as confirmed by Professor Wooldridge in his
7 Supplemental Declaration, clustering standard errors where, as here, the entire data population is
8 being used, can result in “severe upward biases” of the standard errors. Sweeney Reply Daubert
9 Decl., Ex. 1 (Wooldridge Supp. Decl.) at 19-20, 26; see also Sweeney Daubert Decl., Ex. 4 (Noll
10 Rebuttal) at 6. This is far from harmless.
11

In his Supplemental Declaration, Professor Wooldridge uses a theoretical calculation and

12 several simulations to demonstrate that clustering after random sampling (or, as in this case, after
13 observing an entire population) biases the standard errors upward. The theoretical calculation
14 “confirms the intuition” expressed in Professor Wooldridge’s first declaration: “If one starts with a
15 random sample, computes residuals, and then averages those residuals with arbitrarily created
16 clusters, the residuals will have means that are systematically different from zero. Therefore,
17 clustering artificially inflates the standard errors, perhaps by a substantial amount.“ Id. at 18.
18 Professor Wooldridge concludes that “clustering after collecting a random sample is far from
19 harmless. The upward bias can be severe, and the situation gets worse as the sample size
20 grows . . . .” Id. at 19.
21

Professor Wooldridge’s simulations also demonstrate that clustering standard errors in a

22 population artificially inflates the standard errors:
23
24
25
26
27
28
912519_1

These simulation results cast serious doubt on the practice of clustering when
the entire population is available. When we take a 10% random sample we know that
the proper standard error is, roughly, .049. Across the 1,000 simulations the largest of
the standard errors is about .051. In other words, the largest standard error we obtain
using 1,000 different 10% random samples is never close to the clustered standard
error using the entire population, .206. Thus, if we insist on clustering the population
standard errors, we have the perverse conclusion that virtually any 10% random
sample produces a much more precise estimate than using the entire population. By
contrast, if we use the usual standard error then this paradox disappears: the standard
error from the population, .016, is much smaller than any of the 1,000 standard errors
obtained from a 10% sample. (The smallest is about .049.)
PLAINTIFFS’ REPLY MEMO IN SUPPORT OF DAUBERT MOTION TO EXCLUDE CERTAIN
OPINION TESTIMONY OF KEVIN M. MURPHY AND ROBERT H. TOPEL - C-05-00037-YGR

-9-

1 Sweeney Reply Daubert Decl., Ex. 1 (Wooldridge Supp. Decl.) at 24.
2

G.

3

As confirmed by Professor Wooldridge’s simulations, when the number of observations in a

The Number of Observations Per Cluster Distorts the Standard Error

4 cluster increases relative to the number of clusters, the standard error becomes distorted. Sweeney
5 Reply Daubert Decl., Ex. 1 (Wooldridge Supp. Decl.) at 20. Running a basic simulation, Professor
6 Wooldridge found that the usual standard errors work well when the total number of observations
7 were at 50. Id. However, when the number of observations increased to 5,000 – with an average of
8 1,000 observations per cluster – the mean cluster standard is approximately .709, which is close to
9 the predicted value of .707. Id. Calculating the ratio of the average cluster standard error to the
10 usual standard error is .709/.033 = 21.5 Id. This indicates that the clustered standard error is
11 expected to be about 21.5 times too large – confirming both Professor Noll and Professor
12 Wooldridge’s theory that clustering is very harmful and produces standard errors that are too
13 conservative. Id.
14

Further, Apple’s attack on Noll’s reliance on Angrist and Pischke’s book is misguided.

15 Apple argues that the relevant chapter demonstrates the opposite of Noll’s opinion – that clustering
16 should be performed when the number of observations per group and the amount of within-group
17 correlation increases because the standard error becomes “increasingly misleading.” Def.’s Mem. at
18 16. However, this chapter of Angrist and Pischke’s book is based on problems when one is already
19 dealing with cluster samples – not when one working is working with an entire population. Sweeney
20 MSJ Opp. Decl., Ex. 3 (Noll Supp. Rebuttal) at 4. When discussing the performance of standard
21 errors in relation to the number of clusters, the authors state that “[i]f 42 is enough for the standard
22 cluster adjustment to be reliable, and if less is too few, then what should you do when the cluster
23 count is low? First best is to get more clusters by collecting more data.” Id. This statement only
24 makes sense if the original number of clusters is less than the population of clusters, thereby
25 denoting the ability to obtain more clusters if necessary. Id.
26
27
28
912519_1

PLAINTIFFS’ REPLY MEMO IN SUPPORT OF DAUBERT MOTION TO EXCLUDE CERTAIN
OPINION TESTIMONY OF KEVIN M. MURPHY AND ROBERT H. TOPEL - C-05-00037-YGR

- 10 -

1 III.

APPLE’S ATTACKS ON PROFESSOR WOOLDRIDGE ARE
UNFOUNDED

2
A.
3
4
5

Professor Wooldridge’s Opinions Are Supported by Generally
Accepted Econometrics and Wooldridge’s Own Peer-Reviewed
Publications and Research and Were Not Manufactured for This
Litigation

All of Apple’s attacks on Professor Wooldridge fail to withstand scrutiny. First, Wooldridge

6 did not “custom-manufacture” his opinion for the purposes of this litigation.11 Def.’s Mem. at 14-15.
7 Professor Wooldridge is on published record criticizing clustering in a context closely related to
8 random sampling: stratified sampling with large sample sizes within strata. Sweeney Reply Daubert
9 Decl., Ex. 1 (Wooldridge Supp. Decl.) at 27. With stratified sampling, one first partitions the
10 population, just as with cluster sampling. Id. But, rather than sample clusters, one obtains random
11 samples within each stratum.

Id.

In Wooldridge (2003, 2010, Section 20.3.4.), Professor

12 Wooldridge explains that treating stratified samples as cluster samples is likely to be too
13 conservative, which is the same issue here: the usual standard errors are correct and clustered
14 standard errors are too conservative. Id.
15

Apple also argues that Professor Wooldridge’s opinion that clustering with random samples

16 is unnecessary and perhaps harmful has no support in economic literature. Besides being incorrect
17 (Wooldridge’s own peer-reviewed publications provide such authority), this criticism actually
18 demonstrates that it is Murphy and Topel’s clustering opinions that lack support. As Professor
19 Wooldridge explains, econometrics literature does not warn against clustering a random sample
20 because “it is well known how to compute standard errors with random samples, and it does not
21 involve clustering.” Id. at 28, see generally id. at 28-29.
22
23
24
25
26
27
28
912519_1

11

The authority Apple relies upon to exclude Professor Wooldridge’s declaration are
inapposite. In Cabreara v. Cordis Corp., 134 F.3d 1418 (9th Cir. 1998), the expert in question could
not support his conclusions with any of his own research or any other known research – unlike here,
where Professor Wooldridge relied upon both econometric theory and simulations to support his
opinion. Similarly, in Lust by & Through Lust v. Merrell Dow Pharms., Inc., 89 F.3d 594, 597 (9th
Cir. 1996), the origination of the expert’s opinion could only be traced to a point after he began
working as a professional plaintiff’s witness. Here, Professor Wooldridge began developing his
theory on clustering years before he was retained in this litigation and has stated that he has seen this
issue come up in other contexts. Kiernan Daubert Decl., Ex. 11 (1/6/14 Wooldridge Dep.) at 91:1016.
PLAINTIFFS’ REPLY MEMO IN SUPPORT OF DAUBERT MOTION TO EXCLUDE CERTAIN
OPINION TESTIMONY OF KEVIN M. MURPHY AND ROBERT H. TOPEL - C-05-00037-YGR

- 11 -

1

Apple also criticizes Professor Wooldridge for clustering in previous work when using the

2 entire population. Def.’s Mem. at 14. However, Professor Wooldridge’s past work with clustering a
3 population data set are limited to particular circumstances not present here. Sweeney Reply Daubert
4 Decl., Ex. 1 (Wooldridge Supp. Decl.) at 30-31. The first circumstance occurs with panel data,
5 where one wants to group the data to control for unobserved effects that can cause bias in the
6 estimates. Clustering is even appropriate when dealing with a random sample of cross-section/time
7 period units here because of the need to control for unobserved factors that are constant over time.
8 Id. The second scenario where Professor Wooldridge clustered a population involved cross sectional
9 data, where there were many groups and relatively small group sizes. Id. Both of these examples of
10 clustering a population are narrow exceptions that have no parallels to the litigation at hand. Id.
11

Finally, it must be noted that if random sampling required the calculation of cluster-robust

12 standard errors, then numerous papers in empirical economics would have to be redone. Sweeney
13 Reply Daubert Decl., Ex. 1 (Wooldridge Supp. Decl.) at 12. Many influential studies, like Angrist
14 and Krueger (1991), Angrist and Evans (1998), and Angrist, Chernozhukov, and Fernandez-Val
15 (2006, Econometricia), use large random samples from the United States Census without using
16 clustering for standard errors. Sweeney Reply Daubert Decl., Ex. 1 (Wooldridge Supp. Decl.) at 1117 12.
18

B.

19

This Court should reject Apple’s argument that Wooldridge’s declaration is untimely.12

Plaintiffs Timely Disclosed Wooldridge

20 First, Plaintiffs complied with the scheduling order when it timely served Professor Noll’s reports in
21 April and November of 2013. Rule 26(a)(2) only requires the disclosure of “the identity of any
22 witness it may use at trial to present evidence.” Fed. R. Civ. P. 26(a)(2).
23
24
25
26
27
28
912519_1

12

All three of the out-of-circuit authorities Apple relies upon in support of its argument to
exclude Professor Wooldridge share a common theme, limiting their applicability. In Reed v. Smith
& Nephew, Inc., 527 F. Supp. 2d 1336, 1348 (W.D. Okla. 2007), Moore v. Napolitano, 926
F. Supp. 2d 8, 25 n.12 (D. D.C. 2013), and Jeffries v. Centre Life Ins. Co., No. 1:02-cv-351, 2004
U.S. Dist. LEXIS 30769, at *1 (S.D. Ohio Jan. 28, 2004), the court ultimately excluded the untimely
expert opinions because the objecting party never had a chance to depose the expert witness, unlike
here. Plaintiffs made Professor Wooldridge fully available for an entire day of deposition, of which
Apple took advantage. Def.’s Mem. at 12.

PLAINTIFFS’ REPLY MEMO IN SUPPORT OF DAUBERT MOTION TO EXCLUDE CERTAIN
OPINION TESTIMONY OF KEVIN M. MURPHY AND ROBERT H. TOPEL - C-05-00037-YGR

- 12 -

1

Professor Wooldridge is offering testimony on a narrow topic – clustering – in connection

2 with Plaintiffs’ Daubert motion. He is not Plaintiffs’ merits expert, and Plaintiffs had no obligation
3 to disclose him in April of 2013. Because Plaintiffs had no obligation to disclose Professor
4 Wooldridge prior to the merits expert disclosure deadlines, his expert report was not untimely. See,
5 e.g., Nightlight Sys. v. Nitelites Franchise Sys., No. 1:04-CV-2112, 2007 U.S. Dist. LEXIS 95538, at
6 *24-*26 (N.D. Ga. May 11, 2007) (permitting the use of an undisclosed expert strictly at the
7 Daubert hearing and not at trial, noting that there was enough time for expert reports and depositions
8 prior to the expert hearing); In re Paoli R.R. Yard PCB Litig., 35 F.3d 717, 739 (3d Cir.
9 1994) (affirming district court’s decision to allow an undisclosed expert to testify at Daubert
10 hearing).
11

Second, under Rule 37(c)(1), even assuming the disclosure was untimely (which it was not),

12 a party may use a late-disclosed witness if the failure to disclose was “substantially justified or is
13 harmless.” Fed. R. Civ. P. 37(c)(1). Because Plaintiffs made Professor Wooldridge available for a
14 full day of deposition on January 6, 2014 (which availability Apple took complete advantage of –
15 using even less than the allotted amount of time agreed upon to complete the deposition), there was
16 no prejudice. Professor Wooldridge also fully responded to Apple’s document subpoena. Further,
17 Apple claims that the arguments proffered by Professor Wooldridge are largely similar to those put
18 forth by Professor Noll, thereby refuting Apple’s suggestion that it did not have enough to time to
19 address Professor Wooldridge’s theories or opinions. Def.’s Mem. at 12. Courts regularly permit
20 expert testimony under similar circumstances.13 See, e.g., Wendt v. Host Int’l, 125 F.3d 806, 814
21 (9th Cir. 1997) (vacating district court’s preclusion of expert testimony because parties had time “to
22 begin the expert disclosure procedure anew”); Baxter Healthcare Corp. v. Fresenius Med. Care
23 Holding, Inc., No. C 07-1359, 2009 U.S. Dist. LEXIS 32380, at *8 (N.D. Cal. Apr. 2,
24 2009) (denying motion to preclude evidence and witnesses introduced after fact discovery
25
13
That Apple’s former expert
raised
irrelevant.
26 Plaintiffs successfully refuted those(used for class certification only)the factclustering isWare ruled
arguments, as demonstrated by
that Judge
demonstrated class-wide impact and damages based on Professor
27 that Plaintiffs had adequatelyin turn rejecting Dr. Burtis’ opinion (which included her clustering
Noll’s methodologies, while

28 opinions). Dkt. No. 694 (Order Granting Plaintiffs’ Motion for Class Certification) at 6-7.
912519_1

PLAINTIFFS’ REPLY MEMO IN SUPPORT OF DAUBERT MOTION TO EXCLUDE CERTAIN
OPINION TESTIMONY OF KEVIN M. MURPHY AND ROBERT H. TOPEL - C-05-00037-YGR

- 13 -

1 concluded, noting that the opposing party was able to depose the witness); In re Enron Corp. Sec.,
2 Derivative & ERISA Litig., MDL No. 1446, 2007 U.S. Dist. LEXIS 98619, at *133-*134 (S.D. Tex.
3 Feb. 1, 2007) (untimely expert report permitted where the expert’s theory and data were already
4 available to the opposing party and thus there was no unfair surprise; court noted that there was
5 plenty of time to schedule depositions without disrupting the trial schedule).
6 IV.

CONCLUSION

7

Clustering the error residuals in this case is both inappropriate and harmful, as confirmed

8 through Professor Noll’s Supplemental Rebuttal Report and Professor Wooldridge’s Supplemental
9 Declaration. Apple’s assertions to the contrary falter when analyzed against the facts of the case,
10 economic and econometric theory, and statistical analysis and simulations that Murphy and Topel’s
11 clustering opinion fails to fit the facts of this case and should be excluded from the summary
12 judgment record and at trial.
13 DATED: January 31, 2014

Respectfully submitted,

14

ROBBINS GELLER RUDMAN
& DOWD LLP
BONNY E. SWEENEY
THOMAS R. MERRICK
ALEXANDRA S. BERNAY
CARMEN A. MEDICI
JENNIFER N. CARINGAL

15
16
17
18
19
20

s/ Bonny E. Sweeney
BONNY E. SWEENEY

22

655 West Broadway, Suite 1900
San Diego, CA 92101
Telephone: [phone]
619/231-7423 (fax)

23

Class Counsel for Plaintiffs

24

THE KATRIEL LAW FIRM
ROY A. KATRIEL
1101 30th Street, N.W., Suite 500
Washington, DC 20007
Telephone: [phone]
202/330-5593 (fax)

21

25
26
27
28
912519_1

PLAINTIFFS’ REPLY MEMO IN SUPPORT OF DAUBERT MOTION TO EXCLUDE CERTAIN
OPINION TESTIMONY OF KEVIN M. MURPHY AND ROBERT H. TOPEL - C-05-00037-YGR

- 14 -

1
2
3
4
5

BONNETT, FAIRBOURN, FRIEDMAN
& BALINT, P.C.
ANDREW S. FRIEDMAN
FRANCIS J. BALINT, JR.
ELAINE A. RYAN
2325 E. Camelback Road, Suite 300
Phoenix, AZ 85016
Telephone: [phone]
602/274-1199 (fax)

6
7
8
9
10
11
12

BRAUN LAW GROUP, P.C.
MICHAEL D. BRAUN
10680 West Pico Blvd., Suite 280
Los Angeles, CA 90064
Telephone: [phone]
310/836-6010 (fax)
GLANCY BINKOW & GOLDBERG LLP
BRIAN P. MURRAY
122 East 42nd Street, Suite 2920
New York, NY 10168
Telephone: [phone]
212/382-3944 (fax)

13

16

GLANCY BINKOW & GOLDBERG LLP
MICHAEL GOLDBERG
1925 Century Park East, Suite 2100
Los Angeles, CA 90067
Telephone: [phone]
310/201-9160 (fax)

17

Additional Counsel for Plaintiffs

14
15

18
19
20
21
22
23
24
25
26
27
28
912519_1

PLAINTIFFS’ REPLY MEMO IN SUPPORT OF DAUBERT MOTION TO EXCLUDE CERTAIN
OPINION TESTIMONY OF KEVIN M. MURPHY AND ROBERT H. TOPEL - C-05-00037-YGR

- 15 -

APPLE TYING
Service List - 1/31/2014
Page 1 of 1

(06 0171)

Counsel for Defendant(s)
Robert A. Mittelstaedt
David Craig Kiernan
Amir Q. Amiri
Jones Day
555 California Street, 26th Floor
San Francisco, CA 94104
415/626-3939
415/875-5700(Fax)

Counsel for Plaintiff(s)
Andrew S. Friedman
Francis J. Balint
Elaine A. Ryan
Bonnett, Fairbourn, Friedman & Balint, P.C.
2325 E. Camelback Road, Suite 300
Phoenix, AZ 85016
602/274-1100
602/274-1199(Fax)

Michael D. Braun
Marc L. Godino
Braun Law Group, P.C.
10680 West Pico Blvd., Suite 280
Los Angeles, CA 90064
310/836-6000
310/836-6010(Fax)

Brian P. Murray
Glancy Binkow & Goldberg LLP
122 East 42nd Street, Suite 2920
New York, NY 10005
212/682-5340
310/201-9160(Fax)

Michael Goldberg
Glancy Binkow & Goldberg LLP
1925 Century Park East
Suite 2100
Los Angeles, CA 90067
310/201-9150
310/201-9160(Fax)

Bonny E. Sweeney
Alexandra S. Bernay
Thomas R. Merrick
Robbins Geller Rudman & Dowd LLP
655 West Broadway, Suite 1900
San Diego, CA 92101
619/231-1058
619/231-7423(Fax)

Roy A. Katriel
The Katriel Law Firm
1101 30th Street, N.W., Suite 500
Washington, DC 20007
202/625-4342
866/373-4023(Fax)