Direct Marketing Association, The v. Huber

Filing 63

MOTION to Exclude the Testimony of Plaintiff's Expert Witnesses F. Curtis Barry, Thomas Adler, and Kevin Lane Keller by Defendant Roxy Huber. (Attachments: # 1 Exhibit A- Barry Dep., # 2 Exhibit B- Barry Report, # 3 Exhibit C- Barry Dep. Exhs., # 4 Exhibit D- Adler Report and Decl., # 5 Exhibit E-1of5- Adler Dep. Exhs, # 6 Exhibit E-2of5- Adler Dep. Exhs, # 7 Exhibit E-3of5- Adler Dep. Exhs, # 8 Exhibit E-4of5- Adler Dep. Exhs, # 9 Exhibit E-5of5- Adler Dep. Exhs, # 10 Exhibit F- Adler Dep., # 11 Exhibit G- Lichtenstein Decl. and Report, # 12 Exhibit H- Keller Dep., # 13 Exhibit I- Keller Report and Decl., # 14 Exhibit J-1of3- Keller Dep. Exhs., # 15 Exhibit J-2of3- Keller Dep. Exhs., # 16 Exhibit J-3of3- Keller Dep. Exhs.)(Snyder, Melanie)

IN THE UNITED STATES DISTRICT COURT FOR THE DISTRICT OF COLORADO Civil Action No. 10-cv-01546-REB-CBS The Direct Marketing Association, Plaintiff, v. Roxy Huber, in her capacity as Executive Director, Colorado Department of Revenue, Defendant. DEFENDANT'S MOTION TO EXCLUDE THE TESTIMONY OF PLAINTIFF'S EXPERT WITNESSES F. CURTIS BARRY, THOMAS ADLER, AND KEVIN LANE KELLER Defendant, Roxy Huber in her capacity as Executive Director, Colorado Department of Revenue ("Department"), moves to exclude all testimony whether by declaration, deposition excerpt, or live testimony of Plaintiff's expert witnesses F. Curtis Barry, Thomas Adler, and Kevin Lane Keller pursuant to Federal Rule of Evidence 702. INTRODUCTION In this action, Plaintiff, the Direct Marketing Association (“DMA”) raises constitutional challenges to Colorado H.B. 1193 and implementing regulations (“the Law”), which impose reporting requirements on retailers who do not collect sales tax. See Defendant’s Response to Plaintiff’s Motion for Preliminary Injunction, filed November 19, 2010 [Dkt #50, pp.1-4], for a description of the Law’s requirements. DMA sought a preliminary injunction on the grounds that the Law violates the Commerce Clause of the United States Constitution. In support of its motion, DMA offered the testimony of three experts - F. Curtis Barry on the subject of the alleged costs DMA 1 members will incur to comply with the Law, Thomas Adler on a survey he conducted of Colorado consumers regarding their reactions to the Law, and Kevin Lane Keller interpreting that survey. Each of these witnesses' testimony should be excluded pursuant to Rule 702 for failure to employ reliable methodology in forming their opinions. ARGUMENT I. RULE 702 DOES NOT PERMIT EXPERT OPINIONS WHICH ARE NOT BASED ON RELIABLE METHODOLOGY. Rule 702 governs the admissibility of expert witness testimony: If scientific, technical, or other specialized knowledge will assist the trier of fact to understand the evidence or to determine a fact in issue, a witness qualified as an expert by knowledge, skill, experience, training, or education, may testify thereto in the form of an opinion or otherwise, if (1) the testimony is based upon sufficient facts or data, (2) the testimony is the product of reliable principles and methods, and (3) the witness has applied the principles and methods reliably to the facts of the case. Fed.R.Evid. 702. In assessing a Rule 702 motion, the Court must determine: 1) whether the expert is qualified to render the opinion; and 2) whether the opinion is reliable. 103 Investors I, L.P. v. Square D. Co., 470 F.3d 985, 990 (10th Cir. 2006). Exercising its “gatekeeper” role, the Court should exclude the opinions of all three Plaintiff experts because they are not reliable. See Kumho Tire Co, Ltd. v. Carmichael, 526 U.S. 137, 147 (1999). As the proponent of the expert testimony, DMA bears the burden of proving the admissibility of the testimony under Rule 702 by a preponderance of the evidence. United States v. Crabbe, 556 F. Supp. 2d 1217, 1220 (D. Colo. 2008). DMA, therefore, must prove that each expert has sufficient expertise to choose and apply a 2 methodology, that the methodology applied was reliable, that sufficient facts and data as required by the methodology were used, and that the methodology was otherwise reliably applied. Id. at 1221; see also Toni's Alpacas, Inc. v. Evans, 2010 WL 3730382, at *1 (D. Colo. Sept. 16, 2010) (the proponent of the testimony must show that “the expert has employed a method that is scientifically sound and that the opinion is based on facts which enable [him] to express a reasonably accurate conclusion as opposed to conjecture or speculation.”) (internal citations omitted). An opinion is reliable if it is based on scientific knowledge, grounded in the methods and procedures of science. Id. (citing Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579, 58-90 (1993)). Such knowledge must be more than subjective belief or unsupported speculation. Id. The critical determination is “whether the evidence is genuinely scientific, as distinct from being unscientific speculation offered by a genuine scientist.” Goebel v. Denver & Rio Grande W. R.R. Co., 346 F.3d 987, 992 (10th Cir. 2003). Under Daubert, “any step that renders the analysis unreliable … renders the expert’s testimony inadmissible. Mitchell v. Gencorp, Inc., 165 F.3d 778, 782 (10th Cir. 1999). In the reliability assessment, the inquiry is whether the witness employed “the same level of intellectual rigor that characterizes the practice of an expert in the relevant field." Toni's Alpacas, 2010 WL 3730382, at *2 (quoting Goebel, 346 F.3d at 992). Applying these principles, each of DMA's experts' testimony fails the reliability test and his testimony should be excluded. 3 II. F. CURTIS BARRY’S OPINIONS ARE NOT RELIABLE BECAUSE HE DID NOT PERFORM ANY STUDIES OR FOLLOW ANY ESTABLISHED METHODOLOGY, AND HIS OPINIONS SET FORTH ONLY HIS SUBJECTIVE BELIEFS ABOUT COMPLIANCE COSTS. DMA offers the testimony of Mr. F. Curtis Barry to establish the costs for DMA members to comply with each of the three regulatory requirements of the Law. Mr. Barry, however, did not perform any studies of his own (see, e.g., Exh. A., Barry Dep. 53:25-55:8; 126:15-21; 152:18-20), 1 he did not refer to any published literature in his field (see, e.g., id. at 55:8-11; 126:12-14; 152:21-22), and he did not even refer to any documents relating to his own experience with clients to establish his opinions. See, e.g., id. at 52:19-53:14; 128:25-129:4; 164:24-165:7. Instead, he basically eyeballed the requirements using his “best judgment” and information “in his head” to come up with dollar figures as to what it will cost DMA members to comply with the Law. See, e.g., id. at 33:4-9; 119:17-120:11; 131:25-132:6. An expert’s testimony must be based on facts that enable the expert to express reasonably accurate conclusions. Cartel Asset Mgm't v. Ocwen Fin. Corp., 2010 WL 3613819, at *2 (D. Colo. Sept. 7, 2010). Mr. Barry’s opinions are not based on ascertainable facts. For example: • He did not rely on any labor studies or published documents to determine the per hour cost for computer programming or management. Rather, he used numbers “in his head.” Exh. A., Barry Dep. 32:9-36:10. • He did not “do any math” to come up with the $5,000-$10,000 estimate for companies to comply with the transactional notice. Id. at 100:19-101:2. He also did not attempt to break up the estimate into its components other than to “kind of go through that mentally.” Id. at 101:13-20. 1 The one exception is that Mr. Barry’s assistant contacted two printers for quotes on the cost of mailing the annual customer notice. Id. at 176:12-177:18. 4 • He did not base his estimate on the cost to modify an invoice on anything other than “my experience in going through what I think would have to be considered and the time to develop an estimate...” He did not put anything in writing, only calculated it in his head. Id. at 114:18-117:6. Although Barry claimed to have done some mathematical calculations, he did not have any notes or documents reflecting his calculations, and his alleged calculations were based on assumptions of hourly rates drawn from his experience. Id. at 115:7-119:16. 2 • In describing how he arrived at an estimate that it would cost $2,000-$3,000 to develop call center procedures, he estimated that it would be “pretty easy to chew up a bunch of time.” He then calculated his estimate by “just kind of the number of hours and then multiplying by, you know, $40 to $60 an hour” to arrive at “it would take probably a week or so.” Id. at 123:25-127:5. 3 • He could not identify particular data used in estimating that 50 percent of customers will call a call center to inquire about the transactional notice. The estimate is based on his “experience with issues that are not similar in terms of legislation, but for example, when we make changes and we don’t fully think them through and customers have their first reactions…” Id. at 130:8-131:3. Ultimately, his estimate was based on his “best judgment.” Id. at 131:25-132:6. 4 • His estimate that companies will need $3,000-$5,000 of professional assistance from lawyers and others to comply with the transactional notice was not based on any methodology other than “[j]ust from working through it, best judgment, experience.” Id. at 132:15-22. • When asked what data he considered to arrive at an estimate of ongoing compliance costs for the transactional notice, he said “just because this is a sensitive customer service area” and companies would need a whole day of professional time each year to review their compliance. Id. at 147:18-148:24. • In estimating the amount for companies to voluntarily calculate sales tax (a discretionary cost), he initially thought the number would be $20,000-$25,000, 2 Although Mr. Barry claims to have come up with an hourly rate for internal programming by averaging the range he earlier provided based on his experience (Id. at 116:4-15), when estimating external programming costs, he plucked the number of “150, 160, somewhere in there” out of his initially estimated range of $140-$225 per hour. Id. at 118:17-119:16; see also Exh. B, Barry Report, p.4. 3 His call center cost estimate is based on his assumption that a call would last three to four minutes. Although there may be published literature in the field as to the average call length, he said “I’m working with call centers all the time so I have a pretty good idea of these numbers.” Id. at 127:6-128:2. He estimated that a call to a call center costs $1.50-$2.00; however, he did not refer to any literature in using this figure, but stated “I just know that it’s an accurate conservative number.” Id. at 128:3-129:4. 4 This was reached after his email to Plaintiff’s counsel stating, “How about if we figure 50 percent?” Id. at 132:7-14. 5 but then revised the number to $5,000-10,000. He arrived at the final number by estimating that it would take companies anywhere from “a couple days work” to modify existing software at a cost of $3,000-$4,000 to $30,000-$40,000 to buy new software, and then picking a number in the middle that he “thought [] was on the low end of the range, and I think it’s reasonable.” Id. at 149:17-19; 151:1152:25; 154:2-157:4. • He estimated the amount of time to modify computer systems for the annual customer report either internally or by using an external vendor without employing any methodology other than his experience. Id. at 162:1-163:4; 164:3-165:7. • As to the facts underlying his conclusions on the cost of the annual customer report, although he is sure there are published studies about the amount of an average retail order, he did not refer to them, relying instead on his experience. Id. at 166:3-21; see also 166:22-167:13 (same for the average number of customer purchases); 168:13-169:4 (same for the number of repeat buyers); 180:17-182:7 (same as to the cost of mailing the annual customer report) . 5 • He did not follow any particular methodology in determining that less than 20 percent of customers (or as few as 10 percent) would buy more than $500 from a particular retailer. Id. at 169:11-171:17. • His estimated cost of $1,000-$3,000 to comply with the Department’s specifications for submitting the annual revenue report is “some kind of a placeholder” in case companies have to buy software. This is again based on no methodology other than his experience. Id. at 190:7-191:17. • His estimate of the number of management hours it would take to create the annual report to the Department of Revenue is based on “a conservative estimate for the number of people and the salaries that we see people paying.” Id. at 192:3-21. He couldn’t remember, however, what hourly rate he used for management personnel, and he did not refer to any existing literature or perform any studies to come up with a rate. Id. at 192:22-193:25. • He estimated that “somewhere between 25 and 50 percent” of customers will abandon their purchases based on the Law, but he admitted that “I can’t mathematically” quantify how much retailers would lose. Id. at 195:3-197:20. 5 Similarly, he did not contact any particular marketing service bureaus to determine what they charge for mailing processing. Id. at 175:12-176:5. 6 In this case, actual facts exist to demonstrate DMA members’ costs to comply with the Law. Retailers are currently complying with the Law, and Mr. Barry could have researched actual compliance costs. While Mr. Barry contacted a total of 17 retailers, software vendors, and marketing service bureaus, he failed to ask any of them about their current compliance costs. Id. at 57:8-58:13; 63:19-64:4; 64:18-21; 79:4-14. 6 Rather, he asked them to validate what the process for compliance would be. Id. at 63:19-64:4. Even as to this level of effort, Mr. Barry failed to employ any reliable methodology. He did not come up with standard questions to ask the retailers, did not track their responses, and did not keep notes. Id. at 58:14-19 (admitting “I wasn’t trying to do something methodically.”); 65:16-25. Remarkably, Mr. Barry could not remember the names of any of the retailers with whom he spoke, and he could only remember the names of a few of the software vendors. Id. at 60:2-61:22; 67:19-68:12; 79:15-18. Mr. Barry admitted, however, that actual costs of compliance could have been relevant and helpful in forming his opinions. Id. at 80:24-81:23. For an opinion to be based on scientific knowledge, it must be more than “subjective belief or unsupported speculation.” Mitchell, 165 F.3d at 780. Experts must “express a reasonably accurate conclusion as opposed to conjecture or speculation.” Goebel, 346 F.3d at 987 (internal citations omitted); see also James River Ins. Co. v. Rapid Funding, LLC, 2009 WL 481688, at *7 (D. Colo. Feb. 24, 2009) (an expert cannot “jump to an opinion without factual support”) (citing Hathaway v. Bazany, 507 F.3d 312, 6 Mr. Barry proposed to Plaintiff’s counsel that he obtain quotes from several software vendors as to what it would take to comply with the three regulatory requirements of the Law, but never did so. Id. at 71:922; 73:4-8; 80:12-17. 7 318 (5th Cir. 2007) (expert testimony should be excluded when the “proposed expert offered little more than personal assurances based on experience.”)). Mr. Barry’s speculation is evident from the progression of his opinions. He went through several drafts, each of which dramatically reduced his estimated compliance costs. Some of these reductions were based on encouragement from DMA’s counsel, who wanted the numbers to be conservative. Id. at 75:25-76:17 (regarding statement to DMA’s counsel that “I took your advice and cut the costs down considerably to avoid issues.”); 134:20-135:15 (explaining revision of numbers that DMA’s counsel “felt [] might be too high.”). In making these revisions, Mr. Barry did not perform any additional studies or do additional work to come up with the new numbers. See, e.g., id. at 105:11-106:25; 107:23-108:21; 109:2-21; 122:7-22. Examples of the progression of his analysis are as follows: Initial numbers Transactional Notice Professional fees for Transactional Notice Annual Revenue Report Drafts $30,000$50,000 (Exh. A. Barry Dep. 103:22106:25) $5,000-$7,000 (Id. at 134:15135:15) $15,000$25,000 (Id. at 107:722) Final Opinion (Exh. B, Exhibits A1; C1 to Barry Report) $5,000-$10,000 (Id. at 107:23-108:21) $3,000-$5,000 “None” (Exh. C, Barry Dep. Exhs. 5, 11) $8,500-$13,000 As Mr. Barry explained: 5 6 7 Q A So you had initial estimates which were based on your own experience of 50- or 30- to 50-? Right 8 8 9 10 11 12 13 14 15 16 17 Q A Q A Q A And then you had estimates of 15- to 25- based on your experience? Based on thinking about it more. And then you had your final estimate of 5- to 10- also based on your experience? Yes, and a limited outside validation. Did anything that the 17 companies you interviewed say to you factor into your decision to revise these numbers downwards? I would say no. … Exh. A, Barry Dep. 109:5-17. In James River, the Court excluded expert testimony when a real estate appraiser’s opinions as to the cost to rehabilitate a property and the cost to replace it after a fire were based only on his feelings about how much it would cost. 2009 WL 481688, at *8, 10 (noting that “[f]eelings, however, are not scientifically testable.”). Likewise, in Milne v. USA Cycling Inc., 575 F.3d 1120, 1134 (10th Cir. 2009), the Tenth Circuit upheld the exclusion of an expert’s testimony, which was based only on his experience with safety precautions, when the expert did not employ any studies, or refer to any other empirical of quantitative studies and the expert’s experience was with a different type of race. Mr. Barry’s opinions should be excluded for the same reasons. His failure to perform any studies, rely on any published literature, or conduct any sort of systematic methodology in estimating costs, renders his opinions unreliable, and they should be excluded pursuant to Rule 702. 9 III. DR. THOMAS J. ADLER’S OPINIONS ARE NOT RELIABLE BECAUSE HIS SURVEY METHODOLOGY WAS FATALLY FLAWED AND YIELDED UNRELIABLE DATA ON WHICH HE BASED HIS OPINIONS. Dr. Adler is President of Resource Systems Group, Inc. ("RSG"), which conducted a survey of Colorado consumers’ reaction to the Law (“the survey”). Exh. D, Adler Report and Decl. ¶¶1, 3. Dr. Adler offers testimony as to the propriety of the survey - that its methodology was appropriate, that it was conducted properly, and that the results were gathered properly and accurately reported. Id. at ¶¶4, 6-10. Dr. Adler also offers testimony summarizing the results of the survey. Id. at ¶ 5. Because the survey was not designed and implemented with the "same level of intellectual rigor" that would be used by an expert in the field, Dr. Adler's testimony regarding the survey and its results should be excluded. See Toni's Alpacas, 2010 WL 3730382 at *2. The creation of the survey was heavily influenced by DMA counsel. DMA counsel drafted the initial survey questions and sent two versions of them to Dr. Adler and RSG before the survey expert prepared any questions. Exh. F, Adler Dep. 16:2 18:16; Exh. E, Adler Dep. Exhs. 42, 43, 44. Dr Adler used those drafts for the survey's "core questionnaire." Exh. E, Adler Dep. Exh. 47, p.1. 7 Thereafter, he sent his draft questionnaire to a colleague, Nelson Whipple, for review. Exh. F, Adler Dep. 23:14 25:9; Exh. E, Adler Dep. Exhs. 48, 49. 8 Dr. Adler then sent a draft, known as Version 3, to DMA counsel. Exh. F, Adler Dep. 30:6 - 32:10. The survey questionnaire 7 Dr. Adler stated in his deposition that he was referring to the Law as well as the draft questionnaire, Exh. F, Adler Dep. 21:15 – 22, but his use of the word questionnaire in his email response to counsel calls into question this explanation. See also Exh. E, Adler Dep. Exh. 28. 8 Comments in the various redline versions reflect the initials of the commenter as follows: MS= Matthew Schaefer, TJA=Thomas J. Adler, and NJW=Nelson J. Whipple. 10 subsequently underwent additional changes suggested by DMA counsel, 9 who finally gave Dr. Adler the go-ahead to conduct the survey using the final Version 10. Exh. F, Adler Dep. 51:11 - 52:5; Exh. E, Adler Dep. Exhs. 67, 68. Despite all the revisions to the questionnaire, the survey suffers from four fatal flaws that render the methodology entirely unreliable because it: 1) contained a fatal confound, 2) contained false information, 3) suggested a response, and 4) did not follow survey standards. First, the survey does not permit an inference of causality because it contains a significant confound. Exh. G, Lichtenstein Decl. and Report, pp.12-17. The survey conflates consumer privacy concerns with consumer resistance to paying a higher price, due to tax owed. Id. Dr. Adler recognized that the survey needed to be designed to determine whether a seller's reporting of information to the Department would cause consumers not to make future purchase or whether this result would be caused by a resistance to paying taxes and thus higher prices on purchases. As he explained: 15 16 17 18 Q A Okay. Next is May 19th. Can you tell me what that note is? So this looks like kind of a repeat of the specific objectives, what we wanted to find out. 9 Version 3 was further revised after another phone conversation with DMA counsel, resulting in Version 4. Exh. E, Adler Dep. Exhs. 52, 53, 54; Exh. F, Adler Dep. 33:6 - 35:11. DMA counsel then made additional changes to the survey questionnaire and sent Version 5 to Dr. Adler. Exh. E, Adler Dep. Exhs. 9 53, 55; Exh. F, Adler Dep. 35:13 - 36:10. Dr. Adler revised the questionnaire again and sent Version 6 to counsel, who had yet more comments and changes. Exh. E, Adler Dep. Exhs. 56, 57; Exh. F, Adler Dep. 36:13 - 37:16. Counsel sent Version 7 to Dr. Adler with more comments and revisions and Adler incorporated these in Version 8. Exh. F, Adler Dep. 43:1 - 44:1; 44:16 -23; Exh. E, Adler Dep. Exhs. 58, 60. The next day, Dr. Adler sent a "final review version" of the questionnaire, Version 9, to DMA counsel incorporating changes discussed between them. Exh. F, Adler Dep. 45:5 – 21; Exh. E, Adler Dep. Exh. 62. Version 9a was created to address word processing issues. Exh. F, Adler Dep. 46:4 -23; Exh. E, Adler Dep. Exhs. 61, 63. Still unsatisfied with the questionnaire, DMA counsel requested a substantive change to the wording of Question 7 in Version 10. Exh. F, Adler Dep. 50:10 - 52:10; Exh. E, Adler Dep. Exhs. 65, 66. 11 19 20 21 22 23 Q A So you wanted to find out whether turning over the information will cause people to not purchase? Right, versus not paying taxes. We wanted to avoid getting into the issue of people not liking to pay taxes because we know nobody does. Exh. F, Adler Dep. 137:15 - 23; Exh. E, Adler Dep. Exh. 92, p.4. Nevertheless, Dr. Adler made no effort to factor tax avoidance or higher prices out of his survey methodology. The survey did not follow established methodologies for factoring out this confound. Exh. G, Lichtenstein Decl. and Report, p.15. With price/tax being a potential cause for consumer behavior, it should have been measured and “covaried” out. Id. Because the privacy concern cannot be disentangled from a price/tax concern, no conclusions may be drawn from the survey results. Id. at 13. Due to this serious confound alone, the survey methodology is “fatally flawed” and no inference of causality can be drawn from the data. Id. at 13. Second, the survey contained false information. One of the early survey questions asked respondents on an agree-disagree scale whether they minded “the State of Colorado knowing the kinds of products I buy, from whom I buy them, where I have them shipped, and how much I spend." Exh. D, Adler Report and Decl., Exh. B thereto, Final Survey Results, p.5 (emphasis supplied). Because the Law does not allow the reporting of products or even categories of products purchased, this question is misleading. Exh. G, Lichtenstein Decl. and Report, pp.17-18. Attempting to defend the question, Dr. Adler posited that since a retailer's name is disclosed, in some cases an inference might be drawn about the kind of product purchased. Exh. F, Adler Dep. 105:10-17. When pressed, however, Dr. Adler conceded that the question could have 12 been phrased in a way that was more consistent with the true requirements of the Law. Id. at 104:7 - 107:7. Third, the survey questions themselves suggest a response by prompting concerns about invasion of privacy. Exh. G, Lichtenstein Decl. and Report, pp.21-22. This type of “reactivity bias” in the methodology of the survey can result in a response that never would have been in the consumer’s head had it not been suggested by the question. Id. Further, suggestive questions in the survey likely influenced answers to later questions, rendering the entire survey unreliable. Id. at 18-22. Fourth, the survey failed to follow accepted survey standards because it did not provide sufficient information and did not provide a “don’t know” or “no opinion” response option. Id. at 24-27. For example, the survey asks “If you were to make a similar purchase in the future, but with this new disclosure requirement in place, what would you most likely do?” Exh. E, Adler Report and Decl., Exh. B, Final Survey Results, p.19. Based on this question, Dr. Adler reports that 63% of respondents would not make a subsequent purchase from an affected out-of state retailer. Id. at Decl. ¶5; 10 10 DMA's counsel’s hand is present again in Dr. Adler's declaration. It was initially drafted by counsel and later revised by counsel. Exh. F, Adler Dep. 73:10 - 75:6; Exh. E, Adler Dep. Exh. 82. In the final version of Dr. Adler's declaration, he describes the objectives of the survey as: 1) to determine whether Colorado consumers consider the Law's requirement that out-of-state retailers must report their purchasing information to the Department to be an invasion of privacy, and 2) to determine whether the reporting requirement will affect in any way Colorado consumer's future purchases from out-of-state retailers who are required to report such information to the Department. Exh. D, Adler Report and Decl. ¶ 3. This differs substantially from the stated survey objective. See Exh. E, Adler Report and Decl., Exh. B, Final Survey Results, p.3 (“Objective- Understand how a new Colorado law, and requirements imposed by [the Department] under the law, will impact Colorado consumers' decisions about purchasing from Internet and catalog retailers that do not collect Colorado sales tax.”). 13 Exh. B, Final Survey Results, p. 6. 11 However, the question does not provide sufficient contextual information that can affect a consumer’s purchasing decisions. Exh. G, Lichtenstein Decl. and Report, pp.23-24. Further, the absence of a “don’t know” or “no opinion” response option fails to reduce non-meaningful responses. Id. at pp.24-26. Due to the flawed survey methodology and the overriding influence of counsel at all stages of the expert process, Dr. Adler’s survey is unreliable and should be excluded. See Goebel, 346 F.3d at 992 (“Any step that renders the analysis unreliable... renders the expert’s testimony inadmissible. This is true whether the step completely changes a reliable methodology or merely misapplies that methodology.”). IV. PROFESSOR KEVIN LANE KELLER’S OPINIONS ARE NOT RELIABLE BECAUSE HE DID NOT USE A RELIABLE METHODOLOGY AND HIS OPINIONS ARE BASED UPON UNRELIABLE DATA. As discussed supra Part III, Dr. Adler’s survey, upon which Prof. Keller relied for his opinions, was fatally flawed. Because Prof. Keller relied upon this survey and performed no independent research of his own, his opinion is unreliable and should be excluded. Prof. Keller provides testimony that "out-of -state retailers affected by the Law will likely suffer lost sales and permanent injury to their relationships with customers as a result of complying with the requirement that they disclose their 11 After the survey was completed, the results were sent from RSG to DMA counsel asking if counsel needed any revisions. Exh. F, Adler Dep. 64:21- 65:4; Exh. E, Adler Dep. Exh. 74. A process similar to the survey questionnaire drafting process ensued. DMA counsel tailored and streamlined the report from the survey expert who admitted he "typically accepts comments and suggestions from [his] clients to do it to meet their purposes." Exh. F, Adler Dep. 68:6 - 69:2; Exh. E, Adler Dep. Exh. 76. The report was modified and a new version of the report sent to DMA counsel. Exh. F, Adler Dep. 70:15 - 71:2; Exh. E, Adler Dep. Exh. 79, p.2. Another draft of the report was created incorporating DMA's counsels' modifications. Exh. E, Adler Dep. Exh. 81. When the survey report was finalized, it contained all the changes suggested by DMA counsel. Exh. F, Adler Dep. 71:4 - 73:8; Exh. E, Adler Dep. Exhs. 21, 80, 82). 14 customers' purchasing information to the Department." Exh. I, Keller Report and Decl., Decl. ¶1. Rule 702, however, requires that the proponent demonstrate that the testimony is the product of reliable principles and methods and that the expert applied those principles and methods reliably to the case. Cartel Asset Mgm't, 2010 WL 3613819, at *2. Prof. Keller’s opinions do not meet this standard. Prof. Keller's opinions are not grounded in scientifically sound methods. See Toni’s Alpacas, 2010 WL 3730382, at *1. Rather, his methodology consisted of no more than discussing the case with DMA counsel 12 and reviewing the Complaint, Law, and survey. Exh. H, Keller Dep. 25:5 -14. He spent very little time formulating the substance of his opinions in this case 13 and did not review any literature, treatises, documents on survey methodology, nor any documents or writings on consumer behavior related to stated intentions. Id. at 25:14 - 29:23. Prof. Keller relied solely on the survey results and his past work done in preparing a revision of his textbook. When asked, however, to identify any material from his textbook on which he relied, he could 12 Prof. Keller’s only notes are from conversations with DMA counsel which demonstrate that he was simply given DMA's theory of the case. See Exh. J, Keller Dep. Exh. 17. (notes with terms such as "threatened irreparable harm, "nexus," and "physical presence - employees/facilities, agents"). 13 Prof. Keller worked a total of eight hours on this matter. Exh. H, Keller Dep. 24:17 - 25: l. Of those eight hours, some 40 minutes were spent reviewing DMA's complaint when he first received it, Exh. H, Keller Dep. 82:23 - 83:7, and after that, he read the complaint several additional times. Id. at 83:8-17. Prof. Keller also spent additional time reviewing version 9a of the survey questionnaire, id. at 67:4 72:23; Exh. J, Keller Dep. Exh. 17, and reviewing the declaration of Dr. Adler to be sure it aligned with his own declaration, Exh. H, Keller Dep. 100: 2 -14; Exh. J, Keller Dep. Exh. 29. He spent a significant amount of time on the telephone discussing the case with DMA's counsel. Exh. H, Keller Dep. 73:2 80:8, Exh. J, Keller Dep. Exh. 17. Further Prof. Keller "easily spent an hour or two" discussing his declaration, which DMA's counsel drafted, Exh. H, Keller Dep. 84:1 - 86:20, Exh. J, Keller Dep. Exhs. 2426, 16, and additional time corresponding with counsel via e-mail concerning the declaration, Exh. H, Keller Dep. 85:12 - 86:5; 96:13 - 100:14; Exh. J, Keller Dep. Exhs. 25, 29-31. He spent additional time conferring with DMA counsel via telephone and e-mail regarding drafts of his report prepared by DMA counsel. Exh. H, Keller Dep. 100:12 - 104:20, 107:12 - 109:17; Exh. J, Keller Dep. Exhs. 22, 32, 34, 3637. Given all this, Prof. Keller had a negligible period of time in which to formulate a reliable methodology or reliably apply that methodology to the data. 15 think of none. Id. Prof. Keller did not undertake any research to support any conclusion, or make any independent ascertainment that the data gathered by Dr. Adler was accurate or that the survey methodology was appropriate. Id. at 25:14 - 29:23. As discussed supra Part III, the survey questions were the product of DMA's counsel and Prof. Keller, who claims an expertise in survey methodology, provided little or no input. Exh. H, Keller Dep. 11:19 – 22. Further, Prof. Keller was unfamiliar with the methodology employed by Knowledge Networks, the company that conducted the survey for RSG, and before his deposition never saw the documents describing that methodology, including the calibration and weighting process applied to the results. Id. at 11:23 - 14:9; Exh. J, Keller Dep. Exhs. 18, 19. He simply took the survey report at face value, without inquiring about the survey process or conducting any research or investigation to test the results. Because he accepted the survey results at face value, he concluded that the perceived loss of privacy would change consumer purchasing behavior by the percentages announced in the survey results. Exh. H, Keller Dep. 11:23 - 14:9; 44:1 – 8; Exh. I, Keller Report and Decl., Decl. ¶¶5-9,13. Prof. Keller’s wholesale acceptance of the survey cannot form a reliable basis for his opinions due to the survey’s many flaws. See Crabbe, 556 F.Supp.2d at 1224 (when an expert relies on other opinions, “the component opinion must also be fullysupported under Rule 702 as well”). Professor Keller was unaware of the significant confound and improperly relied on the survey for a causational analysis. He testified that "we know two things and we know that [survey respondents] consider the act to 16 have negative effects on their privacy and we also know that they would subsequently be less likely to purchase from retailers and from that you can draw the conclusion of the relationship between the privacy and the ultimate, and its importance in their decisions to purchase from a retailer outside the state." Exh. H, Keller Dep. 53:13 - 21. At most, Dr. Adler’s survey established a correlation between an event (the Law) and consumer behavior (decreased purchases from affected retailers). The survey, however, simply was not appropriately designed to draw a causal connection between retailers’ reporting obligation and consumers' future purchasing behavior. Exh. G, Lichtenstein Decl. and Report, pp.13-17. As a result, no inference of causality can be drawn. Id. Dr. Keller thus lacked sufficient facts and data to support his opinion requiring exclusion of his expert testimony. See James River Ins. Co., 2009 WL 481688 at *7 (an expert cannot jump to an opinion without factual support). Dr. Keller's superficial involvement is further illustrated by his reliance on DMA counsels' statement of the objectives of the survey. His declaration prepared by counsel sets forth those objectives. Exh. J, Keller Dep. Exh. 17. Those stated objectives, however, differ from the announced objectives in the survey. Id. at Exh. 21. Further, Prof. Keller was unaware of the Department’s statutory obligation to keep information gathered in connection with tax collection confidential. Exh. H, Keller Dep. 42:7 - 22. Moreover, Dr. Keller did not rule out alternative causes for the consumer purchasing behavior measured by the survey. While Prof. Keller acknowledges that lost sales depend upon a number of factors, Exh. I, Keller Report and Decl.¶ 7, he never 17 referenced or analyzed those factors or what influence they may have on consumer behavior. See also Exh. H, Keller Dep. 42:23-44:8 (acknowledging the many factors that can affect consumer purchasing behavior). Neither Dr. Adler nor Prof. Keller investigated or explained why potential causes other than concern for privacy would not account for the predicted consumer behavior change. As a result, Prof. Keller’s opinions are not reliable. See Crabbe, 556 F.Supp.2d at 1223-4 (discussing one factor in the reliability analysis as whether the expert adequately accounted for obvious alternative explanations). Given the limited amount of time he spent analyzing the survey methodology and data, his failure to support or challenge his conclusions with any literature in the field, his failure to consider and rule out alternative explanations for predicted consumer behavior, and his wholesale reliance on a survey that is fundamentally flawed, Prof. Keller did not employ in his expert duties "the same level of intellectual rigor that characterizes the practice of an expert in his field." Toni's Alpacas, 2010 WL 3730382, at *2 quoting Goebel v. Denver and Rio Grande Western RR Co. at 992. Consequently, there is simply “too great an analytical gap” between the survey data and opinions proffered by Prof. Keller. See Crabbe, 556 F. Supp. 2d at 1223-4. While generally, the focus of a 702 inquiry is on an expert’s methodology rather than conclusions, nothing in Daubert or the Federal Rules of Evidence requires a district court to admit opinion evidence “connected to existing data only by the ipse dixit of the expert.” Goebel, 346 F.3d at 992 (citing General Electric Co. v. Joiner, 522 U.S., 136, 142 (1997)). 18 Accordingly, the Court should exercise its gatekeeper function and exclude Prof. Keller's testimony. WHEREFORE, Defendant respectfully requests this Court enter an Order excluding Plaintiff’s expert witnesses F. Curtis Barry, Thomas Adler, and Kevin Lane Keller pursuant to Federal Rule of Evidence 702. Respectfully submitted this 16th day of December, 2010. JOHN W. SUTHERS Attorney General s/ Melanie J. Snyder MELANIE J. SNYDER, 35835* Assistant Attorney General JACK M. WESOKY, 6001* Senior Assistant Attorney General Business & Licensing Section 1525 Sherman Street, 7th Floor Denver, Colorado 80203 Telephone: (303) 866-5273 (Snyder) Telephone: (303) 866-5512 (Wesoky) FAX: (303) 866-5395 E-Mail: E-Mail: STEPHANIE LINDQUIST SCOVILLE, 31182* Senior Assistant Attorney General Civil Litigation and Employment Law Section Telephone: 303.866.5241 FAX: 303.866.5443 E-Mail: *Counsel of Record Attorneys for Defendant 19 CERTIFICATE OF SERVICE I hereby certify that on December 16, 2010, I electronically filed the foregoing DEFENDANT’S MOTION TO EXCLUDE THE TESTIMONY OF PLAINTIFF'S EXPERT WITNESSES F. CURTIS BARRY, THOMAS ADLER, AND KEVIN LANE KELLER with the Clerk of the Court using the CM/ECF system which will send notification of such filing to the following e-addresses: Attorney for Plaintiff s/ Melanie J. Snyder 20