Rosetta Stone LTD v. Google Inc.

Filing 131

Memorandum in Support (Redacted) re 120 MOTION to Exclude Expert Report and Opinion of Dr. Kent Van Liere filed by Google Inc.. (Frieden, Jonathan) (klau, ). Modified text on 4/12/2010 re Under Seal copy received on 4/12/10 (UNDER SEAL)(klau, ).

IN THE UNITED STATES DISTRICT COURT
FOR THE EASTERN DISTRICT OF VIRGINIA
(Alexandria Division)
ROSETTA STONE LTD.
Plaintiff,
v.

CIVIL ACTION NO. 1:09cv736 (GBL / TCB)

GOOGLE INC.
Defendant.
GOOGLE INC.’S MEMORANDUM
IN SUPPORT OF ITS MOTION TO
EXCLUDE EXPERT REPORT AND OPINION OF DR. KENT VAN LIERE

TABLE OF CONTENTS
Page
I.

INTRODUCTION AND BACKGROUND ........................................................................1

II.

LEGAL STANDARD..........................................................................................................3

III.

ARGUMENT .......................................................................................................................4
A.

Dr. Van Liere’s Survey Tests Theories Of Confusion That Are Not At
Issue In This Case And Is Therefore Irrelevant .......................................................4

B.

Dr. Van Liere’s Survey Contains Fatal Methodological Flaws That Render
Its Conclusions Unreliable. ......................................................................................8
1.

Dr. Van Liere’s Survey Failed To Sufficiently Replicate Actual
Marketplace Conditions. ..............................................................................8

2.

Dr. Van Liere’s Survey Failed To Target The Appropriate
Universe Of Consumers. ............................................................................12

3.

Dr. Van Liere’s Survey Is Unreliable Because It Failed To Use An
Adequate Control Stimulus. .......................................................................12

4.

Dr. Van Liere’s Survey Results Were Premised On A Faulty
Definition Of Endorsement Confusion ......................................................17

CONCLUSION ..............................................................................................................................19
CERTIFICATE OF SERVICE ......................................................................................................20

i

TABLE OF AUTHORITIES
Page
Cases
CareFirst of Md., Inc. v. First Care, P.C.,
434 F.3d 263 (4th Cir. 2006) .................................................................................................6, 7
Cooper v. Smith & Nephew Inc.,
259 F.3d 194 (4th Cir. 2001) .....................................................................................................3
Daubert v. Merrell Dow Pharm., Inc.,
509 U.S. 579 (1993) ......................................................................................................... passim
Frosty Treats, Inc. v. Sony Computer Entm’t Am., Inc.,
426 F.3d 1001 (8th Cir. 2005) ...................................................................................................5
Holmes v. Wing Enterprises, Inc.,
2009 WL 1809985 (E.D. Va. 2009) ...........................................................................................3
KP Permanent Make-Up, Inc. v. Lasting Impression I, Inc.,
543 U.S. 111 (2004) ...............................................................................................................6, 7
Mary Kay, Inc. v. Weber,
601 F.Supp.2d 839 (N.D.Tex. 2009) ............................................................................... passim
Newman v. Motorola Inc.,
218 F. Supp. 2d 769 (D. Md. 2002) ...........................................................................................5
Sherman v. Westinghouse Savannah River Co.,
263 Fed. Appx. 357 (4th Cir. 2008) ...........................................................................................4
Simon Property Group L.P. v. mySimon, Inc.,
104 F. Supp. 2d 1033 (S.D. Ind. 2000) ............................................................................ passim
Smith v. Wal-Mart Stores, Inc.,
537 F. Supp. 2d 1302 (N.D. Ga. 2008) ............................................................................ passim
Starter Corp. v. Converse, Inc.,
170 F.3d 286 (2d Cir. 1999).......................................................................................................5
THOIP v. Walt Disney Co.,
__ F. Supp. 2d ___, 2010 WL 447049 (S.D.N.Y. Feb. 9, 2010) ...................................9, 12, 13
Scotts Co. v. United Indus. Corp.,
315 F.3d 264 (4th Cir. 2002) .....................................................................................................5
Trouble v. Wet Seal, Inc.,
179 F. Supp. 2d 291 (S.D.N.Y. 2001)..........................................................................12, 13, 15
U.S. v. Iskander,
407 F.3d 232 (4th Cir. 2005) .................................................................................................5, 7

ii

U.S. v. Simpson,
910 F.2d 154 (4th Cir. 1990) .....................................................................................................8
Wells Fargo & Co. v. WhenU.com, Inc.,
293 F. Supp. 2d 734 (E.D. Mich. 2003)...............................................................................9, 15
Statutes
Fed. R. Evid. 403 .....................................................................................................................1, 3, 4
Fed. R. Evid. 702 .........................................................................................................................1, 3

iii

INTRODUCTION AND BACKGROUND
The confusion opinion of Rosetta Stone’s purported survey expert, Dr. Kent Van Liere,
should be excluded, along with any testimony regarding his survey. The law is clear that expert
testimony must be useful to a jury to be admissible at trial. Daubert v. Merrell Dow Pharm.,
Inc., 509 U.S. 579 (1993); Fed. R. Evid. 403 and 702. It is Rosetta Stone’s burden to show that
Dr. Van Liere’s survey and testimony would be helpful to the jury. Rosetta Stone cannot meet
this burden.
Conducted before Rosetta Stone filed its original Complaint (and before Google adopted
its current trademark policy), Dr. Van Liere’s survey inquired whether respondents understood
sponsored links on a purported screen shot of a Google search results page either to be a Rosetta
Stone “company website” or “endorsed by” Rosetta Stone. Neither measure offers proof that
consumers are likely to be confused. The “company site” measure yielded a “net confusion” rate
of -2%. In other words, respondents who saw the Sponsored Links were less “confused” than
those who did not. The “endorsed” measure yielded a “net confusion” rate of 19%, which Dr.
Van Liere merged with the other result for a rate of 17%. However, apparently unbeknowst to
Dr. Van Liere at the time of his deposition, this Court dismissed Rosetta Stone’s endorsement
claim in September 2009, rendering all evidence on the “endorsement” measure irrelevant.
In addition, Dr. Van Liere’s study suffers from serious methodological flaws, which
render its results unreliable and inadmissible. Dr. Van Liere (1) utilized an altered screen shot
that failed to depict an accurate representation of a real world Google search results pages;
(2) used a control stimulus that had fewer referential links that related to Rosetta Stone software
than the test; (3) used a test population that was much less familiar with Rosetta Stone than
actual consumers searching for information about Rosetta Stone on Google would be; and (4)
used inconsistent definitions of “confused” to calculate his result. Dr. Van Liere’s survey and
1

testimony regarding his survey would, therefore, be unhelpful and confusing to the jury and
should be excluded.
Dr. Van Liere’s Survey
In the survey that Dr. Van Liere supervised, consumers were asked an initial set of
screening questions in eight malls around the country. Van Liere Report ¶¶ 17,19 attached to the
Declaration of Margret M. Caruso ("Caruso Decl."), attached as Exhibit A to Google Inc.'s
Motion for Summary Judgment, Ex. 45. Those individuals who passed were presented with
either a test stimulus (an altered Google search results page that included sponsored links) or a
control stimulus (an altered Google search results page that contained no sponsored links). Id. at
¶¶ 33, 37. To access the search results pages, users were shown a card with the words “Rosetta
Stone” on it and asked to type the words into a Google search engine. Id. at ¶¶ 24-25. The
Google search engine was linked to either the test stimulus screen shot or the control stimulus
screen shot. Id. As part of the study’s design, users could not click on the displayed links to see
what websites those links would take them to as they would have been able to do in an actual
real world search. Van Liere Deposition Transcript 49:8-19., attached to the Declaration of
Cheryl A. Galvin ("Galvin Decl.") attached hereto as Ex. A, Ex. 2.
After entering the words “Rosetta Stone” in the stimulated search engine, respondents
were asked whether they thought any of the links on the search results page sold the Rosetta
Stone product, even though they could not click on the link to see what web page it linked to.
Galvin Decl., Ex.2 ¶¶ 25, 26. Those who identified links other than the Rosetta Stone website in
response to that question, were then asked the remaining questions. Caruso Decl., Ex. 45,
Exhibit C. First, the remaining respondents were asked which, of those links they identified as
selling the Rosetta Stone product that they thought were the Rosetta Stone company website. Id.

2

The respondents were then asked, of the links they had identified as selling the Rosetta Stone
product, “which link or links, if any, do you think are endorsed by the Rosetta Stone company?”
Id. The respondents were not provided with a definition of “endorsed.” Galvin Decl.. Ex. 2
85:21-23; cf. Mary Kay, Inc. v. Weber, 601 F.Supp.2d 839, 848 (N.D.Tex. 2009) (striking Dr.
Van Liere’s confusion conclusion and noting that Dr. Van Liere did not provide a definition of
“affiliation” to respondents in his survey testing for affiliation confusion). After answering these
two questions, the respondents were asked to identify the reasons for their answers. Galvin
Decl.. Ex. 2 85:21-23. However, Dr. Van Liere did not use the responses to those open ended
questions to adjust his “confusion” calculation in any way. Id. 75:11-76:8. When asked why he
included those questions, he replied that courts expect to see them. Id. 75:11-22.
LEGAL STANDARD
Rosetta Stone must establish the admissibility of Dr. Van Liere’s survey and opinion
testimony by a preponderance of the evidence. Cooper v. Smith & Nephew Inc., 259 F.3d 194,
199 (4th Cir. 2001). Rosetta Stone cannot meet this burden.
Under Federal Rule of Evidence 702, a trial judge acts as a gatekeeper to “ensure that any
and all scientific testimony . . . is not only relevant, but reliable.” Id. (quoting Daubert v.
Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579, 588 (1993)). In its role as a gatekeeper, the
trial judge must conduct a “preliminary assessment of whether the reasoning or methodology
properly can be applied to the facts in issue.1” Daubert, 509 U.S. at 592-593. When analyzing
the reliability of an expert’s opinion, the inquiry “must be flexible and case-specific.” Holmes v.
1

The Supreme Court gave a non-exhaustive list of four factors that may be helpful to a
trial court when examining the reliability of an expert’s opinion: (1) whether a theory or
technique can be or has been tested; (2) whether it has been subjected to peer review and
publication; (3) whether a technique has a high known or potential rate of error and whether
there are standards controlling its operation; and (4) whether the theory or technique enjoys
general acceptance within a relevant scientific community. Daubert, 509 U.S. at 593-94.

3

Wing Enterprises, Inc., No. 1:08-cv-822., 2009 WL 1809985, *3 (E.D. Va. June 23, 2009)
(internal quotations and citations omitted).
“The court need not and should not respond reflexively to every criticism by saying it
merely ‘goes to the weight’ of the survey rather than its admissibility.” Simon Property Group
L.P. v. mySimon, Inc., 104 F. Supp. 2d 1033, 1039 (S.D. Ind. 2000). Where the flaws are “too
great,” the court may find under Federal Rule of Evidence 403 that “the probative value of the
survey is substantially outweighed by prejudice, waste of time, and confusion it will cause at
trial.” Id. In the Daubert context, courts must be acutely aware of the potential for prejudice:
“Expert evidence can be both powerful and quite misleading because of the difficulty in
evaluating it. Because of this risk, the judge in weighing possible prejudice against probative
force under Rule 403 of the present rules exercises more control over experts than over lay
witnesses.” Daubert, 509 U.S. at 595 (citation and quotation signals omitted). Moreover, in a
jury trial, “[t]he court has a responsibility to the jurors not to waste their time or to make their
task unduly difficult by admitting evidence that is likely to be complex and time-consuming . . .
when it offers essentially nothing of real probative value.” Simon Property Group, 104 F. Supp.
2d at 1039 n.3.
ARGUMENT
This Court should exclude Dr. Van Liere’s report because it is both irrelevant and
unreliable.
I.

DR. VAN LIERE’S OPINION IS BASED ON IRRELEVANT SURVEY
INQUIRIES
A.

Expert Opinion Must Be Relevant.

To be admissible, expert testimony must “assist the trier of fact to understand the
evidence or to determine a fact in issue.” Fed. R. Evid. 702; Daubert, 509 U.S. at 591. Where

4

experts opine on issues not relevant to the case, courts do not hesitate to exclude their testimony.
See Sherman v. Westinghouse Savannah River Co., 263 Fed. Appx. 357, 368-369 (4th Cir. 2008)
(excluding expert testimony when it did not address the narrow issue presented at trial); U.S. v.
Iskander, 407 F.3d 232, 238 (4th Cir. 2005) (excluding competent expert testimony because it
did not pertain to the personal income tax charge at issue); see also, Newman v. Motorola Inc.,
218 F.Supp. 2d 769, 781 (D. Md. 2002) (excluding expert testimony as irrelevant when the
factual assumptions relied upon did not mirror the facts of the case). In Daubert, the court
described this consideration as one of “fit,” noting that “‘[f]it’ is not always obvious, and
scientific validity for one purpose is not necessarily scientific validity for other, unrelated
purposes.” Daubert, 509 U.S. at 591.
In the context of Lanham Act claims, “fit” requires that an expert conducting a confusion
survey target the relevant question of confusion. Scotts Co. v. United Indus. Corp., 315 F.3d
264, 278-80 (4th Cir. 2002) (holding that a district court abused its discretion by crediting a
survey that failed to establish consumer confusion on a relevant question); Starter Corp. v.
Converse, Inc. 170 F.3d 286, 297 (2d Cir. 1999) (excluding survey evidence where the survey
was “little more than a memory test” and didn’t test whether there was a likelihood of
confusion); Frosty Treats, Inc. v. Sony Computer Entm’t Am., Inc., 426 F.3d 1001, 1010 (8th Cir.
2005) (holding that a survey that “fails to address the relevant inquiry” did not create a fact issue
for summary judgment); Mary Kay, 601 F.Supp.2d at 849 (striking survey results from Dr. Van
Liere where results failed to weed out irrelevant confusion).
In Scotts, for example, the relevant question facing the court was whether the defendant’s
packaging falsely conveyed the message the defendant’s product killed mature crabgrass. 315
F.3d at 279. However, the survey asked respondents whether they thought, based on viewing

5

the packaging, that the product would “prevent the growth of crabgrass.” Id. The court held that
this question failed to adequately target the critical issue because the phrase “prevent the growth
of” was ambiguous and answers to the question “shed no light on the question that is key to
Scotts’ false advertising claims.” Id.
Similarly, in Mary Kay, the court struck the confusion statistic calculated by Dr. Van
Liere because it included “legally irrelevant” confusion. 601 F.Supp.2d at 849. There, the issue
facing the court was whether defendant’s sale of Mary Kay cosmetics through her store on eBay
was likely to cause consumer confusion as to the affiliation between defendant’s store and Mary
Kay. Id. Dr. Van Liere calculated a 45% confusion rate as to such affiliation. Id. The court
struck this conclusion because it included respondents who reported believing defendant’s store
and Mary Kay were affiliated solely because defendant’s store sold Mary Kay products. Id. The
court agreed and held that because it was lawful for the defendant to re-sell her genuine Mary
Kay products, “responses of interviewees who believed affiliation existed solely because the
website sells Mary Kay products are inadmissible.” Id at 848. The court further held that [t]he
legally irrelevant confusion must be weeded out before the evidence can be presented to the jury,
and “confusion that stems solely from the fact that the [defendants] are reselling Mary Kay
products is not legally relevant and might confuse the jury.” Id at 849.
B.

Dr. Van Liere’s Opinion Is Not Based On Relevant Information.

Dr. Van Liere’s conclusions about his survey data “shed[] no light on the question that is
key to” Rosetta Stone’s trademark infringement claim—confusion as to source or origin of
goods. KP Permanent Make-Up, Inc. v. Lasting Impression I, Inc., 543 U.S. 111, 117 (2004)
(holding that “proof of infringement as defined in section 1114 . . . requires a showing that the
defendant’s actual practice is likely to produce confusion in the minds of consumers about the
origin of the goods or services in question.”); CareFirst of Md., Inc. v. First Care, P.C., 434 F.3d
6

263, 267 (4th Cir. 2006) (“Likelihood of confusion exists if ‘the defendant’s actual practice is
likely to produce confusion in the minds of consumers about the origin of the goods or services
in question.’”). Instead, Dr. Van Liere’s survey focused on whether respondents thought links on
an altered screen shot of a Google search results page were endorsed by Rosetta Stone.
Although Rosetta Stone initially brought a false endorsement claim under 15 U.S.C.
1125(a), the Court dismissed that claim. Galvin Decl., Ex.1 28:24-29:3. Thus, the question that
remains in this case—and the question that would be relevant to a survey or opinion testimony—
is whether users are confused as to the source or origin of the goods advertised. See KP
Permanent Make-Up, Inc., 543 U.S. 111 at 116; CareFirst of Md., Inc., 434 F.3d at 267. Dr.
Van Liere’s opinion fails to address this question.
Dr. Van Liere’s analysis of the survey data yields a “net confusion” rate of 17%--which
derives entirely from the “endorsement” measure. Caruso Decl., Ex. 45 ¶ 44; Expert Report of
Edward A. Blair, Ph.D., attached to Declaration of Edward A. Blair, Ph.D. ("Blair Decl.")
attached hereto as Ex. B, Ex. A 5-6. Because Rosetta Stone’s operative pleading no longer
contains a claim based on “endorsement,” Dr. Van Liere’s survey and any opinion testimony
about it lack any probative value and should be excluded. Further, Dr. Van Liere repeats the
same mistake he made in Mary Kay, 601 F.Supp.2d at 848-49—failing to exclude from his
confusion count those whose replies were based simply on the fact that respondents attributed
some affiliation, here, “endorsement,” to their assumption that the links sold Rosetta Stone
products, offered discounts on it, or were reputable merchants—in other words, were
commercially affiliated. Those respondents who gave such an answer in their open ended
questions should have been “weeded out,” Id. at 849, based on the first sale doctrine and body of

7

case law referenced in Google’s Memorandum In Support of Summary Judgment, Section I.A.
This adjustment would drop the “net confusion” rate to 5%. Blair Decl., ¶¶ 13-14.
Even if there were some probative value to the survey, any such value is far outweighed
by the potential for unfair prejudice to Google and confusion of the jury. See, U.S. v. Iskander,
407 F.3d 232 at 238-39 (affirming a district court’s exclusion of expert testimony that was
“potentially confusing to the jury” because it did not relate to the issues to be decided at trial);
See also, U.S. v. Simpson, 910 F.2d 154, 158 (4th Cir. 1990) (reversible error to allow evidence
that presented a danger that the jury would “make a decision on the basis of a factor unrelated to
the issues properly before it”); Mary Kay, 601 F.Supp.2d at 849 (holding that irrelevant
confusion “might confuse the jury”). Dr. Van Liere’s inclusion of “endorsement” confusion
results in his survey poses a risk that the jury will be confused by the survey’s general conclusion
of 17% net confusion and fail to draw distinctions between endorsement confusion, which is not
at issue in this case, and confusion as to the source or origin of goods, which is at issue. Thus, his
survey and any opinion testimony about his survey should be excluded.
II.

DR. VAN LIERE’S SURVEY CONTAINS FATAL METHODOLOGICAL
FLAWS
Even if the survey did measure relevant confusion, its results are wholly unreliable

because it (1) failed to adequately approximate actual market conditions, (2) used a control
stimulus that materially differed from the test stimulus; (3) failed to target the appropriate
universe of consumers; and (4) used a definition of endorsement confusion that biased the
results.
A.

Dr. Van Liere’s Survey Failed To Sufficiently Replicate Actual Marketplace
Conditions.

Dr. Van Liere’s failure to replicate actual market conditions in designing his survey
renders it unreliable. A valid likelihood of confusion survey must “take into account
8

marketplace conditions and typical consumer behavior so that the survey may as accurately as
possible measure the relevant thought processes of consumers encountering the disputed mark
. . . as they would in the marketplace.” Smith v. Wal-Mart Stores, Inc., 537 F. Supp. 2d 1302,
1327 (N.D. Ga. 2008) (excluding a survey “so flawed that it does not establish a genuine issue of
material fact with regard to actual confusion, much less prove actual confusion”). “A survey that
fails to adequately replicate market conditions is entitled to little weight, if any.” Wells Fargo &
Co. v. WhenU.com, Inc., 293 F. Supp. 2d 734, 765 (E.D. Mich. 2003) (denying a preliminary
injunction in part because the surveys failed to “provide reliable evidence of likelihood of
confusion”). Failure to adequately approximate actual market conditions can form a proper basis
for exclusion. THOIP v. Walt Disney Co., __ F. Supp. 2d ___, 2010 WL 447049, *12 (S.D.N.Y.
Feb. 9, 2010) (excluding a survey that “failed to sufficiently replicate the manner in which
consumers encountered the parties’ products in the marketplace”); Simon Property Group, 104 F.
Supp. 2d 1033, 1052 (excluding a survey where it bore “no reasonable relation to situations in
which consumers might actually be exposed to the parties’ trademarks in the marketplace”).
A survey relating to Internet shopping must accurately reflect normal consumer online
shopping behavior, rather than forcing respondents to engage in scripted, unnatural website
interactions. See Smith, 537 F. Supp. 2d at 1319-20; Simon Property Group, 104 F. Supp. 2d at
1052. In Smith, for example, the court criticized a survey that artificially instructed respondents
to type in specific search terms that would take them to the plaintiff’s site, and then forced the
respondents to take a series of scripted steps designed to bring them to a page displaying the
plaintiff’s product, where the respondent was asked a series of confusion questions. Smith, 537
F. Supp. 2d at 1319-20. Similarly, in Simon Property Group, the court excluded a survey that
artificially presented the plaintiff’s and defendant’s websites sequentially to the respondent

9

without any showing that consumers would normally view these sites in such a manner. 104 F.
Supp. 2d at 1052. Though the plaintiff argued that consumers could encounter the two sites
together on search engine results pages, the court found that such results pages would present the
consumer with countless other unrelated pages. Id. at 1042-43. Accordingly, “[t]he survey
would distort that experience by presenting only those two home pages together” and “by
removing the additional information available to help sort through those results.” Id. at 1044.
Here, as in Smith and Simon Property Group, Dr. Van Liere’s survey conditions failed to
adequately approximate normal online shopping behavior. Respondents in the survey were first
shown the Google search page and told to enter “Rosetta Stone” as a search term. Caruso Decl.,
Ex. 45 ¶ 25. The survey then presented respondents with a manipulated image of a Google
search results page that had no clickable links. Id. ¶ 25 n.10, Exhibit D. While looking only at
that image, the respondents were asked the survey questions. Id. ¶¶ 26-28.
As the court in Smith recognized, “[a] great majority of Internet users arrive at a
particular website after searching specific terms via an Internet search engine or by following
links from another website” and the “user makes a judgment based on contextual cues . . . in
determining where to surf next.” Smith, 537 F. Supp. 2d at 1328 (emphasis added). Similarly,
when searching for information on Google, consumers can, and do, click through sponsored links
to determine if an ad is relevant to their search and to find more information about the advertised
product or service. The context of the web site being advertised can provide the consumer with
important information that can dispel confusion about the advertised product or services.
Depriving the consumer of this important information “distorts the experience” in a way that
undermines the reliability of the survey. See Simon Property Group, 104 F. Supp. 2d at 1044

10

(“The obvious effect of these distortions would be to exaggerate any confusion that might be
detected, which thoroughly undermines the reliability of the surveys.”).
Moreover, Dr. Van Liere represents that the test stimulus was “an actual search results
page,” but he concedes that he manipulated the image by removing the Rosetta Stone sponsored
link from the top position in the test stimulus. Caruso Decl., Ex. 45 ¶ 33 n.12. Not deleting that
sponsored link would have better replicated an actual marketplace condition and would have
provided context and reference for the other paid listings. Smith, 537 F. Supp. 2d at 1328; Simon
Property Group, 104 F. Supp. 2d at 1044. Dr. Van Liere’s interference with the actual
sponsored link thus likely influenced the survey results. Blair Decl., Ex. A 10. Indeed, Rosetta
Stone’s own witnesses acknowledge the importance of including the Rosetta Stone sponsored
link. For example, Eric Duehring, Rosetta Stone’s General Manager and Vice President for
Consumer Sales in the United States, testified that Rosetta Stone’s presence as the top sponsored
link helps dispel consumer confusion. Duehring Deposition Transcript, Galvin Decl, Ex. 4
111:25-112:22.
Nino Ninov, Rosetta Stone’s Vice President of Strategic Research, who is the senior
manager at Rosetta Stone responsible for creating and administering surveys for Rosetta Stone
also testified about the importance of replicating actual market conditions as closely as possible
and the importance of including the Rosetta Stone sponsored link. Mr. Ninov has significant
experience in market research, including conducting such research for the Marine Corps
Community Services at Quantico as a senior research analyst. Ninov Deposition Transcript,
Galvin Decl., Ex. 5 10:14-12:11. Mr. Ninov has associate degrees in financial management and
accounting, a masters in business administration from University of Virginia, and a masters
degree in sociology. Id. 8:25-9:24. Mr. Ninov began at Rosetta Stone as Director, Market

11

Research in 2004, was promoted to Senior Director, then to Vice President of Strategic Research,
reporting directly to Rosetta Stone’s CEO. Id. 14:16-15:23.
Mr. Ninov did not personally review the methodology employed by Dr. Van Liere in this
case, but he assumed the Rosetta Stone sponsored link was included in the experiment. Galvin
Decl., Ex. 5 68:22-70:25. When asked why that was his understanding, he responded, “It must
be.” Id. 69:1. He then explained the importance of representing a search page accurately,
testifying that it is very important to pay attention to details when administering surveys because
“people[’s] perceptions and reactions change sometimes based on fairly small things.” Id. 70:2372:3. He also said that if he had done the study, he would have included a Rosetta Stone
sponsored link, elaborating: “It’s not rocket science at the end of the day. It’s just fair
representation. Take [a] snapshot and put it there.” Id. 70:23-24, 72:6-8. “If you’ve done this,
you’ve done your job.” Id. 72:2-3.
Yet Dr. Van Liere chose not to follow this obvious survey design choice. Instead, as in
the excluded survey in Simon Property Group, he “distort[ed]” the user’s experience “by
removing the additional information available to help sort through those results,” thus rendering
the survey unreliable. 104 F. Supp. 2d at 1044.
B.

Dr. Van Liere’s Survey Is Unreliable Because It Failed To Use An Adequate
Control Stimulus.

A fundamental flaw in Dr. Van Liere’s survey was his failure to use an adequate control
stimulus to filter out the background noise in the survey. This alone is grounds for excluding Dr.
Van Liere’s opinions. THOIP, __ F. Supp. 2d. ___, 2010 WL 447049 at *14. A proper control
should “share as many characteristics with the experimental stimulus as possible, with the key
exception of the characteristic whose influence is being assessed.” Id. (citing Shari Seidman
Diamond, Reference Guide on Survey Research, in Reference Manual on Scientific Evidence at

12

258 (Federal Judicial Center 2d ed. 2000)). In THOIP, the court excluded a survey testing
whether the defendant’s use of the particular words “Miss” and “Little Miss” in conjunction with
cartoon characters on a t-shirt was confusing. 2010 WL 447049 at *14 The court held that the
control shirts were too dissimilar from the test shirts because, where the test shirts had words and
cartoon characters on them, the control shirts only had cartoon characters. Id. Thus, although
the control was quite similar to the test, with the exclusion of allegedly infringing words, the
control failed because it did not substitute noninfringing words for the allegedly infringing
words. Id. As such, the survey in THOIP did not provide meaningful evidence of whether the
allegedly infringing words themselves were likely to cause confusion or whether it was merely
the presence of similar words. Id. So too here. Dr. Van Liere’s control did not sufficiently
isolate the allegedly infringing activity from other circumstances that could contribute to
“confusion.” Blair Deposition Transcript, Galvin Decl., Ex. 7 99:17-100:21.
Dr. Van Liere’s survey purported to measure the effect of the presence of sponsored links
on consumer confusion. Caruso Decl., Ex. 45 ¶ 8. To create the control stimulus, Dr. Van Liere
simply removed the sponsored links from the screenshot. Id. ¶ 37. This was not a proper control
stimulus because it failed to include a representative depiction of links actually referring to
Rosetta Stone (the plaintiff) and links not referring to Rosetta Stone (the plaintiff), such as third
parties that use “Rosetta” in their own marks or on websites referring to the Rosetta Stone
artifact.
Given the screenshot Dr. Van Liere selected and his methodology, far fewer organic links
refer to the plaintiff Rosetta Stone in the control condition than in the test condition—two versus
seven. Blair Decl., Ex. A 3-4; Caruso Decl., Ex. 45 Exhibit D. Moreover, the two organic links
that refer to the plaintiff Rosetta Stone are (1) the actual Rosetta Stone website and (2) the

13

Wikipedia page discussing the Rosetta Stone software, both of which Dr. Van Liere deemed
ineligible confusion responses. Galvin Decl., Ex. 2 84:18-85:7. Thus, the control that Dr. Van
Liere created by removing the sponsored links failed to contain any uses of “Rosetta Stone” that
were not either actually “endorsed” by Rosetta Stone or referred to the plaintiff Rosetta Stone or
its products. As a result, Dr. Van Liere’s control stimulus was highly unlikely to produce any
evidence of net confusion pursuant. See Blair Decl., Ex. A 6-7.
Dr. Van Liere defends this design by saying that the screen shot simply depicts actual
search results without sponsored links. Galvin Decl., Ex. 2 58:18-21. However, this is not
accurate. Dr. Van Liere was provided with a selection of twelve different screen shots from
which he could have chosen a control stimulus. Caruso Decl., Ex.8-19. Four of those screen
shots contained links to Amazon.com in both the sponsored links and organic links. Caruso
Decl., Ex. 11, 13, 16, 17. Thus, there were screen shots that had a more balanced content, but
Dr. Van Liere chose to use as a control one that had no commercial referential uses of “Rosetta
Stone” except for the company’s actual site.
This fundamental flaw in the design of the control stimulus practically dictated a high
level of “net confusion” based on the content of the links. Indeed, analysis of the survey results
on an ad-by-ad basis confirms this, as the “confusion rates” among the different sponsored links
vary substantially from each other and range from 21% endorsement and 15% company website
confusion for Amazon.com, an authorized Rosetta Stone reseller, to 1% endorsement and 0%
company website confusion for About.com, an information site. Blair Decl., Ex. A 8. Dr Van
Liere’s lack of an adequate control and failure to account for it renders the results of this survey
wholly unreliable.

14

The extent of this flaw is evident when Dr. Van Liere’s filter question of “Which link or
links if any do you think sells Rosetta Stone language software products?” is properly taken into
account. Caruso Decl., Ex. 45 Exhibit C; Blair Decl. ¶¶ 8-11. Dr. Van Liere testified in his
deposition that he asked this filter question to focus the respondents’ attention on commercial
listings. Galvin Decl., Ex. 2 89:23-90:14. Only those respondents who answered “Yes” were
then asked the confusion questions for that link, while the rest were filtered out of the survey.
Caruso Decl., Ex. 45 ¶¶ 27-28. Yet Dr. Van Liere failed to account for this filter question when
calculating the confusion rates. Instead he counted the filtered-out individuals towards the notconfused population, even though they were never given the opportunity to say they were
confused. Id. ¶¶ 41-43; Blair Decl. ¶ 9. Since only two of the eleven links in the control
stimulus referred to Rosetta Stone, control respondents were much more likely to be filtered out
of the survey.
When the filtering is properly taken into account, the survey actually demonstrates
minimal confusion, with 75% confusion in the test condition and 73% confusion in the control
condition, resulting in 2% net confusion. Blair Decl. ¶ 10. These numbers show that the flawed
control design substantially affected the survey results and render it wholly unreliable.
Admission of the survey into evidence would only serve to confuse the jury and unfairly
prejudice Google.
C.

Dr. Van Liere’s Survey Failed To Target The Appropriate Universe Of
Consumers.

Selection of the proper universe is “one of the most important factors in assessing the
validity of a survey and the weight that it should receive because the persons interviewed must
adequately represent the opinions which are relevant to the litigation.” Smith, 537 F. Supp. 2d at
1323.; See Wells Fargo & Co. v. WhenU.com, Inc., 293 F. Supp. 2d at766 ([t]o have substantial

15

probative value, a survey . . . must . . . test for confusion by replicating marketplace conditions.”)
(citations omitted); Trouble v. Wet Seal, Inc., 179 F.Supp. 2d 291, 307-308, n.11 (S.D.N.Y.
2001) (conducting survey in mall where defendant’s stores were located surveys the wrong
universe because the parties cater to different socioeconomic markets; survey was excluded).
Dr. Van Liere’s selection of the universe for the Rosetta Stone survey was overbroad and
included respondents who were less familiar with the Rosetta Stone brand than appropriate.
Blair Decl., Ex. A 11-12.
The appropriate universe should have been limited to those individuals who would
normally search for “Rosetta Stone” on Google to find information about Rosetta Stone products.
This necessarily requires that such consumers be able to independently recall Rosetta Stone’s
brand. Blair Decl.,Ex. A 12. The screening questions used in the survey did not require
independent brand recall. Caruso Decl., Ex. 45 Exhibit C at 4. Instead, they simply asked
whether the respondent had heard of Rosetta Stone before. Id. The population of respondents
who could independently recall the Rosetta Stone brand would generally be more familiar with
the brand, which, in turn, could have had a material effect on the level of confusion measured by
the survey. Accordingly, this error diminishes the probative value of the survey. Blair Decl., Ex.
A 12.
Dr. Van Liere also failed to sample a population that was representative of consumers
who actually might be looking for information about the Rosetta Stone product. Dr. Van Liere
states that his survey sampled people from malls and that “[o]nly malls that did not include a
Rosetta Stone kiosk or stand-alone cart were included.” Caruso Decl., Ex. 45 Report ¶
17(emphasis added). No reason is given for why only malls without Rosetta Stone products
offered for sale were chosen, and malls that do not sell the Rosetta Stone product are not likely to

16

draw the population of individual consumers who typically seek out Rosetta Stone’s products.

Selecting a mall that actually
had a Rosetta Stone kiosk would naturally have drawn a sample more closely akin to the actual
population interested in Rosetta Stone and who could actually identify, unaided, the Rosetta
Stone brand. Dr. Van Liere’s seemingly arbitrary choice not do so when he could have further
undermines the reliability of his survey results.
D.

Dr. Van Liere’s Survey Results Were Premised On A Faulty Definition Of
Endorsement Confusion

Dr. Van Liere failed to provide his respondents with a definition of “endorsement” when
he asked them whether certain links were endorsed by Rosetta Stone. Galvin Decl., Ex. 2 85:2123. However, Dr. Van Liere testified that “endorsement generally includes the notion of
supporting or approving another party’s work or activity.” Id. 85:8-20. Dr. Van Liere did not,
however, apply that definition when analyzing the results of his survey to determine confusion,
instead adopting an inconsistent definition of endorsement confusion which, on the one hand,
included sponsored links of authorized resellers and affiliates as “confused,” while on the other,
excluded the organic search link of the Wikipedia page about Rosetta Stone (the plaintiff).
Dr. Van Liere explained in his report that the Wikipedia link is “endorsed by Rosetta
Stone,” and thus, respondents who thought it was endorsed by Rosetta Stone were not counted as
confused. Caruso Decl., Ex. 45 ¶¶ 38-40. However, before being asked the endorsement
confusion question, respondents must have first answered that they thought the Wikipedia page

17

sold Rosetta Stone software – an incorrect answer that indicates (deep) confusion.2 Yet Dr. Van
Liere counted them as not confused.
Accepting Dr. Van Liere’s definition of confusion with respect to the Wikipedia link was
appropriate, he erred in failing to consistently apply that definition to all sponsored links. Dr.
Van Liere testified in his deposition that Rosetta Stone was “endorsing” the content of the
Wikipedia link because it was “monitoring it and contributing to it.” Galvin Decl., Ex. 2 84:2385:7. Under such a definition, Amazon.com and CouponCactus should have been considered
“endorsed” as well.

Given these official relationships, Dr. Van Liere should have considered them endorsed as well.
Blair Decl. ¶ 16. Had Dr. Van Liere appropriately treated Amazon.com and CouponCactus as
endorsed by Rosetta Stone, his survey would have revealed -3% net confusion. Id. ¶ 17. In
other words, with this adjustment, it would be clear that the control respondents were more
confused than the test group who were shown the sponsored links.
Dr. Van Liere’s error in calculating confusion plainly resulted in a substantially inflated
net endorsement confusion percentage and seriously calls into question the results of the survey.

2

Dr. Van Liere attempts to explain his error away by pointing out that he also made the
same error with respect to the sponsored links. Galvin Decl. Ex. 2 89:14-90-2. These two errors
do not necessarily cancel each other out, however, because of the imbalance in referential versus
non-referential, non-commercial links in the test and control stimuli.

18

CONCLUSION
Dr. Van Liere’s focus on irrelevant confusion and the litany of errors in the design and
methodology of his study render it unreliable, unhelpful and will be confusing to the jury. Dr.
Van Liere’s expert report and testimony regarding likelihood of confusion therefore fail to
satisfy the requirements of Daubert and Federal Rules of Evidence 702 and 403. Accordingly,
this Court should exclude Dr. Van Liere’s survey and any testimony regarding such survey.
Respectfully Submitted,
GOOGLE INC.
By counsel
/s/
Jonathan D. Frieden, Esquire (VSB No. 41452)
Stephen A. Cobb, Esquire (VSB No. 75876)
ODIN, FELDMAN & PITTLEMAN, P.C.
9302 Lee Highway, Suite 1100
Fairfax, Virginia 22031
[phone]
[phone] (facsimile)
[e-mail]
[e-mail]
Margret M. Caruso, Esquire (Admitted Pro Hac
Vice)
QUINN, EMANUEL, URQUHART &
SULLIVAN, LLP
555 Twin Dolphin Drive, 5th Floor
Redwood Shores, California 94065
[phone]
[phone] (facsimile)
[e-mail]
Counsel for Defendant Google Inc.

19

CERTIFICATE OF SERVICE
I HEREBY CERTIFY that on the 26 day of March, 2010, I will electronically file the
foregoing with the Clerk of Court using the CM/ECF system, which will then send a notification
of such filing (NEF) to the following:
Warren T. Allen II
Clifford M. Sloan
Jennifer L. Spaziano
Skadden, Arps, Slate, Meagher, & Flom, LLP
1440 New York Avenue, N.W.
Washington, D.C. 20005-2111
[e-mail]
[e-mail]
[e-mail]
Counsel for Plaintiff Rosetta Stone Ltd.

/s/
Jonathan D. Frieden, Esquire (VSB No. 41452)
Stephen A. Cobb, Esquire (VSB No. 75876)
ODIN, FELDMAN & PITTLEMAN, P.C.
9302 Lee Highway, Suite 1100
Fairfax, Virginia 22031
[phone]
[phone] (facsimile)
[e-mail]
[e-mail]
Counsel for Defendant Google Inc.

20