Apple Inc. v. Samsung Electronics Co. Ltd. et al

Filing 278

Reply Declaration of Ravin Balakrishnan, Ph.D. in Support of #86 Apple's Motion for Preliminary Injunction filed by Apple Inc.. (Attachments: #1 Exhibit A - Manual Filing Notification, #2 Exhibit B - Manual Filing Notification, #3 Exhibit C - Manual Filing Notification, #4 Exhibit D, #5 Exhibit E, #6 Exhibit F, #7 Exhibit G, #8 Exhibit H, #9 Exhibit I, #10 Exhibit J, #11 Exhibit K, #12 Exhibit L, #13 Exhibit M, #14 Exhibit N, #15 Exhibit O)(Jacobs, Michael) (Filed on 9/30/2011) Modified on 10/3/2011 linking entry to document #86 EXHIBIT A, B AND C HAVE NOT BEEN MANUALLY SUBMITTED TO THE CLERK'S OFFICE FOR FILING (dhm, COURT STAFF).

1
2
3
4
5
6
7
8

UNITED STATES DISTRICT COURT

9

NORTHERN DISTRICT OF CALIFORNIA

10

SAN JOSE DIVISION

11
12

APPLE INC., a California corporation,

13
14
15
16
17
18
19

Plaintiff,
v.
SAMSUNG ELECTRONICS CO., LTD., A
Korean business entity; SAMSUNG
ELECTRONICS AMERICA, INC., a New York
corporation; SAMSUNG
TELECOMMUNICATIONS AMERICA, LLC, a
Delaware limited liability company,

Case No.

11-cv-01846-LHK

REPLY DECLARATION OF
RAVIN BALAKRISHNAN, PH.D.
IN SUPPORT OF APPLE’S
MOTION FOR A PRELIMINARY
INJUNCTION

Defendants.

20
21
22
23
24
25
26
27
28
REPLY DECLARATION OF RAVIN BALAKRISHNAN, PH.D. IN SUPPORT OF MOTION FOR A PRELIMINARY INJUNCTION
Case No. 11-cv-01846-LHK

1

I, RAVIN BALAKRISHNAN, do hereby declare as follows:

2

I.

3

INTRODUCTION
1.

I am a tenured Professor in the Department of Computer Science at the University

4

of Toronto, and have been asked to provide a reply expert declaration on behalf of Apple Inc.

5

(“Apple”) in the above-captioned case. I understand that Apple has alleged that Defendants

6

Samsung Electronics Co. Ltd., Samsung Electronics America, Inc., and Samsung

7

Telecommunications America, LLC (collectively, “Samsung”) have engaged in patent

8

infringement. I also understand that Samsung has alleged that Apple’s patent is invalid and not

9

infringed.

10

2.

I submit this reply declaration in response to the declarations of Drs. Andries van

11

Dam, Benjamin B. Bederson, and Jeffrey Johnson. I reserve the right to supplement or amend

12

this declaration if additional data or other information that affects my opinions becomes available.

13

I may testify at a hearing regarding the matters expressed in this declaration and any supplemental

14

declarations that I may prepare for this litigation. I also may prepare and rely on audiovisual aids

15

to demonstrate various aspects of my testimony at a hearing. I also may testify with respect to

16

any matters addressed by any expert testifying on behalf of Samsung, if asked to do so. If I am

17

asked to testify at trial, I intend to prepare an expert report in compliance with Federal Rule of

18

Civil Procedure 26(a)(2) according to the schedule to be set by the Court.

19

3.

I am being compensated at my standard consulting rate of $430 per hour for my

20

work in connection with this action. My compensation is not based in any way on the outcome of

21

the litigation.

22

II.

23

QUALIFICATIONS
4.

A current, true, and correct copy of my curriculum vitae was attached to my

24

opening declaration in support of Apple’s Motion for a Preliminary Injunction (“Opening

25

Declaration”) as Exhibit 1. Rather than repeat the detailed description of my qualifications

26

outlined in my Opening Declaration, I incorporate that description by reference.

27
28

REPLY DECLARATION OF RAVIN BALAKRISHNAN, PH.D. IN SUPPORT OF MOTION FOR A PRELIMINARY INJUNCTION
Case No. 11-cv-01846-LHK

1

1

III.

MATERIALS CONSIDERED
5.

2

In forming my opinions and views expressed in this reply declaration, I reviewed

3

(1) the materials listed in paragraphs 14-16 of my Opening Declaration, (2) the non-confidential

4

declarations, with non-confidential exhibits, of Andries Van Dam, Benjamin Bederson, and

5

Jeffrey Johnson in support of Samsung’s Opposition to Apple’s Motion for a Preliminary

6

Injunction, and (3) the non-confidential deposition transcripts, with non-confidential exhibits, of

7

Andries Van Dam, Benjamin Bederson, Jeffrey Johnson, and Bas Ording.
6.

8
9

I have also reviewed the materials submitted by Dr. Bederson related to the

LaunchTile and XNav software, WO 03/081458 and U.S. Patent No. 7,872,640 to Lira (“Lira”),

10

International Publication Number WO 01/29702 and U.S. Patent No. 7,152,210 to Van Den

11

Hoven (“Van Den Hoven”), and U.S. 2008/0104544 A1 (“Collins”). Finally, I have reviewed and

12

considered U.S. Patent No. 7,965,263 and U.S. 2007-0080907. Attached hereto as Exhibits A-C

13

are videos demonstrating certain features of the LaunchTile software. True and correct copies of

14

WO 03/081458, WO 01/29702, US 2008/0104544 A1, U.S. Patent No. 7,965,263, and U.S. 2007-

15

0080907 are attached hereto as Exhibits D-H. True and correct copies of excerpts from the

16

Bederson and Van Dam depositions are attached hereto as Exhibits I & J. True and correct copies

17

of Bederson Deposition Exhibits 211, 212, 213 & 222 are attached hereto as Exhibits K-N.

18

Lastly, a true and correct copy of Bederson Declaration Exhibit A is attached hereto as Exhibit O.

19

IV.

20

LEGAL PRINCIPLES
7.

I have not been asked to offer an opinion on the law; however, as an expert

21

assisting the Court in determining infringement, I understand that I am obliged to follow existing

22

law. I have therefore been asked to apply the legal principles outlined in paragraphs 18-20 of my

23

Opening Declaration to my analysis of infringement. Those paragraphs are incorporated herein

24

by reference.

25

8.

I have also been asked to apply the following legal principles to my analysis of

26

patent validity under § 102 (anticipation) and § 103 (obviousness). For a claim to be anticipated,

27

every element and limitation of the claimed invention must be found in a single prior art

28

reference, arranged as in the claim. Lindemann Maschinenfabrik GMBH v. American Hoist &

REPLY DECLARATION OF RAVIN BALAKRISHNAN, PH.D. IN SUPPORT OF MOTION FOR A PRELIMINARY INJUNCTION
Case No. 11-cv-01846-LHK

2

1

Derrick Co., 730 F.2d 1452, 1459 (Fed. Cir. 1984). Inherent anticipation requires that the

2

missing descriptive material is “necessarily present,” not merely probably or possibly present, in

3

the prior art.

4

9.

A claim is invalid for obviousness if “differences between the subject matter

5

sought to be patented and the prior art are such that the subject matter as a whole would have

6

been obvious at the time the invention was made to a person having ordinary skill in the art to

7

which said subject matter pertains.” 35 U.S.C. § 103(a). The mere existence of each and every

8

element of the claimed invention in the prior art does not necessarily prove obviousness, since

9

inventions typically rely on building blocks of prior art. In considering whether a claimed

10

invention is obvious, one may consider whether there was a reason that would have prompted a

11

person having ordinary skill in the field of the invention to combine known elements in a way the

12

claimed invention does, taking into account such factors as: (1) whether the claimed invention

13

was merely the predictable result of combining familiar prior art elements according to their

14

known function(s); (2) whether the prior art teaches or suggests the desirability of combining

15

elements claimed in the invention; (3) whether the prior art teaches away from combining

16

elements in the claimed invention; and (4) whether a person of skill in the art had used a known

17

technique to improve similar devices in the same way. It is not sufficient to show that it was

18

obvious to try a combination. Copying by others is one secondary consideration of non-

19

obviousness.

20

V.

DETAILED OPINION

21

A.

Introduction

22

10.

As stated in my opening declaration, the ’381 patent provides an elegant solution

23

to a vexing user interface problem. To my knowledge, until the Apple iPhone was launched,

24

touch screen devices did not have a visually intuitive way to alert a user when he or she had

25

reached the edge of the document when scrolling or panning it. Now, this feature is nearly

26

ubiquitous, including in Samsung’s own devices.

27
28

11.

Samsung’s alleged prior art does not disclose the edge-responsive functions of the

’381 patent. Its cited references at best disclose automatic re-centering features when scrolling or

REPLY DECLARATION OF RAVIN BALAKRISHNAN, PH.D. IN SUPPORT OF MOTION FOR A PRELIMINARY INJUNCTION
Case No. 11-cv-01846-LHK

3

1

panning within the document, which have a different purpose than the ’381 patent. Because these

2

references are not concerned with what should happen when the edge of the document is reached,

3

they still embody the main problem that the ’381 patent solved. Users either are not allowed to

4

scroll past the edge (i.e. they hit a “hard stop”) or are allowed to scroll endlessly into empty areas

5

devoid of any content.

6

12.

In an attempt to dismiss these key differences, Samsung arbitrarily treats the lines

7

within the boundaries of a document in the same manner as external “edges.” It also sets up a

8

demonstration to simulate supposed edge-responsive behavior with software that in fact is merely

9

re-centering items. While that may be easy to do with the ’381 patent already in hand, a person of

10

skill in the art at the time of the invention would not have recognized the edge-responsive

11

advantages of the ’381 patent in Samsung’s prior art.

12

13.

My declaration begins by showing that Samsung’s alleged prior art was trying to

13

solve a different problem than the ’381 patent, and that it still suffers from the principal

14

limitations and constraints that the ’381 patent was designed to solve. Next, I discuss the faults

15

and shortcomings in Dr. Van Dam’s invalidity and inequitable conduct opinions, concluding that

16

the references he cites do not disclose key elements of the asserted claims or render them obvious.

17

Finally, I address Dr. Johnson’s infringement analysis, which is based on strained claim

18

interpretations that defy common sense.

19

B.

20
21

Invalidity and Inequitable Conduct
1.

14.

Summary of Opinion

The ’381 patent provides an elegant and visually intuitive solution to a discrete

22

issue: what to do when a user scrolls to the edge of an electronic document. In the prior art, when

23

a user scrolled to the edge of a document, one of two scenarios would play out. Either she would

24

scroll continuously past the edge of the document into nothingness (i.e. beyond a place where

25

there was any meaningful content), or she would hit a “hard stop” and not be allowed to scroll

26

any further.

27
28

15.

Each of these scenarios has its own disadvantages. Allowing a user to move

through virtual space going absolutely anywhere, including beyond a place that has any

REPLY DECLARATION OF RAVIN BALAKRISHNAN, PH.D. IN SUPPORT OF MOTION FOR A PRELIMINARY INJUNCTION
Case No. 11-cv-01846-LHK

4

1

meaningful content, can cause the user to become disoriented. (See Ex. N (Desert Fog Article) at

2

p. 4; Ex. I, Bederson Dep. at 204:6-24; 205:6-207:5; 213:2-214:1.) In a paper he wrote in 2011,

3

Dr. Bederson referred to this as the “Desert Fog” citing an earlier articled written by Jul and

4

Furnas. (Id; see also, Ex. J, Van Dam Dep. at 63:3-17 (referring to the empty area as “no man’s

5

land”).) Users who navigate into these empty spaces may get lost and not know how to find their

6

way back. (Id.; see also, Van Dam Dec. (D.I. 168) at ¶ 144.)

7

16.

Most user interfaces avoided the “Desert Fog” problem by inserting a hard stop at

8

the edge of a document. But that solution has its own disadvantages. If the user does not realize

9

he has hit the edge of a document, he may keep trying to move the document in vain. No matter

10

how hard he tries, however, the device will not allow the document to move. As a result, the user

11

may think his device has frozen or locked up, or that it is otherwise not registering his input. In

12

any case, the user could become frustrated when the scrolling or translating does not reflect his

13

intent. (’381 Patent at 2:26-28; see also, Van Dam Dec. (D.I. 168) at ¶ 144 (one way to prevent a

14

user from moving an electronic document beyond the edge is “to prevent the document from

15

moving beyond the edge by ignoring further requests for any such movement”).)

16

17.

The inventors of the ’381 patent recognized these disadvantages and created a

17

novel solution to overcome them. When the user attempts to scroll past the edge of the document,

18

an area beyond the edge is displayed. (’381 patent at 26:59-63 & Fig. 5.) This area provides the

19

user with an instant visual cue informing him that the edge of the document has been reached.

20

(Id.) Importantly, in an exemplary embodiment shown in Figures 8A-D, this area beyond the

21

edge is displayed adjacent to a portion of the electronic document, enabling the user to maintain

22

context and avoid the “Desert Fog” problem. Once the user lifts his finger (or other object) from

23

the surface of the screen, the document moves or snaps back into place, and the area beyond the

24

edge disappears. (See, e.g., ’381 patent at Claim 1.) The elegant solution proposed by the ’381

25

patent significantly enhances the user’s experience in viewing photos, web pages, lists, and other

26

electronic documents. To my knowledge, touch screen devices in the prior art did not have a

27

visually intuitive way to alert a user when he or she had reached the edge of the document when

28

scrolling or panning.

REPLY DECLARATION OF RAVIN BALAKRISHNAN, PH.D. IN SUPPORT OF MOTION FOR A PRELIMINARY INJUNCTION
Case No. 11-cv-01846-LHK

5

1

18.

The prior art cited by Samsung is not concerned with problems that occur when

2

reaching, crossing or falling off the “edge” of the document. Rather, these references at best

3

disclose auto-centering functions not designed to be responsive to the edge of the document. In

4

LaunchTile and XNav, when the user interacts with the device, the content of the screen is

5

automatically re-centered to a pre-defined reference point called “Blue.” (Ex. O (“LaunchTile

6

Article”) at p. 204.) In the e-mail application view (shown below), “Blue” is a translucent blue

7

“highlight” bar, which doubles as an e-mail selection tool. The user can move this bar up and

8

down to select e-mails. (Ex. O, LaunchTile Article at p. 205 (“In cases where limited display real

9

estate necessitates smaller targets, the central Blue widget serves as a moveable tool glass which

10

can be positioned over the target object. . . .”).) If the highlight bar does not line up with an e-

11

mail, the bar and list are automatically re-aligned, thus allowing the user to more easily select a

12

desired e-mail.

13
14
15
16
Moveable blue highlight
& selection tool

17
18
19

Fig. 1 - Screenshot from Bederson Dec. Ex. L (D.I. 165-12) showing e-mail application view

20
21

19.

In the 6X6 “World” and 2X2 “Zone” views (shown below), Blue is depicted as a

22

blue dot or dots on the screen. (Id.; Ex. I, Bederson Dep. at 189:13-190:3.) If the user scrolls a

23

sufficient distance towards an adjacent zone, the user interface automatically navigates to that

24

zone, re-centering the page on the Blue dot. This re-centering feature ensures that the user is

25

never caught in between two zones. (See, e.g., Ex. O, LaunchTile Article at p. 205; Ex. I,

26

Bederson Dep. at 190:21-192:10.)

27
28

REPLY DECLARATION OF RAVIN BALAKRISHNAN, PH.D. IN SUPPORT OF MOTION FOR A PRELIMINARY INJUNCTION
Case No. 11-cv-01846-LHK

6

Central reference points
(“Blue”)

1
2
3
4
5
6
7
8
9

Figs. 2 & 3 - Screenshots from Bederson Dec. Ex. L (D.I. 165-12) showing “World” and “Zone” view

20.

The inventors of Lira similarly recognized that as a user navigates around a large

10

webpage on a device with a small-screen, “such ‘touch-and-drag’ scrolling can result in

11

information 610 that is positioned in the display window 605 but is difficult to view or read since

12

the user may inadvertently navigate to a position where only a portion of a column or an image is

13

visible in the PDA display window 605.” (Ex. D, Lira at p. 11, l. 27 - p. 12, l. 2; see also, p. 12, l.

14

30 - p. 13, l. 2.) They therefore devised methods for reconfiguring the webpage into columns so

15

that the page could be more easily viewed on a small screen. (See, e.g., id. at p. 13, ll. 14-17; p.

16

14, l. 29 - p. 15, l. 5; p. 15, ll. 18 - 21.) For convenience, the PDA display would be re-centered

17

to a column, so that the column aligned perfectly with the display. This re-centering method is

18

illustrated in Figure 14B, shown below.

19
20
21
22
23
24
25
26
27
28

REPLY DECLARATION OF RAVIN BALAKRISHNAN, PH.D. IN SUPPORT OF MOTION FOR A PRELIMINARY INJUNCTION
Case No. 11-cv-01846-LHK

7

1
2
3
4

The PDA screen 1205
is re-centered to
column 1220.

5
6
7
8
9
10
11
12
13
14
15
Fig. 4 - Lira Figure 14B

16
17

21.

Finally, Van Den Hoven merely discloses a user interface that scrolls through a

18

sequence of thumbnail images automatically. (Ex. E, Van Den Hoven at p.2, ll. 21-22 & Fig 2.)

19

The sequence of images is designed to look like a roll of film to make the display more

20

recognizable to the user. (Ex. E, Van Den Hoven at p. 5, ll. 17-21 & Fig. 2.) The user can speed

21

up or slow down the automatic scrolling by swiping his finger on the screen. (Ex. E, Van Den

22

Hoven at p. 6, ll. 28-31.) When the last image in the sequence is reached, the sequence can either

23

loop back to the beginning, or it hits a hard boundary and reverses direction. (Ex. E, Van Den

24

Hoven at 6, 1-4.)

25

22.

Significantly, these references embody the very same disadvantages that the ’381

26

patent is designed to solve. In LaunchTile, individuals are not allowed to scroll past the borders

27

of the “World.” Attempting to do so will result in a hard stop. (See Ex. A, Video No. 1; Ex. I,

28

Bederson Dep. at 148:25-149:4.) Conversely, the LaunchTile e-mail application is unbounded,

REPLY DECLARATION OF RAVIN BALAKRISHNAN, PH.D. IN SUPPORT OF MOTION FOR A PRELIMINARY INJUNCTION
Case No. 11-cv-01846-LHK

8

1

allowing users to scroll into the empty white area past all e-mails (into the “Desert Fog” or “no

2

man’s land”). If the user lifts his finger while navigating in this space, the e-mail application does

3

not snap back to the e-mail list. (See Ex. B, Video No. 2; Ex. I, Bederson Dep. at 113:2-13.) In

4

Van Den Hoven, the scrolling happens automatically and hits a hard boundary when the last

5

image is reached. (Ex. E, Van Den Hoven at p. 6, 1-4.) There is no discussion about displaying

6

an area beyond the edge of the document or moving the document back into place. Finally, Lira

7

is concerned with what happens while navigating within a webpage. It does not disclose what

8

will or should happen if and when the user tries to scroll past the edge of the page.

9
10

When user
selects this
zone…

… he/she
cannot move
up or left.

11
12
13
14
15
Fig. 5 - Screenshots from LaunchTile “World” and “Zone” views

16
17
18
19

No re-alignment
when e-mail list
and blue
highlight bar do
not overlap

User can scroll
into “no man’s
land” with no
snap-back

20
21
22
Fig. 6 - Screenshots from LaunchTile E-mail view

23
24

23.

The behavior in Samsung’s prior art occurs in response to misalignment, i.e. when

25

content needs to be re-centered to a point within the display window. The actions in the ’381

26

patent occur in response to the edge of the document being reached. As a result, the prior art does

27

not disclose at least two critical edge-responsive features of the asserted claims: (1) displaying an

28

area beyond the edge of the document, and (2) moving the portion of the document back into

REPLY DECLARATION OF RAVIN BALAKRISHNAN, PH.D. IN SUPPORT OF MOTION FOR A PRELIMINARY INJUNCTION
Case No. 11-cv-01846-LHK

9

1

view when the user’s finger (or other object) is no longer detected on the display. To one of skill

2

in the art, the re-centering solutions in Samsung’s prior art are very different from the edge-

3

specific functions described and claimed by the ’381 patent. Dr. Van Dam’s discussion of

4

physics based metaphors fails to address this distinction.

5
6

2.
24.

LaunchTile / XNav1

The LaunchTile / XNav art actually consists of four separate references: (1) the

7

XNav source code; (2) the LaunchTile publications (articles and PowerPoint slides); (3) Dr.

8

Bederson’s LaunchTile videos; and (4) public demonstrations at trade shows.2 Dr. Bederson

9

cannot recall what features were publicly demonstrated before the priority date, so there is no

10

evidence that the demonstration disclosed any of the allegedly anticipating features. (Ex. I,

11

Bederson Dep. at 158:21-159:2; 160:18-23.) Having now had the opportunity to review the

12

remaining references, in my opinion these references fail to disclose all of the elements of the

13

asserted claims or render them obvious.

14

a.

15
16

25.

The XNav Source Code Demonstrates that XNav and
LaunchTile Are Performing an Internal Re-Centering
Procedure.

I have reviewed a copy of the XNav source code, which is attached as Exhibit G to

17

Dr. Bederson’s Declaration. The XNav source code confirms that LaunchTile and XNav do not

18

disclose the edge-specific functions of the ’381 patent but are instead performing an internal re-

19

centering procedure. In the case of the XNav e-mail application, that re-centering function is

20

called “SnapObjectToHighlight.” (See Ex. L (“E-mail source code”) at pp. 29-30;3 Ex. I,

21
1

22
23
24
25
26
27
28

According to Dr. Bederson, the allegedly anticipating features of LaunchTile and XNav
are identical. (Ex. I, Bederson Dep. at 103:4-20.) Accordingly, for the purposes of this report, I
will also treat LaunchTile and XNav interchangeably.
2

I understand that there also are serious questions about whether the source code qualifies
as prior art. The XNav source code was not distributed publicly, but instead was sent via an
encrypted message to Microsoft. (Ex. I, Bederson Dep. at 172:6-174:12; see also, 181:5-183:3.)
Because the code was kept private, a person of skill in the art would not have access to it.
Moreover, Dr. Bederson does not “have a precise time stamp of that source code that correlates
with the video that shows that it was done exactly the same day and that it was exactly the same
code.” (Ex. I, Bederson Dep. at 98:18-23; see also, id. at 97:6-98:17.)
3

True and correct copies of the XNav source code, which were marked as Bederson
Deposition Exhibits 211, 212 and 213, are attached hereto as Exhibits K-M.

REPLY DECLARATION OF RAVIN BALAKRISHNAN, PH.D. IN SUPPORT OF MOTION FOR A PRELIMINARY INJUNCTION 10
Case No. 11-cv-01846-LHK

1

Bederson Dep. at 56:14-58:2; 64:16-21.) The code simply determines which e-mail header is

2

overlapping with the blue highlight bar the most, calculates the distance between these items, then

3

realigns the two. (Id.; see also, Ex. I, Bederson Dep. at 80:2-82:2.)4 As Dr. Bederson admits, the

4

exact same code causes the e-mail header to realign to the highlight bar regardless of whether it is

5

positioned in between e-mails or after the last e-mail. (See Ex. I, Bederson Dep. at 78:17-79:8.)

6

In my opinion, a person of skill in the art reviewing this code would not see any edge-responsive

7

functions. Rather, the purpose of this code is to ensure that the e-mail header stays aligned to the

8

movable blue bar, which also acts as an e-mail selection tool. (See Ex. C, Video No. 3; see also,

9

Ex. J, Van Dam Dep. at 60:5-63:17 (noting that there is no snap back when the e-mail headers

10

and highlight bar do not overlap); 157:15-161:24 (admitting that he did not review the source

11

code, and that a benefit of the functionality is to center the header row with the highlight).) This

12

helps the user more easily select desired e-mails.

13

26.

As demonstrated in Figure 6 above, the LaunchTile e-mail program has no

14

reaction to scrolling past the terminus of the list of email headers. Indeed, it allows the user to

15

continue scrolling well past the terminus of the e-mail list. The software does not realign the list

16

because there is no overlap between the e-mail header and the blue highlight bar. (See Figure 6

17

above; Ex. B; Ex. L, E-mail Source Code at pp. 29-30; Ex. I, Bederson Dep. at 84:20-86:21.) In

18

short, all of the “snapping” features in the e-mail application are tied to the position of this

19

highlight bar and do not occur in response to the edge of the document being reached.

20

27.

The zoomed-in “Zone” view discloses a similar method for auto-centering. (See

21

Ex. M, Bederson Dep. Ex. 213 (XNav Source Code) at pp. 19-20; Ex. I, Bederson Dep. at 115:10-

22

117:11; 124:17-126:8; 133:15-135:16; 189:13-190:3; Ex. J, Van Dam Dep. at 154:12-155:25.)

23

Specifically, if the user lifts his finger after moving more than a sixth of the way towards an

24

adjacent zone, the program will continue scrolling to that zone. (Id.) This auto-centering feature

25

works only when scrolling within the perimeter, or inside the “edges” of the “World,” since as

26
4

27
28

This code relies on a second function called “GetIntersectingEmailItemBounds” which
determines the bounds of an e-mail header that most overlaps with the blue highlight bar. (Ex. I,
Bederson Dep. at 29:4-6; 30:1-14; 56:14-25; Ex. K, Bederson Dep. Ex. 211 (Source Code) at 3.)

REPLY DECLARATION OF RAVIN BALAKRISHNAN, PH.D. IN SUPPORT OF MOTION FOR A PRELIMINARY INJUNCTION 11
Case No. 11-cv-01846-LHK

1

shown in Figure 5 above it is impossible to scroll beyond the “edges” of these tiles. (See also,

2

Ex. I, Bederson Dep. at 144:20-145:7; 146:5-14.) As a result, the software cannot display an area

3

beyond the edge of the tiles, and the array of tiles cannot move back in the opposite direction after

4

the user lifts his finger. XNav thus fails to disclose the main user-friendly features of the ’381

5

patent.

6

28.

Dr. Van Dam attempts to avoid these flaws by treating the internal gridlines within

7

the “World” and “Zone” views as “edges.” These internal lines cannot be treated as “edges,”

8

however, because there is still content outside of these boundaries that the user can scroll to. (See

9

also, Ex. I, Bederson Dep. at 205:18-206:11 (stating that most document browsers limit

10

navigation to the available content).)5 If I draw a line through a piece of paper (the physical

11

counterpart for the electronic document), the edges of that page remain the same. Similarly, the

12

internal grid lines within the “World” or “Zone” of LaunchTile are not the same as the external

13

boundaries that define the “edge” of the electronic document. With the ’381 patent in front of

14

you, it is easy to draw arbitrary lines within a document and refer to those lines as edges. But in

15

my opinion, a person of skill in the art at the time of the invention would not have understood the

16

edge-specific advantages of the ’381 patent while navigating at the center of the “World.”

17

Rather, the software at best teaches re-aligning screen objects to a central “Blue” UI element.

18

29.

In sum, the key relationship in XNav is between the objects on the screen and the

19

“Blue” element. The source code does not teach the main advantage of the ’381 patent which

20

involves performing the edge-responsive functions of the claims. As a result, I do not believe that

21

the XNav code discloses all the limitations of the asserted claims or render them obvious.

22
23
24
25
26
27
28

5

I was asked to evaluate the LaunchTile program in the context of the ’381 patent for the
first time at my deposition. As I made clear at the deposition, I had not had the opportunity to
study the program closely, and did not know what the purpose of the Blue reference point was. In
addition, I did not have access to the source code. Now that I have had the opportunity to review
LaunchTile and the XNav source code, I understand that these programs are performing a realignment function and not the edge-responsive functions of the ’381 patent.

REPLY DECLARATION OF RAVIN BALAKRISHNAN, PH.D. IN SUPPORT OF MOTION FOR A PRELIMINARY INJUNCTION 12
Case No. 11-cv-01846-LHK

1
2

b.
30.

LaunchTile Articles and Slides

I have reviewed the articles attached as Exhibits A and B and the slides attached as

3

Exhibits E and H to Dr. Bederson’s declaration. Dr. Van Dam did not rely on Exhibit A as an

4

anticipatory reference, and admitted in his deposition that that article did not disclose all of the

5

limitations of the claims of the ’381 patent. (Ex. J, Van Dam Dep. at 147:9-148: 7; see also, id. at

6

150:10-151:12.) Indeed, these articles and slides do not show or discuss the edge-responsive

7

behaviors of the ’381 patent nor do they disclose any of the supposedly anticipating behaviors

8

shown in Dr. Van Dam’s 2011 video.

9

31.

Because each of these articles and slides disclose the same material features as

10

Exhibit A, I will focus my discussion on that article. This article briefly describes the e-mail

11

application discussed above, including the blue highlight bar which the user moves up and down

12

to select an e-mail. (Ex. O, LaunchTile Article at p. 205-206.) However, the article fails to

13

mention any of the e-mail re-alignment capabilities shown in Dr. Van Dam’s videos. (Id.) The

14

article explicitly describes the features of the “World” and “Zone” view as re-aligning to the

15

“Blue” element—not as edge-related features: “Upon thumb release, the zoomspace animates to

16

align Blue with the closest zone’s empty hub. The visual and automated guidance ensures the

17

user is never caught between zones.” (Id. at p. 205.) The article does not mention or suggest

18

what can or should happen if the user attempts to scroll beyond the edge of the “World” view.

19

Accordingly, these articles and slides do not disclose all of the elements of the asserted claims or

20

render them obvious.

21
22

c.
32.

Dr. Bederson’s LaunchTile Videos

I have viewed the videos attached as Exhibits D and L to Dr. Bederson’s

23

declaration. I understand that Dr. Van Dam does not rely on these videos to demonstrate

24

anticipation. (Ex. J, Van Dam Dep. at 151:21-152:5.) Like the articles discussed above, these

25

videos do not disclose any of the specific examples depicted in Dr. Van Dam’s 2011 videos. The

26

videos only show the zones moving in a single, forward direction. There is no edge-related

27

behavior or bounce-back action shown, a fact Dr. Bederson readily admitted. (Ex. I, Bederson

28

REPLY DECLARATION OF RAVIN BALAKRISHNAN, PH.D. IN SUPPORT OF MOTION FOR A PRELIMINARY INJUNCTION 13
Case No. 11-cv-01846-LHK

1

Dep. at 198:6-25 (stating that videos do not show any bounce back).) Thus the videos do not

2

disclose all the limitations of the asserted claims or render them obvious.

3
4

d.
33.

Dr. Van Dam’s 2011 Videos

Finally, I have reviewed the videos attached to Dr. Van Dam’s declarations, which

5

I understand are not prior art. Unlike persons of skill in the art at the time of the invention, Dr.

6

Van Dam had the ’381 patent in hand when creating these videos, and can use the patent to

7

simulate certain behaviors in the asserted claims. However, as confirmed above, the LaunchTile

8

and XNav software is not behaving in response to the edge of a document being reached. It

9

instead is realigning the view to a central “blue” reference point / selection tool. At best this

10

behavior teaches one of skill in the art that a user can select an item on a list more easily if the

11

item is aligned with a selection tool. Tellingly, none of the allegedly anticipating features

12

demonstrated in Dr. Van Dam’s videos were actually disclosed or discussed in the alleged prior

13

art source code, articles, slides or videos. For these reasons, the LaunchTile and XNav references

14

do not anticipate the ’381 patent or render it obvious.

15
16

3.
34.

Lira

Like LaunchTile, Lira discloses methods for realigning or reformatting webpage

17

content so that it can fit more easily into a small screen. (Ex. D, Lira at p. 1, l. 15 - p. 2, l. 3; p.

18

15, ll. 18-25; Claim 1.) Figure 5, for instance, shows a method for reformatting the web page into

19

columns that are viewable on a PDA screen. In Figure 14B (shown below), the display window

20

1205 is realigned with content of column 1220 so that the window can show the entire column.

21

(Ex. D, Lira at Fig. 14B; p. 15, ll. 18-31.)

22

35.

The reference does not discuss what will or should happen if the user attempts to

23

move the display window beyond the edge of the webpage. Nor does it disclose the specific

24

solution in the ’381 patent: displaying an area beyond the edge of the webpage in response to

25

reaching the edge. On the contrary, the display windows in each figure never move beyond the

26

edges of the webpage; they are always shown within the outer boundaries of the page. (See, e.g.

27

id. at Fig. 5.) In my opinion, the reference teaches away from the solution of the ’381 patent.

28

REPLY DECLARATION OF RAVIN BALAKRISHNAN, PH.D. IN SUPPORT OF MOTION FOR A PRELIMINARY INJUNCTION 14
Case No. 11-cv-01846-LHK

1

36.

Dr. Van Dam again attempts to avoid these flaws by redefining the “edges” of an

2

electronic document to include column borders within the webpage. According to Dr. Van Dam,

3

when the display moves beyond the “edge” of the column, it is displaying an “area beyond the

4

edge” of the electronic document. When the display is re-centered to the column, the “document”

5

is translated in a second direction and the area beyond the edge disappears. He illustrates this

6

analysis with the following figure:

7
8
9
10
11
12
13
14
15
16
17
Fig. 7 - Van Dam Dec. Ex. 4 (D.I. 168-4) at 11.

18
19

37.

As discussed in paragraph 24 above, Dr. Van Dam’s definition of “edge” is not

20

only incorrect but suffers from hindsight bias. As Dr. Van Dam’s figure shows, the display is

21

centered in the middle of the webpage, and the so-called “Area Beyond the Edge” is still

22

displaying a portion of the webpage. Indeed, Dr. Van Dam admitted that a user would not know

23

for sure whether he had moved from one column to another, and that there is nothing preventing

24

the user from doing so. (See Ex. J, Van Dam Dep. at 172:2-173:8.) In my opinion, this figure

25

would not disclose the advantages of the ’381 patent to a person of skill at the time of the

26

invention.

27
28

REPLY DECLARATION OF RAVIN BALAKRISHNAN, PH.D. IN SUPPORT OF MOTION FOR A PRELIMINARY INJUNCTION 15
Case No. 11-cv-01846-LHK

1
2

4.
38.

Van Den Hoven

I have reviewed Dr. Van Dam’s discussion of the Van Den Hoven reference (WO

3

01/029702 / U.S. Patent No. 7,152,210). Dr. Van Dam claims that Apple committed inequitable

4

conduct by not disclosing this allegedly anticipating reference to the Patent Office during

5

prosecution of the ’381 patent. (Van Dam Dec. (D.I. 168) at ¶ 149.) However, his reliance on

6

Van Den Hoven is misplaced. In my opinion, that reference is cumulative of other references

7

before the Examiner and does not anticipate or obviate the asserted claims.

8
9

39.

Van Den Hoven is cumulative of at least the Collins reference (US 2008/0104544

A1), which was cited and considered by the Examiner during prosecution of the ’381 patent.

10

Like Van Den Hoven, Collins discloses portable device software that displays a series of icons

11

substantially identical to the sequence of images in Van Den Hoven. (Compare Ex. E, Van Den

12

Hoven Fig. 2 to Ex. F, Collins Figs. 1 and 2A.) The user of the Collins system can cause the list

13

of icons to rotate by sliding his or her finger across the touch screen. (See Ex. F, Collins at

14

[0034].) This is similar to the scrolling feature of Van Den Hoven, except that in Van Den Hoven

15

the scrolling occurs automatically. (Ex. E, Van Den Hoven at p. 5, ll. 17-21.) According to

16

Collins, when the user attempts to move past the end of the icon list, the list bounces back, i.e. the

17

direction of the scrolling is automatically reversed. (See Ex. F, Collins at [0038].) Collins and

18

Van Den Hoven thus teach identical behaviors: scrolling through a list of images and reversing

19

direction when the last image is reached. Dr. Van Dam does not point out any material

20

differences between Van Den Hoven and Collins (or, for that matter, any other prior art reference

21

considered by the Examiner).6

22

40.

Moreover, just like Collins, the Van Den Hoven reference does not disclose at

23

least two key limitations of the asserted claims. First, Van Den Hoven does not disclose

24

displaying an area beyond the edge of the electronic document in response to the edge of the

25

electronic document being reached. As discussed above, when the final image in the sequence is

26
6

27
28

Dr. Van Dam admits that he did not review all of the references cited on the face of the
’381 patent; as a result, he cannot determine whether the references are cumulative. (See Ex. J,
Van Dam Dep. at 21:16-22:6.)

REPLY DECLARATION OF RAVIN BALAKRISHNAN, PH.D. IN SUPPORT OF MOTION FOR A PRELIMINARY INJUNCTION 16
Case No. 11-cv-01846-LHK

1

reached, Van Den Hoven discloses that the list can reverse direction. (Ex. E, Van Den Hoven at

2

p. 6, ll. 1-4.) This features, however, embodies the very “hard stop” issue that the ’381 patent is

3

designed to solve. Dr. Van Dam again attempts to avoid this flaw by asserting that the “border

4

area” around each image thumbnail—that is, the area that makes the thumbnails look like a film

5

strip—corresponds to the area beyond the “edge” of the document. (Van Dam Dec. Ex. 5 (D.I.

6

168-5) at p. 3.) However, this border area is not an area beyond the “edge” of the document, and

7

thus does not satisfy the limitations of the asserted claims.

8

41.

Second, Van Den Hoven does not disclose translating the electronic document in a

9

second direction until the area beyond the edge of the electronic document is no longer displayed

10

in response to detecting that the object is no longer on or near the touch screen display. Dr. Van

11

Dam himself admits that there is no such disclosure in Van Den Hoven. (Van Dam Dec. Ex. 5

12

(D.I. 168-5) at p. 5.) Although he suggests that the disclosure may be pieced together from

13

disparate elements within the patent, the reversing operation he points to is not in response to the

14

user lifting his finger.

15

42.

Like the other references discussed by Dr. Van Dam, Van Den Hoven does not

16

disclose or discuss the functionality described in the ’381 patent that occur in response to the edge

17

of an electronic document being reached. Indeed, the reference includes the very hard stop

18

problem that the ’381 patent seeks to ameliorate. Accordingly, Van Den Hoven would not teach

19

or inspire one of skill in the art to perform the limitations of the ’381 patent.

20

C.

21
22

Infringement
1.

43.

The ’381 Patent Claims Do Not Require Inhuman Precision

I understand that Samsung contends that it does not infringe the claims of the ’381

23

patent. I have reviewed Dr. Johnson’s two non-infringement arguments and find them

24

unpersuasive for the reasons stated below

25

44.

The plain language of the patent claims, in light of the specification, the

26

understanding in the art, and common sense, does not require that a human finger or stylus move

27

in a mathematically precise line without variation. Rather in my opinion, a person of skill in the

28

art would have understood the term “first direction” to refer to the general course or path on

REPLY DECLARATION OF RAVIN BALAKRISHNAN, PH.D. IN SUPPORT OF MOTION FOR A PRELIMINARY INJUNCTION 17
Case No. 11-cv-01846-LHK

1

which the object is translated, and not, as Dr. Johnson suggests, an “exact straight line.” Indeed,

2

Samsung’s own expert Dr. Van Dam directly contradicts Dr. Johnson and acknowledges that

3

movement in a general direction is sufficient to constitute a first direction. (Ex. J, Van Dam Dep.

4

at 128:5-129:17.)

5

45.

By the time the ’381 Patent was conceived and reduced to practice (and likely well

6

before), it was well understood in the art that a human finger generally did not move in a

7

mathematically precise straight line. The inventors of the ’381 Patent themselves recognized the

8

limitation of human finger precision. The specification states:

9

13

The user may make contact with the touch screen in the display
system 112 using any suitable object or appendage, such as a stylus,
a finger, and so forth. In some embodiments, the user interface is
designed to work primarily with finger-based contacts and gestures,
which are much less precise than stylus-based input due to the
larger area of contact of a finger on the touch screen. In some
embodiments, the device translates the rough finger-based input
into a precise pointer/cursor position or command for performing
the actions desired by the user.

14

’381 Patent at 12:66 to 13:8 (emphasis supplied). Mathematical precision is simply not necessary

15

in order to achieve the purpose of the invention. Because the patent involves moving a finger or

16

other object on a touch screen, a person of ordinary skill would understand that the movements

17

claimed are those that are typically made and are capable of being made by a human. I have

18

reviewed the portions of the specification cited by Dr. Johnson. (Johnson Decl. (D.I. 174) at ¶

19

24.) They do not contradict this common sense understanding of “direction.” To the contrary,

20

they demonstrate that the inventors were aware of the possibility of finger imprecision. (See, e.g.,

21

’381 patent at 29:21-40.)

10
11
12

22

46.

Moreover, even if a high degree of precision were required, the devices would still

23

infringe the claims. Dr. Johnson acknowledges that the accused devices will not infringe

24

“[u]nless the document is somehow constrained to translate in only a fixed linear direction.”

25

(Johnson Decl. (D.I. 174) at ¶ 37.) Each of the accused phones contains a Contacts application

26

that is constrained to translate documents in only a fixed linear direction. Specifically, the list of

27

contacts in the Galaxy S 4G, Infuse 4G and Droid Charge can only be translated vertically.

28

Likewise, the accused Browser application in the Galaxy Tab 10.1 will only translate vertically in

REPLY DECLARATION OF RAVIN BALAKRISHNAN, PH.D. IN SUPPORT OF MOTION FOR A PRELIMINARY INJUNCTION 18
Case No. 11-cv-01846-LHK

1

at least two situations: (1) when displaying a mobile website (e.g. mobile.nytimes.com), or (2)

2

when a website fits horizontally within the browser window (e.g. www.cnn.com). Thus, these

3

accused applications infringe claim 1 of the ’381 patent even under Samsung’s unjustifiably

4

exacting definition.7

5

47.

Finally, claims 19 and 20 claim devices containing instructions for translating

6

documents in a particular direction. (See ’381 Patent at Claims 19 & 20.) As stated in my

7

opening report, the accused devices contain such instructions. They thus infringe these claims.8

8
9

2.
48.

Showing a Black Area on the Screen Does “Display” Black

Samsung’s defense that the area beyond an edge of a document is not “displayed”

10

if it is shown to the user as a black expanse, because the color black is displayed using unlit

11

pixels, is also unpersuasive. In my opinion, those of skill in the art would easily understand that

12

the term “display” was being used by the ’381 Patent in its ordinary sense and that the type of

13

screen technology used does not matter.

14

49.

The “displaying” step in claim 1 is not linked to individual screen pixels, but

15

instead refers to whether the screen shows an area beyond the edge of an electronic document.

16

When a user of an accused device scrolls past the edge of a photograph or (on the accused

17

phones) the edge of the contacts list, a black area is shown in the active portion of the screen

18

display. The screen thus “displays” that area, along with a portion of a document, to the user.

19

Contrary to Dr. Johnson’s analogy, a device would not be able to display this image if it were

20

turned off.

21

50.

This common sense understanding of the word “display” is supported by the

22

specification. The ’381 patent will serve the same purpose regardless of what specific color is

23

displayed, and regardless of the type of display used. Indeed, the patent expressly discloses

24
25
26

7

It is irrelevant that a user is able move his finger in multiple directions. Claim 1 merely
requires that the electronic document be translated in a first direction in response to detecting
movement on the touch screen. (See ’381 Patent at Claim 1.)
8

27
28

I also note that Dr. Johnson has not looked to see whether the source code ignores minor
variations in finger positioning. XNav, for instance, does contain such code. (See Ex. I,
Bederson Dep. at 128:22-130:10.)

REPLY DECLARATION OF RAVIN BALAKRISHNAN, PH.D. IN SUPPORT OF MOTION FOR A PRELIMINARY INJUNCTION 19
Case No. 11-cv-01846-LHK

1

“displaying” the area beyond the edge of an electronic document using a solid color, including

2

black. (’381 patent at 27:30-36 & Claim 13.) The specification also expressly states that any

3

suitable display technology may be used: “The touch screen in the display system 112 may use

4

LCD (liquid crystal display) technology, or LPD (light emitting polymer display) technology,

5

although other display technologies may be used in other embodiments.” (’381 Patent at 12:42 -

6

45.) This same passage specifically discloses the use of light emitting polymer displays as well as

7

other display technologies. These teachings flatly contradict Dr. Johnson’s non-infringement

8

opinion.

9

51.

Moreover, those of ordinary skill in the art know that OLED screens do “display”

10

black. As one of Samsung’s own patents succinctly puts it, “black is displayed on the screen of

11

the OLED display when no current flows through the corresponding OLED LD and the OLED

12

LD does not emit light.” (Ex. G, U.S. 7,965,263 at 9:56 - 59; see also, Ex. H, U.S. 2007-0080907

13

at [0107] & Fig. 8 (“Next, when the second field T2 is started, a black color is displayed from the

14

top end of the screen. . . .”).) Samsung’s contention that it avoids infringement because certain of

15

its screens, like other types of screens, displays black by not emitting light is not persuasive in my

16

opinion.

17

52.

Finally, it should be noted that this argument does not avoid infringement of the

18

Galaxy Tab 10.1, which uses an LCD screen and displays a grey border when the user scrolls past

19

the edge of a webpage. Moreover, claims 19 and 20 require that the device contain instructions

20

for performing various tasks. As I stated previously the devices do contain instructions for

21

displaying.

22
23
24
25
26
27
28

REPLY DECLARATION OF RAVIN BALAKRISHNAN, PH.D. IN SUPPORT OF MOTION FOR A PRELIMINARY INJUNCTION 20
Case No. 11-cv-01846-LHK

1

53.

I have reviewed the portions of the specification cited by Dr. Johnson. (Johnson

2

Decl. (D.I. 174) at ¶ 38.) In general, they simply state that a touch screen can display images. A

3

person of skill in the art would understand that such images may or may not contain black. These

4

passages do not show that the inventors adopted an uncommon definition for the common word

5

“display.”

6

I declare under penalty of perjury under the laws of the United States of America that the

7

forgoing is true and correct and that this Declaration was executed this 29th day of September,

8

2011, at Washington, DC.

9
10
11

Dated: September 29, 2011

/s/
RAVIN BALAKRISHNAN

12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

REPLY DECLARATION OF RAVIN BALAKRISHNAN, PH.D. IN SUPPORT OF MOTION FOR A PRELIMINARY INJUNCTION 21
Case No. 11-cv-01846-LHK