Apple Inc. v. Samsung Electronics Co. Ltd. et al

Filing 278

Reply Declaration of Ravin Balakrishnan, Ph.D. in Support of #86 Apple's Motion for Preliminary Injunction filed by Apple Inc.. (Attachments: #1 Exhibit A - Manual Filing Notification, #2 Exhibit B - Manual Filing Notification, #3 Exhibit C - Manual Filing Notification, #4 Exhibit D, #5 Exhibit E, #6 Exhibit F, #7 Exhibit G, #8 Exhibit H, #9 Exhibit I, #10 Exhibit J, #11 Exhibit K, #12 Exhibit L, #13 Exhibit M, #14 Exhibit N, #15 Exhibit O)(Jacobs, Michael) (Filed on 9/30/2011) Modified on 10/3/2011 linking entry to document #86 EXHIBIT A, B AND C HAVE NOT BEEN MANUALLY SUBMITTED TO THE CLERK'S OFFICE FOR FILING (dhm, COURT STAFF).

EXHIBIT I

Confidential Attorneys' Eyes Only
Outside Counsel

Page 1
1

UNITED STATES DISTRICT COURT
NORTHERN DISTRICT OF CALIFORNIA
SAN JOSE DIVISION

2

3
4

APPLE INC., a California
corporation,

5

Case No.
11-cv-01846-LHK

Plaintiff,
6

v.
7

8

9

10

11
12
13
14
15
16
17
18
19
20

SAMSUNG ELECTRONICS CO.,
LTD., a Korean business
entity; SAMSUNG ELECTRONICS
AMERICA, INC., a New York
corporation; SAMSUNG
TELECOMMUNICATIONS AMERICA,
LLC, a Delaware limited
liability company,
Defendants.
C O N F I D E N T I A L
A T T O R N E Y S'
E Y E S
O N L Y
O U T S I D E
C O U N S E L
VIDEOTAPED DEPOSITION
BENJAMIN B. BEDERSON, Ph.D.
Washington, D.C.
Saturday, September 17, 2011
9:30 a.m.

21
22

Job No. 41965

23
24
25

Reporter: Linda S. Kinkade, RDR, CRR, RMR, CSR
Videographer: Conway Barker
TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 2
1
2
3
4
5

The following is the videotaped deposition

6

of BENJAMIN B. BEDERSON, Ph.D. held at the offices

7

of:

8
9
10

Morrison & Foerster

11

2000 Pennsylvania Avenue, N.W.

12

Washington, DC 20005

13
14
15
16

Taken pursuant to applicable Rules of Civil

17

Procedure, before Linda S. Kinkade, Registered

18

Diplomate Reporter, Certified Realtime Reporter,

19

Registered Professional Reporter, Registered Merit

20

Reporter, Certified Shorthand Reporter (CA), and

21

Notary Public, in and for the District of Columbia.

22
23
24
25

TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 3
1

APPEARANCES:

2
3
4

On Behalf of Plaintiff APPLE INC., a
California corporation:

5

MICHAEL A. JACOBS, ESQUIRE

6

DEOK KEUN AHN, ESQUIRE

7

Morrison & Foerster

8

425 Market Street

9

San Francisco, California 94105

10
11
12
13
14
15

On Behalf of Defendant SAMSUNG ELECTRONICS
CO.:

16

ERIC HUANG, ESQUIRE

17

AARON KAUFMAN, ESQUIRE

18

Quinn Emanuel Urquhart & Sullivan

19

51 Madison Avenue

20

22nd Floor

21

New York, New York 10010

22
23
24
25

TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 29
1

me to explain what it does or how it does it?

2

Q.

Let's start with what it does.

3

A.

I believe that this finds the -- this

4

determines the bounds of an email header that

5

most overlaps with the specified cursor

6

rectangle, which is a -- in the application I

7

believe is a blue highlight.

8
9

Q.

So let's -- these will be concepts I

think we'll be using as we go through the rest

10

of the code, so let's make sure we understand

11

them clearly.

12

We're talking about the actual set of characters

13

that is associated with an email on a list of --

14

on a list display of emails, correct?

15
16

A.

We talk about an email header.

Is it all right if I just describe

what it does in my words?

17

Q.

Sure.

18

A.

So when you are zoomed into the email

19

tile application in X-node, one of the parts of

20

that display is a list of email headers where

21

each header is actually an image where the image

22

visually looks like the kind of information

23

commonly included in an email header, such as, I

24

believe, it includes who it's from and what the

25

subject is.
TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 30
1

So there is a list of these email headers

2

and there is also a partially transparent

3

highlight -- I'll just -- the code refers to it as a

4

cursor.

5

that's -- that highlight can be moved up and down

6

among the different email headers, and the email

7

headers can be moved up and down under the highlight

8

in interaction of this application tile.

9

I'll just call it a highlight.

I think

And so this code is given a particular

10

position of the list and a particular position of

11

the highlight.

12

rectangular bounds of a header that is the header

13

that is most under the highlight of which there is

14

only one.

15

Q.

It determines the rectangle -- the

And at this stage of the code all it's

16

going to do is answer that question, which

17

header is closest; it's not necessarily going to

18

take any action based on that decision.

19

Correct?

20

MR. HUANG:

21

THE WITNESS:

Objection to the form.
I don't believe that

22

this code has any what are sometimes called side

23

effects.

24

bounds, the calculated bounds.

25

BY MR. JACOBS:

I believe all it does is return those

TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 56
1

their finger beyond a certain number of pixels,

2

the email list will then follow the finger up

3

and down.

4

down, when the finger lets -- lifts off the

5

screen, the dragging will stop and specialized

6

method application specific method

7

SnapObjectToHighlight will be called.

As they move their finger up and

8

Q.

And SnapObjectToHighlight is where?

9

A.

SnapObjectToHighlight is defined in

10
11
12

the class EmailAppNode.
Q.

So it's not in the code that we've

reviewed so far.

13

A.

Correct.

14

Q.

Let me see if I can sum this up.

We

15

have seen a determination made in this code, in

16

Exhibit 211, EmailListNode cs, we saw a

17

determination made of which header image is

18

closest to the cursor.

19

steps that early on in the code we saw, correct?

That was one of the

20

MR. HUANG:

21

THE WITNESS:

Objection to form.
We found a mechanism to

22

determine -- well, to compute the bounds and to

23

determine which email header node -- the

24

language I would use would be most overlaps the

25

highlight cursor.
TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 57
1

BY MR. JACOBS:

2

Q.

Most overlaps is the key there, right?

3

A.

Yes.

4

MR. HUANG:

5

THE WITNESS:

Objection.
That's the actual

6

calculation that gets computed.

7

BY MR. JACOBS:

8

Q.

9

And then another function we saw was

the function of determining whether the finger

10

or other input device has moved far enough such

11

that, depending on the constants that have been

12

input, the displayed item should itself move in

13

response to that input.

14

MR. HUANG:

15

THE WITNESS:

Objection to the form.
Since -- that way you

16

said was long.

17

same thing in my words?

18

BY MR. JACOBS:

19

Q.

Please.

20

A.

When the user moves their finger, the

Can I just try and restate the

21

email header list will follow their finger after

22

the finger has moved a certain definable number

23

of pixels.

24
25

Q.

And then when the finger -- when the

user lifts his finger, a function called
TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 58
1

SnapObjectToHighlight is called.

2

A.

Correct.

3

Q.

And that's defined in, I believe it's

4

defined in Email.cs, which is Exhibit 212, if

5

you want to take a quick look at 212.

6

you we'll get a break in a second.

7
8

MR. HUANG:

I promise

Objection to the form.

Go

ahead.

9

THE WITNESS:

I think there was a

10

question that I was going to answer.

11

read back the question?

12

BY MR. JACOBS:

13

Q.

14

Yes.

Sure.

Can you

SnapObjectToHighlight

defined in Email.cs.

15

A.

So to clarify, assuming that Exhibit

16

212 is -- represents the contents of the file

17

Email.cs and that this is the file that I gave

18

to my counsel --

19

Q.

Yes.

20

A.

-- then, yes, that method is defined

21

inside of this source code.

22

this source code very briefly, I need to make

23

one further clarification to something I had

24

said.

25

And in looking at

I said that EmailAppNode appears to be the
TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 64
1

then.

2

SnapObjectToHighlight at the bottom of page 29

3

does?

4

Can you describe what public

A.

Yes.

So this is the method that gets

5

called in the OnMouseUp method that I described

6

previously when the user lifts their finger off

7

the device after they finish dragging.

8
9

It takes two parameters, Node and Animate.
The first parameter that is passed in from the

10

caller OnMouseUp is DraggedNode, which I believe is

11

the node that was being dragged up and down, which I

12

believe is slidingNode -- refers back to slidingNode

13

in the original code.

14
15
16

And the second parameter is called Animate,
which is passed in as true.

So Animate is true.

So the definition of SnapObjectToHighlight

17

takes the overall email header list that is

18

displayed on the screen and moves it with animation

19

over time so that the email header object that most

20

overlaps the highlight cursor ends up completely

21

underneath the highlight cursor.

22

Q.

Can you explain how that occurs?

23

A.

Yes.

The first line determines the

24

bounds of the highlight cursor in the

25

appropriate coordinate system.
TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 78
1

BY MR. JACOBS:

2

Q.

And is it the very same code that

3

executes that functionality in the following two

4

conditions:

5

bar is between email header images in the list;

6

condition 2, the email header is in the white

7

display space underneath the last -- below the

8

last of the email headers in the list?

9
10

Condition 1, the cursor highlight

MR. HUANG:

Objection to the form of

the question.

11

THE WITNESS:

You used the word email

12

header where I think you meant highlight cursor.

13

BY MR. JACOBS:

14

Q.

I think I did, yes.

15

again.

16

So let me ask it

can do it more clearly.

17

Maybe now that I have stated it orally I

There are two possibilities for the email

18

highlight cursor to be out of alignment with

19

email headers.

20

between email headers; the other possibility is

21

it's after the last of the email headers.

22

Correct?

23

A.

Yes.

24

Q.

Is it the exact same code that causes

25

One possibility is it's in

the email header to snap into alignment with the
TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 79
1

email cursor bar in either of those two cases?

2

MR. HUANG:

3

THE WITNESS:

Same objection.
So there is only one

4

code sequence of flow that performs snapping,

5

and that same sequence is used wherever the

6

email list is positioned vertically, including

7

when the bottom-most email header is above the

8

bottom of the screen.

9

BY MR. JACOBS:

10

Q.

And the -- it is possible that, when

11

the user lifts -- in the case of the depiction

12

on page 6 of your declaration -- when the user

13

lifts his finger, that the blue cursor bar and

14

email header image are in alignment, correct?

15

MR. HUANG:

16

THE WITNESS:

Objection to the form.
So at the time the user

17

lifts off, it's possible that one of the email

18

headers is already completely aligned underneath

19

the highlight cursor -- highlight cursor.

20

BY MR. JACOBS:

21

Q.

That's my question.

22

A.

Yes, that's possible.

23

Q.

So I think you did this before, but if

24

you could just again point us to the code that

25

tests whether that condition has been met.
TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 80
1

A.

Okay.

2

Q.

Or, to state the obvious, or that the

3

condition has not been met and that the snap

4

functionality must be invoked in order to create

5

alignment.

6

MR. HUANG:

7

THE WITNESS:

Objection to the form.
So I believe those --

8

the code does not distinguish between those two

9

conditions and it does exactly the same thing in

10

the case where the email header is already

11

aligned with the highlight.

12

same animation over the same period of time, but

13

the calculated ydiff, the amount that it moves,

14

is zero.

15

it will do the animation, it will render many

16

times with several in-between states, and those

17

in between states will all be the same.

18

actually does do a kind of degenerate snapping.

19

BY MR. JACOBS:

20

Q.

It performs the

And so it will actually do the code,

So it

So the calculation that needs to be

21

done in order to decide whether it's -- in order

22

to decide whether there is any kind of visual

23

movement that has to occur is the calculation of

24

ydiff?

25

A.

So ydiff calculates the amount of
TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 81
1

movement.

2

executed is the same, but the user will

3

experience that nothing changes on the screen.

4

Q.

If ydiff is zero, then the code

Let's just go through again the

5

calculation of ydiff with that case in mind.

6

think that was on 212 at 28.

7

A.

I

On pages 28 and 29 there is the method

8

SnapPositionToObject, which we've already

9

discussed, and at the top on page 29, near the

10

top, it calculates ydiff to be

11

sourceRectangle.top minus Rectangle.Top.

12

So in this case where the top of the email

13

header is the same position as the top of the

14

highlight cursor, then these two top values will

15

be the same, whatever they are, and so when you

16

subtract one from the other, they will have the

17

value of zero.

18

zero.

19

OffsetY property of the Matrix variable gets

20

decreased by an amount whose value is ydiff.

21

ydiff is zero, then it won't be decreased.

22

Ydiff will then have a value of

Three lines later Matrix -- sorry -- the

If

So then, when it does the actual animation

23

by calling Animate to Matrix and it passes in that

24

Matrix variable, if Matrix has not been changed from

25

its initial value, then the result of calling
TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 82
1

Animate to Matrix will be that nothing on the screen

2

changes position.

3

Q.

And then looking -- thinking about

4

this from the other end of the email list,

5

thinking about the top of the email list, is it

6

possible that the user ended his -- the user

7

experience with the blue bar -- by lifting the

8

finger at the point where the blue cursor bar is

9

above the first email header?

10

MR. HUANG:

11

THE WITNESS:

12

Q.

I don't recall.

BY MR. JACOBS:

13

Objection to the form.

14

that would stop that from happening?

15
16

Is there some -- is there something

MR. HUANG:

Same objection, objection

to the form.

17

THE WITNESS:

I haven't seen anything

18

in the code we looked at today that would stop

19

the highlight cursor from ending up above the

20

top-most email header.

21

BY MR. JACOBS:

22

Q.

And if there is no obstacle to that

23

occurring, then the same SnapTo functionality

24

should cause the blue bar to go to the top email

25

header, correct?
TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 84
1

Do you see that?

2

A.

Yes, I do.

3

Q.

What's the reference to depending on

4

the degree of the over-pan?

5

MR. HUANG:

6

THE WITNESS:

Objection, form.
If in the example that's

7

described here with these images, for a concrete

8

example, if the user has dragged -- moves --

9

touches the screen, drags their finger up so

10

they are moving the email list up, and the

11

bottom-most email header is above the bottom of

12

the screen, if -- so this is the over-pan

13

position -- if they have over-panned to a degree

14

such that that bottom email header is partially

15

overlapping with the highlight cursor, then it

16

will snap back so that the bottom-most email

17

header is aligned with the bottom of the screen

18

in this situation.

19

BY MR. JACOBS:

20

Q.

And if -- so in order for the snap

21

back to occur, there must be some partial

22

overlap at the end -- when the user lifts his

23

finger?

24

MR. HUANG:

25

THE WITNESS:

Objection to the form.
If the -- it depends on

TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 85
1

how much the bottom email header is above the

2

bottom of the screen.

3

I said, depending on the degree of the over-pan,

4

if the degree is such that there is some

5

overlap, then it will snap back.

6

BY MR. JACOBS:

7

Q.

8

A.

I believe then it does not snap back.

Then I believe it just stays in that position.

11
12

And if the degree is such that there

is no overlap, what happens?

9
10

So if it's -- that's why

Q.

And that's just a function of the

state of the code as -- let me start over again.

13

That is because the code in its -- in the

14

state in which you've provided it to us doesn't

15

have a case for no overlap or beyond overlap; is

16

that correct?

17

A.

Well, the code -- it does what it

18

does.

19

features and interactions, as we talked about,

20

and that's what it does.

21

I mean, it does a very specific set of

Q.

So --

Let me ask it this way.

Point us

22

where in the code the test is set forth in a way

23

that such that that constraint, that there must

24

be some overlap is present.

25

A.

So in Exhibit 212 -TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 86
1

Q.

Email.cs.

2

A.

-- Email.cs, pages 29 to 30, method

3

SnapObjectToHighlight, around the fifth line of

4

code it calls, GetIntersectingEmailItemBounds.

5

As we discussed earlier, this returns the

6

rectangle representing the bounds of the email

7

header that most overlaps the highlight cursor

8

implemented by that

9

GetIntersectingEmailItemBounds method.

If we're

10

in this condition where there is no email header

11

that overlaps -- sorry.

12

condition where the bounds of every email header

13

does not overlap the bounds of the highlight

14

cursor, then this method will return a

15

sourceRectangle whose value is empty.

16

If we're in the

The next line of code in

17

SnapObjectToHighlight says, if sourceRectangle is

18

not empty, then it calls SnapPositionToObject.

19

in the case we're talking about, sourceRectangle

20

would be empty and this SnapPositionToObject method

21

would not get called.

22

Q.

So

So I may not have been tracking your

23

description.

24

well, four lines up from the bottom, if you

25

include the brace, there is if sourceRectangle

On the bottom of page 29, three --

TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 97
1

Microsoft in this time period.

2

Q.

And Amy Karlson was with you at HCIL?

3

A.

Correct, she was.

4

Q.

Same for Aaron Clamage?

5

A.

Correct, yes, he was.

6

Q.

Now, if you turn to paragraph 8, it

7

says, attached as Exhibit F is an executable

8

version of LaunchTile -- line 18, 19, 20 --

9

which is identical or substantially similar to

10

the version of LaunchTile that we demonstrated

11

in our video and discussed in our paper and

12

presentation at the CHI conference.

13

that?

Do you see

14

A.

I do.

15

Q.

The reference to the possibility that

16

it is substantially similar, what are you -- why

17

did you -- why might it not be identical but

18

only substantially similar to the version that

19

you demonstrated?

20

MR. HUANG:

21

THE WITNESS:

Objection to the form.
If you don't mind, I'm

22

just going to read this section of my

23

declaration to get the whole context.

24

BY MR. JACOBS:

25

Q.

Sure.
TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 98
1

A.

There is at least two reasons that I

2

said that.

3

development of LaunchTile after we submitted the

4

paper, and so -- and the -- we made the video at

5

the time that we submitted the paper.

6

made the video with the version of the code that

7

was available right at the time we submitted the

8

paper.

9

that time.

One is that I recall that we stopped

So we

We stopped development just about after
So I don't think there was any

10

further development.

11

executable version that I attached works in the

12

same way that the video does, and so there

13

was -- I couldn't see any difference between the

14

executable that we ran and the demonstration in

15

the video.

16

my memory that these were all done

17

contemporaneously.

18
19
20

Q.

And then, in addition, the

So it's further corroboration with

But the reason for the qualification

of substantially similar is what?
A.

That I don't have a precise time stamp

21

of that source code that correlates with the

22

video that shows that it was done exactly the

23

same day and that it was exactly the same code.

24
25

Q.

The source code control system that we

discussed earlier this morning could give you
TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 103
1

THE WITNESS:

Inasmuch as it refers to

2

snapping and over-panning and under-panning,

3

yes.

4

BY MR. JACOBS:

5

Q.

Is there functionality in -- I need to

6

understand what that might exclude in your

7

declaration.

8

paragraphs 10 through 18, are there any

9

paragraphs that are describing LaunchTile that

10
11
12

So if you take a look at

you're unsure of how they apply to XNav?
A.

I'm going to have to read those

paragraphs.

13

So the features described in paragraphs 10

14

through 18 of my declaration LaunchTile and XNav

15

do in fact behave in identical or substantially

16

similar ways.

17

Q.

So, again, that phrase "substantially

18

similar" comes up.

19

which they are not identical?

Is there any respect in

20

A.

I'm not aware of any.

21

Q.

The phrase, "over-pan or under-pan

22

command," do you see that?

23

A.

Where are you referring to?

24

Q.

In paragraph 20.

25

A.

Yes, I do.
TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 113
1

BY MR. JACOBS:

2

Q.

Let me start over.

Can you please

3

demonstrate the case where the highlighting bar

4

is above the email list because you've pulled

5

the email list down and there is no overlap

6

between the highlighting bar and the top-most

7

email header?

8

MR. HUANG:

9

THE WITNESS:

Same objection.
Okay.

So I'm dragging

10

the email list down so that there is no overlap

11

between any of the email headers and the

12

highlight bar and then I let go.

13

MR. JACOBS:

14

VIDEOGRAPHER:

15

(Brief interruption.)

16

VIDEOGRAPHER:

17

Q.

Off the record at 1:34.

Back on the record at

BY MR. JACOBS:

19

Thank you.

1:34:44.

18

Okay.

Dr. Bederson, in your declaration you

20

refer to some other portions of code as

21

relevant, at paragraph 22 of your declaration.

22

I believe you have ShellForm in front of you,

23

yes, as Exhibit 213.

24

that you referred to at paragraph 22, that being

25

line 746, 760 and 764 at page 19.

We have located the lines

TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 113
1

BY MR. JACOBS:

2

Q.

Let me start over.

Can you please

3

demonstrate the case where the highlighting bar

4

is above the email list because you've pulled

5

the email list down and there is no overlap

6

between the highlighting bar and the top-most

7

email header?

8

MR. HUANG:

9

THE WITNESS:

Same objection.
Okay.

So I'm dragging

10

the email list down so that there is no overlap

11

between any of the email headers and the

12

highlight bar and then I let go.

13

MR. JACOBS:

14

VIDEOGRAPHER:

15

(Brief interruption.)

16

VIDEOGRAPHER:

17

Q.

Off the record at 1:34.

Back on the record at

BY MR. JACOBS:

19

Thank you.

1:34:44.

18

Okay.

Dr. Bederson, in your declaration you

20

refer to some other portions of code as

21

relevant, at paragraph 22 of your declaration.

22

I believe you have ShellForm in front of you,

23

yes, as Exhibit 213.

24

that you referred to at paragraph 22, that being

25

line 746, 760 and 764 at page 19.

We have located the lines

TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 115
1

right near the second line from the bottom, and

2

those are the three lines that are referred to

3

in my declaration.

4
5

Q.

Can you explain what the code is doing

on page 20?

6

MR. HUANG:

7

MR. JACOBS:

8

Actually it was a good

objection.

9

Objection to the form.

BY MR. JACOBS:

10
11

Q.

Can you explain what the code on page

20 is doing?

12

MR. HUANG:

13

THE WITNESS:

Same objection.
So the code on pages 19

14

through 21 are -- all implement a method called

15

OnMouseUp.

16

LaunchPointListener, which is a subclass of Ppan

17

EventHandler.

18

dragging and snapping in the Zone view.

19

method in question, OnMouseUp, gets called when

20

the user lifts their finger off the screen after

21

they have finished dragging.

22

OnMouseUp is part of a class called

The class in general supports
The

And the particular part of the code we're

23

looking at examines what kind of dragging was

24

being done, in particular, there is a condition

25

around line 7 on page 20 that says:
TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 116
1

If direction equals horizontal -- I'm just

2

paraphrasing this code to make it easier to

3

understand -- then it executes one body of code.

4

So this means, if the user was dragging

5

the zones horizontally, that is, left to right,

6

then there is another conditional that I

7

interpret as meaning, if the user has dragged

8

more than one-sixth of a screen width, then it

9

does one thing, and the thing that it does is it

10

will snap to the next Zone in the direction that

11

they were moving, and that is partially

12

displayed on the screen.

13

false, that is, if they dragged less or equal to

14

one-sixth of a screen width, then it would call

15

AdjustToCurrent, which I believe will end up

16

resulting in the Zone snapping back to the Zone

17

that the user started out in.

18

If that condition was

I'm still answering this question.

19

was the case for if the user was dragging

20

That

horizontally.

21

The next conditional is if the user was

22

dragging vertically, and it does parallel code for

23

snapping forward or backward.

24

condition if they were dragging neither horizontally

25

nor vertically, then it would snap back to the Zone
TSG Reporting - Worldwide

And then there is a

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 117
1

they started out at.

2

Q.

Where is that last case coded?

3

A.

It's the last three lines of page 20

4

where it says, else, open brace,

5

landscape.AdjustToCurrent (true) and then closed

6

brace.

7

Q.

So that's if neither the width nor the

8

height condition are satisfied, then it leaves

9

that possibility; is that the way that works?

10
11
12
13

A.

Right.

If the user had dragged

neither horizontally nor vertically.
Q.

So camera.ViewBounds.width or

camera.ViewBounds.height, those are variables?

14

A.

Not quite.

15

Q.

What are they?

16

A.

Those are compound statements.

Camera

17

refers to a variable which represents the camera

18

of the system.

19

the camera sees and the width -- sorry.

20

see if I can say that more clearly.

21

ViewBounds is the bounds of what
Let me

ViewBounds is the bounds of the portion of

22

the abstract space that is displayed on the

23

screen.

24
25

Q.

In the -- when you and I were

discussing the email application of the snap
TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 124
1

A.

I believe that OnMouseUp gets called

2

for each -- not for each -- for many different

3

positions of the finger while the finger is

4

being dragged that include arbitrary positions

5

on the screen, so the finger -- it can report

6

positions or positioning includes an X and a Y

7

position that can be any position on the screen.

8
9

Q.

And how does it -- can you explain how

the code is treating the second phase of the

10

three-step interaction -- interaction

11

sequence -- that you discussed?

12

A.

This code we're talking about is

13

OnMouseUp, so this actually refers to the third

14

part of the interaction sequence when the user

15

lets go of the mouse -- or lets go of the

16

screen.

17

Q.

Excuse me.
So the calculation of one-sixth of

18

this screen width I understood your testimony to

19

be that that's the -- I think I misunderstood

20

your testimony.

21

Now I see what's going on.

This one-sixth of the screen width that's

22

being referred to is a comparison of one-sixth

23

of the screen width with the amount of the --

24

the width of the tile, correct?

25

A.

So in this long statement that we're
TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 125
1

talking about on page 20, inside the horizontal

2

condition, if Math.Abs,

3

MousePressedCanvasPoint --

4
5

THE REPORTER:

If you would read

slower.

6

THE WITNESS:

Sorry.

Let me restate

7

that.

8

in this statement that begins if Math.Abs

9

(MousePressedCanvasPoint.X minus

Just to make sure we're -- to be clear,

10

e.CanvasPosition -- sorry -- CanvasPosition.X,

11

right parenthesis, greater-than sign, left

12

parenthesis, camera.ViewBounds.width/ 6, right

13

parenthesis, in this line that we're talking

14

about, that last part,

15

camera.ViewBounds.width/6, refers to one-sixth

16

of the total display of the width of the screen.

17

BY MR. JACOBS:

18

Q.

And that -- and the comparison is of

19

one-sixth of the total display of the screen to

20

what?

21

A.

So the other side of the condition is

22

the difference between the point the user

23

touched down on the screen and the current

24

point, which is at this point in the code

25

sequence is when the user lifted off on the
TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 126
1
2

screen.
So this is saying between the time the

3

user has touched down and the time the user has

4

lifted up, this looks at how much -- what the

5

difference is in horizontal position on the

6

screen of the finger's -- of the finger, and it

7

compares that to one-sixth of the width of the

8

screen.

9

Q.

So this algorithm depends on the first

10

step and the last step of the interaction

11

sequence and not on the intermediate phase of

12

the interaction sequence, correct?

13

A.

Not quite.

And the reason is because

14

this condition only gets executed if the

15

direction is horizontal that was computed in a

16

previous step.

17

executed only for the particular user

18

interaction sequences where the application has

19

determined that they are moving horizontally.

20

Q.

So this is -- this code gets

If the -- step one of the interaction

21

sequence is finger is -- touches on the screen

22

at location X/Y and step -- and then in between,

23

in the middle phase of the interaction sequence,

24

the finger moves to the left and then back --

25

right to X/Y, is the amount of movement that is
TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 128
1

think I quite understood.

2

BY MR. JACOBS:

3

Q.

So the landing position of the finger

4

and the finger-up position are identical, but in

5

between the user went back and forth diagonally.

6

MR. HUANG:

7

THE WITNESS:

Same objection.
So the code we're

8

talking about right now only executes if the

9

direction variable is horizontal.

So we haven't

10

yet talked about how that might come to be.

11

the case that the direction is horizontal, if

12

they press -- the user touched down, they moved

13

in such a way that direction ended up being

14

horizontal, and then they also had some diagonal

15

or other movement and then came back to the

16

ending point, the direction was still

17

horizontal, then this calculation would still be

18

zero.

19

BY MR. JACOBS:

20

Q.

21
22

In

So I may need to ask you how the

horizontal condition is measured.
A.

All right.

So I'm going to have to

23

refresh my recollection by looking at some code

24

to answer that question.

25

Okay.

So going back to page 15 in this same
TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 129
1

document, Exhibit 213, ShellForm.cs, we've been

2

talking about the class LaunchPointListener, which

3

is a subclass of PPanEventHandler.

4

described this code as generally supporting the

5

dragging and snapping features in the Zone view.

6

And I've already

The second method in this class, at the

7

bottom of page 15, is a method called

8

ShouldStartDragInteraction.

9

gets called to determine when the user is at the

I believe this method

10

beginning of the interaction sequence -- sorry --

11

when the user has first started dragging during the

12

second part of the interaction sequence, whether the

13

visual display on the screen should start tracking,

14

should start dragging underneath the figure.

15

With the expectation -- the way this code is

16

written is that for the first few -- first some

17

number of pixels of finger movement it does not

18

follow the finger.

19

all.

20

number of pixels, has -- more than some threshold,

21

after which the dragging interaction starts -- the

22

dragging feature starts.

23

this ShouldStartDragInteraction.

24

particular constant, the number of pixels the finger

25

must move, is determined by this constant click

It just -- nothing happens at

And then after the finger has moved a certain

So that's implemented in

TSG Reporting - Worldwide

And that

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 130
1

underscore threshold, which is defined elsewhere.

2

So until the finger has moved enough, that

3

dragging has started, until that point essentially

4

nothing happens on the screen.

5

there is a method starting on page 16 entitled

6

"Pan."

7

user is in the second phase of interaction and they

8

are dragging their screen and that threshold has

9

been reached, that is, they have moved their fingers

10
11

And, in particular,

This Pan method gets called whenever the

more than this threshold.
So once the screen starts following the

12

user's finger, then Pan gets called.

13

thing Pan does is it looks at a variable called

14

Direction.

15

the beginning of the declaration of

16

LaunchPointListener class, around the middle of the

17

screen, to be equal to a constant called none,

18

meaning there was no direction initially specified.

19

The first

Direction was initialized on page 15 in

So going back to the Pan method on page 16,

20

the first thing this Pan method does is it says, if

21

direction is none, then it compares how much the

22

current position of the finger has moved in the Y

23

direction, that is, vertically, compared to how much

24

it has moved in the X direction, that is,

25

horizontal.

If it has moved more vertically than
TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 133
1

display of finger movement from the beginning to

2

the end of the interaction sequence, that

3

movement, what defines the end point of that

4

movement?

5
6
7

A.

I'm sorry.

I don't think I understand

the question.
Q.

On page 20, if there is more

8

horizontal movement than one-sixth of the screen

9

width, then the additional movement occurs,

10

correct, beyond the movement that -- beyond the

11

movement that was in place when the finger was

12

lifted off the screen.

13
14
15

A.

I'm sorry.

I don't think -- what you

said doesn't make sense to me.
Q.

So what happens on page 20 in the --

16

with the horizontal condition being satisfied

17

and finger movement of greater than one-sixth of

18

the screen display size?

19

A.

So if we've gone through this three

20

steps of interaction, the user has touched down,

21

they have dragged more than a threshold, this

22

system has determined that they have been

23

dragging in a more horizontal than vertical

24

direction, and then the user lifts their finger

25

off the screen, at the time the finger lifts off
TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 134
1

the screen this OnMouseUp method is called.

2

it is determined that the amount that they have

3

moved their finger horizontally -- sorry -- if

4

the horizontal amount that they had moved their

5

finger in this drag sequence is greater than

6

one-sixth of the screen, then the contents of

7

this "if" condition will get executed.

8

Otherwise, the "else" condition will get

9

executed.

10
11
12

Q.

If

And when the "if" condition is

executed, what happens?
A.

So in the case that it was positive,

13

so either has moved more than one-sixth of the

14

horizontal -- let me restate that.

15

has moved their finger horizontally more than

16

one-sixth of the screen width, then it looks to

17

see if the user had been -- dragged their finger

18

to the right of the starting point or to the

19

left of the starting point.

If the user

20

In the case that they had dragged their

21

finger to the right of the starting point, it

22

calls the NavigateRight method.

23

dragged their finger to the left of the starting

24

point, it calls the NavigateLeft method.

25

Q.

If they had

Is the NavigateRight method present in
TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 135
1

the ShellForm.cs code?

2

A.

I don't believe so.

3

Q.

What does the NavigateRight method do?

4

A.

My recollection is that it will

5

Animate the screen to the next Zone to the right

6

if there is one.

7

Q.

And when you say Zone, what is the --

8

what is the Zone in the code?

9

beginning and end of a Zone in the code?

10

MR. HUANG:

11

THE WITNESS:

What defines the

Objection to the form.
There is a

12

correspondence in the data structures of the

13

code between what we refer to in the user

14

experience as a Zone.

15

structure in the code that represents each of

16

the nine Zones.

17

BY MR. JACOBS:

18

Q.

So there is a data

In the case in which the finger

19

movement in the interaction sequence is less

20

than one-sixth of the screen dimension, whether

21

it's horizontal or vertical, what does the code

22

do?

23

MR. HUANG:

24

THE WITNESS:

25

Objection to the form.
If you don't mind, I'm

just going to answer for the horizontal just to
TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 144
1

sets a callback method so that when the activity is

2

completed it calls a method called

3

PanToTileFinished.

4

ShowThumbs equals true and SlowInSlowOut should be

5

false.

6

It sets a property to say that

This whole big, long conditional ends, and

7

the last four lines of this long method are

8

executed, which sets some values -- I don't know

9

what they mean -- ActiveQuad equals tile; ActiveApp

10

equals null, commented out line, and then another

11

method tile.XNav.AnimateToMode, and I do not know

12

what this line does.

13

After this method returns, this was the

14

activate method, and the whole calling sequence of

15

other methods that resulted in this getting called,

16

at that point this activity which had been scheduled

17

will start and the animation will occur on the

18

screen, and there will be the visual snapback that

19

I've referred to.

20

Q.

In the case where the greater-than

21

one-sixth test has been met and it's a -- and

22

the horizontal-movement test has been met, and

23

so it's -- greater than one-sixth of the

24

horizontal-screen width has been met, and you

25

are -- but in the direction in which you have
TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 145
1

moved there are no more tiles, what happens?

2
3

A.

So do you mean from the user

experience or from the code?

4

Q.

Let's start from the user experience.

5

A.

So just to clarify, if you -- I think

6

the situation you described cannot happen

7

actually.

8
9
10

Q.

So you can't move past -- you can't

pull an edge of a Zone -- probably better to
create an example.

11

If you have a Zone on a screen and on the

12

right edge of the screen or the right edge of the

13

Zone there are no more tiles --

14
15
16
17

A.

Can we say Zones?

There are no more

Q.

No more Zones, and, hence, no more

Zones?

tiles, right, because tiles fill a Zone?

18

A.

Yes.

19

Q.

So to the right -- to the right of the

20

Zone on the screen there is not another Zone,

21

and the finger movement is more than 20% in a

22

horizontal dimension of the screen width to the

23

left, the Zone image does not move.

24
25

MR. HUANG:

Objection to the form of

the question.
TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 146
1

THE WITNESS:

I'm not sure -- I

2

understood everything you said except for the

3

20% part.

4

BY MR. JACOBS:

5

Q.

Okay.

Actually I think it's

6

regardless of whether it's 20%.

7

no -- if the Zone is filling the screen and to

8

the right of the Zone there is not another Zone

9

and the finger movement is to the left, does

10

the -- regardless of whether it's 20% of the

11

screen width or not, does the image of the Zone

12

move to the left with the finger?

13

If there is

A.

I believe in the set situation it will

15

Q.

And what prevents that from happening?

16

A.

In the code?

17

Q.

Yes.

18

A.

I do not know.

19

Q.

So let's take a look back at

14

not.

20

ShellForm.

21

protected override void OnDrag.

There is a functionality called

22

MR. HUANG:

23

MR. JACOBS:

24
25

Do you have a page?
Yes.

I believe it's page

17.
THE WITNESS:

I see the OnDrag method

TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 148
1
2

panning or attempting to pan the screen.
Q.

And how does it -- what happens when

3

there is no, in our case, where there is no Zone

4

to the right?

5

A.

I'm just going to look at this code

6

for a moment.

7

minute so that it will settle.

I'll figure this out in another

8

Q.

Not a problem.

9

A.

To be honest, I'm having a little

10

trouble understanding exactly how this code

11

works.

12

consistent with how it behaved, so I'm likely

13

misunderstanding something here.

14

Q.

My interpretation of the code is not

Let me come at the topic this way.

15

is the case, as we discussed, that you can't

16

cause the Zone to move to the left when the

17

right-most boundary of the Zone does not have

18

adjacent to it on the right another Zone,

19

correct?

20

MR. HUANG:

21

THE WITNESS:

Objection to the form.
I think -- let me

22

just --

23

BY MR. JACOBS:

24

Q.

Say it your way.

25

A.

There is a grid of 3x3 Zones that's
TSG Reporting - Worldwide

(877)-702-9580

It

Confidential Attorneys' Eyes Only
Outside Counsel

Page 149
1

fixed in the code.

2

right-most column, that is, any of the three

3

right Zones, and you try and drag to the left,

4

you will not be able to drag to the left.

5

Q.

Why?

So if you were in the

Why did you design it that way?

6

MR. HUANG:

7

THE WITNESS:

Objection to the form.
I don't recall our

8

thinking in that specific design decision.

9

BY MR. JACOBS:

10

Q.

Was it a design decision?

11

MR. HUANG:

12

THE WITNESS:

Objection to the form.
Or possibly a lack of a

13

design decision.

14

thinking was for that particular interaction

15

detail.

16

BY MR. JACOBS:

17

Q.

18

I don't remember what our

The source code for LaunchTile, were

you able to locate that?

19

A.

No, I was not.

20

Q.

Any idea what happened to it?

21

A.

Well, I know that Amy Karlson was

22

primarily responsible for writing it.

23

she managed source code, and I don't think I

24

probably followed it in that much detail.

25

that's why I -- when I looked, I didn't have it,
TSG Reporting - Worldwide

(877)-702-9580

I believe

So

Confidential Attorneys' Eyes Only
Outside Counsel

Page 158
1

MR. HUANG:

2

THE WITNESS:

Objection to the form.
I wouldn't characterize

3

it that way, but LaunchTile runs on the pocket

4

PC platform.

5

devices from different manufacturers over the

6

years, so I don't remember which specific one we

7

showed on that day.

8

BY MR. JACOBS:

9

Q.

And we had a number of different

Paragraph 20 says, XNav was meant to

10

run on different mobile touchscreen devices and

11

on different operating systems than LaunchTile.

12

Do you see that?

13

A.

Yes, I see that.

14

Q.

So which operating system was

15
16

LaunchTile intended for?
A.

LaunchTile was intended for pocket PC,

17

and, I believe, and Windows mobile platforms,

18

and XNav was intended for the Windows XP and

19

desktop operating systems.

20

on Windows XP.

21

Q.

I think we tested it

Do you recall any specific

22

demonstrations that you did of LaunchTile at the

23

HCIL symposium in 2005 May?

24
25

A.

Do you mean do I specifically have a

recollection of me, sort of mind's eye, showing
TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 159
1

it to an individual?

2

Q.

Exactly.

3

A.

No, I cannot think of a specific one.

4

Q.

On paragraph 20 -- sorry -- on

5

paragraph 25 you -- the declaration says that

6

your team presented our work, which included

7

LaunchTile and its user interface features

8

including the snap and panning features

9

discussed above.

Do you see that?

10

A.

Yes, I do.

11

Q.

And what do you recall specifically of

12

demonstrations or discussions of the snap and

13

panning features discussed above in your

14

declaration?

15
16
17

A.

I think I -- I got confused in your

question, if you don't mind repeating it.
(Record read.)

18

THE WITNESS:

So I recall in the

19

presentation we showed the video, and the video

20

included the panning features and at least some

21

of the snap features, and I recall generally

22

demonstrating the software -- I recall both me

23

and Amy generally demonstrating the software --

24

at least me and Amy.

25

as well.

There may have been others

Aaron Clamage probably was there as
TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 160
1

well demonstrating it.

2

BY MR. JACOBS:

3

Q.

So that's what I recall.

Were you demonstrating the software in

4

a live basis in conformance with what was

5

demonstrated on the video?

6

map what was on the video to your live

7

demonstration?

8

MR. HUANG:

9

THE WITNESS:

Were you trying to

Objection to the form.
No.

The video was a

10

short, you know, narrow summary, and when we

11

gave live demos it was much more casual.

12

would typically hand the device over to whoever

13

we were showing it to, let them do whatever they

14

want, ask us any questions.

15

seen the video, so they typically would want to

16

go beyond that.

17

BY MR. JACOBS:

18

Q.

We

They had already

Do you recall anything specifically

19

being demonstrated in May 2005 that wasn't in

20

the video?

21

A.

I don't recall the specific details of

22

what was or was not shown to any specific

23

individual.

24
25

Q.

Let me show you an email that you

produced to us.
TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 172
1
2

A.

I don't recall having any disagreement

with that.

3

(Exhibit No. 217 marked for

4

identification.)

5

BY MR. JACOBS:

6

Q.

217 is another email that may fall

7

into the category of a response, although it's

8

not in a string with John's email.

9

refresh your recollection.

It may

So 217 is an email

10

from you to John SanGiovanni dated August 25th,

11

2005, subject, LaunchTile source, and in this

12

email you give him a location from which he can

13

download the source code for LaunchTile for the

14

desktop.

Do you see that?

15

A.

Yes, I see that.

16

Q.

And then you must have sent him an

17

email so that he could decrypt an encrypted

18

file, correct?

19

A.

That's implied by this email.

20

Q.

And the reason it was encrypted was

21

what?

22

MR. HUANG:

23

THE WITNESS:

Objection to the form.
I think I vaguely --

24

well, I guess I can't remember.

25

specific recollection of why I would have
TSG Reporting - Worldwide

I don't have a

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 173
1

encrypted it.

2

BY MR. JACOBS:

3

Q.

Do you have a general sense of why you

4

were encrypting files that you were maybe -- the

5

files in particular that you were sending to

6

Microsoft?

7

A.

I guess the reason I hesitate is I

8

don't remember whether this was my request or

9

somebody else's request.

If it was somebody

10

else's request, then it was because they

11

requested it, and if it was at my initiation, I

12

guess I thought it was a good idea at the time.

13

I just don't really remember why.

14

Q.

You do recall Microsoft receiving the

15

code.

16

when I asked you what became of LaunchTile, that

17

at least you transmitted the code, I guess,

18

maybe to be precise about what you said.

19

Microsoft receive the code?

20

I think you said a few minutes ago that,

A.

Did

I guess I assume they received the

21

code and there may have even been follow-up

22

emails or other communications.

23

specific recollection of them representing that

24

they received the code.

25

Q.

I don't have a

So to expand from that slightly, you
TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 174
1

don't have a recollection of them saying, we got

2

the code, we tried it out, it works really well

3

here, we like it, something like that?

4
5

A.

I don't have any specific

recollection.

6

Q.

How about a general -- any general

7

recollection of their reaction to the product

8

when you -- assuming you delivered it to them?

9

A.

10

with it.

11

had already come and gone.

12

the answer is, I guess, no.

13
14

Q.

Well, they had already been familiar
This was a year later.

The conference

So I wouldn't -- so

Let me show you a couple more emails.

This is going back in time again.

15

(Exhibit Nos. 218, 219 and 220 marked for

16

identification.)

17

BY MR. JACOBS:

18

Q.

19

REDACTED

So 218 is September 22, 2004, right?

The last date on the string.

20

A.

September 22nd, 2004, is that what you

22

Q.

Yes.

23

A.

Yes.

24

Q.

It starts with an email from John

21

25

said?

SanGiovanni to you, legal update, and I know

TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 181
1

part of the thread so I can make more sense of

2

that?

3

Q.

Sure.

4

A.

Okay.

5

Q.

Looking at John SanGiovanni's email to

6

you of May 17th, 2005, Re Enhanced LaunchTile

7

PowerPoint, he says, thanks, yes, making great

8

progress on the IP licensing front.

9

MSR -- I believe that's Microsoft Research,

I convinced

10

correct?

11

A.

Probably.

12

Q.

-- to let me write up the LaunchTile

13

shell concepts together with some of Daniel

14

Robbins' ZoomZone claims into a patent.

15

anxious to see what the U.S. Patent Office

16

thinks.

I'm

17

Do you see that?

18

A.

I do.

19

Q.

What information do you have beyond

20
21
22
23

this email on that patent application?
MR. HUANG:

Objection to the form of

the question.
THE WITNESS:

I believe they had filed

24

an application that is on the -- I believe that

25

there is -- I believe Microsoft has filed an
TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 182
1

application for some of the things related to

2

LaunchTile and this other project that was going

3

on at Microsoft at the time, ZoneZoom.

4

BY MR. JACOBS:

5

Q.

And what was ZoneZoom?

6

A.

My general recollection is that it

7

included the idea that you could break a

8

cellphone user interface into a 3x3 grid and map

9

it to the 3x3 grid of a typical 1 through 9

10

NumPad so that you could press one of the

11

numbers on the numerical keypad and have that

12

interact with the grid on the visual display.

13
14

MR. JACOBS:
break.

15
16
17

Why don't we take a

VIDEOGRAPHER:
4.

This is the end of tape

Off the record at 3:29.
(Proceedings recessed.)

18

VIDEOGRAPHER:

This is the beginning

19

of tape 5 in the deposition of Dr. Bederson.

20

the record at 3:47.

21

BY MR. JACOBS:

22

Q.

23
24
25

On

Dr. Bederson, how are you being

compensated for your work on this case?
A.

How?

At an hourly basis, based on the

number of -- the amount of time I have put into
TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 183
1

work on this case.

2

Q.

What is your hourly rate?

3

A.

$450 an hour.

4

Q.

And that is the rate that you have in

5

your understanding, your agreement, with Quinn

6

Emanuel on behalf of Samsung?

7

A.

Yes, I believe it is.

8

Q.

How many hours have you worked so far

9

on the case?

10

A.

I don't recall exactly.

11

Q.

Roughly?

12

A.

Maybe 40 or 50.

13

Q.

Your understanding is that your work

14

in this case is as a fact witness, correct?

15

A.

That's my understanding.

16

Q.

You haven't been asked to form any

17

expert opinions on the relationship between what

18

you did and Apple's patents, correct?

19

A.

Correct, I have not.

20

Q.

Have you looked at the Apple patent

21

that Samsung is asserting your work against?

22

A.

I've reviewed it.

23

Q.

But not formed any opinions at

24

Samsung's request on the relationship between

25

your work and the patent, correct?
TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 189
1

dragging a Zone, does the blue dot move with the

2

Zone?

3

A.

4

of the screen.

5

Q.

6

No.

The blue dot stays in the center

And that's in fact, if we go back to

the demonstration, we would see that, correct?

7

A.

Correct.

8

Q.

So blue remains centered, the Zone

9

moves, and, as it says here in the article, each

10

Zone is depicted with an empty center hub during

11

dragging, correct?

12

A.

Correct.

13

Q.

And then, upon thumb release, the zoom

14

space animates to align, I guess, leaving aside

15

the 20% issue, what it might have said is, upon

16

thumb release, the zoom space animates such that

17

the zoom space is aligned with blue, right?

18

Blue didn't move.

19

A.

Correct.

It doesn't actually say that

20

blue moves in this, but I agree that it is

21

written in a slightly awkward way.

22

zoom space animates, so it's pretty clear that

23

the zoom space is the thing that's animating and

24

it results in the zoom space being aligned with

25

blue, but the way it's written is to align with
TSG Reporting - Worldwide

It says the

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 190
1

blue, so it's just very -- perhaps slightly

2

awkward, but I think the meaning is pretty

3

clear.

4

Q.

Okay.

So what we are saying here is,

5

upon thumb release, the zoom space animates such

6

that the specified zone's empty hub is aligned

7

with blue, specified by the virtue of the 20%

8

rule.

9
10

A.

Well, you said 20%.

I think dividing

by 6 --

11

Q.

Oh, you're right.

12

A.

That's why I was confused.

It's not

13

quite 20%.

14

terms of hubs.

15

as if we had said center the Zone on the screen

16

because of the fact that the blue dot is

17

centered on the screen and because of the empty

18

center hub is centered within the Zone.

19

described in a different way.

20

same.

21

Q.

This described the interaction in
The effect is exactly the same

So it's

The effect is the

And the next sentence, the visual and

22

animated guidance ensures the user is never

23

caught between zones, what is that sentence

24

driving at?

25

A.

Actually it says automated guidance,
TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 198
1

Then at time 1:15 seconds the user again

2

touched the screen, dragged down, let go, which

3

resulted in snapping forward to the Zone above.

4

And that's the end of the video.

5

BY MR. JACOBS:

6

Q.

In either of the videos did we see the

7

activity that's described in paragraph 14 of

8

your declaration?

9
10
11
12
13

A.

Neither video showed the activity

described in paragraph 14 of my declaration.
Q.

And that's what your declaration calls

the under-panning case, correct?
A.

I don't think -- paragraph 14 doesn't

14

use that term, but I believe this describes the

15

concept that was described earlier as

16

under-panning.

17

Q.

And to get back to the way the source

18

code works, that's the case where in the

19

three-stage interaction sequence where, after

20

landing on the screen, the finger is moved less

21

than 20% in the relevant direction -- sorry --

22

yes, less than one-sixth in the relevant

23

direction such that there is what the

24

declaration describes as a snapback.

25

A.

Correct.
TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 204
1

LaunchTile is motivated by the idea that, if

2

there are some places that are convenient for

3

the interface to go to, then you should make the

4

interface naturally take you to those places and

5

not let you get stuck in inconvenient places.

6
7

Q.

So you published an article recently,

"The Promise of Zoomable User Interfaces."

8

Mark this as the next in order.

9

(Exhibit No. 222 marked for

10

identification.)

11

BY MR. JACOBS:

12

Q.

The Promise of Zoomable User

13

Interfaces by Benjamin B. Bederson, 2011, Taylor

14

& Francis.

15
16

A.

What was this published in?
This was published in a journal named

Behaviour & Information Technology.

17

Q.

In 2011?

18

A.

Yes.

19

Q.

On page 4 you have a discussion of

20

Desert Fog citing Jul and Furnas.

21

labels a phenomena that you describe as allowing

22

users to fly through the space going absolutely

23

anywhere including deep into the spaces between

24

objects.

25

A.

Desert Fog

Do you see that?
No, actually.

Sorry.

TSG Reporting - Worldwide

Where are you?

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 205
1

Q.

It's on the right-hand column of --

2

it's such a vivid image I thought it might just

3

jump from the page.

4

page 4, second paragraph.

The right-hand column of

5

A.

Yes, I see this.

6

Q.

So just to maybe start a little bit

7

earlier, different zoomable user interfaces have

8

also had various navigation mechanisms, which

9

are ways for users to move through the space.

10

Again, there is a trade-off between flexibility

11

and usability.

12

fly through the space going absolutely anywhere,

13

including deep into the spaces between objects,

14

resulting in some researchers labeling this

15

phenomenon Desert Fog, Jul and Furnas.

16

say, very few other applications let a user

17

navigate beyond the actual content.

18
19

Some interfaces allow users to

Then you

Can you explain the contrast you were
drawing there between ZUIs and other applications?

20

MR. HUANG:

21

THE WITNESS:

Objection to the form.
Sure.

So if we continue

22

reading this paragraph, it describes this idea

23

of not letting you navigate between the actual

24

content.

25

document browser and editor limits navigation to

I believe it says, almost every

TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 206
1

the available content with the notable exception

2

of Microsoft Excel's scroll bar arrows, Apple

3

numbers, and Google -- I'm sorry -- with the

4

notable exception of Microsoft Excel's scroll

5

bar arrows.

6

spreadsheet, on the other hand, do limit

7

navigation.

8

allow you only to click on objects to zoom into

9

them and click on a zoom out button to zoom out,

Apple numbers and Google

On the other hand, some interfaces

10

making it impossible to get lost, but also

11

giving less control over exactly where you look.

12

So the point of this paragraph was to

13

describe that there are some applications that let

14

the user navigate in space possibly -- navigating

15

can be simple scrolling or it could be this kind of

16

zooming navigation, which is a little bit more

17

uncommon, or it could be 3-D navigation in a 3-D

18

world.

19

Sorry.

I was describing that sometimes you

20

can navigate to a place where there is no content.

21

If there is no content, then you're kind of in a

22

place that essentially -- typically -- represented

23

with an empty screen.

24

because that would make a user feel disoriented

25

since there is nothing on the screen.

And that was a concern

TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 207
1

And I said that it was more common for

2

applications to stop a user from navigating to a

3

place where there was no content, although it

4

occurred, both in widespread applications like Excel

5

and in many ZUIs, in at least those.

6

Q.

So the basic contrast you were drawing

7

was between those ZUIs that are flexible but

8

haven't addressed this problem of getting lost

9

in Desert Fog, and most applications which do

10

constrain you to the space that's filled by

11

content.

12

of your paragraph correctly?

13
14

Is that -- am I capturing the essence

MR. HUANG:

Objection to the form of

the question.

15

THE WITNESS:

The paragraph said --

16

well, it didn't say "most."

17

few applications that let you move to a place

18

where there is no content, although I did

19

describe some, and many constrained you to

20

navigating only within available, visible

21

content.

22

BY MR. JACOBS:

23

Q.

It said there were

And that -- but you were describing

24

that, as of 2011, there remains this problem in

25

ZUIs of flying through the space going
TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 213
1

BY MR. JACOBS:

2

Q.

Just as you look at some of the

3

applications, the ZUIs that are listed on the

4

table, do you regard the Apple iPhone, year

5

2007, but the iPhone you're referring to, I

6

guess, is the 2010 version of it, do you regard

7

that as having resolved the problem of getting

8

lost in Desert Fog?

9

A.

So in this particular -- in this table

10

I'm to read, the thing that comes closest to the

11

Desert Fog issue is the right-most column that

12

talks about navigation mechanism for zooming,

13

and for the Apple iPhone it says, tap to zoom

14

in, physical button to zoom out.

15

So I think it's probably referring to the

16

home screen application icons where you tap on one

17

of those application icons and it has a zooming

18

transition.

19

icon and you can press the hardware button at the

20

bottom of the device to zoom out.

21

It zooms in to launch the application

So at least in that case it solves -- that

22

interface solves the Desert Fog problem as

23

described.

24

external knowledge, you could interpret this to

25

understand that, but it was not explicit in this --

So this is a case where, with some

TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 214
1

not fully explicit in this table.

2

MR. JACOBS:

Could you just give us

3

like three minutes to make sure -- oh, actually,

4

we need to go one more demo, right?

5

out how to move the blue highlighter in XNav.

6

And so if we can get you over by the video to

7

just demonstrate that mode, that would be great.

8

THE WITNESS:

9

VIDEOGRAPHER:

We figured

Okay.
Off the record at 4:37.

10

(Brief interruption.)

11

VIDEOGRAPHER:

12

4:39.

13

BY MR. JACOBS:

14

Q.

Back on the record at

Dr. Bederson, we've given you again

15

the Sony Vaio with the XNav on it that we were

16

demonstrating earlier in the deposition, and

17

we've now learned how to move the blue

18

highlighter into the center of the screen and

19

illustrate the case where the blue highlighter

20

is between -- it ends up between email headers

21

when the stylus or the finger is lifted from the

22

screen; is that correct?

23

A.

Yes.

24

Q.

Can you demonstrate that for us?

25

A.

Actually right now that highlight is
TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 223
1

A.

2
3

Correct.
MR. JACOBS:

I think we're done.

Thank you.

4

MR. HUANG:

5

THE WITNESS:

6

VIDEOGRAPHER:

Thank you.
Thank you.
This concludes the

7

deposition of Dr. Bederson.

8

4:57 and it consists of five tapes.

9

Off the record at

(Proceedings concluded.)

10
11
12

//
(Signature having not been waived, the

13

deposition of BENJAMIN B. BEDERSON, Ph.D.

14

concluded at 4:57 p.m.)

15
16
17
18
19
20
21
22
23
24
25

TSG Reporting - Worldwide

(877)-702-9580

Confidential Attorneys' Eyes Only
Outside Counsel

Page 224
1

CERTIFICATE OF SHORTHAND REPORTER

2

NOTARY PUBLIC

3
4

I, Linda S. Kinkade, RDR, CRR, RMR, CSR,

5

the notarial officer before whom the foregoing

6

proceedings were taken, do hereby certify that the

7

foregoing transcript is a true and correct record of

8

the proceedings; that said proceedings were taken by

9

me stenographically, to the best of my ability, and

10

thereafter reduced to typewriting; and that I am

11

neither counsel for or related to, nor employed by

12

any of the parties to this case and have no

13

interest, financial or otherwise, in its outcome.

14

IN WITNESS WHEREOF, I have hereunto set my

15

hand and affixed my notarial seal this 17th day of

16

September 2011.

17
18
19

_______________________________
Linda S. Kinkade

20
21
22

NOTARY PUBLIC IN AND FOR

23

THE DISTRICT OF COLUMBIA

24

My commission expires:

July 14, 2012

25

TSG Reporting - Worldwide

(877)-702-9580