Apple Inc. v. Samsung Electronics Co. Ltd. et al

Filing 931

Declaration of Gray in Support of #930 Administrative Motion to File Under Seal Samsung's Motion for Summary Judgment filed bySamsung Electronics Co. Ltd.. (Attachments: #1 Exhibit 1, #2 Exhibit 2, #3 Exhibit 3, #4 Exhibit 4, #5 Exhibit 5, #6 Exhibit 6, #7 Exhibit 7, #8 Exhibit 8, #9 Exhibit 9, #10 Exhibit 10, #11 Exhibit 11, #12 Exhibit 12, #13 Exhibit 13, #14 Exhibit 14, #15 Exhibit 15, #16 Exhibit 16, #17 Exhibit 17)(Related document(s) #930 ) (Maroulis, Victoria) (Filed on 5/17/2012)

1 QUINN EMANUEL URQUHART & SULLIVAN, LLP
Charles K. Verhoeven (Bar No. 170151)
 [e-mail]
50 California Street, 22nd Floor

San Francisco, California 94111
 Telephone: [phone]
Facsimile: [phone]

Kevin P.B. Johnson (Bar No. 177129)
 [e-mail]
Victoria F. Maroulis (Bar No. 202603)

[e-mail]
th
 555 Twin Dolphin Drive, 5 Floor
Redwood Shores, California 94065-2139
 Telephone:
[phone]
Facsimile:
[phone]

Michael T. Zeller (Bar No. 196417)

[e-mail]
 865 S. Figueroa St., 10th Floor
Los Angeles, California 90017
 Telephone: [phone]
Facsimile: [phone]

 Attorneys for SAMSUNG ELECTRONICS CO.,
LTD., SAMSUNG ELECTRONICS AMERICA,
 INC. and SAMSUNG
TELECOMMUNICATIONS AMERICA, LLC



UNITED STATES DISTRICT COURT



NORTHERN DISTRICT OF CALIFORNIA, SAN JOSE DIVISION



APPLE INC., a California corporation,

CASE NO. 11-cv-01846-LHK


Plaintiff,


vs.



DECLARATION OF STEPHEN GRAY IN
SUPPORT OF SAMSUNG'S MOTION
FOR SUMMARY JUDGMENT

 SAMSUNG ELECTRONICS CO., LTD., a
Korean business entity; SAMSUNG
 ELECTRONICS AMERICA, INC., a New
York corporation; SAMSUNG
 TELECOMMUNICATIONS AMERICA,
LLC, a Delaware limited liability company,

Defendant.



, PH.

Case No. 11-cv-01846-LHK
DECLARATION OF STEPHEN GRAY

DECLARATION OF STEPHEN GRAY

1
2

I, Stephen Gray, declare:

3

1.

I have personal knowledge of the facts set forth herein, and am competent to testify

4 to the same.
5

2.

I submit this declaration in support of Samsung's Motion for Summary Judgment of

6 invalidity on U.S. Patents 7,844,915 and 7,864,163. If asked at hearings or trial, I am prepared to
7 testify regarding the matters I discuss in this declaration
8

3.

I reserve the right to supplement or amend this declaration based on any new

9 information that is relevant to my opinions.
10

4.

I am being compensated for my work in this matter at my standard consulting rate

11 of $370 per hour. I am also being reimbursed for expenses that I incur. My compensation is not
12 contingent upon the results of my study or the substance of my testimony.
13
14

I.

PROFESSIONAL BACKGROUND
5.

I am an independent consultant. All of my opinions stated in this Declaration are

15 based on my personal knowledge and professional judgment. In forming my opinions, I have
16 relied on my knowledge and experience in graphical user interfaces and operating systems;
17 software development practices; programming, including C and graphical programming; and on
18 the documents and information referenced in this Declaration. I have attached as Exhibit 1 a true
19 and correct copy of my current curriculum vitae (CV), which details my education and experience.
20 The following thus provides only a brief overview of some of my experience that is relevant to the
21 matters set forth in this Declaration.
22

6.

Since the mid-1970s, I have designed, developed, and deployed computing systems

23 and products that operate in server, desktop, and graphical environments.

As such, I have

24 acquired expertise and am an expert in the areas of server computing architecture and design,
25 graphical user interfaces, operating systems, local area and wide area networks, and various
26 programming languages used in the development of those systems and products. I have been
27 employed by or retained as a consultant, including acting as a litigation consultant, for numerous
28
-1-

Case No. 11-cv-01846-LHK
DECLARATION OF STEPHEN GRAY

1 companies such as Burroughs, Filenet, Fujitsu, Marriott Corporation, MCI, Northern Telecom,
2 Olivetti, TRW, and Xerox, as well as other companies.
3

7.

I have several relevant professional experiences that further demonstrate my

4 expertise in the field of graphical user interfaces. In late-2001 to mid-2002, as Chief Technology
5 Officer for Networld Exchange Inc., I was responsible for the design, development, and
6 deployment of a suite of products that delivered eCommerce functions. These functions were
7 provided over the Internet and included product catalog information display, purchase and/or
8 purchase order creation, order delivery to fulfillment systems, and order status reporting. The
9 products for which I had responsibility provided an electronic shopping graphical user interface
10 for business-to-business and business-to-consumer transactions. The graphical user interface was
11 designed to support both vendors of products as well as customers. Each of these user interfaces
12 were an optimization based on the specific user class.
13

8.

In the mid-1990s I was a consultant for Xerox. One of my assignments there was

14 to develop a graphical interface for network attached office products. For example, one of the
15 graphical user interfaces I designed provided end-user visibility into printer queues supporting
16 distributed network printers.

Another graphical user interface I designed provided network

17 operations distributed job management control.
18

9.

As a software development professional, I have had numerous occasions to review

19 bodies of source code. I have analyzed source code written in several variants of C, SQL,
20 COBOL, RPG, variants of Basic, Java, Perl, several Assembler languages, and others. For
21 example, as an individual contributor at Xerox during the mid-1980s to 1990, I evaluated the
22 quality of source code from third-party software providers for possible inclusion in the Xerox
23 product line. Also, I evaluated the source code of several application software packages for
24 completeness and maintainability, and for possible inclusion into the NTN product line in 200025 2001. During my early career, I spent time maintaining source code written by others. In each of
26 these assignments, I analyzed the source code to identify the data structures, logical flow,
27 algorithms, and other aspects.
28
-2-

Case No. 11-cv-01846-LHK
DECLARATION OF STEPHEN GRAY

1

10.

During my career as a software development professional, I have several relevant

2 professional experiences that demonstrate my expertise in the field of operating system
3 technologies. I have performed operating system programming assignments, I have publicly
4 lectured regarding various operating systems, and I have provided litigation support where
5 operating system technology was central to the matter.
6

11.

Finally, I have been retained by attorneys for plaintiffs and defendants in several

7 matters where the concepts and practice of graphical user interface technology was a central issue.
8 The matters include contract disputes: GTE v. Videotron; Eyefinity, Inc. v. Entigo; HealthFirst v.
9 HealthTrio; Waltrip Associates v. Kevin Kimperlin & Spencer Trask Ventures, as well as patent
10 infringement: WebSide Story v. NetRatings; ICR v. Harpo; Leader v. Facebook; Fotomedia v.
11 Yahoo!; Cisco v. Telcordia; Ampex v. Kodak, et al and ICI v. Red Hat and Novell.
12

II.

13

APPLICABLE LEGAL PRINCIPLES
12.

I am informed by counsel that "prior art" includes public information, public

14 knowledge, and public acts that occur before an application for a patent was filed. Prior art
15 includes patents, journals, Internet publications, systems, and products.
16

13.

I am further informed by counsel that a prior art reference "anticipates" an asserted

17 claim of a patent, and thus renders the claim invalid, if all the elements of the claim are disclosed
18 in that prior art reference, either explicitly or inherently (i.e., necessarily present or implied).
19

14.

I make this Declaration with the understanding that anticipation must be shown by

20 clear and convincing evidence.
21 III.

MATERIALS CONSIDERED

22

15.

In forming my opinions in this Declaration, I reviewed a number of materials,

23 including U.S. Patent Nos. 7,844,915 (the "'915" Patent) and 7,864,163 (the "'163 Patent") as well
24 as their respective file histories, and relevant portions of the record in this case to date. I have
25 reviewed the Expert Infringement and Rebuttal Reports of Dr. Karan Singh as well as the
26 deposition transcript of Dr. Karan Singh. In forming my opinions, I have also relied on the
27 Declaration of Dr. Benjamin Bederson dated May 17, 2012.
28
-3-

Case No. 11-cv-01846-LHK
DECLARATION OF STEPHEN GRAY

1

16.

In addition, I rely on the same materials I reviewed and considered in preparing my

2 Expert Invalidity and Rebuttal Reports. Exhibit 2 is a true and correct copy of the materials I
3 considered in preparing my Expert Invalidity Report and Expert Rebuttal Report in this case.
4

IV.

THE '915 PATENT

5

A.

OVERVIEW OF THE '915 PATENT

6

17.

The '915 Patent, entitled "Application Programming Interfaces for Scrolling

7 Operations," issued on Nov. 30, 2010 from an application filed Jan. 7, 2007. The named inventors
8 of the '915 Patent are Andrew Platzer and Scott Herz. The Patent is assigned to Apple Inc.
9

18.

The '915 Patent relates to the field of application programming interfaces that

10 provide user interface operations, such as scrolling. The '915 Patent specifically concerns the
11 problem of distinguishing among different touch-based user inputs, i.e., gestures, and responding
12 by carrying out an appropriate operation in a computer system. As of the date of invention this
13 was not a new problem, and a number of solutions to this problem already existed in the art.
14

19.

The '915 Patent generally describes a programming interface for recognizing touch-

15 based user input that signals either a "scrolling" operation or a non-scrolling "gesture" operation,
16 including scaling and rotation.
17

20.

I understand that Apple has asserted that Samsung's devices infringe independent

18 Claim 8 of the '915 Patent. Claim 8 is reproduced below:
19
20
21
22
23
24
25
26
27
28

8. A machine readable storage medium storing executable program instructions
which when executed cause a data processing system to perform a method
comprising:
[a] receiving a user input, the user input is one or more input points applied to
a touch-sensitive display that is integrated with the data processing
system;
[b] creating an event object in response to the user input;
[c] determining whether the event object invokes a scroll or gesture operation
by distinguishing between a single input point applied to the touchsensitive display that is interpreted as the scroll operation and two or more
input points applied to the touch-sensitive display that are interpreted as
the gesture operation;
[d] issuing at least one scroll or gesture call based on invoking the scroll or
gesture operation;
[e] responding to at least one scroll call, if issued, by scrolling a window
having a view associated with the event object; and
-4-

Case No. 11-cv-01846-LHK
DECLARATION OF STEPHEN GRAY

[f] responding to at least one gesture call, if issued, by scaling the view
associated with the event object based on receiving the two or more input
points in the form of the user input.

1
2
3
4
5
6
7
8
9
10
11
12

B.

CLAIM CONSTRUCTION

21.

Based on my experience, the plain and ordinary meaning to one of ordinary skill in

the art of the phrase "object invokes" in the context of claim 8 of the '915 Patent is that the object
itself calls a method or function. In other words, a person of ordinary skill in the art would
understand the phrase "the event object invokes a scroll or gesture operation" to mean that the
event object calls a method or function for scrolling or scaling.
22.

At their depositions, both inventors of the '915 Patent gave testimony that supports

my conclusion regarding the plain and ordinary meaning of "the event object invokes." Attached
to this Declaration as Exhibits 8 and 9 are true and correct copies of relevant excerpts from the
deposition testimony of Scott Herz and Andrew Platzer, respectively.

13
14
15
16
17
18
19
20
21
22 Ex. 8: Herz Depo Trans. at 95:6-96:10.
23
24
25
26
27
28
-5-

Case No. 11-cv-01846-LHK
DECLARATION OF STEPHEN GRAY

1
2
3
4
5
6
7
8
9

Ex. 9: Platzer Dep. at 79:7 to 80:5.

10
23.
11
12
13
14

This inventor testimony is consistent with the common usage of this phrase in the

field. Attached to this Declaration as Exhibit 16 is a true and correct copy of a standard computer
science dictionary, which use the phrase "invoke" in the manner referred to in the '915 Patent and
in the same sense used by the '915 inventors. Ex. 16: Microsoft Computer Dictionary (5th ed.
2002) at 287 (defining "invoke" as "to call or activate; used in reference to commands and
15
16

subroutines").
24.

17
18
19
20
21
22
23
24
25
26

Attached to this Declaration as Exhibit 13 is a true and correct copy of relevant

excerpts from the Expert Infringement Report of Apple Expert Karan Singh. Attached to this
Declaration as Exhibit 14 is a true and correct copy of Exhibit 17 to Dr. Singh's Expert
Infringement Report. The Singh Report asserts that the Android system's MotionEvent object
represents the "event object" required in Claim 8. Ex. 13: Singh Report at ¶¶ 320-324, 387. But
instead of asserting that the MotionEvent object invokes a scroll or gesture operation, the Singh
Report

maintains

that

another,

different

object

includes

a

method,

WebView.handleQueuedMotionEvent(), that invokes a scroll or gesture operation (e.g.,
handleTouchEventCommon() for a single input point and handleMultiTouchInWebView() for two
or more input points). Ex. 13: Singh Infringement Report at ¶¶ 331-332, 388. Dr. Singh also
testified that the event object does not have to invoke the scroll or gesture operation. Attached to

27
28
-6-

Case No. 11-cv-01846-LHK
DECLARATION OF STEPHEN GRAY

1 this Declaration as Exhibit 7 is a true and correct copy of the deposition transcript of Karan Singh,
2 Volume II.
3
4
5
6
7
8
9
10
11 Ex. 7: Singh Dep. Trans. Vol. II at 319:16-320:6.
12

25.

Dr. Singh's proposed definition ignores the plain meaning of the term "invoke" as

13 used in the phrase "event object invokes" – namely, "the event object invokes a scroll or gesture
14 operation." Dr. Singh argues that certain gestures "invoke" an operation, whereas the claim
15 requires something else: Claim 8 says that the "event object invokes," and not the "user input
16 invokes." Yet Dr. Singh claims that the Epic 4G manual shows that "a Swipe, Slide, or Drag, all
17 of which invoke a scroll operation, are distinguished from a Pinch or Spread, which invoke a
18 gesture operation." Ex. 13: Singh Infringement Report at ¶¶ 311-312.
19

26.

As described above, the '915 claims provide context for the claimed use of the

20 phrase "object invokes." A person of ordinary skill in the art would understand that Claim 8
21 describes a process whereby an "event object invokes" – specifically, claim 8 recites "executable
22 program instructions" in a computer. In this context, one of skill would understand that an
23 "object" is a programmatic construct, a basic building block used in computer programming
24 languages, including object-oriented programming languages - and a person of ordinary skill
25 would therefore understand that "invokes" here is being used to refer to invocation in the
26 programming context. As the inventors recognized, this meaning of "invokes" is the commonly
27 understood meaning of the term in the field, particularly in the context of program instructions for
28 software.
-7-

Case No. 11-cv-01846-LHK
DECLARATION OF STEPHEN GRAY

1

27.

I therefore disagree with Apple's apparent claim construction and interpretation, as

2 it is inconsistent with the plain and ordinary meaning of the term. A person of ordinary skill in the
3 art would understand that the claims require "determining whether the event object invokes a
4 scroll or gesture operation" and so the event object must perform the indicated action (i.e., an
5 invocation). An event object that is simply passed to other methods or modules does not take any
6 action; it is passive data, and one of ordinary skill would not understand such an interaction as an
7 "object invoking" an operation.
8

C.

9
10

The Accused Android Products Do Not Perform The Claimed "Event Object
Invokes A Scroll Or Gesture Operation"

28.

In my opinion, the Accused Products do not infringe independent Claim 8 of the

11 '915 patent, either literally or under the doctrine of equivalents.
12
13

1.
29.

Claim 8 recites the following requirement:
[c] determining whether the event object invokes a scroll or gesture operation
by distinguishing between a single input point applied to the touchsensitive display that is interpreted as the scroll operation and two or more
input points applied to the touch-sensitive display that are interpreted as
the gesture operation;

14
15
16
17

Literal Infringement

(emphasis added.)

18
19
20
21
22
23
24
25
26
27
28

30.

As discussed above, a person of ordinary skill in the art would understand the

phrase "the event object invokes a scroll or gesture operation" to mean that the event object calls a
method or function for scrolling or scaling displayed content. Applying the plain and ordinary
meaning of the phrase "the event object invokes a scroll or gesture operation," Samsung's Accused
Products do not perform this limitation.
31.

The claim limitation relating to the event object invoking a scroll or gesture

operation in Claim 8[c] is preceded by the limitation, "creating an event object in response to the
user input" in Claim 8[b]. Therefore, both limitations refer to the same "event object."
32.

At paragraph 322 of Exhibit 13, the Singh Report asserts that the Android system's

MotionEvent object represents the event object described in the claim. See also Ex. 13: Singh
-8-

Case No. 11-cv-01846-LHK
DECLARATION OF STEPHEN GRAY

1 Infringement Report at ¶ 387. However, Apple fails to show that the MotionEvent object invokes
2 a scroll or gesture operation. In fact, it does not.
3

33.

I note that Android's MotionEvent object is used to "report movement (mouse, pen,

4 finger, trackball) events. The MotionEvent object does not call a method or function to scroll or
5 scale content. Rather, the MotionEvent object may hold either absolute or relative movements and
6 other

data,

depending

on

the

type

of

device.

See

7 http://developer.android.com/reference/android/view/MotionEvent.html.
Some devices can report multiple movement traces at the same time.
Multi-touch screens emit one movement trace for each finger. The
individual fingers or other objects that generate movement traces are
referred to as pointers. Motion events contain information about all
of the pointers that are currently active even if some of them have
not moved since the last event was delivered.

8
9
10
11

12 Id. Thus, the MotionEvent object is a passive container of information with respect to scroll or
13 gesture operations, which does not call, initiate, cause or take any other actions with respect to
14 scroll or gesture operations.
15

34.

Rather than alleging that the MotionEvent object invokes a scroll or gesture

16 operation, which would be inaccurate, the Singh Report maintains that another Android object,
17 called WebView, includes a method called handleQueuedMotionEvent() that invokes a scroll or
18 gesture operation. Ex. 13: Singh Infringement Report ¶¶ 331, 388.
19

35.

The Singh Report goes on to state that the handleTouchEventCommon() method is

20 invoked by the WebView for a single touch input point, while the handleMultiTouchInWebView()
21 method is invoked for two or more touch input points. Id. Attached to this Declaration as Exhibit
22 17 is a true and correct copy of produced source code from the Accused Products.
23
24
25
26

36.

The Singh Report provides no additional discussion of how the "event object

27 invokes" the scroll or gesture operation, as required by this limitation of Claim 8. See Ex. 13:
28 Singh Report ¶¶ 321-323, 388.
-9-

Case No. 11-cv-01846-LHK
DECLARATION OF STEPHEN GRAY

1

37.

For at least these reasons, the Accused Products do not literally infringe Claim 8 of

2 the '915 Patent.
3
4

2.
38.

Doctrine of Equivalents

Dr. Singh asserts that "[t]o the extent that this limitation is not met literally, in my

5 opinion it is met under the doctrine of equivalents because each of the Accused Products perform
6 steps insubstantially different from determining whether the event object invokes a scroll or
7 gesture operation by distinguishing between a single input point applied to the touch-sensitive
8 display that is interpreted as the scroll operation and two or more input points applied to the touch9 sensitive display that are interpreted as the gesture operation, and accomplishes the same function
10 in the same way to achieve the same result." Ex. 13: Singh Infringement Report at ¶ 333.
11

39.

I disagree. In my opinion, the Accused Products do not infringe under the doctrine

12 of equivalents.
13

40.

I am informed that under the "function-way-result" test, an element in an accused

14 product is equivalent to a claimed limitation only if the element of the accused product performs
15 substantially the same function as the claim limitation, in substantially the same way, and achieves
16 substantially the same result.
17

41.

The function of this limitation is "determining whether the event object invokes a

18 scroll or gesture operation by distinguishing between a single input point applied to the touch19 sensitive display that is interpreted as the scroll operation and two or more input points applied to
20 the touch-sensitive display that are interpreted as the gesture operation." As stated above, this
21 function is not performed by the accused MotionEvent object.
22
23

Thus,

24 MotionEvent objects do not perform the same function as the event objects of the '915 patent.
25

42.

The Singh report fails to particularly address how MotionEvent objects perform the

26 recited function in the same "way" as the event objects disclosed in the '915 patent. As discussed
27 above, there is no "way" for MotionEvent objects to invoke scroll or gesture operations. The
28 alleged scrolling and scaling methods are not implemented by MotionEvent objects and are not
-10-

Case No. 11-cv-01846-LHK
DECLARATION OF STEPHEN GRAY

1 called by methods that MotionEvent implements. Thus, the differences between the "way" the
2 Accused Products operate and the way in which the claim limitation operates are not insubstantial.
3

43.

The claimed result is invocation of a scroll or gesture operation by an event object,

4 based on the event object's interpretation of one-finger input as a scroll and multi-finger input as a
5 scaling operation. As explained above, this does not occur in the Accused Products. Thus, the
6 differences between the "result" the Accused Products and claim limitation are not insubstantial.
7

44.

Moreover, in my opinion, Dr. Singh advances an equivalents argument that

8 expands the scope of the claim language to the point that claim elements are eliminated. Claim 8
9 states that "the event object invokes a scroll or gesture operation." If something other than the
10 event object may invoke the scroll or gesture operation, this would eliminate the "event object"
11 identified in this limitation. Dr. Singh's equivalents argument effectively rewrites "determining
12 whether the event object invokes a scroll or gesture operation" as "determining whether to invoke
13 a scroll or gesture operation," effectively eliding "the event object" from Claim 8[c].
14

V.

THE '163 PATENT

15

A.

16

56.

OVERVIEW OF THE '163 PATENT
The '163 Patent, entitled "Portable Electronic Device, Method and Graphical User

17 Interface for Displaying Structured Electronic Documents," issued on January 4, 2011 from an
18 application filed on September 4, 2007. The Patent purports to claim priority to a provisional
19 application filed as early as September 6, 2006. The named inventors of the '163 Patent are Bas
20 Ording, Scott Forstall, Greg Christie, Stephen O. Lemay, Imran Chaudhri, Richard Williamson,
21 Chris Blumenberg, and Marcel Van Os. The Patent is assigned to Apple Inc. on its face. A
22 review of the file history shows that Apple filed a certificate of correction on January 14, 2011 to
23 remove Bas Ording as an inventor and add Andre M.J. Boule as an inventor.
24

57.

The '163 Patent relates to methods and systems for navigating a large information

25 space on portable electronic devices with limited display screens. This limitation of portable
26 electronic devices was well-known by persons in the art prior to the filing of the application for
27 the '163 Patent.
28
-11-

Case No. 11-cv-01846-LHK
DECLARATION OF STEPHEN GRAY

1

58.

The independent claims of the '163 Patent generally cover a two-step process for

2 zooming and panning to areas of interest contained within a structured electronic document.
3

59.

I understand that Apple has asserted that Samsung's devices infringe independent

4 Claim 50 of the '163 Patent. Claim 50 is reproduced below:
50. A portable electronic device, comprising:

5
6
7
8
9

[a] a touch screen display; one or more processors; memory; and one or
more programs, wherein the one or more programs are stored in the memory and
configured to be executed by the one or more processors,
[b] the one or more programs including: instructions for displaying at least
a portion of a structured electronic document on the touch screen display, wherein
the structured electronic document comprises a plurality of boxes of content;

10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

[c] instructions for detecting a first gesture at a location on the displayed
portion of the structured electronic document; instructions for determining a first
box in the plurality of boxes at the location of the first gesture; instructions for
enlarging and translating the structured electronic document so that the first box is
substantially centered on the touch screen display;
[d] instruction[s] for, while the first box is enlarged, detecting a second
gesture on a second box other than the first box; and instructions for, in response to
detecting the second gesture, translating the structured electronic document so that
the second box is substantially centered on the touch screen display.
B.

CLAIM CONSTRUCTION

60.

I understand that the Court has not yet issued a claim construction order affecting

the '163 Patent. For purposes of this Declaration, I understand that the term "structured electronic
document" from Claim 50 of the '163 Patent will likely require construction. In my analysis, I
have applied the definition of structured electronic document I offered in paragraph 274 of my
Expert Invalidity Report:
At the time of the '163 Patent, persons skilled in the art would have been
familiar with structured electronic documents and their various applications. As
understood by those in the art, a "structured electronic document" refers to any type
of two-dimensional information space containing embedded coding that provides
some meaning or "structure" to the document. The coding is embedded within the
content of the document and specifies how elements or objects are to be arranged
within the information space and relative to one another. Thus, the comingling of
data providing structure and data providing content in the code of the document is
a distinguishing feature of a structured electronic document.

28
-12-

Case No. 11-cv-01846-LHK
DECLARATION OF STEPHEN GRAY

1

61.

Attached to this Declaration as Exhibit 6 is a true and correct copy of volume I of

2 the deposition transcript of Apple's Expert Karan Singh.
3
4
5
6

C.

7

62.

OVERVIEW OF PRIOR ART
I understand that Benjamin Bederson and his colleagues created a graphical user

8 interface for mobile devices in 2004 known as LaunchTile (also, sometimes referred to as
9 LaunchPoint). This user interface is described in an indexed publication entitled AppLens and
10 LaunchTile: Two Designs for One-Handed Thumb Use on Small Devices (hereafter "LaunchTile
11 Publication"), which was published no later than April 7, 2005 and was prepared by Dr. Bederson
12 for the ACM Conference on Human Factors in Computing Systems (known as the CHI
13 Conference). I also understand that during the CHI Conference in April 2005 (and later at a May
14 2005 symposium at the Human-Computer Interaction Lab at the University of Maryland) Dr.
15 Bederson and his team discussed their work on LaunchTile and gave live demonstrations.
16

63.

In forming my opinion, I have personally used a HP Compaq ipaq h1900 series

17 model 1950 PocketPC device running LaunchTile. I have reviewed the Declaration of Benjamin
18 Bederson dated May 17, 2012. I have also reviewed the source code for a variant of LaunchTile
19 (called XNav) which appears to have identical or substantially similar functionality to the relevant
20 LaunchTile functionality. I have personally used the XNav program running on a Sony VGN21 U750P touch-screen device.
22

64.

In the LaunchTile Publication, Bederson describes the use of gestures on a touch

23 screen user interface for navigation within an information or content space.

The space is

24 constrained by the form factor of the smart phone:
25
26
27
28

For device interaction when using a touch-sensitive screen, both designs utilize a
gestural system for navigation within the application's zoomspace. While our
designs do not directly address one-handed text entry, they are compatible with a
variety of existing single-handed text input techniques, including single- and
multi-tap alphanumeric keypad input, as well as miniature thumb keyboards and
unistroke input systems executed with a thumb (e.g., Graffiti [6], Quikwriting
[17]).
-13-

Case No. 11-cv-01846-LHK
DECLARATION OF STEPHEN GRAY

1 Declaration of Benjamin Bederson in Support of Samsung's Motion for Summary Judgment
2 (hereafter "Bederson Decl.") Ex. A at 202.
3

65.

LaunchTile consists of an "interactive zoomspace" containing 36 application tiles,

4 divided into nine zones of four tiles each.

When the entire zoomspace was in view, the

5 LaunchTile Publication referred to the view as "World View."
6
"Zone" – a
4 tile cluster

7
8
9
10
11
12
World View
13
14

66.

The zoomspace includes a blue button ("Blue") in the center of each 4-tile "Zone."

15 Any portion of a Zone can be selected to enlarge and translate the zoomspace such that the four16 tiled Zone fills the touch-screen display. After this enlarging of the zoomspace, the four tiles in
17 view are referred to as the "Zone View":
18

"Application tile"

19
20
21
22
23
24
25

Zone View

26
27

67.

From the Zone View, LaunchTile permits the user to select any one of the four

28 Application tiles and thereby once again enlarge and translate the zoomspace such that the
-14-

Case No. 11-cv-01846-LHK
DECLARATION OF STEPHEN GRAY

1 Application tile fills the touch-screen display. Additionally, from Zone View, a user can pan to
2 neighboring 4-tile clusters by "dragging" the zoomspace with his or her thumb, either vertically or
3 horizontally on the "rails" separating each application tile. As the user initiates the pan process,
4 the "zoomspace moves with the thumb during dragging." Bederson Decl. Ex. A at 205.
5

68.

As described by Bederson, LaunchTile was designed to provide the user with an

6 interface "to navigate through a group of embedded electronic elements (Application tiles)
7 contained in the single interactive zoomspace." Bederson Decl. at ¶ 11. To this end, Bederson
8 describes LaunchTile running on a portable electronic device as consisting of a single, hierarchical
9 object-oriented data structure. Bederson Decl. at ¶ 13. My analysis of the XNav source code
10 confirms this overall design architecture.
11

D.

THE '163 PATENT IS ANTICIPATED

12

69.

After considering the evidence in the record, it is my opinion that Claim 50 of the

13 '163 Patent is anticipated by the LaunchTile System, the LaunchTile Video (see Bederson Decl.
14 Ex. D), and by the LaunchTile Publication, (see Bederson Decl. Ex. A).

Attached to this

15 Declaration as Exhibit 3 is a detailed claim chart demonstrating how LaunchTile discloses each
16 and every element of Claim 50. Along with my analysis, this claim chart includes relevant
17 citations to the evidentiary record. Attached to this Declaration as Exhibits 4 and 5 are videos
18 representing two alternative behaviors of the LaunchTile System that I believe anticipate the '163
19 Patent.
20
21

1.
70.

Element [a] to Claim 50 recites:
"A portable electronic device, comprising: a touch screen display;
one or more processors; memory; and one or more programs,
wherein the one or more programs are stored in the memory and
configured to be executed by the one or more processors."

22
23
24
25

Claim 50, Element [a]

71.

The Compaq ipaq h1900 series model 1950 Pocket PC is a portable electronic

26 device with a touch screen display.
27
28
-15-

Case No. 11-cv-01846-LHK
DECLARATION OF STEPHEN GRAY

1

72.

The HP ipaq 1950 Pocket PC Quick Specs describe the device as including a

2 Samsung SC32442 processor and main memory of 32 MB SDRAM. See HP ipaq 1950 Quick
3 Specs (Dkt 168-13).
4

73.

Finally, the LaunchTile program, when running on the ipaq device, "was stored in

5 the memory and configured to be executed by the one or more processors of the portable
6 electronic device." Bederson Decl. at ¶ 9.
7

74.

For these reasons, it is my opinion that the LaunchTile System discloses this

8 element of Claim 50.
9
10

2.
75.

Element [b] to Claim 50 recites:
"the one or more programs including: instructions for
displaying at least a portion of a structured electronic document on
the touch screen display, wherein the structured electronic document
comprises a plurality of boxes of content."

11
12
13
14

Claim 50, Element [b]

76.

When running on an ipaq device, the LaunchTile System initially displays an

15 interactive zoomspace consisting of a 6x6 grid of active tiles. It is my opinion that this 6x6
16 zoomspace is a "structured electronic document" with 36 embedded Application tiles, each of
17 which is also a structured electronic document. The 36 embedded Application tiles as well as 2x2
18 groupings of these Application tiles ("Zones") are visually significant areas of interest and
19 constitute the "plurality of boxes" of content.
20

77.

The zoomspace satisfies the construction I have offered for "structured electronic

21 document," because it is a two-dimensional information space – the visual manifestation of the
22 single, hierarchical object oriented data structure Bederson describes. The zoomspace contains
23 "structure" in the form of embedded coding that provides how the various elements (i.e.
24 Application tiles and Zones) are to be arranged within the zoomspace and relative to one another.
25 Bederson Decl. at ¶ 13. The "content" (derived from the Application level of the hierarchy) and
26 the "structure" (information dictating how the Application tiles are to be visually arranged) exist
27 together in the hierarchical data structure that is visually depicted to the user as a structured
28 electronic document.
-16-

Case No. 11-cv-01846-LHK
DECLARATION OF STEPHEN GRAY

1

78.

The advantages of using such a hierarchical design structure are apparent: such a

2 design ensures a logical connection between each of the elements in the hierarchy. This permits
3 the zoomspace to utilize and display appropriate data from the embedded Applications, regardless
4 of the level of zoom. See Bederson Decl. at ¶ 15.
5

79.

The Bederson Declaration expressly recognizes that LaunchTile's creation of the

6 object-oriented data structure is very similar to the process by which a browser interprets an
7 HTML page. Bederson Decl. at ¶ 13. Because the '163 Patent expressly states that a "web page"
8 is an example of structured electronic document, this reinforces my opinion that LaunchTile's
9 interactive zoomspace is a structured electronic document.
10

80.

Attached to this Declaration as Exhibit 15 is a true and correct copy of relevant

11 excerpts from the Expert Rebuttal Report of Apple's Expert Karan Singh. I would note that the
12 Singh Rebuttal Report does not appear to dispute my opinion that the zoomspace constitutes a
13 "structured electronic document." Ex. 15: Rebuttal Report of Apple Expert Karan Singh at ¶ 33
14 n.1 ("I express no opinion as to whether the portions of the World View, Zone View, and
15 Application View displayed by LaunchTile . . . individually constitute 'structured electronic
16 documents' within the meaning of the '163 Patent.").
17

81.

Attached to this Declaration as Exhibits 10, 11, and 12 are true and correct copies

18 of relevant excerpts from the deposition testimony of '163 Patent inventors Scott Forstall, Richard
19 Williamson, and Greg Christie, respectively.
20
21
22
23
24
25
26
27
28
-17-

Case No. 11-cv-01846-LHK
DECLARATION OF STEPHEN GRAY

1
2
3
4

82.

For these reasons, it is my opinion that the LaunchTile System discloses this

5 element of Claim 50.
6

3.

7

83.

8

Claim 50, Element [c]

Element [c] to Claim 50 recites:
"instructions for detecting a first gesture at a location on the
displayed portion of the structured electronic document; instructions for
determining a first box in the plurality of boxes at the location of the first
gesture; instructions for enlarging and translating the structured electronic
document so that the first box is substantially centered on the touch screen
display."

9
10
11
84.

12

LaunchTile contains instructions for detecting a first gesture at a location on the

13 displayed portion of the zoomspace (i.e., structured electronic document) and instructions for
14 determining a first box in the plurality of boxes (i.e., a 2x2 Zone). Bederson Decl. at ¶ 16. This is
15 consistent with my opinion based on my use of LaunchTile running on an ipaq device and an
16 analysis of the similar XNav source code.
85.

17

It is further my opinion that LaunchTile's instructions for displaying an animated

18 transition from World view to Zone view are "instructions for enlarging and translating the
19 structured electronic document [i.e. the zoomspace] so that the first box [i.e. 2x2 Zone] is
1
20 substantially centered on the touch screen display." In other words, it would not be correct to

21 describe the Zone view as a different "structured electronic document." The Zone view is merely
22 an "enlarged" and "translated" view of the same zoomspace that existed at the World view.
23 Although the four embedded tiles that remain in view render additional content from the
24
25

1

I have also opined that the term "substantially centered" is indefinite because it fails to
26 reasonably apprise persons of skill in the art as to what is being claimed. For purposes of this
Declaration, however, I assume that LaunchTile's enlarging and translating of the zoomspace such
27 that a Zone or Application tile fills the touch-screen display is at least an example of "substantially
centering" a "box of content."
28
-18-

Case No. 11-cv-01846-LHK
DECLARATION OF STEPHEN GRAY

1 Application-level of the hierarchical data

structure, the two-dimensional information space

2 presented to the user is still derived from the same underlying collection of content and structure.
3

86.

At the display level, Video Exhibits 4 and 5 in support of this Declaration make

4 clear that LaunchTile's animated transitions are designed to give the user the impression that the
5 zoomspace itself is "enlarged" and "translated." Additionally, with respect to the transition from
6 Zone view to Application view, the LaunchTile Publication describes the process as "[a]n
7 animated zoom [that] draws the zoomspace toward the user until the target application fills the
8 entire display." Bederson Decl. Ex. A at 205. It is significant that the LaunchTile user interface
9 includes consistent visual metaphors such as a large, blue on-screen button in the middle of each
10 Zone and dividing "rails" between each Application tile. These visual metaphors, which exist in
11 World view and in an enlarged form in Zone view, reinforce to the user that a single, unified
12 information space is at all times being viewed and displayed.
13

87.

Additionally, at the code level, LaunchTile's overall hierarchical design structure

14 clearly indicates that during each zooming transition, "it is still fundamentally the same
15 hierarchical object oriented data structure that is visually displayed to the user. The four tiles that
16 happen to be displayed in Zone view are the same embedded Application tiles (albeit rendered in
17 further detail) that were present at World view." Bederson Decl. at ¶ 17. This is precisely what I
18 would expect from a program employing a top-down hierarchical data structure. Indeed, the
19 whole purpose of employing such a structure is to leverage the inherent logical connections
20 between the lower-level elements and the top-most element in the hierarchy. When a hierarchical
21 data structure (like LaunchTile) consists of many lower-level elements, each containing a large
22 amount of data, it is sometimes the case that the available screen space may limit the amount of
23 content that can be rendered from the lower-level elements.2 But this does not change the fact that
24
25

2

By contrast HTML documents (which are also transformed into a hierarchical structure) are
26 specifically designed such that the amount of content associated with any one element does not
overwhelm the structure within which it is placed. In other words, HTML documents are coded so
27 that the entire "document" can be rendered at once. However, there is no requirement in Claim 50
that the entire "document" be rendered upon the initial displaying step.
28
-19-

Case No. 11-cv-01846-LHK
DECLARATION OF STEPHEN GRAY

1 all the content exists as one cohesive data structure, represented to the user as a "structured
2 electronic document."
3

88.

The Singh Rebuttal Report's primary criticism of my position is that "moving to a

4 different layer in LaunchTile . . . does not merely enlarge or translate a structured electronic
5 document, but instead displays different and additional content." Ex. 15: Singh Rebuttal Report at
6 ¶ 31. However, as described above, LaunchTile's rendering of additional content during the step
7 from World view to Zone view does not mean that the original, embedded tiles have been
8 replaced. In fact, during deposition questioning, even Dr. Singh appears to have recognized that
9 there is no requirement that an "electronic document" (a term encompassing a "structured
10 electronic document," Ex. 6: Singh Dep. Trans. Vol. I at 73:14-15) be all at once visually
11 manifested to the user:
12
13
14
15
16
17
18
19
20
21
22
23
24
25
Ex. 6: Singh Dep. Trans. Vol. I at 178:22-179:22 (emphasis added).
26
27
28
-20-

Case No. 11-cv-01846-LHK
DECLARATION OF STEPHEN GRAY

1

This is exactly how Dr. Bederson describes the zoomspace and its

2 embedded elements in LaunchTile. Bederson Decl. at ¶ 14 ("[W]hile the zoomspace did consist of
3 a collection of embedded tiles that were distinct areas of interest . . . those embedded tiles were
4 always part of one unified zooomspace that was dependent on a single object-oriented data
5 structure for its content during the rendering process.").
6

89.

Finally, even if the tiles (embedded structured electronic documents) were entirely

7 replaced during the enlarging and translating step, this would not change my opinion that the
8 zoomspace, within which the tiles exist at any particular level of zoom, is the same structured
9 electronic document throughout the navigation process. In a somewhat analogous situation, I am
10 aware that web pages (structured electronic documents encoded in HTML) sometimes contain
11 embedded objects displaying live content in the form of advertising material, stock quotes, or
12 "breaking news" headlines. When a user manually refreshes the web page or, in some cases, after
13 some pre-set amount of time, the embedded object will be updated or even replaced with entirely
14 new different content. However, no person of ordinary skill in the art would believe that they
15 were viewing a different webpage (i.e., "structured electronic document") merely because the
16 content in one embedded element had changed.
17

90.

For these reasons, it is my opinion that the LaunchTile System discloses this

18 element of Claim 50.
19
20

4.
91.

21

Element [d] to Claim 50 recites:
"instruction[s] for, while the first box is enlarged, detecting a
second gesture on a second box other than the first box; and instructions
for, in response to detecting the second gesture, translating the structured
electronic document so that the second box is substantially centered on the
touch screen display."

22
23
24

Claim 50, Element [d]

92.

LaunchTile contains instructions for detecting a second gesture at a location on a

25 second box (i.e. Application tile). Bederson Decl.
at ¶ 19. This is consistent with my opinion
26 based on my use of the LaunchTile running on an ipaq device and an analysis of the similar XNav
27 source code.
28
-21-

Case No. 11-cv-01846-LHK
DECLARATION OF STEPHEN GRAY

1

93.

It is my opinion that in Zone view, each of the four Application tiles constitute

2 second "boxes" of content "other than the first box" notwithstanding the fact that the second
3 "boxes" happen to be within the boundaries of the first box.
4

94.

It is further my opinion that a gesture on one of the four Application tiles within the

5 Zone view (as depicted in Video Exhibit 4) "enlarges" and "translates" the zoomspace such that
6 the "second box" (i.e. Application tile) is at least "substantially centered" on the touch-screen
7 display. For the same reasons described in Section V.D.3, it is my opinion that this operation
8 meets the requirements of the claim limitation notwithstanding the fact that LaunchTile renders
9 additional content. Additionally, it is my opinion that this operation meets the requirements of the
10 claim limitation notwithstanding the fact that the process involves an "enlarging" step in addition
11 to the required "translating" step.
12

95.

Alternatively, if the claim language is construed to preclude the "second box" from

13 existing within the boundaries of the "first box," I believe the many other commonly used
14 operations in LaunchTile anticipate Claim 50. For instance, the LaunchTile Publication describes
15 how from Zone view, "the user can use his or her thumb directly to drag the zoom space" (i.e.
16 translate the "structured electronic document") in order to bring another Zone into view. Bederson
17 Decl. Ex. A at 205. Therefore, as shown in Video Exhibit 5 attached to this Declaration, after a
18 gesture at the location of a first box (i.e. first Zone), the user is able to drag a second Zone with
19 four additional "second boxes" of content into view. A gesture at any one of these second boxes
20 would then cause the translation of the zoomspace to bring the Application tile fully into view, as
21 described by the '163 Patent. It is my opinion that this operation meets the requirements of the
22 claim limitation notwithstanding the fact that the user performs an "interim gesture" between the
23 "first" and "second" gestures explicitly called for by the claim.
24

96.

For these reasons, it is my opinion that the LaunchTile System discloses this

25 element of Claim 50.
26

97.

Because the LaunchTile System and the LaunchTile Publication disclose each and

27 every limitation of Claim 50, it is my opinion that Claim 50 is anticipated by these prior art
28 reference.
-22-

Case No. 11-cv-01846-LHK
DECLARATION OF STEPHEN GRAY

1
2

VI.

OTHER COMMENTS
98.

The opinions expressed in this Declaration are my opinions based on my review to

3 date of the evidence in the record. I reserve the right to amend or update my opinions as
4 appropriate in response to future developments regarding claim construction.
5

99.

At a hearing or trial, I reserve the right to use as exhibits various documents

6 produced in this case that refer or relate to the matters discussed in this Declaration or to either of
7 my Expert Reports. I have not yet selected particular exhibits that might be used. I also reserve
8 the right to create or assist in the creation of certain demonstrative evidence that will assist me in
9 testifying.
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
-23-

Case No. 11-cv-01846-LHK
DECLARATION OF STEPHEN GRAY