Elan Microelectronics Corporation v. Apple, Inc.

Filing 282

Declaration of Ravin Balakrishnan in Support of 281 Opposition/Response to Motion, for Partial Summary Judgment of Infringement of U.S. Patent No. 5,825,352 (PUBLIC VERSION) filed byApple, Inc.. (Related document(s) 281 ) (Greenblatt, Nathan) (Filed on 6/10/2011)

1
2
3
4
5
6
7
8

MATTHEW D. POWERS (Bar No. 104795)
[e-mail]
JARED BOBROW (Bar No. 133712)
[e-mail]
SONAL N. MEHTA (Bar No. 222086)
[e-mail]
DEREK C. WALTER (Bar. No. 246322)
[e-mail]
NATHAN GREENBLATT (Bar No. 262279)
[e-mail]
WEIL, GOTSHAL & MANGES LLP
Silicon Valley Office
201 Redwood Shores Parkway
Redwood Shores, CA 94065
Telephone: [phone]
Facsimile: [phone]

9
10

Attorneys for Defendant and Counterclaim Plaintiff
Apple Inc.

11

UNITED STATES DISTRICT COURT

12

NORTHERN DISTRICT OF CALIFORNIA

13

SAN FRANCISCO DIVISION

14
15

ELAN MICROELECTRONICS
CORPORATION,
Plaintiff and Counterclaim
Defendant,

16
17
18
19
20

Case No. C-09-01531 RS (PSG)

v.
APPLE INC.,
Defendant and Counterclaim
Plaintiff.

21

DECLARATION OF RAVIN
BALAKRISHNAN IN SUPPORT OF
APPLE INC.’S OPPOSITION TO
ELAN MICROELECTRONICS
CORP.’S MOTION FOR PARTIAL
SUMMARY JUDGMENT OF
INFRINGEMENT OF U.S. PATENT
NO. 5,825,352
DATE:
TIME:
JUDGE:
CTRM:

July 14, 2011
1:30 p.m.
Hon. Richard Seeborg
3, 17th Floor

22
23
24
25

FILED UNDER SEAL

26

SUBJECT TO PROECTIVE ORDER

27
28

DECLARATION OF RAVIN BALAKRISHNAN

Case No. C-09-01531 RS (PSG)

1

I.

INTRODUCTION

2

I, Ravin Balakrishnan, declare and state as follows:

3

1.

I have been retained by Respondent Apple Inc. (“Apple”) to investigate and opine

4

on certain issues related to U.S. Patent 5,825,352 (“the ’352 Patent”) as applied to the accused

5

Apple iBook G4, PowerBook G4, MacBook, and MacBook Pro that use the
touchpads.

6
7
8
9
10

or

II.

QUALIFICATIONS
2.

I am a tenured Professor in the Department of Computer Science at The University

of Toronto, where I also serve as the Canada Research Chair in Human-Centered Interfaces.
3.

I received my B.Sc. (1st Class Honours) degree in Computer Science at the

11

University of New Brunswick in May 1993. Subsequently, I received my M.Sc. and Ph.D.

12

degrees in Computer Science in January 1997 and February 2001, respectively, both from the

13

University of Toronto.

14

4.

Since 1993, I have either trained or worked in the field of human-computer

15

interfaces, including inter alia interfaces for touch sensitive input devices, multi degree-of-

16

freedom input devices, two-handed input, multi-touch input, haptic feedback interfaces, tablet-

17

based input, large and small scale displays, and interactive 3D graphics.

18

5.

During my undergraduate studies at the University of New Brunswick, I worked as

19

a research assistant in the human interface lab, working with different kinds of novel input

20

technologies, including touch input systems for three dimensional data interaction.

21

6.

Since my undergraduate years, I have continued to work in this and other related

22

areas. I have published 13 refereed journal papers, 98 refereed conference full-length papers,

23

three refereed conference notes and short papers, and two technical reports in the field of human-

24

computer interfaces. I have further presented numerous conference abstracts, posters, talks, and

25

demonstrations in my field. I am a named inventor on nine issued patents in my area of work,

26

plus an additional eleven filed (though not yet issued) patents.

27
28

7.

I have also served on the organizing and paper reviewing committees of many

leading conferences in my field, and have taken on editorial roles for leading technical journals in

DECLARATION OF RAVIN BALAKRISHNAN

1

Case No. C-09-01531 RS (PSG)

1

fields pertinent to my research. For example, I am currently an Associate Editor of the ACM

2

Transactions on Computer-Human Interfaces (the premier journal in the field), and of the IEEE

3

Transactions on Visualization and Computer Graphics. Similarly, I have been the paper’s chair

4

for the ACM UIST Symposium on User Interface Software and Technology, and have served

5

multiple times as an associate chair for the premier ACM CHI Conference on Human-Computer

6

Interaction.

7

8.

I joined The University of Toronto in July 2001 as an Assistant Professor. In

8

2006, I was promoted to Associate Professor with tenure, and in 2011 was promoted to full

9

Professor. As a professor, I have taught nine undergraduate courses and nine graduate courses in

10

topics related to human-computer interaction. Nine Ph.D. students and twenty research masters

11

students have completed their degrees and research under my supervision, and five postdoctoral

12

fellows have completed their research training under my supervision. In addition to these

13

graduated students and postdoctoral fellows, I currently supervise 4 Ph.D. and 7 masters students,

14

and 2 postdoctoral fellows.

15

9.

My research at The University of Toronto has involved nearly every broad aspect

16

of human-computer interaction and data visualization. For instance, I have done significant work

17

in the areas of input devices, sensing technologies, and interaction techniques, in particular touch

18

and multi-touch interaction, gestural, sketching, and multi degree-of-freedom interaction,

19

interfaces to small and/or mobile computers, and interfaces to displays of the future. As another

20

example, I have done work in the evaluation of user interfaces, including associated metrics and

21

predictive models of human performance. My research program has been funded by leading

22

companies such as Microsoft, IBM and Hewlett-Packard and also organizations such as the

23

National Sciences and Engineering Research Council of Canada and also the Sloan Foundation.

24

10.

I have also received major awards and honors in my field, including:

25

•

Alfred P. Sloan Research Fellowship.

26

•

Nine best paper awards and honorable mentions at the leading conferences
in my field.

•

Ontario Premier’s Research Excellence Award, which included a $100,000
research grant.

27
28

DECLARATION OF RAVIN BALAKRISHNAN

2

Case No. C-09-01531 RS (PVT)

1
2

11.

A more detailed description of my work experience and other qualifications can be

found in my Curriculum Vitae, which is attached as Appendix A to this Report.

3

12.

I have previously testified in the ITC Investigation In re Certain Video Game

4

Machines and Related Three-Dimensional Pointing Devices, Inv. No. 337-TA-658 on behalf of

5

respondent Nintendo.

6

13.

I have also testified in the recent ITC Investigation instituted by Elan on behalf of

7

Respondent Apple in which Elan asserted U.S. Patent 5,825,352, the same patent at issue here.

8

The Investigation name was In re Certain Electronic Devices With Multi-Touch Enabled

9

Touchpads And Touchscreens, No. 337-TA-714. During those proceedings, the parties agreed

10

that I was an expert in the field of computer user input devices, an agreement CALJ Luckern

11

accepted.

12

14.

As set forth in my CV, I have over twenty years of experience studying and

13

teaching computer programming. I have been a professor of computer science for over nine

14

years. I can read and program using object-oriented programming languages fluently, including

15

the C++ language in which Apple’s code is written.

16

15.

I am currently being compensated at a rate of $400 per hour. My compensation is

17

not contingent upon the outcome of this case.

18

III.

19

MATERIALS CONSIDERED
16.

My conclusions are based on my review and analysis of various sources of

20

information relating to the structure, function and operation of the accused Apple products,

21

including source code for the accused legacy products made available to Elan in this Investigation

22

(including specifically those portions of the source code relied upon by Mr. Dezmelyk), and the

23

materials cited and relied upon by Mr. Dezmelyk in his report. A complete list of materials

24

considered is attached hereto as Appendix B.

25

IV.

BACKGROUND OF THE TECHNOLOGY

26

17.

In this section, I provide a background of the applicable technology.

27

18.

The ’352 Patent relates to touch sensitive input devices. A touch sensitive input

28

device typically consists of a flat surface with an underlying sensing mechanism and associated

DECLARATION OF RAVIN BALAKRISHNAN

3

Case No. C-09-01531 RS (PVT)

1

software that can determine the position of a user’s finger or another object on the surface. This

2

touch location information can be used, for example, to control a cursor position on screen or

3

activate virtual buttons. Touch sensitive devices may be opaque and operate independently of the

4

computer display, or transparent and mounted as an overlay on the display.

5

19.

From a hardware perspective, several different technologies have been developed

6

over the years for sensing touch on the surface, with three common technologies being resistive,

7

capacitive, and optical sensing. Resistive touch sensing devices, in a typical instantiation, consist

8

of two stacked sheets of electrically conductive material that are spaced slightly apart such that

9

there is a gap between the two sheets. When a user touches the top sheet, it deforms slightly at the

10

touch point and contacts the bottom sheet, creating an electrical connection between the two

11

sheets at that point. A touch location can be calculated by varying a voltage across one sheet and

12

measuring the voltage at the contact point on the other sheet. This measurement is used to derive

13

the position of the contact along one axis (x or y). A similar voltage is placed across the second

14

sheet in a direction perpendicular to the first, and the position on the second axis (x or y) is

15

measured accordingly—thus providing an x, y position for the contact:

16

Resistive Sensing

17
18
19
20
21
22
23
24
25
26

20.

Commercially viable resistive touch sensing devices will typically include

additional circuitry to, for example, remove electrical noise and improve sensing accuracy.
21.

The advantages of resistive touch sensing devices are that they can be

27

manufactured at low cost, in sizes ranging from a square inch to several square feet, and can sense

28

contact from either human fingers or inanimate objects such as a stylus. Disadvantages are that

DECLARATION OF RAVIN BALAKRISHNAN

4

Case No. C-09-01531 RS (PVT)

1

they typically cannot provide a correct position reading if the device is touched at more than one

2

location concurrently, and they can only detect a finger or inanimate object when it comes in

3

actual contact with the device and cannot provide information regarding the position or proximity

4

of a finger/object as it approaches the device.

5

22.

Capacitive touch sensing devices rely on measuring changes in capacitance on a

6

conductive surface when it is touched. Because the human body is electrically conductive,

7

touching or coming in close proximity to a conductive surface will influence that surface’s

8

capacitance, which is measured and analyzed to determine the location of the touch. There were

9

many different implementations of capacitive touch sensing devices known in the art prior to

10

January 1996. In one such instantiation, sometimes referred to as surface capacitive touch

11

sensors, a small voltage is applied on the conductive layer that coats just one side of an insulating

12

material. When an electrical conductor such as a human finger touches the surface, the finger and

13

surface dynamically form a capacitor. The location of the touch is calculated by measuring the

14

change in capacitance measured across the sensor’s surface. While simple, fast, and durable, the

15

accuracy of this approach is highly dependent on how uniformly the conductive layer is applied

16

on the insulating substrate and is also prone to errors due to stray capacitances arising from

17

electrical noise in the environment.

18

23.

A more accurate capacitive touch sensing implementation, which was also known

19

in the art prior to January 1996, is often referred to as projected self capacitive touch sensors.

20

This method improves upon surface capacitive touch sensors by etching the conductive layer to

21

form a grid of electrodes. Each line within the grid is typically called a trace. A finger touching

22

or in close proximity to the sensor will cause a change in capacitance that will vary at each trace

23

by an amount that depends on the trace’s distance to the touch location. In a common

24

implementation, the grid is arranged along the Cartesian X-Y axes, with each row or column of

25

the grid forming a trace, as shown below:

26

Capacitive Sensing

27
28

DECLARATION OF RAVIN BALAKRISHNAN

5

Case No. C-09-01531 RS (PVT)

1
2
3
4
5
6
7
8
9
10
11

24.

Determining the touch location is done by first scanning the capacitance values at

each trace, resulting in a set of trace values along each of the X and Y axes as depicted below:

12
13
14
15
16
17
18
19
20
21
22
23
24

25.

These trace values are then subsequently analyzed to determine where one or more

touches are occurring.
26.

While traditional projected self capacitive touch sensors increase the accuracy of

25

locating a contact, projected self capacitive touch sensors only provide one dimensional views of

26

the contact (a view along the x axis and a view along the y axis). A more advanced type of

27

capacitive touch sensor, called a mutual capacitive touch sensor, yields capacitive values at each

28

x-y node on the touch sensor as illustrated below:

DECLARATION OF RAVIN BALAKRISHNAN

6

Case No. C-09-01531 RS (PVT)

1
2
3
4
5
6
7
8
9
10
11
12

27.

Briefly, this may be accomplished by, for example, sequentially measuring the

13

difference in capacitance among pairs of perpendicularly oriented conductors until a measurement

14

has been made for each pair of perpendicularly oriented conductors.

15

28.

The advantages of capacitive touch sensors are that they can be made from a

16

variety of materials including ones that are truly transparent, they enable the sensing of more than

17

one touch point, and they can also detect position of fingers that are not actually touching but are

18

in close proximity to the device. Disadvantages are that capacitive touch sensors can typically

19

only sense touch from a human and cannot detect electrically inert objects such as a stylus.

20

Furthermore, mutual capacitive touch sensors generate more data (n2) than that provided by self

21

capacitive touch sensors (2n), thus requiring more computational processing on the part of the

22

touch sensor or computer.

23

29.

Optical touch sensing devices are those that employ an imaging sensor, such as a

24

camera, to create an image of the touch surface. A key advantage is that they are not limited to

25

sensing just points of contact but rather can be used to sense touch and objects of more complex

26

shapes; for example, a palm pressed against an optical touch sensing device can be detected in its

27

entirety and not just as five finger touch points.

28

DECLARATION OF RAVIN BALAKRISHNAN

7

Case No. C-09-01531 RS (PVT)

1

30.

Hardware is but one component of a touch sensing device. The signals produced

2

by whatever hardware is used have to be read and analyzed by software to ultimately determine

3

how the touch information is interpreted and presented to the host computer. Software for a touch

4

sensing device would typically perform two high-level functions. First, the signals from the

5

hardware will have to be read or scanned. This scanning step is in essence a data-acquisition

6

process that is relatively straightforward in that no attempt is made to analyze data. After the

7

signal data has been acquired, it is followed by an analysis step where the data is then interpreted.

8

The analysis step can vary in complexity depending on the comprehensiveness of the scanned

9

data and the kinds of information one might want to extract from it.

10

31.

Because of the different sensing technologies available, the algorithms used in the

11

analysis step to determine the desired touch information may vary. When only data along two

12

axes is provided, the analysis step is typically simpler in that a much smaller data set has to be

13

analyzed than when data across the entire matrix of the sensed surface is provided. The downside

14

is that the smaller data set also limits the type of information that can be inferred. For example,

15

recognizing the outline of several fingers touching the surface can be readily done with the rich

16

data set from an optical touch sensor as opposed to the limited data from a resistive touch sensor

17

that provided data along two axes.

18

32.

During the timeframe of the ’352 Patent (i.e., January 1996), the amount of

19

computational processing power that could be dedicated to the analysis of the touch device data

20

was generally limited. The analysis could be performed either on an embedded microprocessor in

21

the touch device itself, with limited memory and processing power, or on the host computer’s

22

main processor, in which case it would be competing for resources with all the other software

23

running on that host computer. Thus, simpler devices that reported smaller amounts of data, such

24

as only along the x and y axes, and which consequently required a simpler analysis step, were

25

more common even if they could not provide the sophisticated touch interpretations of the more

26

complex devices.

27

V.

28

OVERVIEW OF THE ’352 PATENT
33.

In this section, I will provide an overview of the ’352 Patent.

DECLARATION OF RAVIN BALAKRISHNAN

8

Case No. C-09-01531 RS (PVT)

1

34.

As explained in the specification, at the time of filing of the ’352 Patent there

2

already existed a wide variety of touch pads that used any number of different sensing

3

technologies including those described above. See, e.g., Exh. 1 [’352 Patent] at 1:18-26. Thus,

4

one of ordinary skill in the art would recognize that the claimed invention of the ’352 Patent is

5

not about new touch pad devices or new hardware for detecting contact with a touch pad. Rather

6

the ’352 Patent is about a specific, allegedly novel method for utilizing pre-existing touch pad

7

technology to detect the presence of multiple simultaneous finger contacts. As the ’352 Patent

8

puts it, the ’352 Patent “relates generally to touch pad devices, and more particularly relates to

9

touchpad devices which detect at least the presence of one or more objects such as fingers . . . .”

10

See, e.g., id. at 1:12-15. In other words, at a high level, the ’352 Patent pertains to the detection

11

of one or more simultaneous contacts with a touch pad. See, e.g., id. at Title (“Multiple Fingers

12

Contact Sensing Method For Emulating Mouse Buttons And Mouse Operations On A Touch

13

Sensor Pad”); see also id. at Abstract (“Method and apparatus for detecting an operative coupling

14

between one or more fingers or other appropriate objects and a touch pad . . . .”).

15

35.

As explained in the specification, it is desirable to detect two finger contacts with a

16

touch pad so that a touch pad may be used to perform both the cursor movement and button

17

actuation functions of a conventional mouse, thus obviating the need for a traditional computer

18

mouse. Within the context of the ’352 Patent, this is accomplished by allowing the first touch pad

19

contact and subsequent movement to control a cursor, while the second touch pad contact is

20

interpreted as button actuation. See, e.g., id. at 2:56-3:15 (noting that “the present invention can

21

be described in most of its applications by establishing one finger as controlling movement of the

22

cursor, and the second finger as controlling functions equivalent to a mouse button or switch”);

23

see also, e.g., id. at 4:36-39 (noting that a “further object of the present invention is to provide a

24

method for effecting on a touchpad, through the use of multiple finger contacts, a plurality of

25

conventional mouse button functions”).

26

36.

To perform such functions, the ’352 Patent discloses a specific technique for

27

detecting multiple finger contacts based on the presence of maxima and minima in a capacitance

28

profile of the touch sensitive surface. However both the use of maximum values to detect touches

DECLARATION OF RAVIN BALAKRISHNAN

9

Case No. C-09-01531 RS (PVT)

1

to the surface and the detection of two fingers to a touch sensitive surface were known at the time.

2

For example, at the time of filing of the ’352 Patent, it was known that the presence of a single

3

finger contact could be determined by identifying a maximum in the signal data acquired from the

4

touch pad, including in the prior art cited on the face of the ’352 Patent. See, e.g., Exh. 2 [U.S.

5

Patent No. 4,686,332] (cited prior art determining finger contact based on the identification of

6

two matrix wires carrying the maximum signal); Exh. 3 [U.S. Patent No. 5,149,919] (determining

7

stylus position based on sets of matrix wires containing the highest signals); Exh. 4 [U.S. Patent

8

No. 4,733,222] (interpolating the three largest signals in a set of "drive" electrodes and to identify

9

finger position). As shown below, the use of peak detection to identify a finger contact was both

10

known and straightforward:

11
12
13
14
15
16
17
18

37.

Likewise, during prosecution of the ’352 Patent, the patentee explained that to

19

detect the presence of multiple fingers, a well-known centroid “jumping” algorithm was

20

available. The patentee explicitly distinguished this technique from the alleged invention of the

21

’352 Patent during the prosecution of the ’352 Patent. See generally Exh. 5 [Apr. 8, 1998

22

Amendment]. In the centroid “jumping” technique, when one finger is in contact with a set of

23

sensors on the touch pad, the location of the finger may be determined by computing a weighted

24

average of the signal intensities from all touch pad sensors, which is referred to in the art as a

25

“centroid.” Because the signal intensity will be greatest directly below the point of finger contact

26

and will fall off sharply on either side of this point, the “centroid” will naturally correspond to the

27

point of finger contact. However, when a second finger contacts the touch pad, there will, of

28

course, be two points where the signal intensity peaks, each point corresponding to the contact of

DECLARATION OF RAVIN BALAKRISHNAN

10

Case No. C-09-01531 RS (PVT)

1

a finger. As a result, the “centroid” will immediately “jump” to a point that is approximately

2

midway between the two fingers. It was well known in the art that this jumping could be used to

3

determine the presence of two fingers. See, e.g. Exh. 6 [U.S. patent No. 5,495,077]. This

4

“centroid jumping” phenomenon is depicted below:

5
6
7
8
9
10
11
12
13

38.

In view of the above methods already known in the prior art, the alleged novelty of

14

the ’352 Patent is directed to the patent’s disclosure of a specific method that enables the use of

15

peak detection to identify multiple finger contacts in lieu of other methods like “centroid

16

jumping.”

17

39.

The method disclosed in the ’352 Patent relies on the analysis of “finger profiles”

18

that are obtained from a scan of sensors on a touch pad. See, e.g., Exh. 1 [’352 Patent] at 5:20-43

19

(describing the use of a capacitive touch pad with X direction and Y direction traces). As

20

discussed above, at the time of filing of the ’352 Patent, capacitive touch pads relied primarily on

21

a process called projection scanning in which a scan of touch sensor “traces” along an axis results

22

in a profile of the capacitances measured on that axis. See, e.g., id. at 5:56-65 (describing the

23

scanning of traces and measurement of capacitance on each trace in typical traces).

24
25

40.

The ’352 Patent discloses a representative finger profile based on the use of such

capacitive touch pads as set forth below:

26
27
28

DECLARATION OF RAVIN BALAKRISHNAN

11

Case No. C-09-01531 RS (PVT)

1
2
3
4
5
6
7
8

41.

Id. at Fig. 3; see also id. at Fig. 4. The finger profile shown above represents the

9

contact of two fingers with the touch pad surface; each peak shown above is determinative of a

10

single finger contact. See id. at 4:56-67 (“FIG. 3 shows a finger profile for two non-overlapping

11

fingers as sensed by the present invention.”).

12

42.

A finger profile is generated based on the capacitive values that are measured

13

along an axis. For example, in the case of two finger contacts to a touch pad with traces along x

14

and y axes, the touch pad would yield a set of capacitance values along the x axis and another set

15

along the y axis:

16
17
18
19
20
21
22
23
24
25

43.

An X PROFILE is generated from the set of capacitance values along the x axis:

26
27
28

DECLARATION OF RAVIN BALAKRISHNAN

12

Case No. C-09-01531 RS (PVT)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

44.

In the same way, a Y PROFILE is generated from the set of capacitance values

18

along the y axis:

19
20
21
22
23
24
25
26
27
28

DECLARATION OF RAVIN BALAKRISHNAN

13

Case No. C-09-01531 RS (PVT)

1
2
3
4
5
6
7
8
9
10

45.

Consistent with this, the ’352 Patent discloses a pair of profiles generated from the

11

contact of two fingers with a touch pad device that is capable of measuring finger contact in both

12

the X and Y directions:

13
14
15
16
17
18
19
20
21
22
23

46.

Id. at Fig. 7B. Notably, when both fingers are contacting the touch pad at the same

24

vertical position, the Y PROFILE includes only a single peak. In contrast, two peaks appear in

25

the X PROFILE because each finger is contacting the touch pad at a different horizontal position.

26

47.

One of ordinary skill in the art would recognize that the use of a finger profile

27

taken on a straight line is key to the alleged invention of the ’352 Patent. Indeed, the method of

28

the ’352 Patent involves sequentially stepping through the elements of the finger profile one-by-

DECLARATION OF RAVIN BALAKRISHNAN

14

Case No. C-09-01531 RS (PVT)

1

one to identify extrema in order to determine the presence of fingers. Briefly, the method works

2

by stepping through the elements of the finger profile looking only for a first maximum until it is

3

first determined that the values of the finger profile are no longer increasing and then designates

4

the location of that value. At that point, the algorithm has identified a first maximum, an event

5

that is determinative of a first finger contact. After the first maxima has been identified, the

6

algorithm starts looking only for a minimum and continues stepping through the elements of the

7

finger profile until the values of the finger profile are no longer decreasing and then designates

8

the location of that value. At this point, the first minima following the first maxima has been

9

identified. Then, the algorithm starts looking only for a second maximum and once again steps

10

through the elements of the finger profile until they are no longer increasing and designates its

11

location such that a “second maxima” following the first minima is identified. Similar to the

12

detection of the “first maxima,” which is determinative of a first finger contact, this identification

13

of the “second maxima” is determinative of a second finger contact. See, e.g., generally id. at

14

9:18-10:65; Claim 1. This method is depicted below:

15
16
17
18
19
20
21
22
23

48.

Having discussed the background of touch sensor technology and the ’352 Patent,

24

I will next provide an overview of the Apple Products accused in this case of infringement.

25

VI.

26

THE ACCUSED LEGACY PRODUCTS
49.

I understand that Elan’s Motion for Partial Summary Judgment dated May 5, 2011,

27

relates to models of Apple’s iBook, MacBook, MacBook Pro, and PowerBook laptop computers

28

released between 2005 and 2009. Each of these products incorporates what is referred to as a

DECLARATION OF RAVIN BALAKRISHNAN

15

Case No. C-09-01531 RS (PVT)

1

or

touchpad. For purposes of this declaration, I will refer to these products

2

as the “accused legacy products,” to distinguish them from the current Apple products accused of

3

infringement in the lawsuit.

4

50.

Apple’s

and

touchpads have the ability to recognize gestures

5

performed by a user on the touchpad with one or two fingers. In order to distinguish a contact of

6

one finger from a contact of two fingers, the touchpads utilize source code that interprets the

7

touch sensor data. In brief, the source code contains an algorithm that receives data from the

8

touch sensor,

9
10
11
12

Finally, the algorithm interprets gestures performed by a user based on the number of fingers,

13

movement of the fingers, and other information.

14

51.

I understand that the

or

products are not current Apple products

15

and were not involved in the recent ITC Investigation in which Apple was respondent. The ITC

16

Investigation involved Apple’s current products, which all use Apple’s

17

understand that the Chief Administrative Law Judge issued a final initial determination finding no

18

infringement of the ’352 Patent by Apple’s

19

VII.

20

algorithm. I

products.

LEGAL STANDARDS
52.

I am not a legal expert and offer no opinions of the law. However, I have been

21

informed by counsel of the various legal standards that apply, and I have applied those standards

22

in arriving at my conclusions.

23

53.

It is my understanding that the following legal standards apply in assessing

24

infringement. An analysis of infringement requires two steps. First, the language of the patent

25

claims must be properly construed. Next, to infringe a patent claim, the accused product must

26

include elements that correspond to each and every limitation in the claims as they have been

27

properly construed.

28

DECLARATION OF RAVIN BALAKRISHNAN

16

Case No. C-09-01531 RS (PVT)

1

54.

Here, I understand that the certain terms of the asserted claims have been

2

construed in the November 1, 2010 Claims Construction Order, and that those constructions are

3

the constructions to be used for purposes of infringement. I have used the November 1, 2010

4

claim constructions in my analyses.

5

55.

I understand that if an accused product contains elements that fall within the literal

6

meaning of the claims as properly construed, the accused product is said to infringe “literally.”

7

Thus, to infringe literally, the accused product must contain elements that fall literally within the

8

scope of each and every limitation of the claims.

9

56.

I understand that to infringe a means plus function claim, the accused product must

10

perform the identical function and contain identical or equivalent structure. I understand that two

11

structures are equivalent if a person of ordinary skill in the art would consider the differences

12

between them to be insubstantial for performing the required function. I also understand that one

13

way to determine this is to look at whether or not the accused structure performs the identical

14

function in substantially the same way to achieve substantially the same result. I understand that

15

structural equivalence is determined at the time of the issuance of the patent.

16

57.

I understand that every limitation of a claim is essential and that the absence of any

17

one limitation avoids infringement. It is also my understanding that, to infringe a dependent

18

claim, the accused product must include each and every limitation of all claims from which it

19

depends.

20

58.

I understand that to prove infringement, the patent owner must show specific

21

instances of direct infringement or that the accused device necessarily infringes the patent in suit.

22

VIII. CLAIM CONSTRUCTION

23

59.

I understand that the Court construed certain terms of the ’352 Patent in a Claims

24

Construction Order dated November 1, 2010 (“Claims Construction Order”), and made non-

25

substantive corrections to that order on January 25, 2011.

26

60.

I understand that the Court concluded that the claim language requires a temporal

27

order for the identification steps. Claims Construction Order at 9-10. Specifically, I understand

28

that the Court will instruct the jury that:

DECLARATION OF RAVIN BALAKRISHNAN

17

Case No. C-09-01531 RS (PVT)

1

(a) “identify a first maxima in a signal corresponding to a first
finger” means: “Identify a first peak value in a finger profile taken
on a line obtained from scanning the touch sensor;”

2
3

(b) “identify a minima following the first maxima” means: “Identify
the lowest value in the finger profile taken on said line that occurs
after the first peak value and before another peak value is
identified;” and

4
5

(c) “identify a second maxima in a signal corresponding to the
second finger following said minima” means: “After identifying the
lowest value in the finger profile taken on said line, identify a
second peak value in the finger profile taken on said line.”

6
7
8
9
10
11

61.

With respect to the term “identify,” I understand that the Court did not issue a

construction, but instead concluded that “there is no dispute between the parties that the term
‘identify’ includes the concept of recognition.” Claims Construction Order at 11.
62.

With respect to the term “in response to,” I understand that the Court did not issue

12

a construction, but concluded that “Apple has persuasively shown that the invention claimed in

13

the ’352 Patent utilizes the identification of a first and second maxima, without some amalgam of

14

additional information, to determine and indicate the simultaneous presence of two fingers.”

15

Claims Construction Order at 11.

16

63.

I next set forth the relevant terms from asserted means-plus-function claims 18, 21,

17

and 30. As explained below, Elan and Mr. Dezmelyk apply different and broader constructions to

18

certain terms here than were determined to be correct in the ITC proceeding. For example,

19

asserted claim 30 was found to be indefinite in the ITC. The Court here has not yet construed all

20

of the necessary limitations of claims 18, 21, and 30 or addressed the indefiniteness issue with

21

respect to claim 30. As such, my infringement analysis is based on the parties’ agreed

22

construction for one term, and my understanding of the correct constructions for the other terms

23

set forth below.

24

“Means for Scanning the Touch Sensor”

25

64.

I understand that the parties’ agreed construction for “means for scanning the

26

touch sensor” specifies a function of scanning the touch sensor as “the corresponding structure is

27

an analog multiplexer, a circuit to measure changes in capacitance of sensor conductors, an

28

analog to digital converter, a microcontroller, and equivalents thereof.”

DECLARATION OF RAVIN BALAKRISHNAN

18

Case No. C-09-01531 RS (PVT)

1

“Means for Scanning the Touch Sensor to Identify . . .”

2

65.

While I believe that the parties’ agreed construction for “means for scanning the

3

touch sensor” is not inaccurate, I note that the parties have not agreed upon and the Court has not

4

issued a claim construction for remainder of claim 18. While I have not previously been involved

5

in this district court proceeding, I was asked to opine on behalf of Apple during claim

6

construction proceedings in the ITC.

7

66.

In the ITC, the Chief ALJ construed the complete means-plus-function portion of

8

claim 18. Specifically, the Chief ALJ construed the term “means for scanning the touch sensor to

9

(a) identify a first maxima in a signal corresponding to a first finger, (b) identify a minima

10

following the first maxima, and (c) identify a second maxima in a signal corresponding to a

11

second finger following said minima.” Exh. 7 [ITC Claim Construction Order] at 34-38. The

12

Chief ALJ concluded that the recited function of this term is “scanning the touch sensor to (a)

13

identify a first maxima in a signal corresponding to a first finger, (b) identify a minima following

14

the first maxima, and (c) identify a second maxima in a signal corresponding to a second finger

15

following said minima.” Id. at 38. The Chief ALJ further concluded that the corresponding

16

structure “is an analog multiplexer, a circuit to measure changes in capacitance of sensor

17

conductors, an analog to digital converter, a microcontroller, and Fig. 5 (items, 400-440) and Fig.

18

6-1 or Fig. 9-1 (items 200-278).” Id. I believe that the Chief ALJ’s construction is correct for the

19

following reasons.

20

67.

One of ordinary skill in the art would understand based on the plain language of

21

the claim that the claimed function is not merely scanning the touch sensor, but “scanning the

22

touch sensor to (a) identify a first maxima in a signal corresponding to a first finger, (b) identify a

23

minima following the first maxima, and (c) identify a second maxima in a signal corresponding to

24

a second finger following said minima.” The corresponding structure must accomplish the

25

entirety of the claimed function. The parties’ agreed construction only relates to the hardware

26

structure involved in scanning the touch sensor.

27
28

68.

In the specification, the algorithm disclosed to accomplish the identification

function of claim 18 is Figure 5 (items 400-440) and Figure 6-1 (items 200-278). At item 400 of

DECLARATION OF RAVIN BALAKRISHNAN

19

Case No. C-09-01531 RS (PVT)

1

Figure 5, the process for scanning the touch sensor to identify the claimed extrema begins. Exh. 1

2

[’352 Patent] at 7:34-36 (“Referring still to FIG. 5, the cyclical process begins at step 400, and

3

contnues at step 410 by scanning the conductor sensors.”). At item 410, the touch sensor scans

4

the conductors and stores their values in memory. Id. at Fig. 5 (“Scan Conductors: Store in

5

Ram”); id. at 7:38-46 (“The scan process measures the values of finger-induced capacitance for

6

each of the conductors, and stores the values in RAM at step 420.”). At items 430 and 440, the

7

Xcompute and Ycompute processes shown in Figure 6 identify the required first maxima,

8

minimum, and second maxima in the order recited in the claims. Id. at 7:40-43 (“The cycle

9

process continues by performing the Xcompute loop of FIG. 6 discussed hereinafter, and also the

10

Ycompute loop analogous to FIG. 6, at step 430 and 440, respectively.”). The specification

11

makes clear that the function of the Xcompute and Ycompute processes is to “evaluate the current

12

measurements . . . by detecting whether a second finger is touching the pad—which determines

13

the button state.” Id. at 7:43-48; see also id. at 8:46-11:15 (explaining the operation of the

14

Xcompute and Ycompute loops, and stating that their function is to “identif[y] [] minima and

15

maxima in the X and Y directions.”). Indeed, the specification states that the Xcompute process

16

includes variables such as “Xpeak1” which “store[s] the value of the first peak X value.” Id. at

17

8:64. Thus, the specification makes clear that the corresponding algorithmic structure that

18

accomplishes the scanning and identification function of claim 18 is Figure 5 (items 400-440) and

19

Figure 6-1 (items 200-278).

20

69.

Consistent with the above, in the ITC Claim Construction order, the Chief ALJ

21

determined that claim 18 “recites two separate means: a means for scanning the touch sensor to

22

identify the claimed maxima and minima, and a means for providing an indication of the

23

simultaneous presence of two fingers in response to said identification.” Exh. 7 [ITC Claim

24

Construction Order No. 17] at 35. The ITC judge further concluded that flow diagrams in Figure

25

5 and Figure 6 in the patent recite the structure for identifying extrema in the values obtained

26

from scanning the touch sensor. Specifically, the Xcompute and Ycompute processes, executed

27

at steps 430 and 440, detect whether a second finger is touching the pad. Id. at 37. Thus, the

28

corresponding structure for the “means for scanning to identify . . .” limitation includes the

DECLARATION OF RAVIN BALAKRISHNAN

20

Case No. C-09-01531 RS (PVT)

1

hardware elements agreed-upon by the parties, as well as the corresponding algorithmic structure

2

of Figure 5 (items 400-440) and Figure 6-1 (items 200-278).

3

“Means for Providing an Indication in Response to . . .”

4

70.

I further note that the parties have not agreed upon and the Court has not issued a

5

claim construction for the limitation “means for providing an indication of the simultaneous

6

presence of two fingers in response to the identification of said first and second maxima” in claim

7

18. In the ITC, the Chief ALJ concluded that the recited function is “providing an indication of

8

the simultaneous presence of two fingers.” Exh 7. [ITC Claim Construction Order] at 39-45. The

9

Chief ALJ further concluded that the corresponding structure “is step 860 of Figure 8 where the

10

value of the ‘Finger’ variable is set to equal two.” Id. at 43. I believe that the Chief ALJ’s

11

construction is the correct construction of the means for providing an indication limitation of

12

claim 18.

13

71.

14

“[M]eans for providing an indication” first appears in claim 18:
A touch sensor for detecting the operative coupling of multiple
fingers comprising:

15

means for scanning the touch sensor to (a) identify a first maxima
in a signal corresponding to a first finger, (b) identify a minima
following the first maxima, and (c) identify a second maxima in a
signal corresponding to a second finger following said minima, and

16
17
18
19

means for providing an indication of the simultaneous presence
of two fingers in response to identification of said first and second
maxima.

20

’352 Patent (Exh. 1) at Claim 18 (emphasis added).

21

72.

I understand that the parties agree that this is a means-plus-function term. I further

22

understand that the parties agree that the claimed function is “providing an indication of the

23

simultaneous presence of two fingers in response to identification of said first and second

24

maxima.”

25
26
27
28

73.

As to the corresponding structure, as set forth below, in the algorithm of Fig. 8-1, a

variable is set to indicate the number of fingers contacting the touch sensor:
At step 850, a determination is made whether two fingers are in
contact with the touchpad by evaluating both Xcompute and
Ycompute. If neither Xcompute nor Ycompute indicate the
DECLARATION OF RAVIN BALAKRISHNAN

21

Case No. C-09-01531 RS (PVT)

1

presence of two fingers, the answer is NO and the process drops to
step 855. However, if either the Xcompute routine or the Ycompute
routine indicates the presence of two fingers [i.e., identified a first
maxima, minima, and a maxima], the answer at step 850 is YES
and the process moves to step 860, where the value of the variable
FINGER is set to 2.

2
3
4

Id. at 14:8-17.
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

74.

This conclusion (i.e., that there are two fingers) is thereafter used by the system to

perform gestures. See id. at 14:28-15:11 (describing button actuation based on the conclusion
concerning the number of fingers). For example, after the number of fingers is determined, if the
value of the ButtonPrevious variable is down, and zero fingers are determined to be currently on
the touch sensor, then “the process moves to step 925 and reports to the host that Button is Up,
while also setting the variable ButtonPrevious to Up.” Id. at 14:56-65 & Fig. 8-2. In other words,
if the button was previously down but no fingers are currently detected, the algorithm recognizes
the button as being up. Similarly, if the button was previously up and two fingers are currently
detected, the algorithm recognizes the button as being down. Id. at 14:48-55 & Fig. 8-2.
75.

Accordingly, by setting the variable to 2, the algorithm provides an indication to

other software or software modules capable of accessing the FINGER variable that two fingers
are simultaneously present. Indeed, in January 1996, setting a variable to a value was one of
many well-known techniques for providing an indication in the context of computer
programming. Thus, one of ordinary skill in the art would recognize that the algorithm set forth
in Fig. 8-1 is the corresponding structure for this claim element.
76.

Mr. Dezmelyk opines that a preliminary step, the Xcompute algorithm, provides

an indication of the simultaneous presence of two fingers. See Dezmelyk Decl. ¶ 42 (citing step
310 in Fig. 6-2 and step 980 in Fig. 9-2). In the ITC, Elan similarly argued that a preliminary
step, step 980 of Figure 9-2, which sets “Xfinger = 2” within the “Xcompute” loop, provides an
indication of the simultaneous presence of two fingers. Chief ALJ Luckern rejected Elan’s
argument, finding that “if step 980 within the Xcompute loop of Figure 9 were sufficient alone to
determine the simultaneous presence of two fingers, then the algorithm would skip the Ycompute
loop as well as steps 850-870 where a determination of the number of fingers is made.” Exh. 7

28

DECLARATION OF RAVIN BALAKRISHNAN

22

Case No. C-09-01531 RS (PVT)

1

[ITC Claim Construction Order No. 17] at 43. Chief ALJ Luckern also cited testimony by Elan’s

2

expert Mr. Dezmelyk to the effect that the number of fingers is determined in steps 850, 855, 870,

3

860 or 865 of Figure 8-1, in concluding that “because the indication of the simultaneous presence

4

of two fingers requires analyzing both profiles, step 980 of Figure 9-2 does not accomplish the

5

claimed function.” Id. at 43. In other words, Chief ALJ Luckern concluded that a preliminary

6

step of setting a variable equal to two fingers for the X profile does not provide an indication of

7

the simultaneous presence of two fingers. I agreed with Chief ALJ Luckern’s reasoning and

8

conclusions for several reasons.

9

77.

The preliminary results of either the Xcompute or Ycompute algorithm alone does

10

not determine the final number of fingers. As shown in Figure 8-1 of the patent, the final number

11

of fingers is based on the consideration of both Xcompute and Ycompute, not one or the other in

12

isolation. For example, if one finger is detected in the X direction, but two fingers are detected in

13

the Y direction by the algorithm in the patent, then two fingers are reported, and that number is

14

used to determine the button state in the remainder of the algorithm. See Exh. 1 [’352 Patent] at

15

Fig. 8-1 & 8-2. Likewise, the conclusion of the algorithm shown in Figure 9 of the patent merely

16

sets the value of the Xfinger variable, which is then used in Figure 8-1 along with the value of the

17

Yfinger variable to determine the number of fingers.

18

78.

Moreover, the patent describes performing gestures based on the results of the

19

number of fingers detected, but this is not done with the preliminary result of Xcompute or

20

Ycompute alone. See id. at 2:38-41 (“Operation of the present invention includes two aspects:

21

detection of multiple objects, typically fingers, and assignment of various functions to particular

22

actions by the movement of one or both fingers.”). Setting Xbutton to Down in step 310 of

23

Figure 6-2 only sets the Xbutton variable. The state of the Xbutton variable is one of four

24

variables used to make a final determination as to whether the “Button” is up or down. See id. at

25

7:34-65 (showing that step 465 of Figure 5 depends on four variables, Xbutton, Ybutton,

26

XbuttonPrevious, and YbuttonPrevious). For example, if Xbutton is Up, the Button variable can

27

still be set to Down if Ybutton is down and YbuttonPrevious is Up. Therefore, I disagree with

28

Mr. Dezmelyk’s opinion that the preliminary steps of step 310 in Fig. 6-2, or step 980 in Fig. 9-2

DECLARATION OF RAVIN BALAKRISHNAN

23

Case No. C-09-01531 RS (PVT)

1

disclose the corresponding structure for providing an indication of the simultaneous presence of

2

two fingers for claim 18.

3

79.

I further disagree with Mr. Dezmelyk’s opinion that setting a button equal to down

4

in step 465 of Figure 5 of the patent provides an indication of the simultaneous presences of two

5

fingers. In general, one of ordinary skill in the art would not understand a button being down as

6

indicating the presence of two fingers. Indeed, the fact that BUTTON is DOWN in step 465 of

7

Figure 5 does not necessarily mean that two fingers are present on the touch sensor. That is

8

because the indication of button state in Figure 5 is in response to not just the presence of two

9

maxima, but the absence of two maxima in a previous scan. For example, if XbuttonPrevious and

10

YbuttonPrevious are Down, then BUTTON will not be set to DOWN in step 465 regardless of

11

how many fingers are on the touch sensor. See ’352 Patent at 7:57-65 (“If both compares are

12

true, then the variable Button is set to Down at step 465.”) & Fig. 5. In short, the indication of

13

button state in step 465 of Figure 5 is just that—an indication of button state. It is neither an

14

indication of the simultaneous presence of two fingers nor an in response to the presence of two

15

maxima.

16

80.

The claim language requires that the indication must be “in response to” the

17

identification of two maxima. One of ordinary skill in the art would not understand this as

18

allowing for the involvement of several additional factors beyond than the presence of two

19

maxima in providing the indication of two fingers. See Exh. 8 [Balakrishnan ITC Claim

20

Construction Decl.] ¶¶ 96-103. As the Court stated, “the invention claimed in the ‘352 patent

21

utilizes the identification of a first and second maxima, without some amalgam of additional

22

information, to determine and indicate the simultaneous presence of two fingers.” Claims

23

Construction Order at 11. Accordingly, I do not agree with Mr. Dezmelyk that Figure 9, which

24

includes 4 separate tests in block 305 that must be passed before setting Xfinger = 2, is the

25

corresponding structure for this claim element. To the extent Mr. Dezmelyk contends that Figure

26

8 requires the use of Figure 9, I disagree. See Dezmelyk Decl. ¶ 85. While Figure 8 represents

27

the general case, Figure 9 represents a specific manner of determining the presence of two

28

DECLARATION OF RAVIN BALAKRISHNAN

24

Case No. C-09-01531 RS (PVT)

1

maxima that includes additional steps. It is not required that this specific method be used in

2

conjunction with Figure 8.

3

81.

Accordingly, I cannot agree with Mr. Dezmelyk that step 465 in Fig. 5, step 310 in

4

Fig. 6-2, or step 980 in Fig. 9-2 is the structure corresponding to an indication of the simultaneous

5

presence of two fingers. Instead, one of ordinary skill in the art would recognize that the

6

corresponding structure is the algorithm of Figure 8-1 of the ’352 Patent wherein a variable is set

7

to indicate the number of fingers contacting the touch sensor.

8
9

82.

In the ITC, Elan and Mr. Dezmelyk took the position that the corresponding

structure is “firmware or software that provides data indicating the presence of two fingers in

10

response to the identification of two maxima and equivalents thereof.” I disagree with this view.

11

In my opinion, Elan’s proposed corresponding structure is vague and ambiguous and does not

12

provide a link to any structure in the specification. Indeed, Elan’s proposed structure makes no

13

reference to anything in the specification and instead just recites the claimed function verbatim as

14

something that can be done by generic firmware or software. Thus, under Elan’s proposed

15

constructions, the claims apparently cover every conceivable method of carrying out the claimed

16

function. This view, however, does not provide any guidance to one of ordinary skill in the art

17

regarding the scope of the claims. Rather, it leaves them boundless. As a result, under Elan’s

18

proposed construction, one of ordinary skill in the art would be unable to determine the scope of

19

the claims, a situation that one of ordinary skill in the art would find unacceptable.

20

83.

In the ITC, Elan and Mr. Dezmelyk also identified portions of the specification as

21

supporting its proposed corresponding structure, including Figures 5 (steps 465, 540), 6-1, 6-2

22

(step 310), 8-1 (steps 860, 915), 9-1 and 9-2 (steps 980) and 7:1-6, 7:49-8:15, 14:3-55 and 15:12-

23

31. Although neither Elan nor Mr. Dezmelyk explained how these portions of the specification

24

support its proposed corresponding structure, I have reviewed these passages and disagree that

25

they provide any support for the generic corresponding structure Elan proposed. A person of

26

ordinary skill in the art would not understand the passages Elan and Mr. Dezmelyk identified as

27

providing corresponding structure for the function of “providing an indication of the simultaneous

28

DECLARATION OF RAVIN BALAKRISHNAN

25

Case No. C-09-01531 RS (PVT)

1

presence of two fingers in response to identification of said first and second maxima,” but instead,

2

as relating to other functions such as computing motion and setting “button” states.

3

Claim 30 Is Indefinite

4

84.

It is my opinion that claim 30 of the ’352 Patent is indefinite.1 One of ordinary

5

skill in the art in January 1996 would not have been able to identify the corresponding structure of

6

“means for calculating first and second centroids corresponding to said first and second fingers”

7

in the specification of the ’352 Patent because the specification fails to disclose any

8

corresponding structure for performing this function.

9

85.

“[M]eans for calculating first and second centroids corresponding to said first and

10

second fingers” first appears in claim 30: The sensor of claim 18 further comprising means for

11

calculating first and second centroids corresponding to said first and second fingers. Exh.1

12

[’352 Patent] at Claim 30 (emphasis added).

13

86.

I understand that the parties agree that this is a means-plus-function term. I further

14

understand that the parties agree that the claimed function is “calculating first and second

15

centroids corresponding to said first and second fingers.”

16

87.

One of ordinary skill in the art would recognize that although the specification

17

discloses an algorithm to calculate a single centroid and recognizes the prior art problem

18

associated with attempting to calculate two centroids simultaneously (a separate centroid for each

19

of two fingers contacting the touch sensor), the specification fails to disclose an algorithm to

20

calculate both centroids.

21

88.

22

For instance, the specification explains as follows:
In an exemplary embodiment, the Xcompute process then continues
by calculating the centroid for the fingers detected, so long as the
maxima exceed a threshold value. In accordance with the present
invention, two approaches may be used in calculating centroid
values. In a first implementation, only a single centroid value is
calculated for the combination of one or more fingers. In this
arrangement, it will be apparent that, when a second finger contacts

23
24
25
26
1

27
28

I note that I previously opined on and provided live testimony as to the indefiniteness of
claims 19, 24, and 30 of the ’352 Patent in the ITC Investigation. Subsequently, the Chief ALJ
found those claims indefinite in an order dated September 28, 2010, and the ITC declined to
review that order.
DECLARATION OF RAVIN BALAKRISHNAN

26

Case No. C-09-01531 RS (PVT)

1

the touchpad, the centroid ‘jumps’ laterally approximately to the
midpoint of the two fingers. In a second implementation, a
centroid value may be calculated for each maxima, yielding
multiple centroid values when multiple fingers interact with the
pad. For purposes of clarity, the following description will be
limited to the first implementation.

2
3
4

Id. at 10:31-45 (emphasis added). Thus, according to the bolded
text above, the specification specifically omits the description of
detecting both centroids.

5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

89.

In my opinion, this omission is critical because one of ordinary skill in the art in

January 1996 would have understood that detecting two centroids does not involve simply
applying the same algorithm used for detecting a single centroid twice. One reason for this was
the well-known “segmentation” problem. Briefly, in certain situations where two fingers contact
the touch pad, it is difficult to determine whether a capacitance reading should be grouped as part
of the first finger’s contact area (first centroid) or the second finger’s contact area (second
centroid). Thus, an algorithm is required to “segment” the two contact areas. My review of the
specification reveals that it fails to disclose any algorithm that would accomplish this or any of
the other tasks attendant to calculating two separate centroids. As a result, one of ordinary skill in
the art would be unable to determine the scope of the claims.
90.

Elan contends that the corresponding structure is “analog multiplexer 45:

Capacitance measuring circuit 70: A to D convertor 80, Microcontroller 60 and/or software,
firmware or hardware performing the claimed function.” First Amended Joint Claim
Construction Statement, Exh. A at 11-12. I disagree with this view. In my opinion, Elan’s
proposed corresponding structure is vague and ambiguous and does not provide a link to any
structure in the specification. Indeed, Elan’s proposed structure does nothing more than recite
hardware elements related to the scanning function and further recite generic “software, firmware,
or hardware” that has no specific relation to the calculation of centroids. Thus, under Elan’s
proposed constructions, the claims apparently cover every conceivable method of carrying out the
claimed function. This view, however, does not provide any guidance to one of ordinary skill in
the art regarding the scope of the claims. Rather, it leaves them boundless. Accordingly, one of
ordinary skill in the art would find Elan’s proposed construction unacceptable.

28

DECLARATION OF RAVIN BALAKRISHNAN

27

Case No. C-09-01531 RS (PVT)

1

91.

In the ITC Investigation, Elan and Mr. Dezmelyk identified portions of the

2

specification as supporting its proposed corresponding structure for claim 30, including Figs 6

3

and 9 and 10:31-51. Although neither Elan nor Mr. Dezmelyk explained how these portions of

4

the specification support its proposed corresponding structure, I have reviewed these passages and

5

disagree that that they provide any support for the generic corresponding structure Elan has

6

proposed. In fact, even if Elan were proposing that these passages were themselves

7

corresponding structure for the “means for calculating first and second centroids” limitation, I

8

would disagree. A person of ordinary skill in the art would not understand the passages Elan

9

identifies as providing corresponding structure for the function of “calculating first and second

10

centroids corresponding to said first and second fingers.” To the contrary, the patent specifically

11

describes an implementation in which one centroid value is calculated while omitting a

12

corresponding description for implementations in which more than one centroid value is

13

calculated. Id. at 10:31-51 (describing two implementations and noting that “[f]or purposes of

14

clarity, the following description will be limited to the first implementation”). In sum, it is my

15

opinion that claim 30 of the ’352 Patent is indefinite.

16
17
18
19
20

92.
IX.

I apply the foregoing claim constructions in my analysis below.

THE ACCUSED LEGACY PRODUCTS DO NOT INFRINGE ASSERTED
METHOD CLAIMS 1, 2, 6, 7, OR 16
A.

The Accused Legacy Products Do Not Identify Maxima Or Minima Values
Obtained From Scanning The Touch Sensor

93.

I understand that the claims require “identif[ication] of a first peak value in a

21

finger profile obtained from scanning the touch sensor,” “identif[ication] of the lowest value in

22

the finger profile taken on said line that occurs after the first peak value and before another peak

23

value is identified,” and “after identifying the lowest value in the finger profile taken on said line,

24

identif[ication] of a second peak value in the finger profile taken on said line.” Claims

25

Construction Order at 10. For the following reasons, it is my opinion that the accused legacy

26

products do not identify maxima or minima values in a finger profile obtained from scanning the

27

touch sensor.

28

DECLARATION OF RAVIN BALAKRISHNAN

28

Case No. C-09-01531 RS (PVT)

1

94.

Elan and Mr. Dezmelyk take the position that the “obtained from scanning the

2

touch sensor” limitation is met because the “[t]he [a]ccused Apple [p]roducts ‘scan the touch

3

sensor.’” Elan Motion at 10; Dezmelyk Decl. ¶ 23. In my opinion, this position improperly

4

ignores the word “obtained.” I understand that the complete claim limitation requires that the

5

maxima and minima values of the identification steps of the claims must be values obtained from

6

scanning the touch sensor, and not simply that the method scan the touch sensor and then identify

7

maxima and minima in values other than those obtained from scanning the touch sensor.

8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

DECLARATION OF RAVIN BALAKRISHNAN

29

Case No. C-09-01531 RS (PVT)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

DECLARATION OF RAVIN BALAKRISHNAN

30

Case No. C-09-01531 RS (PVT)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

DECLARATION OF RAVIN BALAKRISHNAN

31

Case No. C-09-01531 RS (PVT)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

DECLARATION OF RAVIN BALAKRISHNAN

32

Case No. C-09-01531 RS (PVT)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

109.

Therefore, because any maxima or minima values identified by the accused legacy

18

products are significantly different than the values obtained from scanning the touch sensor, it is

19

my opinion that the accused legacy products do not identify maxima or minima values in a finger

20

profile “obtained from scanning the touch sensor.”

21

B.

The Accused Legacy Products Do Not Identify Extrema In The Requisite
Temporal Order Because The Algorithm Does Not Search For A First
Maximum, A Minimum, And A Second Maximum In That Order

110.

In the context of the patent, identification of extrema in the requisite temporal

22
23
24

order is accomplished by searching for a first maximum, then searching for a minimum, and

25

finally searching for a second maximum. This requirement is reflected in the claim language that

26

requires “scanning the touch sensor to identify” a first maximum, minimum, and second

27

maximum in the requisite order.

28

DECLARATION OF RAVIN BALAKRISHNAN

33

Case No. C-09-01531 RS (PVT)

1

111.

This is consistent with the patent specification, which discloses a state variable

2

named “Xstate” is used “to indicate which part of the finger profile we are currently searching

3

for.” Exh. 1 [’352 Patent] at 9:10-14. The “Xstate” variable “can have values Peak1, Valley,

4

Peak2 or Tail,” which correspond to the first maximum, minimum, and second maximum of the

5

claims, as well as “the remainder of the scan after a second peak (in the exemplary embodiment)

6

has been identified.” Id. Unless the Xstate variable in Figure 9-1 is set to Valley, the algorithm

7

will not identify a minimum. Id. at Fig. 6-1, 9-1. In other words, the ’352 Patent discloses a

8

method that methodically looks for a first maximum. After identifying the first maximum, it only

9

then looks for a minimum. Similarly, after identifying a minimum, it only then looks for a second

10
11

maximum.
112.

I understand that using a state variable in the patent relates to one embodiment and

12

that there might be a number of ways in which the method could search for the extrema in the

13

requisite order. However, it is my understanding of the claims that the search must be done in the

14

requisite order in some manner pursuant to the claim language that requires “scanning the touch

15

sensor to identify” a first maximum, minimum, and second maximum in the requisite order.

16
17
18
19
20
21
22
23
24
25
26
27
28

DECLARATION OF RAVIN BALAKRISHNAN

34

Case No. C-09-01531 RS (PVT)

1
2
3
4
5
6
7
8
9

115.

Thus, it is my opinion that the accused legacy products do not infringe because

10

they do not scan the touch sensor to identify a first maximum, then a minimum following that

11

first maximum, and then a second maximum following that minimum.

12
13
14
15

C.

The Accused Legacy Products Do Not Provide An Indication Of The
Simultaneous Presence Of Two Fingers In Response To The Identification Of
Said First And Second Maxima

116.

I understand that the Court did not adopt a construction for the term “in response

16
17
18

to.” However, I understand that the Court stated that “Apple has persuasively shown that the

19

invention claimed in the ’352 Patent utilizes the identification of a first and second maxima,

20

without some amalgam of additional information, to determine and indicate the simultaneous

21

presence of two fingers.” Claims Construction Order at 11.

22
23
24
25
26
27
28

DECLARATION OF RAVIN BALAKRISHNAN

35

Case No. C-09-01531 RS (PVT)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

DECLARATION OF RAVIN BALAKRISHNAN

36

Case No. C-09-01531 RS (PVT)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

DECLARATION OF RAVIN BALAKRISHNAN

37

Case No. C-09-01531 RS (PVT)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

DECLARATION OF RAVIN BALAKRISHNAN

38

Case No. C-09-01531 RS (PVT)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

DECLARATION OF RAVIN BALAKRISHNAN

39

Case No. C-09-01531 RS (PVT)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

DECLARATION OF RAVIN BALAKRISHNAN

40

Case No. C-09-01531 RS (PVT)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

DECLARATION OF RAVIN BALAKRISHNAN

41

Case No. C-09-01531 RS (PVT)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

DECLARATION OF RAVIN BALAKRISHNAN

42

Case No. C-09-01531 RS (PVT)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

143.

In sum, it is my opinion that the accused legacy products do not provide an

21

indication of the simultaneous presence of two fingers in response to the detection of said first

22

and second maxima.

23
24
25
26

D.

Evidence Of Direct Infringement

27
28

DECLARATION OF RAVIN BALAKRISHNAN

43

Case No. C-09-01531 RS (PVT)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

DECLARATION OF RAVIN BALAKRISHNAN

44

Case No. C-09-01531 RS (PVT)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

X.

THE ACCUSED LEGACY PRODUCTS DO NOT INFRINGE ASSERTED
MEANS-PLUS-FUNCTION CLAIMS 18, 21, OR 30
A.

The Accused Legacy Products Do Not Perform The Identical Function

148.

It is my opinion that the accused legacy products do not infringe claim 18 for

20

several reasons. First, the identical function is not performed for the “means for scanning the

21

touch sensor to identify” limitation. Claim 18 recites the same function as claim 1. As explained

22

above, the accused legacy products do not identify maxima or minima values obtained from

23

scanning the touch sensor.

24
25
26

See Section IX.A.
149.

Second, as explained above, the accused legacy products do not perform the

27

identical function for the “means for scanning the touch sensor to identify” limitation because

28

they do not scan the touch sensor to identify a first maximum, then a minimum following that

DECLARATION OF RAVIN BALAKRISHNAN

45

Case No. C-09-01531 RS (PVT)

1

first maximum, and then a second maximum following that minimum.

2
3
4
5
6

See Section IX.B.
150.

Third, the accused legacy products do not perform the identical function for the

“means for providing an indication” limitation of claim 18.

7
8
9
10
11
12
13

Id.

14

B.

The Accused Legacy Products Do Not Contain Corresponding Structure

15

151.

Besides not performing the identical function, the accused legacy products do not

16

include the same or equivalent structure for the “means for scanning the touch sensor to identify

17

[the claimed extrema]” limitation. In his declaration, Mr. Dezmelyk only addresses part of this

18

limitation and never offers an opinion as to (a) the structure disclosed in the patent for performing

19

the full limitation; or (b) whether the accused legacy products contain that structure.

20

152.

As explained above, in my view, the Chief ALJ in the ITC correctly construed the

21

limitation of “means for scanning the touch sensor to identify [the claimed extrema].” Applying

22

that construction, Mr. Dezmelyk has offered no opinion that the accused Apple Products perform

23

the algorithms identified by the Chief ALJ as corresponding structure or equivalents thereto. Mr.

24

Dezmelyk fails to address whether the accused legacy products include blocks 400-440 of Fig. 5

25

or equivalents. Mr. Dezmelyk also fails to address whether the accused legacy products include

26

blocks 205, 210, 215, 220, and 225 of Fig. 6-1 or Fig. 9-1 or equivalents. The accused legacy

27

products do not include the required corresponding structures for several reasons.

28

DECLARATION OF RAVIN BALAKRISHNAN

46

Case No. C-09-01531 RS (PVT)

1
2

153.

With respect to Fig. 5, the accused Apple products include neither a Y compute

nor an X compute, much less both.

3
4
5
6

The
X compute and Y compute loops in the patent lack any such structure.
154.

With respect to the algorithm of Figs. 6-1 and 9-1 (items 200-278), the algorithm

7

begins by setting the variable Xstate = Peak1 at block 205. The algorithm then proceeds to Xloop

8

Start, block 210, and computes the centroid of the finger profile taken on the X axis. See Exh. 1

9

[’352 Patent] at 9:24-38 (“At step 210, a loop referred to as ‘Xloop’ starts. The purpose of Xloop

10

is to calculate the X centroid, by accumulating the sum and weighted sum of the X values for all

11

the X conductors from one to Xcon. . . .”).

12
13

155.

After computing the centroid of the X axis, the corresponding structure in the

14

patent determines which extrema to seek by testing the Xstate variable and then proceeds to

15

iterate across the X axis (i.e., the finger profile taken on a straight line) to locate the sought after

16

extrema. Peak 1 is identified on the X axis and then the algorithm specifically searches for

17

Xvalley. Following the identification of Xvalley, the algorithm specifically searches for Xpeak2.

18

Once Xpeak2 is found, the algorithm sets the Xstate variable to Tail, which effectively terminates

19

the search for the extrema.

20
21

See Section IX.B.
156.

With respect to the last step of the algorithms disclosed in Figs. 6-1 and 9-1 is

22

block 278, which stops the search for another peak after the second peak is identified. The

23

accused Apple Products include no such step.

24
25

Mr. Dezmelyk provides no evidence that the

26

accused Apple products conclude after a second maximum is detected as required by Figs. 6-1

27

and 9-1, nor has Mr. Dezmelyk offered any opinion that the accused Apple Products include any

28

equivalent algorithmic steps.

DECLARATION OF RAVIN BALAKRISHNAN

47

Case No. C-09-01531 RS (PVT)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

158.

In sum, it is my opinion that the accused legacy products do not infringe claims 18,

20

21, or 30 because the accused legacy products do not perform the identical function and the

21

accused legacy products do not contain corresponding structure.

22
23
24
25
26
27
28

DECLARATION OF RAVIN BALAKRISHNAN

48

Case No. C-09-01531 RS (PVT)

1

Executed on June 2, 2011 in Toronto, Canada

2

I declare under penalty of perjury under the laws of he United States of America that the

3

foregoing is true and correct.

4
RAVIN BALAKRISHNAN

5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

DECLARATION OF RAVIN BALAKRISHNAN

49

Case No. C-09-01531 RS (PVT)