Apple Inc. v. Samsung Electronics Co. Ltd. et al

Filing 1013

Administrative Motion to File Under Seal Documents Under Seal Re Apples Opposition To Samsungs Motion For Summary Judgment filed by Apple Inc.. (Attachments: #1 Proposed Order, #2 Declaration Of Karl Kramer In Support Of Apples Opposition To Samsungs Motion For Summary Judgment, #3 Exhibit Kramer Decl. Ex. 1, #4 Exhibit Kramer Decl. Ex. 2, #5 Exhibit Kramer Decl. Ex. 3, #6 Exhibit Kramer Decl. Ex. 4, #7 Exhibit Kramer Decl. Ex. 5, #8 Exhibit Kramer Decl. Ex. 6, #9 Exhibit Kramer Decl. Ex. 7, #10 Exhibit Kramer Decl. Ex. 8, #11 Exhibit Kramer Decl. Ex. 9, #12 Exhibit Kramer Decl. Ex. 10, #13 Exhibit Kramer Decl. Ex. 11, #14 Declaration Of Michel Maharbiz, Ph.D. In Support Of Apples Opposition To Samsungs Motion For Summary Judgment, #15 Exhibit Maharbiz Decl. Ex. A, #16 Exhibit Maharbiz Decl. Ex. B, #17 Exhibit Maharbiz Decl. Ex. E, #18 Exhibit Maharbiz Decl. Ex. F)(Jacobs, Michael) (Filed on 6/1/2012)

Exhibit B

TO ALL TO WIIOR TII ENE I>It EN ENTS SHALII CO3I E

UNITED STATES DEPAR w NT OF COMws ·<CE
United States Patent and Trademark Office
June 22, 2011
· mm IS TO CER = w THAT ^=EXED HERETO IS A TRUE COPY FROM
THE RECORDS OF I nla OFFICE OF:

U.S. PATENT: 7,663,607
ISSUE DATE: February 16, 2010

By Authority of the
Under Secretary of Commerce for In llectual Property
and Director of the United State ate and Trademark orace

P. SWA
Certifying Officer

APLNDC00022435

lillllllllllllllllllllllllllllllllllllllllllllllllllllllllllli
(12) United States Patent

(10) Patent No.:

Hotelling et al.

(45) Date of Patent:

US 7,663,607 B2
Feb. 16, 2010

(54) MULTIPOINT TOUCH- - - N

4,734,685 A
4,746,770 A

3/1988 Watanabe ................... 340/7 0
5/1988 McAvinney .................. 178| 8

(75)

4,771,276 A
4,788,384 A
4,806,846 A

9/1988 Parks ......................... 178/ 8
11/1988 Bruere-Dawson et al. .....340/7 2
2/1989 Kerber .................. 324/60 CD

4,898,555 A
4,914,624 A
4,968,877 A

2/1990 Dunthorn ..................... 445/22
4/1990 Sampson et al.
11/ 990 McAvinney et al. ........ 250/221

5,003,519 A
5,017,030 A
5,189,403 A

3/
5/
1/
2/

5,194,862 A
5,224,861 A

3/ 993 Edwardsal. ................... 434/35
7/ 993 Glass et ...................... 341/20

5,241,308 A

8/ 993 Young ......................... 341/34

Inventors: Steve Hotelling, San Jose, CA (US);
Joshua A. Strickon, San Jose, CA (US);
Brian Q. Huppi, San F
'
CA
(US)

(73) Assignee: Apple Inc., Cupertino, CA (US)

( * ) Notice:

Subject to any disclaimer, the term of this
patent is extended or adjusted under 35
U.S.C. 154(b) by 754 days.

5,178,477 A

(21) Appl. No.: 10/840,862
(22) Filed:
(65)

FOREIGN P

Prior Publication Data
US 2006/0097991 Al
May 11, 2006

CA

- · DOCUMENTS

1243096

10/1988

................. 340/180

(Continued)
OTHER PUBLICATIONS

(2006.01)

(52) U.S. Cl. ..................... 345/173; 345/174; 178/18.01
(58) Field of Classification Search ......... 345/173-179;
178/18.01-18.04
See application file for complete search history.
(56)

Noirjean ...................... 368/73
Crews ........................ 400/485
Garnbaro
Franz et al..................... 400/489
................. 340/711

(Continued)

May 6, 2004

(51) Int. Cl.
GOdF 3/041

991
991
993
993

References Cited

U.S. Appl. No. 10/654,108, filed Sep. 2, 2003.

(Continued)
Primary Examiner-Richard Hjerpe
Assistant Examiner-Kimnhung Nguyen

(74) Attorney, Agent, or Firm-Morrison & Foerster LLP

U.S. PATENT DOCUMENTS

(57)

ABS· -= ·

3,333,160 A
3,541,541 A
3,662,105 A

7/1967 Gorski
11/1970 Englebart
5/1972 Hurst et al. ................... 178/18

A touch panel having a transparent capacitive sensing

3,798,370 A
4,246,452 A

3/ 974 Hurst .......................... 17W18
1/ 981 Chandler ....................... 200/5

medium configuredto detectmultiple touches ornear touches
that occur at the same time and at distinct locations in the

6/ 987 Beckes ...................... 340/365 P
6/1987 Lucas et al. ............... 364/518

resentative of the location of the touches on the plane of the
touch panel for each of the multiple touches is disclosed.

4,550,221 A

10/ 985 Mabusth ...................... 178/18

4,692,809 A
4,695,827 A

9/1987 Beining et al. .............. 358/247
9/1987 Beining et al. ........... 34W365 P

4,733,222 A

plane of the touch panel and to produce distinct signals rep-

3/1988 Evans ..................... 340/365 C

4,672,364 A
4,672,558 A

11 Claims, 14 Drawing Sheets

100
102

108

110104

110

116

110
110

110

104
112
106

Conv provided by USPTO from the PIRS Image Database on 06/20/2011

APLNDC00022436

US 7,663,607 B2
Page 2
U.S. P
5,252,951 A

5,281,966 A

-· •- -

-

•~

10/ 993 Tannenbaum et al. ....... 345/156

1/ 994 Walsh ......................... 341/22

6,002,389 A
6,002,808 A
6,020,881 A

6,031,524 A

5,305,017
5,345,543
5,376,948
5,398,310
5,442,742
5,463,388

A
A
A
A
A
A

4/
9/
12/
3/
8/
10/

994
994
994
995
995
995

Gerpheide .................. 345/174
Capps et al. ................ 395/ 37
Roberts ...................... 345/ 73
Tchao et al. ................ 395/ 44
Greyson et al. ............. 395/ 46
Boie et al. .................... 341/33

6,037,882
6,050,825
6,052,339
6,061,177
6,072,494
6,084,576

A
A
A
A *
A
A

5,463,696
5,483,261
5,488,204
5,495,077
5,513,309
5,523,775

A
A
A
A
A
A

10/
1/
1/
2/
4/
6/

995
996
996
996
996
996

Beernink et al. ............ 382/186
Yasutake .................... 345/173
Mead et al. ................... 178/18
Miller et al. .................. 178/18
Meieretal. ................. 395/155
Capps ........................ 345/179

6,107,997
6,128,003
6,131,299
6,135,958
6,144,380
6,188,391

A
A
A
A
A
B

5,530,455 A
5,543,590 A

6/996 Gillicketal.al................345/163
8/ 996 Gillespie et .............. 178/18

5,543,591
5,563,632
5,563,996
5,565,658
5,579,036
5,581,681
5,583,946
5,589,856
5,590,219

A
A
A
A
A
A
A
A
A

8/
_0/
0/
0/
1/
12/
12/
12/
12/

5,592,566
5,594,810
5,596,694
5,612,719
5,631,805
5,633,955
5,634,102
5,636,101

A
A
A
A
A
A
A
A

1/ 997
1/ 997
1/1997
3/ 997
5/ 997
5/ 997
5/ 997
6/ 997

5,642,108 A

996
996
996
996
996
996
996
996
996

6,198,515 B
6,208,329 B

12/1999 Kasser ....................... 345/173
12/1999 Freeman ..................... 382/288
2/2000 Naughton et al. ........... 345/327

2/2000 Kunert ....................... 345/173

3/2000
4/2000
4/2000
5/2000
6/2000
7/2000
8/2000
10/2000
10/2000
10/2000
11/2000
2/200

Levy
Nicho
1
Frenkel et al
Fujimoto ......
Nguyen ......................
Leuet al. ....................

341/20
434/227
368/230
359/443
345/358
345/ 68

Ure ............................ 345/ 73
Smith et al. ................. 345/ .57
Raab et al. .................... 33/503
Mikula-Curtis et al. ..... 600/443
Shwartsetal. .............. 345/350
Seely et al.

3/200 Ballare ....................... 345/173
3/200 Cole ..........................348/836

Gillespie et al. .............. 178/18
Roberts ...................... 345/173
Tchao ........................ 395/144
Gerpheide et al. ............ 178/19
Yates, IV .................... 345/173
Tchao et al. ................ 395/804
Gourdol ..................... 382/187
Stein et al
Gourdol ....
382/202

6,222,465
6,239,790
6,243,071
6,246,862
6,249,606
6,288,707
6,289,326
6,292,178
6,323,846

B
B
B
B
B
B
B
B
B

4/200
5/200
6/200
6/200
6/200
9/200
9/200
9/200
11/2001

Kumar et al. ................. 341/20
Martinelli et al. ........... 345/174
Shwarts et al. .............. 345/146
Grivas et al. .................. 455/90
Kiraly et al. ................ 382/195
Philipp ....................... 345/168
LaFleur ...................... 705/702
Bernstein et al. ............ 345/173
Westerman et al.

Pagallo et al. ...............
Gourdol .....................
Capps ........................
Beernink et al. ............
Bonsall ......................
Bozinovic et al. ...........
Capps ........................
Bonsall et al. ..............

6,337,678
6,347,290
6,377,009
6,380,931
6,411,287
6,414,671
6,421,234
6,452,514

B
B
B
B
B1
B1
B1
Bl

1/2002
2/2002
4/2002
4/2002
6/2002
7/2002
7/2002
9/2002

Fish
Bartlett ....................... 702/150
Philipp ....................... 318/468
Gillespie et al. ............ 345/173
Scharfetal. ............... 345/177
Gillespie et al. ............ 345/157
Ricks et al. ................. 361/683
Philipp ........................ 341/33

382/ 87
382/ 87
395/ 52
345/ 73
361/681
381/187
395/334
361/681

6/ 997 Gopher et al. ................ 341/22

6,457,355 B1

5,644,657
5,666,113
5,666,502
5,666,552
5,675,361
5,677,710

A
A
A
A
A
A

7/ 997
9/ 997
9/ 997
9/1997
10/ 997
10/ 997

Capps et al. ................ 382/229
Logan ......................... 341/34
Capps ........................ 345/352
Graysonet al. ............. 395/802
Santilli ....................... 345/168
Thompson-Rohrlich .... 345/173

6,466,036
6,515,669
6,525,749
6,535,200
6,543,684
6,543,947

5,689,253
5,710,844
5,729,250
5,730,165
5,736,976
5,741,990
5,745,116
5,745,716
5,746,818
5,748,269
5,764,222
5,767,457
5,767,842
5,790,104
5,790,107
5,802,516
5,808,567
5,809,267
,821,690
,821,930
,823,782
,825,351
5,825,352
5,854,625
5,880,411
5,898,434
5,920,309
-,923,319
,933,134
,942,733
,943,044

A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A

11/ 997
1/ 998
3/1998
3/ 998
4/ 998
4/ 998
4/ 998
4/ 998
5/1998
5/,998
6/'998
6/ 998
6/ 998
8/ 998
8/ 998
9/ 998
9/1998
9/ 998
0/ 998
0/ 998
0/ 998
0/ 998
0/ 998
2/ 998
3/ 999
4/ 999
7/ 999
7/ 999
8/ 999
8/ 999
8/ 999

Hargreaves et al. ........... 341/22
Capps et al. ................ 382/317
Bishop et al. ............... 345/175
Philipp .......................... 137/1
Cheung ...................... 345/168
Davies ..................... 84/423 R
Pisutha-Arnond ........... 345/358
Tchao et al. ................ 395/350
Yatake .................... 106/31.86
Harris et al. .................. 349/58
Shieh ......................... 345/173
Gerpheide et al. ............ 178/18
Korth ......................... 345/168
Shieh ......................... 345/173
Kasser et al. ............... 345/174
Shwarts et al. ................. 707/6
McCloud ..................... 341/20
Moran et al. ................ 395/358
Martens et al. ............. 313/506
Hansen ....................... 345/340
Marcus et al. .............. 434/156
Tam
Bisset et al. ................ 34 /173
Fnsch et al. ................ 345/173
Gillespie et al. ......... 178/ 8.01
Small et al. ................. 345/348
Bisset et al. ................ 345/173
Bishop et al. ............... 345/175
Shieh .......
.......345/173
Allen et al.
Martinelli et al. ........... 345/174

6,570,557
6,593,916
6,610,936
6,624,833
6,639,577
6,650,319
6,658,994
6,670,894
6,677,932
6,677,934
6,723,929
6,724,366
6,757,002
6,803,906
6,842,672
6,856,259
6,888,536
6,900,795
6,927,76
6,942,57
6,965,37
6,972,40
6,977,666
6,985,801
6,992,659
7,030,860
7,031,228
2002/0015024
2002/0118848
2003/0006974
2003/0076301

B1
Bl
Bl
B2
B1
B2

BI
BI
B2
B1
B2
Bl
B1
B2
Bl
BI
B2
B2
B
B
B
B *
B2 *
Bl
B2
Bl
B1
B2
B1
B1
B2
B1 *
B2
A *
A
A
A

10/2002 Philipp ........................ 73/304
10/2002
2/2003
2/2003
3/2003
4/2003
4/2003

5/2003
7/2003
8/2003
9/2003
0/2003
1/2003
2/2003
2/2003
1/2004
1/2004
4/2004
4/2004
6/2004
10/2004
1/2005
2/2005
5/2005
5/2005
8/200
9/200
11/200
12/200
12/2005
1/2006
1/2006
4/2006
4/2006
2/2002
8/2002
1/2003
4/2003

Copy provided by uur .s, from the PIRS Image Database on 06/20/2011

Philipp .......................
Mohri ........................
Moran et al. ................
Philipp
White et
Lee ............ ...... ......

324/678
345/474
345/863
345/168
234/379
400/489

Westerman et al.
Aroyan
Gillespie et al. ......... 178/18.01
Kumar et al. ............... 345/863
Eberhard .................... 345/102
Hurst et al.
McMillan .................... 99/468
Mehring
341/22
Westerman
Blanchard ................... 345/173
Kent
Crawford .................... 345/157
Oross et al. ................. 345/864
Morrison et al. ............ 345/173
Straub et al. ................... 701/3
Sharp
341/5
Westenna et
......... 45/173
Knight, IH et al. .......... 345/173
Badaye et al. .............. 345/173
McAllister et al
463/20
Gettemy et al. ...
345/173
Akitt et al. .................. 250/221
Hedrick ...................... 345/690
Straub et al. ................... 701/3
Gettemy ..................... 345/173
Hsu et al. ................... 345/173
Bom et al. .................... 368/69
Westerman et al. ......... 345/173
Karpenstein ................ 381/119
Clough et al. ............... 345/179
Tsuk et al. .................. 345/159

APLNDC00022437

US 7,663,607 B2
Page 3

2003/0076303 A

2003/0076306
2003/0095095
2003/0095096
2003/0098858

4/2003 Huppi ........................ 345/ 63

A
A
A
A

4/2003
5/2003
5/2003
5/2003

2003/0206202 A

Zadesky et al. .............
Pihlaja .......................
Robbin et al. ...............
Perski et al. ................

345/
345/
345/
345/

73
56
56
73

1/2003 Moriya ....................... 345/846

2003/0234768 A

2/2003 Rekimoto et al. ........... 345/ 69

2003/0234770 A * 2/2003 MacKey ..................... 345/ 73

2004/0227736 A *
2004/0263484 A _
2005/0007349 A _ *

2005/0012723 A _

2005/0052425
2005/0083307
2005/0104867
2005/0110768
2005/0146511
2006/0022955
2006/0022956
2006/0026521
2006/0026535
2006/0026536
2006/0032680
2006/0033724
2006/0053387
2006/0066582
2006/0085757
2006/0097991
2006/0197753

A
A *
A.
A.
A_*
A.
A.
A.
Ai
Al
A
A
A
A
A
A
A.

1/2004 Kamrath et al. ............. 345/ 73
2/2004 Mantysalo et al. .......... 345/ 73
1/2005 Vakil et al. .................. 345/ 73

1/2005 Pallakoff .................... 345/ 73

3/2005
4/2005
5/2005
5/2005
7/2005
2/2006
2/2006
2/2006
2/2006
2/2006
2/2006
2/2006
3/2006
3/2006
4/2006
5/2006
9/2006

FOREIGN P
DE
EP
EP
EP
EP
EP
EP
JP
JP
JP
JP
JP
JP
WO
WO
WO
WO
WO
WO
WO
WO
WO
WO

102 51 296
0 156 593
0 250 931
0 288 692
0 664 504
0 464 908
1 014 295
59-2,4941
60-2 1529
08-0 6307
2000-1 2642
2002-342033
2003-029899
97/18547
97/23738
98/14863
WO-01/27868
03/088176
WO-2004/0 3833
WO-2004/0 3833
WO-2005/1 4369
WO-2005/l 4369
2006/023569

«=

Zadesky et al. ............. 345/ 73
Aufderheide et al. ....... 345/ 73
Westerman et al. ......... 3451 73
Marriott et al. ............. 345/ 73
Hilletal..................... 345/,73
Kennedy .................... 345/173
Lengeling et al. ........... 345/173
Hotelling et al. ............ 715/702
Hotelling et al. ............ 715/863
Hotelling et al. ............ 715/863
Elias et al. ............... 178/ 8.06
Chaudhri et al. ............ 34 /173
Ording ....................... 71 J773
Lyon et al. .................. 345/173
Andre et al. ................ 715/771
Hotelling et al. ............ 345/173
Hotelling .................... 345/173

= =

Al
A2
A3
A2
A3

st

S

5/2004
10/1985
1/1988
7/1993
1/1995
9/1996
1/2002
12/1984
10/1985
1/1996
4/2000
11/2002
1/2003
5/1997
7/1997
4/1998
4/2001
10/2003
2/2004
2/2004
12/2005
12/2005
3/2006

- PUBLICATIONS

U.S. Appl. No. 0/789,676, filed Feb. 27, 2004.
U.S. Apç . No. 0/903,964, filed Jul. 30, 2004.
U.S. Apç . No. 1/015,978, filed Dec. 17, 2004.
U.S. Apç . No. 1/038,590, filed Jan. 18, 2005.
U.S. Apç . No. 1/048,264, filed Jan. 31, 2005.
"Touch Technologies Overview," 2001, 3M Touch Systems, Massachusetts.
"Touchscreen Technology Choices," http://www.elotouch.com/
products/detech2.asp, downloaded Aug. 5, 2005.
Jun Rekimoto, "SmartSkin: An Infastructure for Freehand Manipulation on Interactive Surfaces," CHI 2002, Apr. 20-25, 2002, Minneapolis, Minnesota.
"Wacom
Components-Technology,"
http://www.
wacomcomponents.com/english/tech.asp., downloaded Oct. 10,
2004.

"Comparing Touch Technologies," http://www.touchscreens.com/
intro-touchtypes.html, downloaded Oct. 10, 2004.
"GlidePoint!®,"http://www.cirque.com/technology/technology_gp.
htm, downloaded Aug. 5, 2005.
"Captive Position Sensing," http://www.synaptics.com/technology/
cps.cfm, downloadedAug. 5, 2005.
"How do touchscreen monitors know where you're touching?,"
http:#electronics.howstuffworks.com/question716.htm,
downloaded Aug. 5, 2005.
"How Does a Touchscreen Work?," http:Hwww.touchscreens.com/
intro-anatomy.html, downloaded Aug. 5, 2005.
"4-Wire Resistive Touchscreens," http://www.touchscreens.com/
intro-touchtypes-4resistive.html, downloaded Aug. 5, 2005.
"S-Wire Resistive Touchscreens," http://www.touchscreens.com/
intro-touchtypes-resistive.html, downloaded Aug. 5, 2005.
"Capacitive Touchscreens," http://www.touchsereens.com/introtouchtypes-capacitive.html, downloaded Aug. 5, 2005.
"PenTouch Capacitive Touchscreens," http://www.touchscreens.
com/intro-touchtypes-pentouch.html, downloaded Aug. 5, 2005.
"Surface Acoustic Wave Touchscreens," http://www.touchscreens.
comlintro-touchtypes-saw.html, downloaded Aug. 5, 2005.
"Near Field Imaging Touchscreens," http://www.touchscreens.com/
intro-touchtypes-nfi.html, downloaded Aug. 5, 2005.
"Infrared Touchscreens," http://www.touchscreens.com/introtouchtypes-infrared.html, downloaded Aug. 5, 2005.
"WatershedAlgorithm,"http:Hrsb.info.nih.govlijlplugins/watershed.
html, downloadedAug. 5, 2005.
U.S. Appl. No. 10/654, 108, filed Sep. 2, 2003 entitled "Ambidextrous
Mouse".
U.S. Appl. No. 10/789,676, filed Feb. 27, 2004 entitled "Shape
Detecting Input Device".
"4-Wire Resistive Touchscreens" obtained from http://www.
touchscreens.com/intro-touchtypes-4resistive.html generated Aug.
5, 2005.
"S-Wire Resistive Touchscreens" obtained from http://www.
touchscreens.com/intro-touchtypes-resistive.html generated Aug. 5,

2005.
"A Brief Overview of Gesture Recognition" obtained from http://
www.dai.ed.ac.uk/Cvonline/LOCA_COPIES/COHEN/
gesture_overview.html, generated Apr. 20, 2004.
"Capacitive Touchscreens" obtained fromhttp://www.touchscreens.
com/intro-touchtypes-capacitive.html generated Aug. 5, 2005.
"Capacitive Position Sensing" obtained fromhttp://www.synaptics.
com/technology/cps.cfm generated Aug. 5, 2005.
"Comparing Touch Technologies" obtained from http://www.
touchscreens.com/intro-touchtypes.html generated Oct. 10, 2004.
"Gesture
Recognition"
http://www.fingerworks.com/
gesture_recognition.html.
"GlidePoinar®" obtained from http://www.cirque.com/technology/
technology_gp.html generated Aug. 5, 2005.
"How do touchscreen monitors know where you're touching?"
obtained from http://www.electronics.howstuffworks.com/question716.html generated Aug. 5, 2005.
"How does a touchscreen work?" obtained from http://www.
touchscreens.comlintro-anatomy.html generated Aug. 5, 2005.
"iGesture Products for Everyone (learn in minutes) Product Overview" FingerWorks.com.
"Infrared Touchscreens" obtained from http://www.touchscreens.
com/intro-touchtypes-infrared.htmlgenerated Aug. 5, 2005.
"Mouse Emulation" FingerWorks obtained from http://www.
fingerworks.com/gesture_guide_mouse.html generated Aug. 30,

2005.
"Mouse Gestures in Opera" obtained from http://www.opera.com/
products/desktop/mouselindex.dml generated Aug. 30, 2005.
"Mouse Gestures," Optim oz, May 21, 2004.
"MultiTouch Overview" FingerWorks obtained from http://vvvvw.
fmgerworks.com/multoverview.html generated Aug. 30, 2005.
"Near Field Imaging Touchscreens" obtained from http://www
touchscreens.com/intro-touchtypes-nfi.htm1generatedAug. 5, 2005.
"PenTouch Capacitive Touchsereens" obtained from http://www.
touchscreens.com/intro-touchtypes-pentouch.html generated Aug.

5, 2005.

Conv nrovided bv USPTO from the PERS Imaae Database on 06/20/2011

APLNDC00022438

US 7,663,607 B2
Page 4
"Surface Acoustic Wave Touchscreens" obtained from http://www.
touchscreens.com/intro-touchtypes-saw.html generated Aug. 5,
2005.
"Symbol Commander" obtained from http://www.-- com/
symbolcomanderl, generated Aug. 30, 2005.
"Tips for Typing" FingerWorks http://www.fingerworks.com/
mini_typing.html generated Aug. 30, 2005.
"Touch Technologies Overview" 2001, 3M Touch Systems, Massachusetts.
"Wacom Components-Technology" obtained from http://www.
wacom-components.com/english/tech.asp generated on Oct. 10,
2004.
"WatershedAlgorithm"http://rsb.info.nih.govlij/plugins/watershed.
html generated Aug. 5, 2005.
"FingerWorks-Gesture Guide-Application Switching," obtained
from http://www.fingerworks.com/gesture_guide_ apps.html, generated on Aug. 27, 2004, 1-pg.
"FingerWorks-Gesture Guide-Editing," obtained from http://
www.fmgerworks.com/gesure_guide_editing.html, generated on
Aug. 27, 2004, 1-pg.
"FingerWorks-Gesture Guide-File Operations," obtained from
http://www.fingerworks.com/gesture_guide_files.html, generated
on Aug. 27, 2004, 1-pg.
"FingerWorks-Gesture Guide-Text Manipulation," obtained from
http://www.fingerworks.com/gesture_guide_text_manip.html,
generated on Aug. 27, 2004, 2-pg.
"FingerWorks-Gesture Guide-Tips and Tricks," obtained from
http://www.fmgerworks.com/gesture_guide_tips.html, generated
Aug. 27, 2004, 2-pgs.
"FingerWorks-Gesture Guide-Web," obtained from http://www.
fmgerworks.com/gesture_guide_web.html, generated on Aug. 27,
2004, 1-pg.
"FingerWorks-Guide to Hand Gestures for USB Touchpads,"
obtained from http://www.fmgerworks.comligesture_userguide.
html, generated Aug. 27, 2004, 1-pg.
"FingerWorks-iGesture-Technical Details," obtained from http://
www.fmgerworks.comligesture_tech.html, generated Aug. 27,
2004, 1-pg.
"FingerWorks-The Only Touchpads with Ergonomic Full-Hand
Resting and Relaxation!" obtained from http://www.fingerworks.
com/resting.html, Copyright 2001, 1-pg.
"FingerWorks-Tips for Typing on the Mini," obtained from http:#
www.fmgerworks.com/mini_typing.html, generated on Aug. 27,
2004, 2-pgs.
"iGesture Pad-the MultiFinger USB TouchPad with Whole-Hand
Gestures," obtained from http://www.fingerworks.comligesture.
html, generated Aug. 27, 2004, 2-pgs.
Bier, et al., "Toolglass and Magic Lenses: The see-through interface"
In James Kijiya, editor, Computer Graphics (SIGGRAPH '93 Proceedings), vol. 27, pp. 73-80, Aug. 1993.
Douglas et al., The Ergonomics of Computer Pointing Devices
(1997).
European Search Report received in EP 1 621 989 (@ Beyer Weaver
& Thomas, LLP) dated Mar. 27, 2006.
EVB Elektronik "TSOP6238 IR Receiver Modules for Infrared
Remote Control Systems" dated Jan. 2004 1-pg.
Fisher et al., "Repetitive Motion Disorders: The Design of Optimal
Rate- Rest Profiles," Human Factors, 35(2):283-304 (Jun. 1993).
Fukumoto, et al., "ActiveClick: Tactile Feedback for Touch Panels,"
In CHI 2001 Summary, pp. 121-122, 2001.
Fukumoto and Yoshinobu Tonomura, "Body Coupled Fingering:
Wireless Wearable Keyboard," CHI97, pp. 147-154 (Mar. 1997).
Hardy, "Fingerworks" Mar. 7, 2002; BBC World On Line.
Hillier and Gerald J. Lieberman, Introduction to Operations
Research (1986).
International Search Report dated Mar. 3, 2006 (PCT/US 05/03325;
119-0052WO).
Jacob et al., "Integrality and Separability of Input Devices," ACM
Transactions on Computer-Human Interaction, 1:3-26 (Mar. 1994).
Kinkley et al., "Touch-Sensing Input Devices,"in CHI '99 Proceedings, pp. 223-230, 1999.
Kionx "KXP84 Series Summary Data Sheet" copyright 2005,dated
Oct. 21, 2005, 4-pgs.

Lee et al., "A Multi-Touch Three Dimensional Touch-Sensitive Tablet," in CHI '85 Proceedings, pp. 121-128, 2000.
Lee, "A Fast Multiple-Touch-Sensitive Input Device," Master's Thesis, University of Toronto (1984).
Matsushita et al., "HoloWall: Designing a Finger, Hand, Body and
Object Sensitive Wall," In Proceedings of UIST '97, Oct. 1997.
Quantum Research Group "QTS10 / QWheelm Touch Slider IC"
copyright 2004-2005, 14-pgs.
Quek, "Unencumbered Gestural Interaction," IEEE Multimedia,
3:36-47 (Winter 1996).
Radwin, "Activation Force and Travel Effects on Overexertion in
Repetitive Key Tapping," Human Factors, 39(1):130-140 (Mar.
1997).
Rekimoto "SmartSkin: An Infrastructure for Freehand Manipulation
on Interactive Surfaces" CHI 2002, Apr. 20-25, 2002.
Rekimoto et al., "Toolstone: Effective Use of the Physical Manipulation Vocabularies of Input Devices," In Proc. Of UIST 2000, 2000.
Rubine et al., "Programmable Finger-Tracking Instrument Controllers," Computer Music Journal, vol. 14, No. 1 (Spring 1990).
Rutledge et al., "Force-To-Motion Functions for Pointing," HumanComputer Interaction-· ~ · - - · (1990).
Subatai Ahmad, "A Usable Real-Time 3D Hand Tracker," Proceedings of the 28a' Asilomar Conference on Signals, Systems and
Computers-Part 2 (of2), vol. 2 (Oct. 1994).
Texas Instnunents "TSC2003 / I2C Touch Screen Controller" Data
Sheet SBAS 162, dated Oct. 2001, 20-pgs.
Wellner, "The Digital Desk Calculators: Tangible Manipulation on a
Desk Top Display" in ACM UIST '91 Proceedings, pp. 27-34, Nov.
1991.
Williams, "Applications for a Switched-Capacitor Instrumentation
Building Block" Linear Technology Application Note 3, Jul. 1985,
pp. 1-16.
Yamada et al., "A Switched-Capacitor Interface for Capacitive Pressure Sensors" IEEE Transactions on Instrumentation and Measurement, vol. 41, No. 1, Feb. 1992, pp. 81-86.
Yeh et al., "Switched Capacitor Interface Circuit for Capacitive

Transducers" 1985 IEEE.
Zhai et al., "Dual Stream Input for Pointing and Scrolling," Proceedings ofCHI'97ExtendedAbstracts (1997).
Zimmerman et al., "Applying Electric Field Sensing to HumanComputer Interfaces," In CHI '85 Proceedings, pp. 280-287, 1995.
U.S. Appl. No. 10/774,053, filed Feb. 5, 2004.
U.S. Appl. No. 11/140,529, filed May 27, 2005 which is a Reissue of
6,570,557 listed above.
U.S. Appl. No. 11/381,313, filed May 2, 2006 entitled "Multipoint
Touch Surface Controller".
U.S. Appl. No. 11/332,861, filed Jan. 13, 2006 which is a Reissue of
6,677,932 listed above.
U.S. Appl. No. 11/380,109, filed Apr. 25, 2006 entitled "Keystroke
Tactility Arrangement on Smooth Touch Surface."
U.S. Appl. No. 11/428,501, filed Jul. 3, 2006 entitled "Capacitive
Sensing Arrangement," which is a Continuation ofUS 2005/0104867
listed above.
U.S. Appl. No. 11/428,503, filed Jul. 3, 2006 entitled "Touch Surface" which is a Continuation of US 2005/0104867 listed above.
U.S. Appl. No. 11/428,506, filed Jul. 3, 2006 entitled "User Interface
Gestures" which is a Continuation ofUS 2005/0104867 listed above.
U.S. Appl. No. 11/428,515, filed Jul. 3, 2006 entitled "User Interface
Gestures" which is a Continuation ofUS 2005/0104867 listed above.
U.S. Appl. No, 11/428,522, filed Jul. 3, 2006 entitled "Identifying
Contacts on a Touch Surface" which is a Continuation of US 2005/
0104867 listed above.
U.S. Appl. No. 11/428,521, filed Jul. 3, 2006 entitled "Identifying
Contacts on a Touch Surface" which is a Continuation of US 2005/
0104867 listed above.
U.S. Appl. No. 11/426,078, filed Jun. 23, 2006 entitled "Electronic
Device Having Display and Surrounding Touch Sensitive Bezel for
User Interface and Control" which is a Continuation-In-Partof 2006/
0197753 listed above.

Copy provided by USPTO from the PIRS Imaae Database on 06/20/2011

APLNDC00022439

US 7,663,607 B2
Page 5
U.S. Appl. No. 11/278,080, filed Mar. 30, 2006 entitled "Force Imaging Input Device and System".
U.S. Appl. No. 11/382,402, filed May 9, 2006 entitled "Force and
Location Sensitive Display" which is.a Continuation of U.S. Appl.
No. 11/278,080 listed above.
International Search Report received in corresponding PCT application No. PCT/US2006/008349 Oct 6, 2006.

International search report for International Application No. ITT/
US2005 014364 mailed Jan. 12, 2005.
U.S. Appl. No. 12 267.522. filed Nov. 7. 2008, by Ilotelling et al.
U.S. Appl. No. 12 267.532. filed Nov. 7, 2008. by IIotelling et al.
U.S. Appl. No. 12 267.540, filed Nov. 7, 2008. by llotelling et al.

* cited by

Copy provided by USPTO from the PIRS Image Database on 06/20/2011

APLNDC00022440

- I
U.S. Patent

Feb. 16, 2010

Sheet 1 of 14

US 7,663,607 B2

FIG. 1A

FIG. 1B
Row projection

signal

I

I

Column projection

signal

Copy provided by USPTO from the PIRS Image Database on 06/20/2011

APLNDC00022441

U.S. Patent

Feb. 16, 2010

Sheet 2 of 14

US 7,663,607 B2

30
S

38

36
34

42

FIG. 2

Copy provided by USPTO from the PIRS Image Database on 06/20/20t1

APLNDC00022442

U.S. Patent

Feb. 16, 2010

Sheet 3 of 14

US 7,663,607 B2

38

44

44

44

42A
44

42A

42

FIG. 3

Copy provided by USPTO from the PIRS Image Database on 06/20/2011

APLNDC00022443

U.S. Patent

Feb. 16, 2010

Sheet 4 of 14

US 7,663,607 B2

45
46

RECEIVE MULTIPLE
TOUCHES ON
SURFACE OF TOUCH
SCREEN
47

RECOGNIZE EACH OF
THE MULTIPLE
TOUCHES
-

48

REPORT TOUCH DATA
BASED ON MULTIPLE
TOUCHES

FIG. 4
50
72

70

60

74'

---

/

62

64

68

58

F G. 5

Copy provided by USPTO from the PIRS Image Database on 06/20/2011

APLNDC00022444

U.S. Patent

Feb. 16, 2010

Sheet 5 of 14

US 7,663,607 B2
100

102

108

1101

-------

110

116

116 (

110
.

110

110

104
112

ice

FIG. 6A

110

102

SERIAL DATA
BUS

FIG. 6B
106

130

'

F G. 8A

142
140

8B

Copy provided by USPTO from the PIRS Imaae Database on 06/20/20ti

APLNDC00022445

i Ii

U.S. Patent

Ii

II iI

Feb. 16, 2010

I

I

I I ii

-----..

Sheet 6 of 14

i

I

iI

1 I

i

I

II

I I
i I

i
I

I
I

11
li

i
I

I
I

I
i

11
11

i

US 7,663,607 B2

-...-

I

LL.
I

I

i

I

Illi

I i
I i

i
I

iI
il

i I I --..-III

i I

I

II

111

I I

i I i .........

I I

(O

9-

CO

Copy provided by USPTO from the PIRS Imaae Database on 06/20/2011

APLNDC00022446

192

U.S. Patent

.

Feb. 16, 2010

,

,

'

176

Sheet 7 of 14

174

US 7,663,607 B2

150

156

158B

152B

152A

154
158A

FIG. 9

190

18

170

192
180
182

177

178

172

FIG. 10

Gopy provided by USP TO from the PIRS Image Database on 06/20/2011

APLNDC00022447

204

U.S. Patent

F G. 11 B

Feb. 16, 2010

2ee

Sheet 8 of 14

US 7,663,607 B2
200

©©©©©©©
206

15253E51

ESISISS5SSSESESESS

204

44'swnnnnnnnwnnunnnnnwnannnunnnsnw

206

204

20s

FIG. 11A

fasanas awaariAaA kar iluDTl% freeswa ika DIDO hwany.a filmiaL

204

202

ARI

ftfotadd

APLNDC00022448

U.S. Patent

Feb. 16, 2010

Sheet 9 of 14

US 7,663,607 B2

232
220

222

224

230

236

226
237

237

228

FIG. 12

IN

O
OUT

FIG. 13

Conv orovided bv vr

from the PIRS Image Database on 06/20/2011

APLNDC00022449

266

262

268

CHANNEL (1) '--i

CHANNEL
(N+1)

272

V
MUX

i
\

274

A/D

DSP

¡¯

270
$

262
HOST

264

FIG. 14

APLNDC00022450

U.S. Patent

Feb. 16, 2010

Sheet 11 of 14

US 7,663,607 B2

280

282
DRIVE SENSING POINTS

w

284

RI = 1OUTPUT FROM SENSING
POINTS

286

PRODUCE AND ANALYZE
TOUCHSCREEN DATA

T

288

COMPARE CURRENT DATA TO PAST
DATA
w

290

PERFORM ACTION BASED ON
COMPARISON

FIG. 15

Copy provided by USPTO from the PIRS Image Database on 06/20/2011

APLNDC00022451

U.S. Patent

Feb. 16, 2010

Sheet 12 of 14

US 7,663,607 B2

300

302

RECEIVE RAW DATA

304
FILTER RAW DATA

306

GENERATE GRADIENT DATA

308
CALCULATE BOUNDARIES FOR TOUCH
REGIONS

310

CALCULATE COORDINATES FOR EACH
TOUCH REGION
y

312

PERFORM MULTIPOINT TRACKING

FIG. 16

Copy provided by USPTO from the PIRS Image Database on 06/20/2011

APLNDC00022452

o

ao

U.S. Patent

a

a

a

Feb. 16, 2010

Sheet 13 of 14

US 7,663,607 B2

RAW DATA INCLUDING NOISE

TOUCH REG ONS

F G. 17A
FILTERED DATA
J

FIG. 17D
COORDINATES OF TOUCH REGIONS
a=1500 p=12193
x=172 04, y=234 237288

a=33.00 p=133.97

FIG. 17B
GRADIENT DATA

,

x=707.07.04, y=331.323230

a=9 00 p=

3 33

X=417 29 y=333 ô66667

a=3500 p=13374
x=290 16, y=570 155950

FIG. 17E
i
FIG. 17C

Copy provided by USPTO from the PIRS Image Database on 06/20 2011

APLNDC00022453

366

372 3 0

U.S. Patent

370

376

Feb. 16, 2010

366

378

56

358

Sheet 14 of 14

354

US 7,663,607 B2

350

356 358

37

370

(

360

354

352
382

FIG. 18

350

I

I

i

360

I

352

362 --

FIG. 19

Copy provided by USPTO from the PIRS Image Database on 06/20/2011

APLNDC00022454

US 7,663,607 B2
1

2

MULTIPOINT TOUCHSC--

of all simultaneously -- ng touch points are determined
and a single point which falls somewhere between the touch
points is reported. In surface wave and infrared tecimologies,
it is impossible to discern the exact positionofmultiple touch

BACKGROUND OF THE um v tus 11ON

5 points that fall on the same horizontal or vertical lines due to
1. Field of the Invention
masking. In either case, faulty results are generated.
The present invention relates generally to an electronic
device having a touch ---. More particularly, the present
These problems are particularly problematic in tablet PCs
inventionrelates to a touch screen capable ofsensing multiple
where one hand is used to hold the tablet and the other is used
points at the same time.
to generate touch events. For example, as shown in FIGS. 1A
2. Description of the Related Art
10 and 1B, holding a tablet 2 causes the thumb 3 to overlap the
There exist today many styles ofinput devices forperformedge ofthe touch sensitive surface 4 ofthe touch screen 5. As
ing operations in a computer system. The operations genershown in FIG. 1A, ifthe touch technology uses averaging, the
ally correspond to moving a cursor and/or making selections
teclmique used by resistive and capacitive panels, then a
on a display screen. By way of example, the input devices
single point that falls somewhere between the thumb 3 of the

may include buttons or keys, mice, trackballs, touch pads, joy 15 left hand and the index fmger 6 of the right hand would be
sticks, touch -- and the like. Touch ---, in particular,
reported. As shown in FIG. 1B, if the technology uses proare becoming increasingly popular because of their ease and
jection scanning, the technique used by infra red and SAW
versatility of operation as well as to their declining price.
panels, it is hard to discern the exact vertical position of the
Touch screens allow a user to make selections and move a
index finger 6 due to the large vertical component of the
cursor by simply touching the display screen via a finger or 20 thumb 3. The tablet 2 can only resolve the patches shown in
stylus. In general, the touch screen recognizes the touch and
gray. In essence, the thumb 3 masks out the vertical position
position of the touch on the display screen and the computer
of the index fmger 6.
system interprets the touch and thereafterperforms an action

based on the touch event.
Touch screens typically include a touch panel, a controller

SI IMM^ Y OF THE no -o -ON
25

and a software driver. The touch panel is a clear panel with a
The invention relates, in one embodiment, to a touch panel
touch sensitive surface. The touch panel is positioned in front
having a transparent capacitive sensing medium configured to
of a display screen so that the touch sensitive surface covers
detect multiple touches or near touches that occur at the same
the viewable area of the display screen. The touch panel
time and at distinct locations in the plane of the touch panel
registers touch events and sends these signals to the control- 30 and to produce distinct signals representative of the location
ler. The controller processes these signals and sends the data
of the touches on the plane of the touch panel for each of the
to the computer system. The software driver translates the
multiple touches.
touch events into computer events.
The invention relates, in another embodiment, to a display
There are several types of touch screen technologies
including resistive, capacitive, infrared, surface acoustic 35 arrangement. The display arrangement includes a display
having a screen for displaying a graphical user interface. The
wave, electromagnetic, near field imaging, etc. Each ofthese
display arrangement further includes a transparent touch
devices has advantages and disadvantages that are taken into
panel allowing the screen to be viewed therethrough and
account when designing or configuring a touch screen. In
capable of recognizing multiple touch events that occur at
resistive technologies, the touch panel is coated with a thin
40 different locations on the touch sensitive surface ofthe touch

metallic electrically conductive andresistive layer. When the

panel is touched, the layers come into contact thereby closing

a switch that registers the position of the touch event. This

screen at the same time and to output this information to a host
device.

The invention relates, in another embodiment, to a cominformation is sent to the controller for further processing. In
puter implemented method. The method includes receiving
capacitive technologies, the touch panel is coated with a
material that stores electrical charge. When the panel is 45 multiple touches on the surface of a transparent touch screen
at the same time. The method also includes separately recogtouched, a small amount of charge is drawn to the point of
nizing each of the multiple touches. The method further
contact. Circuits located at each corner of the panel measure
includes reporting touch data based on the recognized multhe charge and send the information to the controller for
tiple touches.
processmg.
The invention relates, in another embodiment, to a comIn surface acoustic wave technologies, ultrasonic waves 50
are sent horizontally and vertically over the touch screen
puter system. The computer system includes a processor configured to execute instructions and to carry out operations
panel as for example by transducers. When the panel is
touched, the acoustic energy of the waves are absorbed. Senassociated with the computer system. The computer also
sors located across from the transducers detect this change
includes a display device that is operatively coupled to the
and send the information to the controller for processing. In 55 processor. The computer system further includes a touch

screen that is operatively coupled to the processor. The touch
infrared technologies, light beams are sent horizontally and
screen is a substantially transparentpanel that is positionedin
vertically over the touch panel as for example by light emitting diodes. When the panel is touched, some of the light
front of the display. The touch screen is configured to track
beams emanating from the light emitting diodes are intermultiple objects, which rest on, tap on or move across the
rupted. Light detectors located across from the light emitting 60 touch screen at the same time. The touch screen includes a
diodes detect this change and send this information to the
capacitive sensing device that is divided irto several indepencontroller for processing.
dent and spatially distinct sensing points that are positioned
throughout the plane of the touch screen. Each sensing point
One problem found in all ofthese technologies is that they
is capable of generating a signal at the same time. The touch
are only capable of reporting a single point even when multiple objects are placed on the sensing surface. That is, they 65 screen also includes a sensing circuit that acquires data from
the sensing device and that supplies the acquired data to the
lack the ability to track multiple points of contact simultapr^^^-r
neously. In resistive and capacitive technologies, an average

Copy provided by USPTO from the PIRS image Database on 06/20/2011

APLNDC00022455

US 7,663,607 B2
3

4

The invention relates, in another embodiment, to a touch

FIGS. 17A-E show touch data at several steps, in accorscreen method. The method includes driving a plurality of
dance with one embodiment of the present invention
sensing points. The method also includes reading the outputs
FIG. 18 is a side elevation view of an electronic device, in
from all the sensing lines connectedto the sensing points. The
accordance with one embodiments of the present invention.
method further includes producing and analyzing an image of 5
FIG. 19 is a side elevation view of an electronic device, in
the touch screen plane at one moment in time in order to
accordance with one embodiments of the present invention.
detc.usi.ie where objects are touching the touch screen. The

DETAILED D- •< = ION OF THE INVENTION

method additionally includes comparing the current image to

a past image in order to determine a change at the objects
touching the touch screen.
io
Embodiments of the invention are discussed below with
reference to FIGS. 2-19. However, those skilled in the art will
The invention relates, in another embodiment, to a digital
readily appreciate that the detailed description given herein
signal processing method. The method includes receiving
withrespect to these figures is for explanatorypurposes as the
raw data. The raw data includes values for each transparent
capacitive sensing node of a touch . The method also
invention extends beyond these limited embodiments.

FIG. 2 is a perspective view of a display arrangement 30, in
includes filtering the raw data. The method further includes 15
accordance with one embodiment of the present invention.
generating gradient data. The method additionally includes
calculating the boundaries for touch regions base on the graThe display arrangement 30 includes a display 34 and a
transparert touch screen 36 positioned in front of the display
dient data. Moreover, the method includes calculating the
34. The display 34 is configured to display a graphical user
interface (GUI) including perhaps a pointer or cursor as well
BRIEF DES= - •ONOF THE DRAWINGS
as other information to the user. The transparent touch screen
36, on the other hand, is an input device that is sensitive to a
The invention will be readily understood by the following
user's touch, allowing a user to interact with the graphical
detailed description in conjunction with the accompanying
user interface on the display 34. By way ofexample, the touch
drawings, wherein like reference numerals designate like 25 screen 36 may allow a user to move an input pointer or make
structural elements, and in which:
selections on the graphical user interface by simply pointing

coordinates for each touch region.

20

FIGS. 1A and 1B show a user holding conventional touch

at the GUI on the display 34.

In general, touch screens 36 recognize a touch event on the
FIG. 2 is a perspective view of a display arrangement, in
surface 38 of the touch screen 36 and thereafter output this
accordance with one embodiment of the present invention. 3o informationto a host device. The host device may for example
FIG. 3 shows an image of the touch screen plane at a
correspond to a computer such as a desktop, laptop, handheld
particular point in time, in accordance with one embodiment
or tablet computer. The host device interprets the touch event
of the present invention.
and thereafter performs an action based on the touch event.

FIG. 4 is a multipoint touch method, in accordance with

Conventionally, touch

-- have only been capable of

one embodiment of the present invention.
35 recognizing a single touch event even when the touch screen
FIG. 5 is a block diagram of a computer system, in accoris touchedatmultiple points at the same time (e.g., averaging,
dance with one embodiment of the present invention.
masking, etc.). Unlike conventional touch screens, however,
FIG. 6 is a partial top view of a transparent multiple point
the touch screen 36 shown herein is configured to recognize
touch screen, in accordance with one embodiment of the
multiple touch events that occur at different locations on the
present invention.
40 touch sensitive surface 38 of the touch screen 36 at the same
FIG. 7 is a partial top view of a transparent multi point
time. That is, the touch screen 36 allows for multiple contact
touch screen, in accordance with one embodiment of the
points T1-T4 to be tracked simultaneously, i.e., iffour objects
present invention.
are touching the touch screen, then the touch screen tracks all
FIG. 8 is a front elevationview, in cross section ofa display
four objects. As shown, the touch screen 36 generates sepaarrangement, in accordance with one embodiment of the 45 rate tracking signals S1-S4 for each touch point T1-T4 that
present invention.
occurs on the surface ofthe touch screen 36 at the same time.
FIG. 9 is a top view of a transparent multipoint touch
The number of recognizable touches may be about 15. 15
screen, in accordance with anotherembodiment ofthe present
touch points allows for all 10 fingers, two palms and 3 others.
invention.
The multiple touch events can be used separately or
FIG. 10 is a partial front elevation view, in cross section of so together to perform singular or multiple actions in the host
a display arrangement, in accordance with one embodiment
device. When used separately, a first touch event may be used
of the present invention.
to perform a first action while a second touch event may be
FIGS. 11A and 11B are partial top view diagrams of a
used to perform a second action that is different than the first
driving layer and a sensing layer, in accordance with one
action. The actions may for example include moving an
embodiment.
ss object such as a cursor orpointer, scrolling or panning, adjustFIG. 12 is a simplified diagram of a mutual capacitance
ing control settings, opening a file or document, viewing a
circuit, in accordance with one embodiment of the present
menu, making a selection, executing instructions, operating a

invention.

peripheral device connected to the host device etc. When used

FIG. 13 is a diagram of a charge amplifier, in accordance

with one embodiment of the present invention.

together, first and second touch events may be used for per-

So forming one particular action. The particular action may for

FIG. 14 is a block diagram of a capacitive sensing circuit,

in accordance with one embodiment of the present invention.

example include logging onto a computer or a computer

network, permitting authorized individuals access to

FIG. 15 is a flow diagram, in accordance with one embodi-

restricted areas ofthe computeror computernetwork, loading
a user profile associated with a user's preferred arrangement
FIG. 16 is a flow diagram of a digital signal processing 65 of the computer desktop, permitting access to web content,
method, in accordance with one embodiment of the present
launching a particular program, encrypting or decoding a
invention.
message, and/or the like.

ment of the present invention.

Copy provided by USPTO trom the PIRS Image Database on 06/20/2011

APLNDC00022456

US 7,663,607 B2
5

6

Recognizing multiple touch events is generally accomplished with a multipoint sensing arrangement. The multi-

changes occur and the magnitude ofthose changes are used to
help recognize the multiple touch events. A change in capacipoint sensing arrangement is capable of simultaneously
tance typically occurs at a capacitive coupling node when a
detecting and monitoring touches and the magnitude ofthose
user places an object such as a fmger in close proximity to the
touches at distinct points across the touch sensitive surface 38 5 capacitive coupling node, i.e., theobject steals charge thereby
of the touch screen 36. The multipoint sensing arrangement
affecting the capacitance.

generally provides a plurality of transparent sensor coordi-

nates or nodes 42 that work independent of one another and

By way ofexample, the signals generated at the nodes 42 of

the touch screen 36 may be used to produce an image of the
touch screen plane at a particular point in time. Referring to
plural objects are pressed against the touch screen 36, one or 10 FIG. 3, each object in contact with a touch sensitive surface 38
more sensor coordinates are activated for each touch point as
ofthe touch screen 36 produces a contact patch area 44. Each
for example touch points T1-T4. The sensor coordinates 42
of the contact patch areas 44 covers several nodes 42. The
associated with each touch point T1-T4 produce the tracking
covered nodes 42 detect surface contact while the
g
signals S1-S4.
nodes 42 do not detect surface contact. As a result, a pixilated
In one embodiment, the touch screen 36 includes a plural- " image of the touch screen plane can be formed. The signals
ity of capacitance sensing nodes 42. The capacitive sensing
for each contact patch area 44 may be grouped together to
nodes may be widely varied. For example, the capacitive
form individual images rep.. .Ltive of the contact patch
sensing nodes may be based on self capacitance or mutual
area 44. The image ofeach contact patch area 44 may include
capacitance. In self capacitance, the "self" capacitance of a
high and low points based on the p--- at each point. The
single electrode is measured as for example relative to 20 shape of the image as well as the high and low points within
ground. In mutual capacitance, the mutual capacitance
the image may be used to differentiate contact patch areas 44
between at least first and second electrodes is measured. In
that are in close proximity to one another. Furthermore, the
either cases, each of the nodes 42 works independent of the
current image, and more particularly the image of each conother nodes 42 so as to produce simultaneously occurring
tact patch area 44 can be compared to previous images to
signals representative of different points on the touch screen 25 det .' .. what action to perform in a host device.
36.
Referring back to FIG. 2, the display arrangement 30 may
In order to produce a transparent touch screen 36, the
be a stand alone unit or it may integrated with other devices.
capacitance sensing nodes 42 are formed with a transparent
When stand alone, the display arrangement 32 (or each of its
conductive medium such as indium tin oxide (ITO). In self
components) acts like a peripheral device (monitor) that
capacitance sensing arrangements, the transparent conducincludes its own housing and that can be coupled to a host
tive medium is patterned into spatially separated electrodes
device through wired or wireless connections. When inteand traces. Each of the electrodes represents a different coorgrated, the display arrangement 30 shares a housing and is
dinate and the traces connect the electrodes to a capacitive
hard wired into the host device thereby forming a single unit.
sensing circuit. The coordinates may be associated with Cartesian coordinate system (x and y), Polar coordinate system 35 By way of example, the display arrangement 30 may be
disposed inside a variety of host devices including but not
(r, 6) or some other coordinate system. In a Cartesian coorlimited to general purpose computers such as a desktop, lapdinate system, the electrodes may be positioned in columns
top or tablet computers, handhelds such as PDAs and media
and rows so as to form a grid array with each electrode
players such as music players, or peripheral devices such as
representing a different x, y coordinate. During operation, the
cameras, printers and/or the like.
capacitive sensing circuit monitors changes in capacitance
FIG. 4 is a multipoint touch method 45, in accordance with
that occur at each of the electrodes. The positions where
one embodiment of the present invention. The method genchanges occur and the magnitude ofthose changes are used to
erally begins at block 46 where multiple touches are received
help recognize the multiple touch events. A change in capacion the surface of the touch screen at the same time. This may
tance typically occurs at an electrode when a user places an
object such as a finger in close proximity to the electrode, i.e., 45 for example be accomplished by placing multiple fingers on
the surface of the touch ........ Following block 46, the prothe object steals charge thereby affecting the capacitance.
cess flow proceeds to block 47 where each of the multiple
In mutual capacitance, the transparent conductive medium
touches is separately recognized by the touch
. This
is patterned into a group ofspatially separated lines formedon
may for example be accomplished by multipoint capacitance
two different layers. Driving lines are formed on a first layer
and sensing lines are formed on a second layer. Although so sensors located within the touch screen. Following block 47,
the process flow proceeds to block 48 where the touch data
separated by being on different layers, the sensing lines
based on multiple touches is reported. The touch data may for
traverse, intersect or cut across the driving lines thereby formexamplebe reportedto a host device such as a general purpose
ing a capacitive coupling node. The manner in which the
computer.
sensing lines cut across the driving lines generally depends on
the coordinate system used. For example, in a Cartesian coor- 55
FIG. 5 is a block diagram of a computer system 50, in
dinate system, the sensing lines are perpendicular to the drivaccordance with one embodiment of the present invention.
ing lines thereby forming nodes with distinct x and y coordiThe computer system 50 may correspond to personal computer systems such as desktops, laptops, tablets or handhelds.
nates. Alternatively, in a polar coordinate system, the sensing
lines may be concentric circles and the driving lines may be
By way of example, the computer system may correspond to
radially extending lines (or vice versa). The driving lines are 60 any Apple or PC based computer system. The computer sysconnected to a voltage source and the sensing lines are contem may also correspond to public computer systems such as
nected to capacitive sensing circuit. During operation, a curinformation kiosks, automated teller machines (ATM), point
rent is driven through one driving line at a time, and because
of sale machines (POS), industrial machines, gaming
of capacitive coupling, the current is carried through to the
machines, arcade machines, vending machines, airline
sensing lines at each of the nodes (e.g., intersection points). 65 e-ticket terminals, restaurant reservation terminals, customer
Furthermore, the sensing circuit monitors changes in capaciservice stations, library terminals, learning devices, and the
like.
tance that occurs at each of the nodes. The positions where

that represent different points on the touch screen 36. When

Copy provided by USPTO from the PIRS Image Database on 06/20/2011

APLNDC00022457

US 7,663,607 B2
7

8

As shown, the computer system 50 includes a processor 56
configured to execute instructions and to carry out operations
associated with the computer system 50. For example, using
instructions retrieved for example from memory, the processor 56 may control the reception and manipulation of input 5

move across the touch sensitive surface ofthe touch screen at
the same time. The multiple objects may for example correspond to fingers and palms. Because the touch screen is
capable of tracking multiple objects, a user may perform
several touch initiated tasks at the same time. For example,

and output data between components of the computing sys-

the user may select an onscreen button with one finger, while

tem 50. The processor 56 can be a single-chip processor or
moving a cursor with another fmger. In addition, a user may
can be implemented with multiple components.
move a scroll bar with one fmger while selecting an item from
In most cases, the processor 56 together with an operating
a menu with another finger. Furthermore, a first object may be
system operates to execute computer code and poduce and 10 dragged with one finger while a second object may be
use data. The computer code and data may reside within a
dragged with another finger. Moreover, gesturing may be

program storage block 58 that is operatively coupled to the

performed with more than one fmger.

processor 56. Program storage block 58 generally provides a
place to hold data that is being used by the computer system

To elaborate, the touch screen 70 generally includes a
sensing device 72 configured to detect an object in close
50. By way of example, the program storage block may 15 proximity thereto and/or the pressure exerted thereon. The
include Read-Only Memory (ROM) 60, Random-Access
sensing device 72 may be widely varied. In one particular
Memory (RAM) 62, hard disk drive 64 and/or the like. The
embodiment, the sensing device 72 is divided into several
computer code and data could also reside on a removable
independent and spatially distinct sensing points, nodes or
storage medium and loaded or installed onto the computer
regions 74 that are positionedthroughoutthe touch screen 70.
system when needed. Removable storage mediums include, 20 The sensing points 74, which are typically hidden from view,
for example, CD-ROM, PC-CARD, floppy disk, magnetic
are dispersed about the touch screen 70 with each sensing
tape, and a network component.
point 74 representing a different position on the surface ofthe
The computer system 50 also includes an input/output
touch screen 70 (ortouch screen plane). The sensingpoints 74
(I/O) controller 66 that is operatively coupledto thep
may be positioned in a grid or a pixel array where each
56. The (I/O) controller 66 may be integrated with the pro- 25 pixilated sensing point 74 is capable of generating a signal at
cessor 56 or it may be a separate component as shown. The
the same time. In the simplest case, a signal is produced each
I/O controller 66 is generally configured to control interactime an object is positioned over a sensing poiñt 74. When an
tions with one or more I/O devices. The I/O controller 66
object is placed over multiple sensing points 74 or when the
generally operates by exchanging data between the processor
object is moved between or over multiple sensing point 74,
and the 1/O devices that desire to communicate with the 30 multiple signals are generated.
processor. The 1/O devices and the I/O controller typically
The numberand configuration ofthe sensing points 74 may
communicate through a data link 67. The data link 67 may be
be widely varied. The number of sensing points 74 generally
a one way link ortwo way link. In some cases, the1/O devices
depends on the desired sensitivity as well as the desired
may be connected to the I/O controller 66 through wired
transparency of the touch screen 70. More nodes or sensing
connections. In other cases, the I/O devices may be connected 35 points generally increases sensitivity, but reduces transparto the I/O controller 66 through wireless connections. By way
ency (and vice versa). With regards to configuration, the
of example, the data link 67 may correspond to PS/2, USB,
sensing points 74 generally map the touch screen plane into a
F ....1., IR, RF, Bluetooth or the like.
coordinate system such as a Cartesian coordinate system, a
The computer system 50 also includes a display device 68
Polar coordinate system or some other coordinate system.
that is operatively coupled to the processor 56. The display 40 When a Cartesian coordinate system is used (as shown), the
device 68 may be a separate component (peripheral device) or
sensing points 74 typically correspond to x and y coordinates.
it may be integrated with the processor and program storage
When a Polar coordinate system is used, the sensing points
to form a desktop computer (all in one machine), a laptop,
typically correspond to radial (r) and angular coordinates (6).
handheld or tablet or the like. The display device 68 is conThe touch screen 70 may include a sensing circuit 76 that
figured to display a graphical user interface (GUI) including 45 acquires the data from the sensing device 72 and that supplies
perhaps a pointer or cursor as well as other information to the
the acquired data to the processor 56. Alternatively, the prouser. By way of example, the display device 68 may be a
cessor may include this functionality. In one embodiment, the
monochrome display, color graphics adapter (CGA) display,
sensing circuit 76 is configured to send raw data to the proenhanced graphics adapter (EGA) display, variable-graphicscessor 56 so that the processor 56 processes the raw data. For
array (VGA) display, super VGA display, liquid crystal dis- so example, the processor 56 receives data from the sensing
play (e.g., active matrix, passive matrix and the like), cathode
circuit 76 and then determines how the data is to be used
ray tube (CRT), plasma displays and the like.
within the computer system 50. The data may include the
The computer system 50 also includes a touch screen 70
coordinates of each sensing point 74 as well as the pressure
that is operatively coupled to the processor 56. The touch
exerted on each sensing point 74. In another embodiment, the
screen 70 is a transparent panel that is positioned in front of 55 sensing circuit 76 is configured to process the raw data itself.
the display device 68. The touch screen 70 may be integrated
That is, the sensing circuit 76 reads the pulses from the
with the display device 68 or it may be a separate camponent.
sensing points 74 and turns them into data that the processor
The touch screen 70 is configured to .-1- input from a
56 can understand. The sensmg circuit 76 may perform filuser's touch and to send this information to the processor 56.
tering and/or conversion processes. Filtering processes are
In most cases, the touch screen 70 recognizes touches and the 60 typically implementedto reduce a busy data stream so that the
position and magnitude of touches on its surface. The touch
processor 56 is not overloaded with redundant or non-essen-

screen 70 reports the touches to the processor 56 and the

tial data. The co-

La processes may be implemented to

processor 56 interprets the touches in accordance with its
adjust the raw data before sending or reporting them to the
programming. For example, the processor 56 may initiate a
processor 56. The conversions may include determining the
task in accordance with a particular touch.
65 center point for each touch region (e.g., centroid).
In accordancewith one embodiment, the touch screen 70 is
The sensing circuit 76 may include a storage element for

capable oftracking multiple objects, which rest on, tap on, or

storing a touch screen program, which is a capable of con-

Copy provided by USPTO from the PIRS Image Database on 06/20/2011

APLNDC00022458

US 7,663,607 B2
9

10

trolling different aspects ofthe touch screen 70. For example,
present invention. By way of example, the touch screen 100
the touch screen program may contain what type of value to
may generally correspondto the touch screen shown in FIGS.
output based on the sensing points 74 selected (e.g., coordi2 and 4. The multipointtouch screen 100 is capable ofsensmg
nates). In fact, the sensing circuit in conjunction with the
the position and the pressure of multiple objects at the same
touch screen program may follow a predetermined commu- 5 time. This particular touch screen100 is based on selfcapacinication protocol. As is generally well known, communicatance and thus it includes a plurality oftransparent capacitive
tionprotocolsare a set ofrules and procedures for exchanging
sensing electrodes 102, which each represent different coordata between two devices. Corununication protocols typidinates in the plane of the touch screen 100. The electrodes
cally transmit information in data blocks or packets that con102 are configured to ....
capacitive input from one or
tain the data to be transmitted, the data required to direct the io more objects touching the touch screen 100 in the vicinity of
packet to its destination, and the data that corrects errors that
the electrodes 102. When an object is proximate an electrode
occur along the way. By way of example, the sensing circuit
102, the object steals charge thereby affecting the capacitance

may place the data in a HID format (Human Interface

at the electrode 102. The electrodes 102 are connected to a

Device).
capacitive sensing circuit 104 through traces 106 that are
The sensing circuit 76 generally includes one or more 15 positioned in the gaps 108 found between the spaced apart
microcontrollers, each of which monitors one or more senselectrodes 102. The electrodes 102 are spacedapart m order to
ing points 74. The microcontrollers may for example correelectrically isolate them from each other as well as to provide
spond to an application specific integrated circuit (ASIC),
a space for separately routing the sense traces 106. The gap

which works with firmware to monitor the signals from the

108 is preferably made small so as to maximize the sensmg

sensing device 72 and to process the monitored signals and to 20 area and to
optical diff
between the space
report this information to the processor 56.
and the transparent electrodes.
In accordance with one embodiment, the sensing device 72
As shown, the sense traces 106 are routed from each elecis based on capacitance. As should be appreciated, whenever
trode 102 to the sides of the touch screen 100 where they are
two electrically conductive members come close to one
connected to the capacitive sensing circuit 104. The capaci-

anotherwithout actually touching, their electric fields interact 25 tive sensing circuit 104 includes one or more sensor ICs 110
to form capacitance. In most cases, the first electrically conthat measure the capacitance at each electrode 102 and that
ductive member is a sensing point 74 and the second electrireports its findings or some form thereof to a host controller.
cally conductive member is an object 80 such as a finger. As
The sensor ICs 110 may for example convert the analog
the object 80 approaches the surface of the touch screen 70, a
capacitive signals to digital data and thereafter transmit the
tiny capacitance forms between the object 80 and the sensing 30 digital data over a serial bus to a host controller. Any munber
points 74 in close proximity to the object 80. By detecting
ofsensor ICs may be used. For example, a single chip may be
changes in capacitance at each of the sensing points 74 and
used for all electrodes, or multiple chips may be used for a
noting the position of the sensing points, the sensing circuit
single or group of electrodes. In most cases, the sensor ICs
can recognize multiple objects, and determine the location,
110 report tracking signals, which are a function of both the
pressure, direction, speed and acceleration of the objects 80 35 position of the electrode 102 and the intensity of the capacias they are moved across the touch screen 70. For example,
tance at the electrode 102.
the sensing circuit can det
when and where each of the
The electrodes 102, traces 106 and sensing circuit 104 are

fingers and palm of one or more hands are touching as well as

generally disposed on an optical transmissive member 112. In

the pressure being exerted by the finger and palm of the

most cases, the optically transmissive member 112 is formed

hand(s) at the same time.
40 from a clear material such as glass or plastic. The electrode
The simplicity of capacitance allows for a great deal of
102 and traces 106 may be placed on the member 112 using
flexibility in design and constructionofthe sensing device 72.
any suitable patterning technique including for example,
By way of example, the sensing device 72 may be based on
deposition, etching, printing and the like. The electrodes 102
self capacitance or mutual capacitance. In self capacitance,
and sense traces 106 can be made from any suitable transpareach of the sensing points 74 is provided by an individual 45 eut conductive material. By way of example, the electrodes
charged electrode. As an object approaches the surface of the
102 and traces 106 may be formed from indium tin oxide
touch screen 70, the object capacitive couples to those elec(ITO). In addition, the sensor ICs 110 of the sensing circuit
trodes in close proximity to the object thereby stealing charge
104 can be electrically coupled to the traces 106 using any

away from the electrodes. The amount ofcharge in each ofthe

suitable techniques. In one implementation, the sensor ICs

electrodes are measured by the sensing circuit 76 to deter- so 110 are placed directly on the member 112 (flip chip). In
mine the positions of multiple objects when they touch the
another implementation, a flex circuit is bondedto the mem-

touch screen 70. In mutual capacitance, the sensing device 72

ber 112, and the sensor ICs 110 are attachedto the flex circuit.

includes a two layer grid of spatially separated lines or wires.
In yet another implementation, a nex circuit is bonded to the
In the simplest case, the upper layer includes lines in rows
member 112, a PCB is bonded to the flex circuit and the
while the lower layer includes lines in columns (e.g., orthogo- 55 sensor ICs 110 are attached to the PCB. The sensor ICs may
nal). The sensing points 74 are provided at the intersectionsof
for example be capacitance sensing ICs such as those manuthe rows and columns. During operation, the rows are charged
factured by Synaptics of San Jose, Calif., Fingerworks of
and the charge capacitively couples to the columns at the
Newark, Del. or Alps of San Jose, Calif.
intersection. As an object approaches the surface ofthe touch
The distribution of the electrodes 102 may be widely varscreen, the object capacitive couples to the rows at the inter- 60 ied. For example, the electrodes 102 may be positioned
sections in close proximity to the object thereby stealing
almost anywhere in the plane of the touch screen 100. The

charge away from the rows and therefore the columns as well.

electrodes 102 may be positioned randomly or in a particular

The amount of charge in each of the columns is measured by
pattern about the touch screen 100. With regards to the later,
the sensing circuit 76 to determine the positions of multiple
the position of the electrodes 102 may depend on the coordiobjects when they touch the touch screen 70.
65 nate system used. For example, the electrodes 102 may be
FIG. 6 is a partial top view of a transparent multiple point
placed in an array of rows and columns for Cartesian coorditouch screen 100, in accordance with one embodiment of the
nates or an array of concentric and radial segments for polar

Copy provided by USPTO from the PIRS Image Database on 06/20/2011

APLNDC00022459

US 7,663 ,607 B2
11

12

coordinates. Within each array, the rows, columns, concentric
electrode 102 is in the form of a parallelogram, and more
particularly a parallelogram with sloping sides.
or radial segments may be stacked uniformly relative to the
others or they may be staggered or offset relative to the others.
FIG. 7 is a partial top view of a transparent multi point
touch screen 120, in accordance with one embodiment of the
Additionally, within each row or column, or within each
-tric or radial segment, the electrodes 102 may be stag- 5 present invention. In this embodiment, the touch screen 120 is
similar to the touch screen 100 shown in FIG. 6, however,
gered or offset relative to an adjacent electrode 102.
unlike the touch screen 100 of FIG. 6, the touch screen 120
Furthermore, the electrodes 102 may be formed from
shown in FIG. 7 includes electrodes 122 with different sizes.
almost any shape whethersimple (e.g., squares, circles, ovals,
As shown, the electrodes 122 located in the center ofthe touch
triangles, rectangles, polygons, and the like) or complex (e.g.,
screen 120 are larger than the electrodes 122 located at the
random shapes). Further still, the shape ofthe electrodes 102
sides of the touch screen 120. In fact, the heig,ht of the elecmay have identical shapes or they may have different shapes.
For example, one set of electrodes 102 may have a first shape
while a second set ofelectrodes 102 may have a second shape
that is different than the first shape. The shapes are generally

trodes 122 gets correspondingly smaller when moving from

the center to the edge of the touch screen 120. This is done to
make room for the sense traces 124 extending from the sides
chosen to maximize the sensing area and to minimize optical 15 of the more centrally located electrodes 122. This arrangement advantageouslyreduces the gap found betweenadjacent
differences between the gaps and the transparent electrodes.
rows 126 of electrodes 122. Although the height of each
In addition, the size ofthe electrodes 102 may vary accordelectrode 122 shrinks, the height H of the row 126 as well as
ing to the specific needs ofeach device. In some cases, the size
the width W of each electrode 122 stays the same. In one
of the electrodes 102 corresponds to about the size of a fmger
configuration, the height of the row 126 is substantially equal
tip. For example, the size of the electrodes 102 may be on the
to the width of each electrode 122. For example, the height of
order of 4-5 mm2. In other cases, the size of the electrodes
the row 126 and thewidth ofeach electrode 122 may be about
102 are smaller than the size of the finger tip so as to improve
4 mm to about 5 mm.
resolution of the touch screen 100 (the finger can influence
FIG.8 is a ll ort elevation view, in cross section ofa display
two or more electrodes at any one time thereby enabling
arrangement 130, in accordance with one embodiment of the
interpolation). Like the shapes, the size ofthe electrodes 102
present invention. The display arrangement 130 includes an
may be identical or they may be different. For example, one
LCD display 132 and a touch screen 134 positioned over the
set of electrodes 102 may be larger than another set of elecLCD display 132. The touch screen may for example corretrodes 102. Moreover, any number of electrodes 102 may be
spond to the touch screen shown in FIG. 6 or 7. The LCD
used. The munber of electrodes 102 is typically determined
display 132 maycorrespondto any conventional LCD display
by the size of the touch screen 100 as well as the size of each
known in the art. Although not shown, the LCD display 132
electrode 102. In most cases, it would be desirable to ...-typically includes various layers including a flu
t
the number of electrodes 102 so as to provide higher resolupanel, polarizing filters, a layer of liquid crystal cells, a color
tion, i.e., more information can be used for such things as
filter and the like.
acceleration.
35
The touch screen 134 includes a transparent electrode layer
Although the sense traces 106 can be routed a variety of
136 that is positioned over a glass member 138. The glass
ways, they are typically routed in
that reduces the
member 138 may be a portion of the LCD display 132 or it
distance they have to travel between their electrode 102 and
may be a portion of the touch screen 134. In either case, the
the sensor circuit 104, and that reduces the size of the gaps
glass member 138 is a relatively thick piece ofclear glass that
108 found between adjacent electrodes 102. The width of the 40 protects the display 132 from forces, which are exerted on the
sense traces 106 are also widely varied. The widths are gentouch screen 134. The thickness ofthe glass member 138 may
erally determined by the amount of charge being distributed
for example be about 2 mm. In most cases, the electrode layer
there through, the number ofadjacent traces 106, and the size
136 is disposed on the glass member 138 using suitable transof the gap 108 through which they travel. It is generally
parent conductivematerials and patterningtechniques such as
desirable to maximize the widths of adjacent traces 106 in 45 ITO and printing. Although not shown, in some cases, it may
order to -, 'Aze the coverage inside the gaps 108 thereby
be necessary to coat the electrode layer 136 with a material of
creating a more uniform optical appearance.
similar refractive index to improve the visual appearance of
In the illustrated embodiment, the electrodes 102 are posithe touch
. As should be appreciated, the gaps located
tioned in a pixilated array. As shown, the electrodes 102 are
between electrodes and traces do not have the same optical
positioned in rows 116 that extend to and from the sides ofthe so index as the electrodes and traces, and therefore a material
touch screen 100. Within each row 116, the identical elecmay be needed to provide a more similar optical index. By
trodes 102 are spaced apart and positionedlaterally relative to
way of example, index matching gels may be used.
one another (e.g., juxtaposed). Furthermore, the rows 116 are
The touch screen 134 also includes a protective cover sheet
stacked on top of each other thereby forming the pixilated
140 disposed over the electrode layer 136. The electrode layer
array. The sense traces 106 are routed in the gaps 108 formed 55 136 is therefore sandwiched between the glass member 138
between adjacent rows 106. The sense traces 106 for each row
and the protective cover sheet 140. The protective sheet 140
are routed in two different directions. The sense traces 106 on
serves to protect the under layers and provide a surface for
one side of the row 116 are routed to a sensor IC 110 located
allowing an object to slide thereon. The protective sheet 140
onthe left side and the sense traces 106 on the other side ofthe
also provides an insulating layer between the object and the
row 116 are routed to another sensor IC 110 located on the 60 electrode layer 136. The protective cover sheet 140 may be
right side of the touch screen 100. This is done to formed from any suitable clear material such as glass and
the gap 108 formed between rows 116. The gap 108 may for
plastic. The protective cover sheet 140 is suitablythin to allow
example be held to about 20 microns. As should be apprecifor sufficient electrode coupling. By way of example, the
ated, the spaces between the traces can stack thereby creating
thickness of the cover sheet 140 may be between about 0.3a large gap between electrodes. If routed to one side, the size 65 0.8 mm. In addition, the protective cover sheet 140 may be
ofthe space would be substantially doubled thereby reducing
treated with coatings to reduce stiction when touching and
the resolution ofthe touch screen. Moreover, the shape ofthe
reduce glare when viewing the underlying LCD display 132.

Conv provided bv USPTO from the PIRS Imaae Database on --,...,..11

APLNDC00022460

US 7,663,607 B2
13

14

By way of example, a low stiction/anti reflective coating 142
technique including for example. deposition, etching. printmay be applied over the cover sheet 140. Although the elecing and the like. Furthermore, the lines 152 can be made from
trode layer 136 is typically patterned on the glass member
any suitable transparent conductive material. By way of
138, it should be noted that in some cases it may be alternaexample, the lines may be formed from indium tin oxide
tively or additionally patterned on the protective cover sheet 5 (ITO). The driving lines 152A are typically coupled to the
140.
voltage source through a flex circuit 158A, and the sensing
FIG. 9 is a top view ofa transparent multipoint touch screen
lines 152B are typically coupled to the sensing circuit, and
150, in accordance with another embodiment of the present
more particularly the sensor ICs through a flex circuit 158B.

invention. By way of example. the touch screen 150 may

The sensor ICs may be attached to a printed circuit board

generally correspond to the touch screen of FIGS. 2 and 4. 10 (PCB). Alternatively, the sensor ICs may be placed directly
Unlike the touch screen shown in FIGS. 6-8, the touch screen
on the member 156 thereby eliminating the flex circuit 158B.
of FIG. 9 utilizes the concept of mutual capacitance rather
The distribution ofthe lines 152 may be widely varied. For

than self capacitance. As shown, the touch screen 150

example, the lines 152 may be positioned almost anywhere in

includes a two layer grid of spatially separated lines or wires
the plane of the touch screen 150. The lines 152 may be
152. Inmost cases, the lines 152 on each layer are parallel one 15 positioned randomly or in a particularpattern about the touch

another. Furthermore, although in different planes, the lines

152 on the different layers are configured to intersect or cross

in order to produce capacitive sensing nodes 154, which each

screen 150. With regards to the later, the position of the lines
152 may depend onthe coordinate system used. For example,

the lines 152 may be placed in rows and columns for Carte-

represent different coordinates in the plane of the touch
sian coordinates or concentrically and radially for polar coorscreen 150. The nodes 154 are configured to - - capaci- 20 dinates. When using rows and columns, the rows and columns
tive input from an object touching the touch screen 150 in the
may be placed at various angles relative to one another. For

vicinity of the node 154. When an object is proximate the

example, they may be vertical, horizontal or diagonal.

node 154, the object steals charge thereby affecting the
Furthermore, the lines 152 may be formed from almost any
capacitance at the node 154.
shape whether rectilinear or curvilinear. The lines on each
To elaborate, the lines 152 on different layers serve two 25 layer may be the same or different. For example, the lines may
different functions. One set of lines 152A drives a current
alternate between rectilinear and curvilinear. Further still, the
therethrough while the second set of lines 152B senses the
shape ofthe opposing lines may have identical shapes or they
capacitance coupling at each of the nodes 154. In most cases,
may have different shapes. For example, the driving lines may
the top layerprovides the driving lines 152A while the bottom
have a first shape while the sensing lines may have a second
layer provides the sensing lines 152B. The driving lines 152A 30 shape that is different than the first shape. The geometryofthe
are connected to a voltage source (not shown) that separately
lines 152 (e.g., linewidths and spacing) may also be widely
drives the current through each ofthe driving lines 152A. That
varied. The geometry of the lines within each layer may be
is, the stimulus is only happening over one line while all the
identical or different, and further, the geometryofthe lines for

other lines are grounded. They may be driven similarly to a

both layers may be identical or different. By way ofexample,

raster scan. The sensing lines 152B are connected to a capaci- 35 the linewidths ofthe sensing lines 152B to driving lines 152A
tive sensing circuit (not shown) that continuouslysenses all of
may have a ratio of about 2:1.
the sensing lines 152B (always sensing).
Moreover, any number of lines 152 may be used. It is
When driven, the charge on the driving line 152A capacigenerally believed that the number of lines is dependent on

tively couples to the intersecting sensing lines 152B through
the desired resolution ofthe touch screen 150. The number of
the nodes 154 and the capacitive sensing circuit senses all of 40 lines within each layer may be identical or different. The
the sensing lines 152B in parallel. Thereafter, the next driving
number of lines is typically determined by the size of the
line 152A is driven, and the charge on the next driving line
touch screen as well as the desired pitch and linewidths ofthe
lines 152.
152A capacitively couples to the intersecting sensing lines
152B through the nodes 154 and the capacitive sensing circuit
In the illustrated embodiment, the driving lines 152A are
senses all of the sensing lines 152B in parallel. This happenS 45 positioned in rows and the sensing lines 152B are positioned
sequential until all the lines 152A have been driven. Once all
in colunms that are perpendicular to the rows. The rows

the lines 152A have been driven, the sequence starts over
extend ho
Llly to the sides of the touch screen 150 and
(continuously repeats). In most cases, the lines 152A are
the columns extend vertically to the top and bottom of the
sequentially driven from one side to the opposite side.
touch screen 150. Furthermore, the linewidths for the set of
TIle capacitive sensing circuit typically includes one or so lines 152A and 152B are different and the pitch for set oflines
more sensor ICs that measure the capacitance in each of the
152A and 152B are equal to one another. In most cases, the
sensing lines 152B and that reports its findings to a host
linewidths of the sensing lines 152B are larger than the lincontroller. The sensor ICs may for example convert the analog
ewidths of the driving lines 152A. By way of example, the
capacitive signals to digital data and thereafter transmit the
pitch ofthe driving and sensing lines 152 may be about 5 mm,
digital data over a serial bus to a host controller. Any number 55 the linewidths of the driving lines 152A may be about 1.05
of sensor ICs may be used. For example, a sensor IC may be
mm and the linewidths of the sensing lines 152B may be
used for all lines, or multiple sensor ICs may be used for a
about 2.10 mm. Moreover, the number of lines 152 in each
single or group of lines. In most cases, the sensor ICs 110
layer is different. For example, there may be about 38 driving
report tracking signals, which are a function of both the
lines and about 50 sensing lines.
position of the node 154 and the intensity of the capacitance 60
As mentioned above, the lines in order to form semi-transat the node 154.
parent conductors on glass, film or plastic, may be patterned

The lines 152 are generally disposed on oneor more optical

transmissive members 156 formed from a clear material such

with an ITO material. This is generally accomplished by

depositing an ITO layer over the substrate surface, and then
by etching away portions ofthe ITO layer in order to form the
placed on opposing sides of the same member 156 or they 65 lines. As should be appreciated, the areas with ITO tend to
may be placed on different members 156. The lines 152 may
have lower transparency than the areas without ITO. This is
be placed on the member 156 using any suitable patterning
generally less desirable for the useras the user can distinguish

as glass or plastic. By way of example, the lines 152 may be

Copy provided by USPTO from the PIRS Image Database on 06/20/2011

APLNDC00022461

US 7,663,607 B2
15
the lines from the spaces therebetween, i.e., the patternedITO
can become quite visible thereby producing a touch screen

16
The touch screen 174 also includes a protective cover sheet

190 disposed overthe driving layer 180. The driving layer 180

is therefore sandwiched between the second glass member
182 and the protective cover sheet 190. The protective cover
that produces a relatively low resistance, and unfortunately 5 sheet 190 serves to protect the under layers and provide a
surface for allowing an object to slide thereon. The protective
low resistance ITO tends to be less transparent than high
cover sheet 190 also provides an insulating layer between the
resistance ITO.
object and the driving layer 180. The protective cover sheet is
In order to prevent the aforementioned problem, the dead
suitably thin to allow for sufficient coupling. The protective
areas between the ITO may be filled with indexing matching
materials. In another embodiment, rather than simply etching 10 cover sheet 190 may be formed from any suitable clear material such as glass and plastic. In addition, the protective cover
away all of the ITO, the dead areas (the uncovered spaces)
sheet 190 may be treated with coatings to reduce stiction
may be subdivided into unconnected electrically floating ITO
when touching and reduce glare when viewing the underlying
pads, i.e., the dead areas may be patterned with spatially
LCD display 172. By way of example, a low stiction/anti
separated pads. The pads are typically separated with a mini-

with undesirable optical properties. To further exacerbate this
problem, the ITO material is typically applied in a

mum trace width. Furthermore, the pads are typically made 15 -Ï! .ive coating may be applied over the cover sheet 190.
Althoughthe line layer is typically patterned on a glass memsmall to reduce their impact on the capacitive measurements.
This technique attempts to minimize the appearance of the

ITO by creating a uniform optical retarder. That is, by seeking

ber, it should be noted that in some cases it may be alternatively or additionally patterned on the protective cover sheet.
The touch screen 174 also includes various bonding layers

to create a uniform sheet of ITO, it is believed that the panel
will function closer to a uniform optical retarderandtherefore 20 192. The bonding layers 192 bond the glass members 178 and
182 as well as the protective cover sheet 190 together to form
non-uniformities in the visual appearance will be minimized.
the laminated structure and to provide rigidity and stiffness to
In yet another embodiment, a combination ofindex matching
the laminated structure. In essence, the bonding layers 192
materials and unconnected floating pads may be used.
help to produce a monolithic sheet that is stronger than each
FIG. 10 is a partial front elevation view, in cross section of

a display arrangement 170, in accordance with one embodi- 25 of the individual layers taken alone. In most cases, the first
and second glass members 178 and 182 as well as the second
ment of the present invention. The display arrangement 170
glass member and the protective sheet 182 and 190 are laminated together using a bonding agent such as glue. The compliant nature of the glue may be used to absorb geometric
variations so as to form a singularcomposite structure with an
overall geometry that is desirable. In some cases, the bonding
display known in the art. Although not shown, the LCD disagent includes an index matching material to improve the
play 172 typically includes - - layers including a fluovisual appearance of the touch screen 170.
rescent panel, polarizing filters, a layer of liquid crystal cells,
With regards to configuration, each of the various layers
a color filter and the like.
The touch screen 174 includes a transparent sensing layer 35 may be formed with various sizes, shapes, and the like. For
example, each of the layers may have the same thickness or a
176 that is positioned over a first glass member 178. The
different thickness than the other layers in the structure. In the
sensing layer 176 includes a plurality of sensor lines 177
illustrated embodiment, the first glass member 178 has a
positioned in columns (extend in and out of the page). The
thickness of about 1.1 mm, the second glass member 182 has
first glass member 178 may be a portion of the LCD display
172 or it may be a portion of the touch screen 174. For 40 a thickness of about 0.4 mm and the protective sheet has a

includes an LCD display 172 and a touch screen 174 positioned over the LCD display 170. The touch screen may for
example correspond to the touch screen shown in FIG. 9. The
LCD display 172 may correspond to any conventional LCD 30

example, it may be the front glass ofthe LCD display 172 or
it may be the bottom glass ofthe touch screen174. The sensor
layer 176 is typically disposed onthe glass member 178 using

thickness of about 0.55 mm. The thickness of the bonding
layers 192 typically varies in order to produce a laminated

structure with a desired height. Furthermore, each of the

layers may be formed with various materials. By way of
suitable transparent conductive materials and patterningtechniques. In some cases, it may be necessary to coat the sensor 45 example, each particular type of layer may be formed from
the same or different material. For example, any suitable glass
layer 176 with material of similar refractive index to improve
or plastic material may be used for the glass members. In a
the visual appearance, i.e., make more uniform.
The touch screen 174 also includes a transparent driving

similar .-.-, any suitable bonding agent may be used for

the bonding layers 192.
layer 180 that is positioned over a second glass member 182.
FIGS. 11A and 11B are partial top view diagrams of a
The second glass member 182 is positioned overthe first glass 50
driving layer 200 and a sensing layer 202, in accordance with
member 178. The sensing layer 176 is therefore sandwiched
one embodiment. In this embodiment, each of the layers 200
betweenthe first and second glass members 178 and 182. The
and 202 includes dummy features 204 disposed between the
second glass member 182 provides an insulating layer
driving lines 206 and the sensing lines 208. The dummy
between the driving and sensing layers 176 and 180. The
driving layer 180 includes a plurality of driving lines 181 55 features 204 are configured to optically improve the visual
appearance of the touch screen by more closely matching the
positioned in rows (extend to the right and left of the page).
optical index of the lines. While index matching materials
The driving lines 181 are configured to intersect or cross the

may improve the visual appearance, it has been found that
there still may exist some non-uniformities. The dummy fealayer 176, the driving layer 180 is disposed on the glass 60 tures 204 provide the touch screen with a more uniform
appearance. The dununy features 204 are electrically isolated
member using suitable materials and patterning techmques.
and positioned in the gaps between each of the lines 206 and
Furthermore, in some cases, it may be necessary to coat the
208. Although they may be patterned separately, the dummy
driving layer 180 with material of similar refractive index to
features 204 are typically pattemed along with the lines 206
improve the visual appearance. Although the sensing layer is
typically patterned on the first glass member, it should be 65 and 208. Furthermore, although they may be formed from
different materials, the dununy features 204 are typically
noted that in some cases it may be alternatively or additionformed with the same transparent conductive material as the
ally patterned on the second glass member.

sensing lines 177 positioned in columns in order to form a

plurality of capacitive coupling nodes 182. Like the sensing

Copy provided by USPTO from the PIRS Image Database on 06/20/2011

APLNDC00022462

US 7,663,607 B2
17

18

lines as for example ITO to provide the best possible index

sensing points 262 (electrode, nodes, etc.), to process the data
and to output processed data to a host controller.

matching. As should be appreciated, the dummy features will
more than likely still produce some gaps, but these gaps are
much smaller than the gaps found between the lines (many

The sensing circuit 260 includes a multiplexer264 (MUX).
The multiplexer 264 is a switch configured to perform time
orders of magnitude smaller). These gaps, therefore have 5 Inultiplexing. As shown, the MUX 264 includes a plurality of
minimal impact on the visual appearance. While this may be
independent input channels 266 for receiving signals from
the case, index matching materials may be additionally
each of the sensing points 262 at the same time. The MUX
applied to the gaps between the dummy features to further

264 stores all of the incoming signals at the same time, but

sequentially releases them one at a time through an output
improve the visual appearance of the touch screen. The distribution, size, number, dimension, and shape of the dununy 10 channel 268.

features may be widely varied.
The sensing circuit 260 also includes an analog to digital
FIG. 12 is a simplified diagram of a mutual capacitance
converter 270 (ADC) operatively coupled to the MUX 264
circuit 220, in accordance with one embodiment of the
through the output channel 268. The ADC 270 is configured
present invention. The mutual capacitance circuit 220 15 to digitize the incoming analog signals sequentially one at a

includes a driving line 222 and a sensing line 224 that are
spatially separated thereby forming a capacitive coupling
node 226. The driving line 222 is electrically coupled to a

time. That is, the ADC 270 converts each of the incoming

voltage source 228, and the sensing line 224 is electrically

retically infinite number ofvalues. The voltage varies accord-

analog signals into outgoing digital signals. The input to the

ADC 270 generally corresponds to a voltage having a theo-

coupled to a capacitive sensing circuit 230. The driving line 20 ing to the amount ofcapacitive coupling at each ofthe sensing
222 is configured to carry a current to the capacitive coupling
points 262. The output to the ADC 270, on the other hand, has
node 226, and the sensing line 224 is configured to carry a
a defined number of states. The states generally have predictcurrent to the capacitive sensing circuit 230. When no object
able exact voltages or currents.
is present, the capacitive coupling at the node 226 stays fairly
The sensing circuit 260 also includes a digital signal proconstant. When an object 232 such as a finger is placed 25 cessor 272 (DSP) operatively coupled to the ADC 270
p, ,,,,«-te the node 226, the capacitive coupling changes
through another channel 274. The DSP 272 is a programthrough the node 226 changes. The object 232 effectively
mable computer processing unit that works to clarify or stanshunts some of the field away so that the charge projected
dardize the digital signals via high speed mathematical proacross the node 226 is less. The change in capacitive coupling
cessing. The DSP 274 is capable of differentiating between
changes the current that is carried by the sensing lines 224. 30 human made signals, which have order, and noise, which is
The capacitive sensing circuit 230 notes the current change
inherently chaotic. In most cases, the DSP performs filtering
and the position of the node 226 where the current change
and conversion algorithms using the raw data. By way of
occurred and reports this information in a raw or in some
example, the DSP may filter noise events from the raw data,
processed form to a host controller. The capacitive sensing
calculate the touch boundaries for each touch that occurs on
circuit does this for each node 226 at about the same time (as 35 the touch screen at the same time, and thereafter determine
viewed by a user) so as to provide multipoint sensing.
the coordinates for each touch event. The coordinates of the
touch events may then be reported to a host controller where
The sensing line 224 may contain a filter 236 for eliminating parasitic capacitance 237, which may for example be
they can be compared to previous coordinates of the touch
created by the large surface area of the row and column lines
events to determinewhat action to perform in the host device.
relative to the other lines and the system enclosure at ground
FIG. 15 is a flow diagram 280, in accordance with one
potential. Generally speaking, the filter rejects stray capaciembodiment of the present invention. The method generally
tance effects so that a clean representation of the charge
begins at block 282 where a plurality of sensing points are
transferred across the node 226 is outputted (and not anything
driven. For example, a voltage is applied to the electrodes in
in addition to that). That is, the filter 236 produces an output
self capacitance touch screens or through driving lines in
that is not dependent on the parasitic capacitance, but rather 45 mutual capacitance touch --. In the later, each driving
on the capacitance at the node226. As a result, a more acculine is driven separately. That is, the driving lines are driven
rate output is produced.
one at a time therebybuildingup charge on all the intersecting
FIG. 13 is a diagram of an inverting amplifier 240, in
sensing lines. Following block 282, the process flow proceeds
accordance with one embodiment of the present invention.
to block 284 where the outputs (voltage) fiom all the sensing
The inverting amplifier 240 may generally correspond to the so points are read. This block may include multiplexing and
filter 236 shown in FIG. 12. As shown, the inverting amplifier
digitizing the outputs. For example, in mutual capacitance
includes a non inverting input that is held at a constant voltage
touch screens, all the sensing points on one row are multi(in this case ground), an inverting input that is coupled to the
plexedand digitizedandthis is repeated until all the rows have

node and an output that is coupled to the capacitive sensing
been sampled. Following block 284, the process flow procircuit 230. The output is coupled back to the inverting input ss ceeds to block 286 where an image or other form of data
through a capacitor. During operation, the input from the
(signal or signals) ofthe touch screen plane at one moment in
node may be disturbed by stray capacitance effects, i.e., paratime can be produced and thereafter analyzed to determine
sitic capacitance. If so, the inverting amplifier is configured to
where the objects are touching the touch screen. By way of
drive the input back to the same voltage that it had been
example, the boundaries for each unique touch can be calcupreviously before the stimulus. As such, the value of the so lated, and thereafter the coordinates thereof can be found.
parasitic capacitance doesn't matter.
Following block 286, the process flow proceeds to block 288
where the current image or signalis compared to a past image
FIG. 14 is a block diagram of a capacitive sensing circuit
or signal in order to determine a change in pressure, location,
260, in accordance with one embodiment of the present
direction, speed and acceleration for each object on the plane
invention. The capacitive sensing circuit 260 may for
example correspond to the capacitive sensing circuits 65 of the touch screen. This information can be subsequently
used to perform an action as for example moving a pointer or
described in the previous figures. The capacitive sensing circursor or making a selection as indicated in block 290.
cuit 260 is configured to receive input data from a plurality of

Copy provided by USPTO from the PIRS Image Database on 06/20/2011

APLNDC00022463

US 7,663,607 B2
19

20

FIG. 16 is a flow diagram of a digital signal processing
method 300, in accordance with one embodiment of the

calculated. This may be accomplished by performing a centroid calculation with the raw data associated with each touch
region. For example, once the touch regions are determined,
the raw data associatedtherewithmay be used to calculate the

present invention. By way of example, the method may generally correspond to block 286 shown and described in FIG.
15. The method 300 generally begins at block 302 where the 5 centroid of the touch region. The centroid may indicate the
raw data is received. The raw data is typically in a digitized

central coordinate of the touch region. By way of example,

form, and includes values for each node of the touch .
.
The values may be between 0 and 256 where 0 equates to the
highest capacitive coupling (no touch p
.) and 256

equations:

the X and Y centroids may be found using the following
Xc=T'*=Ær and

equates to the least capacitive coupling (full touch pressure). 10

An example of raw data at one point in time is shown in FIG17A. As shown in FIG. 17A, the values for each point are
provided in gray scale where points with the least capacitive

vc=Iz*ym,
rrepresents the x ceñtroid of the touch region

coupling are shown in white and the points with the highest
capacitive coupling are shown in black and the points found 15
between the least and the highest capacitive coupling are

Ye represents the y centroid of the touch region
x represents the x coordinate of each pixel or point in the

shoF

yr reesents t

g lock 302, the process flow proceeds to block

304 where the raw data is filtered. As should be appreciated,
the raw data typically includes some noise. The filtering pro- 20

y coordinate of each pixel or point in the

touch region
Z represents the magnitude (capacitance value) at each

cess is configured to reduce the noise. By way of example, a
pixel or point
noise algorithm may be run that removes points that aren't
An example of a centroid calculation for the touch regions
connected to other points. Single or uncomiected points genis shown in FIG. 17E. As shown, each touch region represents
erally indicate noise while multiple connected points genera distinct x and y coordinate. These coordinates may be used
ally indicate one or more touch regions, which are regions of 25 to perform multipoint tracking as indicated in block 312. For
the touch screen that are touched by objects. An example of a
example, the coordinates for each ofthe touch regions may be
filtered data is shown in FIG. 17B. As shown, the single
compared with previous coordinates of the touch regions to
scattered points have been removed thereby leaving several
determine positioning changes of the objects touching the
concentrated areas.
touch screen or whether or not touching objects have been
Following block 304, the process flow proceeds to block 30 added or subtracted or whether a particular object is bemg

306 where gradient data is generated. The gradient data indi.

tapped.

cates the topology of each group of connected points. The
FIGS. 18 and 19 are side elevation views of an electronic
topology is typically based on the capacitive values for each
device 350, in accordance with multiple embodiments of the
point. Points with the lowest values are steep while points
present invention. The electronic device 350 includes an LCD
with the highest values are shallow. As should be appreciated, 3s display 352 and a transparent touch screen 354 positioned
steep points indicate touch points that occurred with greater
over the LCD display 352. The touch screen 354 includes a
pressure while shallow points indicate touch points that
protective sheet 356, one or more sensmg layers 358, and a
occurred with lower pressure. An example of gradient data is
bottom glass member 360. In this embodiment, the bottom

shown in FIG. 17C.
glass member 360 is the front glass ofthe LCD display 352.
Following block 306, the process flow proceeds to block 40 Further, the sensing layers 358 may be configured for either
308 where the boundaries for touch regions are calculated
based on the gradient data. In general, a determination is
made as to which points are grouped together to form each
touch region. An example of the touch regions is shown in
FIG. 133.
45
In one embodiment, the boundaries are determinedusing a
watershed algoritlun. Generally speaking, the algorithm performs image segmentation, which is the partitioning of an
image into distinct regions as for examplethe touch regions of
multiple objects in contact with the touchscreen. The concept so
of watershed initially comes from the area of geography and
moreparticularlytopographywhereadropofwaterfallingon
a relief follows a descending path and eventually reaches a

self or mutual capacitance as described above. The sensing
layers 358 generally include a plurality ofint
ects at the
edge ofthe touch screen for coupling the sensing layer 358 to
a sensing circuit (not shown). By way ofexample, the sensing
layer 358 may be electrically coupled to the sensing circuit
through one or more flex circuits 362, which are attached to
the sides of the touch screen 354.
As shown, the LCD display 352 and touch screen 354 are
disposed within a housing 364. The housing 364 serves to
cover and support these components in their assembledposition within the electronic device 350. The housing 364 provides a space for placing the LCD display 352 and touch
screen 354 as well as an opening366 so that the display screen

minimum, and where the watersheds are the divide lines of
can be seen through the housing 364. In one embodiment, as
the domains of attracting drops of water. Herein, the water- ss shown in FIG. 18, the housing 364 includes a facade 370 for
shed lines represent the location ofpixels, whichbest separate
different objects touching the touch screen. Watershed algorithms can be widely varied. In one particular implementation, the watershed algorithm includes forming paths kom

covering the sides the LCD display 352 and touch screen 354.
Although not shown in great detail, the facade 370 is positionedaround the entire perimeterofthe LCD display 352 and
touch screen 354. The facade 370 serves to hide the intercon-

low points to a peak (based on the magnitude of each point), 60 nects leaving only the active area ofthe LCD display 352 and
classifying the peak as an ID label for a particular touch
touch screen 354 in view.
region, associating each point (pixel) on the path with the
In another embodiment, as shown in FIG. 19, the housing
peak. These steps are performed over the entire image map
364 does not include a facade 370, but rather a mask 372 that
thus carving out the touch regions associatedwith each object
is printed on interior portion of the top glass 374 of the touch
in contact with the touchscreen.
65 screen 354 that extends betweenthe sides ofthe housing 364.
Following block 308, the process flow proceeds to block
This particular arrangement makes the mask 372 look sub310 where the coordinates for each of the touch regions are
merged in the top glass 356. The mask 372 serves the same

Copy provided by USPTO from the PIRS Image Database on 06/20/2011

APLNDC00022464

US 7,663 ,607 B2
21

22

function as the facade 370, but is a more elegant solution. In
one implementation, the mask 372 is a formed from high

4. The touch panel as recited in claim 1 wherein the transparent first conductive lines of the first layer are disposed on
temperature black polymer. In the illustrated embodiment of
a first glass member, and wherein the transparent second
FIG. 19, the touch screen 354 is based on mutual capacitance
conductive lines ofthe second layer are disposed on a second
sensing and thus the sensing layer 358 includes driving lines 5 glass member, the first glass member being disposed over the
376 and sensing lines 378. The driving lines 376 are disposed
second glass member.

on the top glass 356 and the mask 372, and the sensing lines

5. The touch panel as recited in claim 4 further including a

378 are disposed on the bottom glass 360. The driving lines
third glass member disposed over the first glass member, the
and sensing lines 376 and 378 are insulated from one another
first and second glass members being attached to one another
via a spacer 380. The spacer 380 may for example be a clear 10 via an adhesive layer, the third glass member being attached
piece of plastic with optical matching materials retained
to the first glass member via another adhesive layer.
therein or applied thereto.
6. The touch panel as recited in claim 1 wherein the conIn one embodiment and referring to both FIGS. 18 and 19,
ductive lines are formed from indium tin oxide (ITO).
the electronic device 350 corresponds to a tablet computer. In
7. The touchpanel as recited in claim 1, whereinthe capacithis embodiment, the housing 364 also encloses various inte- 15 tive sensing medium is a mutual capacitance sensing
grated circuit chips and other circuitry 382 that provide commedium.
puting operations for the tablet computer. By way ofexample,
8.'Ibe touch panel as recited in claim7, further comprising
the integrated circuit chips and other circuitry may include a
a virtual ground charge amplifier coupled to the touch panel
for detecting the touches on the touch panel.
microprocessor, motherboard, Read-Only Memory (ROM),
9. The touch panel as recited in claim 1, the transparent
Random-Access Memory (RAM), a hard drive, a disk drive, 20
a battery, and various input/output support devices.
capacitive sensing medium formed on both sides of a single
While this invention has been described in terms of several
substrate.
preferred embodiments, there are alterations, permutations,
10. A display arrangement comprising:
and equivalents, which fall within the scope ofthis invention.
a display having a screen for displaying a graphical user
For example, although the touch screen was primarily 25
interface; and
directed at capacitive sensing, it should be noted that some or
a transparent touch panel allowing the screen to be viewed
all of the features described herein may be applied to other
therethrough and capable of recognizing multiple touch
sensing methodologies. It should also be noted that there are
events that occur at different locations on the touch panel
many alternative ways of implementing the methods and
at a same time and to output this information to a host
apparatuses of the present invention. It is therefore intended 30
device to form a pixilated image;
that the following appended claims be interpreted as includwherein the touch panel includes a multipoint sensing
ing all such alterations, permutations, and equivalents as fall
arrangement configured to simultaneously detect and
within the true spirit and scope of the present invention.
monitor the touch events and a change in capacitive
coupling associated with those touch events at distinct
What is claimed is:
points across the touch panel; and
1. A touch panel comprising a transparent capacitive sens- 35
wherein the touch panel comprises:
ing medium configured to detect multiple touches or near
a first glass member disposed over the screen of the
touches that occur at a same time and at distinct locations in
a plane of the touch panel and to produce distinct signals
display;
a first transparent conductive layer disposed over the
representative of a location of the touches on the plane of the
first glass member, the first transparent conductive
touch panel for each of the multiple touches, wherein the *°
layer comprising a plurality of spaced apart parallel
transparent capacitive sensing medium comprises:
lines having the same pitch and linewidths;
a first layer having a plurality of transparent first conduca second glass member disposed over the first transpartive lines that are electrically isolated from one another;
ert conductive layer;
and
a second transparent conductive layer disposed over the
a second layer spatially separated from the first layer and 45
second glass member, the second transparent conduchaving a plurality oftransparent second conductive lines
tive layer comprising a plurality of spaced apart parthat are electrically isolated from one another, the secallel lines having the same pitch and linewidths, the
ond conductive lines being positioned transverse to the
parallel lines of the second transparent conductive
first conductive lines, the intersection of transverse lines
layer being substantially perpendicularto the parallel
being positioned at different locations in the plane of the 5
lines of the first transparent conductive layer;
touch panel, each of the second conductive lines being
a third glass member disposed over the second transparoperatively coupled to capacitive monitoring circuitry;
eut conductive layer; and
wherein the capacitive monitoring circuitry is configured
one or more sensor integrated circuits operatively
to detect changes in charge coupling between the first
coupled to the lines.
conductive lines and the second conductive lines.
2. The touch panel as recited in claim 1 wherein the conductive lines on each of the layers are substantially parallel to
one another.
3. The touch panel as recited in claim 2 wherein the conductive lines on different layers are substantially perpendicular to one another.

11. The display arrangement as recited in claim 10 further
including dununy features disposed in the space between the
parallel lines, the dummy features optically improving the
visual appearance ofthe touch screen by more closely matching the optical index of the lines.

*****

Gopy provided by USPTO from the PIRS Image Database on 06/20/2011

APLNDC00022465

UNITED STATES P^r a I ^Ni l I u ni »Niv!^œK OFFICE

CERTIFICATE OF CORRECTION
PATENT NO.
APPLICATION NO.
DATED
INVENTOR(S)

:
:
:
:

7,663,607 B2
10/840862
February 16, 2010
Hotelling et al.

Page 1 of 1

It is certified that error appears in the above-identified patent and that said Letters Patent is hereby corrected as shown below:

On the Title Page:

The first or sole Notice should read --

Subject to any disclaimer, the term of this patent is extended or adjusted under 35 U.S.C. 154(b)
by 1423 days.

Signed and Sealed this
Twenty-eighth Day of December, 2010

David J. Kappos
Director ofthe United States Patent and TrademarkDjice

Copy provided by USPTO from the PIRS Image Database on 06/20/2011

APLNDC00022466