Apple Inc. v. Samsung Electronics Co. Ltd. et al

Filing 462

Declaration of DEOK KEUN Matthew Ahn IN SUPPORT OF #461 APPLES OPENING CLAIM CONSTRUCTION BRIEF PURSUANT TO PATENT L.R. 4-5 filed by Apple Inc.(a California corporation). (Attachments: #1 Exhibit A, #2 Exhibit B Part 1, #3 Exhibit B Part 2, #4 Exhibit C Part 1, #5 Exhibit C Part 2, #6 Exhibit D Part 1, #7 Exhibit D Part 2, #8 Exhibit D Part 3, #9 Exhibit D Part 4, #10 Exhibit E Part 1, #11 Exhibit E Part 2, #12 Exhibit F, #13 Exhibit G, #14 Exhibit H, #15 Exhibit I, #16 Exhibit J, #17 Exhibit K, #18 Exhibit L, #19 Exhibit M Part 1, #20 Exhibit M Part 2, #21 Exhibit N, #22 Exhibit O, #23 Exhibit P, #24 Exhibit Q)(Jacobs, Michael) (Filed on 12/8/2011) Modified on 12/9/2011 linking entry to document #461 (dhm, COURT STAFF).

Exhibit D

T(LALL TO WII(BI TIIESE PitESIOTS SILtLL C(BIE:
14

se

UNITED STATES DEPAR wmNT OF COMMERCE
United States Patent and Trademark Office

THIS IS TO CERTIFY THAT ANNEXED HpEr
THE RECORDS OF THIS OFFICE OF:

ISIA TRUE COPY FROM

U.S. PATENT: 7,812,828
ISSUE DATE: October 12, 2010

By Authority of the
Under Secretary of Commerce for Intellectual Property
and Director of the Uni
ent nd
mark Office

APLNDC00030326

Illlllllllllllllllllllllllllllllllllllllllllllllllllllllllll
(12) United States Patent

(10) Patent No.:

Westerman et al.
(54) ELLIPSE FITTING FOR MULTI-TOUCH
SUREACES

4,246,452
4,266,144
4,290,061
4,291,303
4,550,221
4,561,002
4,672,364
4,672,558

(75) Inventors: Wayne Westerman, San Francisco, CA
(US); John G. Elias, Townsend, DE
(US)
(73) Assignee: Apple Inc., Cupertino, CA (US)
( * ) Notice:

Subject to any disclaimer, the term of this
patent is extended or adjusted under 35
U.S.C. 154(b) by 707 days.

(65)

CA

1243096

Prior Publication Data
US 2007/0139395 A1
Jun. 21, 2007

Related U.S. Application Data
(63) Continuation of application No. 11/015,434, filed on
Dec. 17, 2004, now Pat. No. 7,339,580, which is a
continuation of application No. 09/236,513, filed on
Jan. 25, 1999, now Pat. No. 6,323,846.
(60) Provisional application No. 60/072,509, filed on Jan.
26, 1998.
(51) Int. Cl.
GO6F 3/041
(52) U.S. Cl.

(2006.01)
..... 345/173; 345/174; 345/175;
178/18.01; 178/18.03
(58) Field of Classification Search ......... 345/173-178;
178/18.01, 18.03, 19.01, 20.01; 715/863
See application file for complete search history.
References Cited
U.S. Pa
3,333,160
3,541,541
3,662,105
],798,370
4,237,421

A
A
A
A
A

7/
11/
5/
3/
12/

i DOCUMENTS
967
970
972
974
980

1/1981
5/1981
9/1981
9/1981
10/1985
12/1985
6/1987
6/1987

Chandler
Bristol
Serrano
Cutler et al.
Mabusth
Chiu
Lucas
Beckes et al.

FOREIGN Pa i n is i DOCUMENTS

Feb. 22, 2007

(56)

A
A
A
A
A
A
A
A

Oct. 12, 2010

(Continued)

(21) Appl. No.: 11/677,958

(22) Filed:

US 7,812,828 B2

(45) Date of Patent:

Gorski
Engelhart
Hurst et al.
Hurst
Waldron

10/1988

(Continued)

* • •-m•<PUBLICATIONS
Chang, C-C. et aL (Aug. 1, 1993). "A Hashing-Oriented Nearest
Neighbor Searching Scheme," Pattern Recognition Letters,
14(8):625-630.

(Continued)
Primary Examiner-Amare Mengistu
AssistantExaminer-KooshaSharifi

(74) Attorne Agent, or Firm-Morrison & Foerster LLP
(57)

ABS • - • T

Apparatus and methods are disclosed for simultaneously
trackingmultiple fingerandpalm contacts as hands approach,
touch, and slide across a proximity-sensing, multi-touch surface. Identification and classification of intuitive hand configurations and motions enables unprecedented integration of
typing, resting, pointing, scrolling, 3D manipulation, and
handwriting into a versatile, ergonomic computer input
device.

35 Claims, 45 Drawing Sheets

Calm DNMB
PRONIWIYlMMIE

CONTM:y
tACKINOMO
IDENTIFICAMON

18

APLNDC00030327

US 7,812,828 B2
Page 2
U.S. PATENT DOCUMENTS
4,692,809 A
4,695,827 A
4,733,222 A
4,734,685 A
4,746,770 A
4,771,276 A
4,788,384 A
4,806,846 A
4,898,555 A
4,968,877 A
5,003,519 A
5,017,030 A
5,178,477 A
5,189,403 A
5,194,862 A
5,224,861 A
5,241,308 A
5,252,951 A
5,270,711 A
5,281,966 A
5,305,017 A
5,327,161 A
5,345,543 A
5,369,228 A
,376,948 A
,379,057 A
,398,310 A
5,442,742 A
5,463,388 A
5,463,696 A
5,471,578 A
5,479,528 A
5,483,261 A
5,488,204 A
5,495,077 A
5,513,309 A
5,523,775 A
5,530,455 A
5,543,590 A
5,543,591 A
5,548,667 A
5,563,632 A
5,563,996 A
5,565,658 A
5,579,036 A
5,581,276 A
5,581,484 A
5,581,681 A
5,583,946 A
5,589,856 A
5,590,219 A
5,592,566 A
5,594,469 A
5,594,810 A
5,596,694 A
5,612,719 A
5,625,715 A
5,631,805 A
5,633,955 A
5,634,102 A
5,636,101 A
5,642,108 A
5,644,657 A
5,666,113 A
5,666,502 A
5,666,552 A
,675,361 A
,675,362 A
,677,710 A
5,689,253 A
5,710,844 A

9/ 987
9/ 987
3/ 988
3/ 988
5/ 988
9/ 988
11/ 988
2/ 989
2/ 990
11/ 990
3/ 991
5/ 991
1/ 993
2/1993
3/1993
7/ 993
8/ 993
10/ 993
12/ 993
1/ 994
4/ 994
7/2994
9/2994
11/2994
12/2994
1/2995
3/2995
8/ 995
10/ 995
10/ 995
11/ 995
12/ 995
1/ 996
1/ 996
2/ 996
4/ 996
6/ 996
6/ 996
8/ 996
8/ 996
8/ 996
20/ 996
20/ 996
20/ 996
1/ 996
-2/ 996
2/1996
2/ 996
2/ 996
2/ 996
2/ 996
/ 997
/ 997
/ 997
/ 997
3/ 997
4/ 997
5/ 997
5/ 997
5/ 997
6/ 997
6/ 997
7/ 997
9/ 997
9/ 997
9/ 997
20/ 997
0/ 997
0/ 997
1/ 997
1/ 998

Beining et al
Beining et al
Evans
Watanabe
McAvinney
Parks
Bruere-Dawsonet al
Kerber
Sampson
McAvinney et al.
Noirjean
Crews
Gambaro
Franz et al·
Edwards
Glass et al·
Young
Tannenbaum et al·
Knapp et al·
Walsh
Gerpheide
Logan et alCapps et alFaust
Roberts
Clough et al·
Tchao et al
Greyson et al.
Boie et alBeernink et al
Moran et al
Speeter
Yasutake
Mead et al
Miller et al
Meier et alCapps
Gillick et al
Gillespie et al
Gillespie et al
Tu
Roberts
Tchao
Gerpheide et al
Yates, IV
Cipolla et al·
Prince
Tchao et al·
Gourdol
Stein et al
Gourdol
Pagallo et al·
Freeman et al
Gourdol
Capps
Beernink et al
Trew et al·
Bonsall
Bozinovic et al
Capps
Bonsall et alGopher et alCapps et alLogan
Capps
Greyson et al·
Santilli
Clough et al.
Thompson-Rohrlich
Hargreaves et al.
Capps et al.

5,729,249 A
5,729,250 A
5,730,165 A
5,736,976 A
5,741,990 A
5,745,116 A
5,745,716 A
5,746,818 A
5,748,184 A *
5,748,269 A
5,764,222 A
5,767,457 A
5,767,842 A
5,790,104 A
5,790,107 A
5,802,516 A
5,808,567 A
5,809,267 A
5,821,690 A
5,821,930 A
5,823,782 A
5,825,351 A
5,825,352 A *
5,835,079 A
5,837,947 A
5,854,625 A
5,856,824 A
5,877,748 A
5,880,411 A
5,883,619 A
,886,697 A
,896,126 A *
,898,434 A
,917,476 A
5,920,309 A
5,923,319 A
5,933,134 A
5,943,043 A
5,943,044 A
5,982,302 A
5,995,106 A
6,002,389 A
6,002,808 A
6,020,881 A
6,031,524 A
6,037,882 A
6,050,825 A
6,052,339 A
6,061,177 A
6,072,494 A
6,084,576 A
6,097,371 A
6,107,997 A
6,128,003 A
6,131,299 A
6, 35,958 A
6 44,380 A
6 47,680 A *
6, 54,209 A
6, 60,551 A
6, 88,391 B
6, 98,515 B
6,208,329 B
6,222,465 B
6,239,790 B
6,243,071 B
6,246,862 B
6,249,606 B
6,255,604 B
6,278,443 B
6,288,707 B1
6,289,326 B1
6,292,178 B1

3/2998
3/1998
3/1998
4/ 998
4/ 998
4/ 998
4/ 998
5/ 998
5/ 998
5/ 998
6/ 998
6/1998
6/1998
8/1998
8/1998
9/ 998
9/ 998
9/ 998
0/ 998
0) 998
0/ 998
0) 998
0) 998
1) 998
11/1998
12/ 998
1/ 999
3/ 999
3/ 999
3/ 999
3/ 999
4/1999
4/1999
6/ 999
7/ 999
7) 999
8/ 999
8/ 999
8/1999
11/1999
11/1999
12/1999
12/1999
2/2000
2/2000
3/2000
4/2000
4/2000
5/2000
6/2000
7/2000
8/2000
8/2000
0/2000
0/2000
0/2000
1/2000
1/2000
1/2000
2/2000
2/2001
3/2001
3/200
4/200
5/200
6/200
6/200
6/200
7/200
8/200
9/200
9/200
9/200

Yasutake
Bishop et al.
Philipp
Cheung
Davies
Pisutha-Amond
Tchao et al.
Yatake
Shieh .........................
Harris et al.
Shieh
Gerpheide et al.
Korth
Shieh
Kasser et al.
Shwarts et al.
McCloud
Moran et al.
Martens et al.
Hansen
Marcus et al.
Tam
Bisset et al. ................
Shieh
Teterwak
Frisch et al.
Shieh
Redlich
Gillespie et al.
Ho et al.
Naughton et al.
Shieh
Small et al.
Czerniecki
Bisset et al.
Bishop et al.
Shieh
Furubata et al.
Martinelli et al.
Ure
Naughton et al.
Kasser
Freeman
Naughton et al.
Kunert
Levy
Nichol et al.
Frenkel et al.
Fujimoto
Nguyen
Leu et al.
Siddiqui et al.
Ure
Smith et al.
Raab et al.
Mikula-Curtis et al.
Schwarts et al.
Tareev
.............
Naughton et al.
Naughton et al.
Seely et al.
Cole
Ballare
Kumar et al.
Martinelli et al.
Shwarts et al.
Grivas et al.
Kiraly et al.
Tokioka et al.
Amro et al.
Philipp
LaFleur
Bernstein et al.

345/173

345/173

345/173

345/173

APLNDC00030328

US 7,812,828 B2
Page 3
6,323,846
6,323,849
6,344,861
6,347,290
6,377,009
6,380,931
6,411,287
6,414,671
6,421,234
6,452,514
6,457,355
6,466,036
6,515,669
6,525,749
6,535,200
6,543,684
6,543,947
6,570,557
6,593,916
6,610,936
6,624,833
6,639,577
6,650,319
6,658,994
6,670,894
6,677,932
6,677,934
6,724,366
6,757,002
6,803,906
6,842,672
6,856,259
6,888,536
6,900,795
6,927,761
6,942,571
6,965,375
6,972,401
6,977,666
6,985,801
6,992,659
7,031,228
7,084,859
7,129,416
7,240,289
RE40,153
7,339,580
2002/0118848
2002/0140668
2003/0006974
2003/0076301
2003/0076303
2003/0076306
2003/0095095
2003/0095096
2003/0098858
2003/0132950
2003/0164820
2003/0206202
2003/0234768
2004/0146688
2004/0263484
2005/0012723
2005/0052425
2005/0104867
2005/0110768
2006/0022955
2006/0022956
2006/0026521
2006/0026535
2006/0026536
2006/0032680
2006/0033724

BI
B
B
B
B
B
B
B
B
B
B
B
B
B
B2
B1
B2
Bl
B1
B2
B1
B2
B1
B1
B2
B1
B1
B2
B
B
B
B
B2
B1
B2
B1
B1
B2
B1
B1
B2
B2
B1
B1
B2
E
B2
A
A
A
A
A
A
A
A
A
A1
Al
A1
Al
Al
Al
Al
A
A
A
A
A
A
A
A
A
A

11/2001
11/2001
2/2002
2/2002
4/2002
4/2002
6/2002
7/2002
7/2002
9/2002
10/2002
10/2002
2/2003
2/2003
3/2003
4/2003
4/2003
5/2003
7/2003
8/2003
9/2003
0/2003
1/2003
2/2003
2/2003
1/2004
1/2004
4/2004
6/2004
10/2004
1/2005
2/2005
5/2005
5/2005
8/200
9/200

11/200
12/2000
12/200 i
1/2006
1/2006
4/2006
8/2006
10/2006
7/2007
3/2008
3/2008
8/2002
10/2002
1/2003
4/2003
4/2003
4/2003
5/2003
5/2003
5/2003
7/2003
9/2003
11/2003
12/2003
7/2004
12/2004
1/2005
3/2005
5/2005
5/2005
2/2006
2/2006
2/2006
2/2006
2/2006
2/2006
2/2006

Usamaas

Westerman et al.
He et al.
Naughton et al.
Bartlett
Philipp
Gillespie et al.
ScharKet al.
Gillespie et al.
Ricks et al.
Philipp
Philipp
Philipp
Mohri
Moran et al.
Philipp
White et al.
Lee
Westerman et al.
Aroyan
Gillespie et al.
Kumar et al.
Eberhard
Hurst et al.
McMillan
Mehring
Westerman
Blanchard
Crawford
Oross et al.
Morrison et al.
Straub et al.
Sharp
Westerman et al.
Knight, III et al.
Badaye et al.
McAllister et al.
Gettemy et al.
Akitt et al.
Hedrick
Straub et al.
Gettemy
Bom et al.
Pryor
Steinfeld et al.
Naughton et al.
Westerman et al.
Westerman et al.
Karpenstein
Crawford
Clough et al.
Tsuk et al.
Huppi
Zadesky et al.
Pihlaja
Robbin et al.
Perski et al.
Surucu et al.
Kent
Moriya
Rekimoto et al.
Treat
Montysalo et al.
Pallakofy
Zadeski et al.
Westerman et al.
Marriott et al.
Kennedy
Lengeling et al.
Hotelling et al.
Hotelling et al.
Hotelling et al.
Elias et al.
Chaudhri et al.

Eaas i IRDTT3

----387
2006/0066582
2006/0085757
2006/0097991
2006/0197753
2006/0232567
mvwv ¤8517
-8518
2006/0238519
2006/0238520
"^^"^" 8521
2006/0238522
2007/0070050
2007/0070051
2007/0070052
2007/0078919
2007/0081726
2007/0229464
2007/0236466
2007/0247429
2007/0257890
2007/0268273
2007/0268274
2007/0268275
2008/0041639
2008/0042986
2008/0042987
2003/0042988
2008/0042989
2008/0128182
onnoinn31489
2009/0160816

A1
Al
A1
Al
A1
Al
A1
Al
A
A
A
A
A
A
A
A
A
A
A
A
A1
A
A
A
A
A
A
A
A
A
A
A

3/2006
3/2006
4/2006
5/2006
9/2006
0/2006
0/2006
0/2006
0/2006
10/2006
10/2006
10/2006
3/2007
3/2007
3/2007
4/2007
4/2007
0/2007
0/2007
0/2007
1/2007
1/2007
1/2007
1/2007
2/2008
2/2008
2/2008
2/2008
2/2008
6/2008
1/2009
6/2009

Ording
Lyon et al.
Andre et al.
Hotelling et al.
Hotelling
Westerman et al.
King et al.
Westerman et al.
Westerman et al.
Westerman et al.
Westerman et al.
Westerman et al.
Westerman et al.
Westerman et al.
Westerman et al.
Westerman et al.
Westerman et al.
Hotelling et al.
Hotelling
Westerman
Hotelling et al.
Westerman et al.
Westerman et al.
Westerman et al.
Westerman et al.
Westerman et al.
Westerman et al.
Westerman et al.
Westerman et al.
Westerman et al.
Westerman et al.
Westerman et al.

2009/0244031
2009/0244032
2009/0244033
9^^ In349236

A
A
A
A

0/2009
0/2009
0/2009
0/2009

Westerman et al.
Westerman et al.
Westerman et al.
Westerman et al.

9^^°in"1435 A1
2009/0251438 A1
2009/0251439 Al

10/2009 Westerman et al.
10/2009 Westerman et al.
10/2009 Westerman et al.

FOREIGN P
DE
EP
EP
EP
EP
EP
EP
EP
EP
EP
EP
EP
GB
JP
JP
JP
JP
JP
JP
JP
JP
JP
WO
WO
WO
WO
WO
WO
WO
WO
WO

DIDO Imm---

10251296
0 546 704
0 546 704
0288692
0 622 722
0 622 722
0 622 722
0664504
0464908
0 817 000
0 817 000
1014295
2330670
02-056017
02-304613
04-233619
06-083523
06-332620
07-057103
07-129312
08-286830
2000-163031
WO-92/02000
WO-96/07981
WO-97/18547
WO-97/23738
WO-97/36225
WO-97/40744
WO-98/14863
WO-99/38149
,,,.,-22,2..176

A1
A2
A3

A2
A3
B1
Al
B1
A
A
A
A
A
A
A
A
A
A

A1

A1

==

S

5/2004
6/1993
6/ 993
7) 993
1 / 994
1 / 994
1 / 994
/ 995
9/ 996
1/ 998
1/ 998
1/2002
4/1999
2/1990
12/1990
8/ 992
3/ 994
12/ 994
3/ 995
5/ 995
11/ 996
6/2000
2/1992
3/1996
5/1997
7/ 997
10/ 997
11/ 997
4/1998
7/1999
10/2003

APLNDC00030329

US 7,812,828 B2
Page 4
WO

WO2006/023569

3/2006

OTHER PUBLICATIONS
Crowley, J.L. (Mar. 1, 1997). "Vision for Man-MachineInteraction,"
Robotics andAutonomous Systems, 19(34):347-358.
Davis, J. et al. (May 2, 1994). "Recognizing Hand Gestures," Euro
pean Conference on Computer Vision, Stockholm, SE, 1:331-340.
European Search Report mailed Dec. 12, 2008, for EP Application
No. 06016855.6 filed Jan. 25, 1999, six pages.
European Search Report mailed Dec. 23, 2008, for EP Application
No. 06016831.7 filed Jan. 25, 1999, seven pages.
European Search Report mailed Jan. 9, 2009, for EP Application No.
06016832.5 filed Jan. 25, 1999, four pages.
Heap, T. et al. (Oct. 14, 1996). "Towards 3D Hand Tracking Using a
Deformable Model," Pmceedings of the 2* International Conference, Killington, VT, USA, Oct. 14-16, 1996, Automatic Face and
Gesture Recognition, IEEE Comput. Soc., pp. 140-145.
Intemational Search Report mailed Dec. 28, 2005, for PCT Application No. PCT/USOS/03325, filed Feb. 3, 2005, two pages.
Nirei, K. et al. (Aug. 5, 1996). "Human Hand Tracking from Binocular Image Sequences," Proceedings ofthe 1996 IEEE IECON 22nd
International Conference, Tarpei, Taiwan, Aug. 5-10, 1996, Indurtrial Electronics, Control, and Instrumentation 1(5):297-302.
Non-Final Office Action mailed Sep. 21, 2007, for U.S. Appl. No.
11/428,515, filed Jul. 3, 2006, seven pages.
Non-Final Office Action mailed Sep. 24, 2007, for U.S. Appl. No.
11/428,506, filed Jul. 3, 2006, six pages.
Non-Final Office Action mailed Sep. 24, 2007, for U.S. Appl. No.
11/428,521, filed Jul. 3, 2006, six pages.
Non-Final Office Action mailed Jan. 28, 2008, for U.S. Appl. No.
11/428,522, fded Jul. 3, 2006, seven pages.
Non-Final Office Action mailed Feb. 4, 2008, for U.S. Appl. No.
11/428,515, filed Jul. 3, 2006, six pages.
Non-Final Office Action mailed Jul. 9, 2008, for U.S. Appl. No.
11/428,521, filed Jul. 3, 2006, 11 pages.
Non-Final Office Action mailed Sep. 2, 2008, for U.S. Appl. No.
11/428,522, filed Jul. 3, 2006, six pages.
Non-Final Office Action mailed Sep. 15, 2003, for U.S. Appl. No.
11/428,506, filed Jul. 3, 2006, eight pages.
Non-Final Office Action mailed Oct. 31, 2008, for U.S. Appl. No.
11/428,515, filed Jul. 3, 2006, seven pages.
Non-Final Office Action mailed Feb. 17, 2009, for U.S. Appl. No.
11/428,522, filed Jul. 3, 2006, six pages.
Notice of Allowability mailed Feb. 11, 2009, for U.S. Appl. No.
11/428,521, filed Jul. 3, 2006, five pages.
U.S. Appl. No. 11/332,861, filed Jan. 13, 2006 which is a Reissue of
6,677,932.
Westerman, W. (Jan. 1, 1999)."HandTracking, Finger Identification,
and Chordic Manipulation on a Multi-Touch Surface," Dissertation,
University of Delaware, pp. 1-333.
Final Office Action mailed Mar. 19, 2009, for U.S. Appl. No.
11/428,506, filed Jul. 3, 2006, seven pages.
Non-Final Office Action mailed Mar. 18, 2009, for U.S. Appl. No.
11/428,515, filed Jul. 3, 2006, 12 pages.
Non-Final Office Action mailed Apr. 2, 2009, for U.S. Appl. No.
11/428,501, filed Jul. 3, 2006, 11 pages.
Non-Final Office Action mailed Apr. 2, 2009, for U.S. Appl. No.
11/428,503, filed Jul. 3, 2006, 12 pages.
Notice of Allowability mailed Jul. 10, 2009, for U.S. Appl. No.
11/428,521, filed Jul. 3, 2006, four pages.
Rubine, D. (May 1992). "Combining Gestures and Direct Manipulation," CHI '92, pp. 659-660.
U.S. Appl. No. 2/342,027, filed Dec. 22, 2008, by Westerman et al.
U.S. Appl. No. 2/422,197, filed Apr. 0, 2009, by Westerman et al.
U.S. Appl. No. 2/422,205, filed Apr. 0, 2009, by Westerman et al.
U.S. Appl. No. 12/422,212, filed Apr. 0, 2009, by Westerman et al.
U.S. Appl. No. 2/422,222, filed Apr. 0, 2009, by Westerman et al.
U.S. Appl. No. 2/422,225, filed Apr. 0, 2009, by Westerman et al.
U.S. Appl. No. 2/434,439, filed May 1, 2009, by Westerman et al.
U.S. Appl. No. 2/479,573, filed Jun. 5, 2009, by Westerman et al.
U.S. Appl. No. 2/479,617, filed Jun. 5, 2009, by Westerman et al.
U.S. Appl. No. 2/479,678, filed Jun. 5, 2009, by Westerman et al.

Cui, Y. et al. (1996). "Hand Segmentation Using Learning-Based
PredictionandVerificationfor Hand Sign Recognition,"Proceedings
ofthe1996]EEE ComputerSociety Conference on Computer Vision
and Pattern Recognition, pp. 88-93.
Final OBce Action mailed Nov. 19, 2009, for U.S. Appl. No.
11/428,515, filed Jul. 3, 2006, 14 pages.
Non-Final Office Action mailed Aug. 25, 2009, for U.S. Appl. No.
11/428,522, filed Jul. 3, 2006, six pages.
Non-Final Office Action mailed Sep. 2, 2009, for U.S. Appl. No.
11/559,736, filed Nov. 14, 2006, 12 pages.
Non-Final Office Action mailed Oct. 5, 2009, for U.S. Appl. No.
11/559,799, filed Nov. 14, 2006, 14 pages.
Non-Final OMce Action mailed Oct. 6, 2009, for U.S. Appl. No.
11/559,763, filed Nov. 14, 2006, 24 pages.
Non-Final OMce Action mailed Oct. 14, 2009, for U.S. Appl. No.
11/428,501, filed Jul. 3, 2006, six pages.
Non-Final OBce Action mailed Oct. 29, 2009, for U.S. Appl. No.
11/559,822, filed Nov. 14, 2006, 11 pages.
Non-Final OMce Action mailed Oct. 30, 2009, for U.S. Appl. No.
11/428,503, filed Jul. 3, 2006, nine pages.
Non-Final Office Action mailed Dec. 22, 2009, for U.S. Appl. No.
11/559,833, filed Nov. 14, 2006, six pages.
Non-Final OMce Action mailed Jan. 27, 2010, for U.S. Appl. No.
11/428,522, filed Jul. 3, 2006, five pages.
Notice of Allowability mailed Sep. 2, 2009, for U.S. Appl. No.
11/428,506, filed Jul. 3, 2006, five pages.
Rubine, D.H. (Dec, 1991). "The Automatic Recognition of Gestures," CMU-CS-91-202, Submitted in Partial Fulfillment of the
Requirements of the Degree of Doctor of Philosophy in Computer
Science at Camegie Mellon University, 285 pages.
U.S. Appl. No. 10/654,108, filedSep. 2, 2003 entitled "Ambidextrous
Mouse".
U.S. Appl. No. 10/789,676, filed Feb. 27, 2004 entitled "Shape
Detecting Input Device".
"4-Wire Resistive Touchscreens" obtained from http://www.
touchscreens.com/intro-touchtypes-4resistive.html generated Aug.
5, 2005.
"5-Wire Resistive Touchscreens" obtained from http://www.
touchscreens.comlintro-touchtypes-resistive.html generated Aug. 5,
2005.
"A Brief Overview of Gestme Recognition" obtained from http://
www.dai.ed.ac.uk/Cvonline/LOCA COPIES/COHEN/
gesture_overview.html, generated Apr. 20, 2004.
"Capacitive Touchscreens" obtained from http://www.touchscreens.
com/intro-touchtypes-capacitive.html generated Aug. 5, 2005.
"Capacitive Position Sensing" obtained from http://www.synaptics.
com/technology/cps.cfm generatedAug. 5, 2005.
"Comparing Touch Technologies" obtained from http://www.
touchscreens.com/intro-touchtypes.html generated Oct. 10, 2004.
"GlidePoint®" obtained from http://www.cirque.com/technology/
technologyg.html generatedAug. 5, 2005.
"How do touchscreen monitors know where you're touching?"
obtained from http //www.electronics.howstuffworks.com/question716.html generated Aug. 5, 2005.
"How does a touchscreen work?" obtained from http://www.
touchscreens.comlintro-anatomy.htmlgenerated Aug. 5, 2005.
"Infrared Touchscreens" obtained from http://www.touchscreens.
com/intro-touchtypes-infrared.htmlgenerated Aug. 5, 2005.
"Mouse Emulation" FingerWorks obtained from http://www.
fmgerworks.com/gesture_guide_mouse.html generated Aug. 30,
2005.
"Mouse Gestures in Opera" obtained from http://www.opera.com/
products/desktop/mouse/index.dml generated Aug. 30, 2005.
"Mouse Gestures," Optim oz, May 21, 2004.
"MultiTouch Overview" FingerWorks obtained from http://www.
fmgerworks.com/multoverview.html generated Aug. 30, 2005.
"Near Field Imaging Touchscreens" obtained from http://www.
touchscreens.com/intro-touchtypes-nfi.htmlgeneratedAug. 5, 2005.
"PenTouch Capacitive Touchscreens" obtained from http://www.
touchscreens.com/intro-touchtypes-pentouch.html generated Aug.
5, 2005.

Conv nrovided hv USPTO from the PIRE Imana natakaan an nal9RIondi

APLNDC00030330

US 7,812,828 B2
Page 5
"Surface Acoustic Wave Touchscreens" obtained from http://www.
touchscreens.com/intro-touchtypes-saw.html generated Aug. 5,

2005.
"Symbol Commander" obtained from http://www --- - com/
symbolcomanderl, generated Aug. 30, 2005.
"Tips for Typing" FingerWorks http://www.fingerworks.com/
mini_typing.html generated Aug. 30, 2005.
"Touch Technologies Overview" 2001, 3M Touch Systems, Massachusetts.
"Wacom Components-Technology" obtained from http://www.
wacom-components.com/english/tech.asp generated on Oct. 10,

2004.
"WatershedAlgorithm"http://rsb.info.nih.gov/ij/plugins/watershed.
html generated Aug. 5, 2005.
"FingerWorksdesture Guide-Application Switching," obtained
from http://www.fmgerworks.com/gesturegide apps.html, generated on Aug. 27, 2004, 1-pg.
"FingerWorksdesture Guide-Editing," obtained from http://
www.fmgerworks.com/gesure_guide_editing.html, generated on
Aug. 27, 2004, 1-pg.
"FingerWorkaesture Guide-File Operations" obtained from
http://www.fmgerworks.com/gesture_guide_files.html, generated
on Aug. 27, 2004, 1-pg.
"FingerWorks-<iestureGuide-Text Manipulation" obtained from
http://www.fmgerworks.com/gesture_guide_text._manip.html,
generated on Aug. 27, 2004, 2-pg.
"FingerWorks-fresture Guide-Tips and Tricks," obtained from
http://www.fmgerworks.com/gesturejde tips.html, generated
Aug. 27, 2004, 2-pgs.
"FingerWorks-Gesture Guide-Web," obtained from http://www.
fmgerworks.com/gesture_guide web.htmi, generated on Aug. 27,
2004, 1-pg.
"FingerWorknuide to Hand Gestures for USB Touchpads,"
obtained from http://www.fmgerworks.com/igesture_userguide.
html, generated Aug. 27, 2004, 1-pg.
"FingerWorkWGesture-Technical Details," obtained from http //
www.fmgerworks.com/igesture_tech.html, generated Aug. 27,
2004, 1-pg.
"FingerWorks-The Only Touchpads with Ergonomic Full-Hand
Resting and Relaxation!" obtained from http://www.fmgerworks.
com/resting.html, Copyright 2001, 1-pg.
"FingerWorks-Tips for Typing on the Mini," obtained from http //
www.fmgerworks.com/mini_typing.html, generated on Aug. 27,
2004, 2-pgs.
"iGesture Padde MultiFinger USB TouchPad with Whole-Hand
Gestures," obtained from http://www.fingerworks.comligesture.
html, generated Aug. 27, 2004, 2-pgs.
Bier, et al., "Toolglass and Magic Lenses: The see-through interface"
In James Kijiya, editor, Computer Graphics (SIGGRAPH '93 Proceedings), vol. 27, pp. 73-80, Aug. 1993.
Buxton, W. et al. (Jul. 22, 1985). "Issues and Techniques in TouchSensitive Tablet Input," Pmceedings ACM Siggraph, pp. 215-224.
Cutler, L.D. et al. (Apr. 27, 1997). "Two-Handed Direct Manipulation on the Responsive Workbench," Proceedings ofthe 1997 Symposium on Interactive 3D Graphics, Providence, RI, Apr. 27-30,
1997, pp. 107-114.
Davis, J. et al. (Oct. 31, 1994). "Determining 3-D Hand Motion,"
Signals, Systems andComputers, 1994 ConferenceRecordofthe 28"'
Asilomar Conference on Pacgic Gove, CA, Oct. 31-Nov. 2, 1994,
Los Alamitos, CA, pp. 1262-1266.
Douglas et al., The Ergonomics of Computer Pointing Devices
(1997).
European Search Report received in EP 1 621 989 (@ Beye
Weaver & Thomas, LLP) dated Mar.27, 2006.
EVB Elektronik "TSOP6238 IR Receiver Modules for Infrared
Remote Control Systems" dated Jan. 2004 l-pg.
Fisher et al., "Repetitive Motion Disorders: The Design of Optimal
Rate-Rest Profiles," Human Factors, 35(2):283-304 (Jun. 1993).
Fukumoto, et al., ''ActiveClicle Tactile Feedback for Touch Panels,"
In CHI 2001 Summary, pp. 121-122, 2001.
Fukumoto and Yoshinobu Tonomura, "Body Coupled Fingering:
Wireless Wearable Keyboard" CHI97, pp. 147-154 (Mar. 1997).
Hardy, "Fingerworks" Mar. 7, 2002; BBC World On Line.

Hillier and Gerald J. Lieberman, Intmduction to Operations

Research (1986).
Jacob et al., "Integrality and Separability of Input Devices," ACM
7Fansactions on Computer-Iluman Interaction, 1:3-26 (Mar. 1994).
Kinkley et al., "Touch-Sensing Input Devices," in CHI '99 Proceedings, pp. 223-230, 1999.
KIONX "KXPS4 Series SummaryData Sheet" copyright 2005,dated
Oct. 21, 2005, 4-pgs.
Lee et al., "A Multi-Touch Three Dimensional Touch-Sensitive Tablet," in CHI '85 Proceedings, pp. 121-128, 2000.
Lee, "A Fast Multiple-Touch-SensitiveInput Device," Master's The-

sis, University of Toronto (1984).
Lee, S.K. et al. (Apr. 1985). "A Multi-Touch Three Dimensional
Touch-Sensitive Tablet," Proceedings of CHI: ACM Conference on
Human Factors in ComputingSystems, pp.21-25.
Matsushita et al., "HoloWall: Designing a Finger, Hand, Body and
Object Sensitive Wall," In Proceedings of UIST '97, Oct. 1997.
Pavlovic, V.I. et al. (Jul. 1997). "Visual Interpretation of Hand Gestures for Human-Computer Interaction: A Review," IEEE Transactions on Pattern Analysis andMachine Intelligence 19(7):677-695.
Quantum Research Group "QTS10 / QV/l-le Touch Slider IC"

copyright 2004-2005, 14-pgs.
Quek, "Unencumbered Gestural Interaction," IEEE Multimedia,
3:3647 (Winter 1996).
Radwin, "Activation Force and Travel Effects on Overexertion in
Repetitive Key Tapping," 11uman Factors, 39(1):130-140 (Mar.
1997).
Rekimoto "SmartSkin An Infrastructure for Freehand Manipulation
on Interactive Surfaces" CHI 2002, Apr. 20-25, 2002.
Rekimoto et al., "ToolStone: Effective Use of the Physical Manipulation Vocabularies of Input Devices," In Proc. of UIST 2000, 2000.
Rubine et aL, "Programmable Finger-Tracking Instrument Controllers," ComputerMusic Journal, vol. 14, No. 1 (Spring 1990).
Rutledge et al., "Force-To-Motion Functions for Pointing," HumanComputer Interacti- mmm wr (1990).
Subatai Ahmad, "A Usable Real-Time 3D Hand Tracker," Proceedings of the 28th Asilomar Conference on Signals, Systems and
Computers
2 (of2), vol. 2 (Oct. 1994).
Texas Instruments - - ----- / I2C Touch Screen Controller" Data
Sheet SBAS 162, dated Oct. 2001, 20-pgs.
Wellner, "The Digital Desk Calculators: Tangible Manipulation on a
Desk Top Display" In ACM UIST '91 Proceedings, pp. 27-34, Nov.

199L
Williams, "Applications for a Switched-Capacitor Instrumentation
Building Block" Linear Technology Application Note 3, Jul. 1985,
pp. 1-16.
Yamada et al., "A Switched-Capacitor Interface for Capacitive Pressure Sensors" IEEE Transactions on Instrumentation and Measurement, vol. 41, No. 1, Feb. 1992, pp. 81-86.
Yeh et al., "Switched Capacitor Interface Circuit for Capacitive
Transducers" 1985 IEEE.
Zhai et al., "Dual Stream Input for Pointing and Scrolling," Pmceedings ofCIH'97ExtendedAbstracts (1997).
Zimmerman et al., "Applying Electric Field Sensing to HumanComputer Interfaces," In CHI '85 Proceedings, pp. 280-287, 1995.
International Search Report received in corresponding PCT application No. PCT
-- - 8349 dated Oct. 6, 2006.
International search report for PCT/US99/01454, mailed May 14,
1999.
European supplementarysearchreport for European Patent Application No. 99904228.6, mailed Feb. 16, 2005.
SupplementaryEuropeansearchreportforU.S.Appl.No.99904228.
6, mailed Jul. 8, 2005.
European examination report for European patent Application No.
99904228.6, mailed Apr. 20, 2006.
European examination report for European Patent Application No.
99904228.6, mailed Mar. 23, 2007.
Extended European search report for European Patent Application
No.06016858.0, mailed Dec. 21, 2007.
Extended European search report for European Patent Application
No.06016856.4, mailed Mar. 14, 2008.
European examination report for European Patent Application No.
06016830.9, mailed Aug. 6, 2008.

Conv nrovidad hv um .o from the PIRE Imana Datakaan on niinic/ 11

APLNDC00030331

US 7,812,828 B2
Page 6
European search report for European Patent Application No.
06016830.9 mailed Dec. 3, 2007.
European examination report for European Patent Application No.
06016856.4 mailed Sep. 16, 2008.
Notice of Allowance mailed Mar. 23, 2010, for U.S. Appl. No.
11/428,501, filed Jul. 3, 2006, eight pages.

European Search Report mailed Feb. 17, 2010, for EP Application
No. 06016857.2, six pages.
European Search Report mailed Feb. 24, 2010, for EP Application
No. 06016833.3, nine pages.

Sun Microsystems. (1992). "The Star7 PDA Prototype," located at
<http://www.youtube.com/watch?v=Ahg8OBYixLO, last visited
Jan. 15, 2010, seven pages.
Tomita, A. Jr. et al. (Nov.9, 1997)."An Image Processing Method for
Use in a GUI for the Visually Impaired," Proceedings of the1ECON
'97: 23"InternationalConference on IndustrialElectronics, Control
and Instrumentation, New Orleans, LA, Nov. 9-14, 1997, pp. 264268.

* cited by

i

Conv nrovided bv USÞTO from tha ÞiRR Imana Dafnhuma nn ORIOKlondi

APLNDC00030332

U.S. Patent

Oct. 12, 2010

Sheet 1 of 45

US 7,812,828 B2

4

| ELECTRODE
i
SCANNING
HARDWARE

2

6

CALIBRATION AND
PROXIMITY IMAGE
FORMATION

CONTACT
TRACKlNG AND
IDENTIFICATION

12
I

E

10

14

YPING
RECOGNIZER

FINGER
SYNCHRONIZATION
DETECTOR

M
COMPONENT
EXTRACTION

16
PENGRIP
DETECTOR
17

CHORD MOTION y 18
RECOGNIZER

24
DISPLAY

22
HOST
COMP•~- •
SYSTEM

HOST
COMMUNICATION
INTERFACE

20

FIG. 1

Conv nrovided bv ----- - from the PIRS Immae Database on 04/25/2011

APLNDC00030333

v

33

U.S. Patent

v

32

Oct. 12, 2010

Sheet 2 of 45

v

35

US 7,812,828 B2

38

46

45

34

58

48

30

31

36

37

FIG. 2

Conv nrovided hv

from tha DIRR Imana

na narecienn

APLNDC00030334

34

y

4

U.S. Patent

Oct. 12, 2010

Sheet 3 of 45

US 7,812,828 B2

38

32

39

33

40

46

45

30

.

31

V

58

41

FIG. 3A

38
32
46

33

34

69
45

39

48
30

31

V

41

FIG. 3B

Copy provided by USPTO from the PIRS Imaae Database on 04/25/2011

APLNDC00030335

F43
337

y

U.S. Patent

Oct. 12, 2010

48

37
35

55

Sheet 4 of 45

58

US 7,812,828 B2

38
32
46
34

30
u_J

31

45

36

U-1

FIG. 4A

55

43

44
TIME

FIG. 4B

Copy provided by USPTO from the PIRS Image Database on 04/25/2011

APLNDC00030336

I

U.S. Patent

/

Oct. 12, 2010

Sheet 5 of 45

46

US 7,812,828 B2
44
43

46

- -

44

47
30

48

31

33

32

45

45

FIG. 5A
46

44

43

43

47

45

45

46

44

FIG. 5B

l

Copy provided by USPTO from the PIRS Image Database on 04/25/2011

APLNDC00030337

44a
4h

47

44b

47

44c

47

44d

47

44e

47

44f

47

44g

47

44h

(D

47

4
c

<

ggggg474747
43e

43c

5

gggg4Ïggg

4

43a

35

ERRERREE

58

34
v

55

46

FIG. 6

9

36

37

oc

APLNDC00030338

'

U.S. Patent

Oct. 12, 2010

I

Sheet 7 of 45

US 7,812,828 B2

58

70

65

6

68

FIG. 7A

71

58a-h
67

65

70
69

3
68

66

64

FIG. 7B

Copy provided by war .« from the PlHS Image Database on 04/25/2011

APLNDC00030339

78
76

g
75

g
a

E

g
b

47

81
c

47

81
4d

47

81 - M
44e

47

44f
47

M
44g

47

78
44h 75

47 45h-

ERREEEEE45a
y

g

g

g

g

41

41

7
58h
58g

E 45f

8f

58d

g

R

.41

41

E

E 5c

8c

ggg41REME45

8b
37 58a

g474747RRRE
43

35
55

34

v

46

oc

APLNDC00030340

75

U.S. Patent

Ri

oct.12, 2010

75

Sheet 9 of 45

78

US 7,812,828 B2

78

V

76

77
10

GI

80
75

75

81

FIG. 9A

44

78

78

76

77

F/G. 98

44

APLNDC00030341

U.S. Patent

85

oct.12, 2010

Sheet 10 of 45

US 7 812,828 B2

1

86 -T............... ..

33

FIG. 10

GODV Drovided bv uur nas from the PIRS Immae Database on 04/25/2011

\7

APLNDC00030342

U.S. Patent

oct. 12, 2010

Sheet 11 of 45

US 7,812,828 B2

85 x
90-"

33

FIG. 11

Conv nrovided bv · ---- - from the PIRS Immae Database on 04/25/2011

APLNDC00030343

U.S. Patent

Oct. 12, 2010

Sheet 12 of 45

US 7,812,828 B2

33
85

88

FIG. 12

APLNDC00030344

u. 6
- O

U.S. Patent

Oct. 12, 2010

Sheet 13 of 45

14 -

20

US 7,812,828 B2

4

12

E 10
o

202

>- 8

20§

209
201

2 4

O

z

O

O

206

u.i -2

207

-4

-6,

0

2

4
6
8
10
12
14
16
HORIZONTAL POSITION ON SURFACE (X AXIS cm)

18

FIG. 13

Copy provided by USPTO from the PIRS Image Database on 04/25/2011

APLNDC00030345

U.S. Patent

Oct. 12, 2010

Sheet 14 of 45

US 7,812,828 B2

1412
E

o 10

202

203

204

205

8
u.I
Le

6

2

4

O

201

2

O
O

O

-4
-6,

·

0

2

4
6
8
10
12
14
16
HORIZONTAL POSITION ON SURFACE (X AXlS cm)

18

FIG. 14

CoDV Drovided bv USPTO from the PIHS Imaae Database on 04/25/2011

APLNDC00030346

-6 ,

'·

U.S. Patent

Oct. 12, 2010

Sheet 15 of 45

·

US 7,812,828 B2

1412

E

202

o10

(

8
203

4

204

205

201

O

-02

207

206

8

1012141618

-4

0

2

4

6

HORIZONTAL POSITION ON SURFACE (X AXIS cm)

FIG. 15

nnu nrnvidad hu I

.a from the WIRR Imana Datahame on OADENB11

APLNDC00030347

U.S. Patent

Oct. 12, 2010

Sheet 16 of 45

US 7,812,828 B2

240
241

243
PATHS FROM
PREVIOUS IMAGES

IMAGE
SEGMENTATION

PARAMe onlZED
LECTRODE GROUP

242
CONTACT PATH
TRACKING

y 245

EW PATHS &

246

UPDATED PATH
PARAMETERS
247

248

HAND
IDENTIFICATION

FINGER & PALM
IDENTIFICATION

IDENTIFIED
CONTACT PATHS

250

HAND POSITION y 251
ESTIMATION
252
ESTIMATED HAND &
FINGER OFFSETS

FIG. 16

C

sidad hu ilRDTf3 fram tha DIRR Imana ¯

nn It&IORIDilid

APLNDC00030348

U.S. Patent

Oct. 12, 2010

Sheet 17 of 45

US 7,812,828 B2

( START )
GET HAND'S
IDENTIFIED PATHS

y 250

COMPUTE OFFSETS BL I V¥EEN

254

EACH FINGER'S MEASURED AND
DEFAULT POSITIONS

COMPUTE AVERAGE OF
OFFSETS WEIGHTED BY
CONTACT PROXIMITY

255

ADJUST FILTER POLE TO
CURRENT IDENTIFICATION
CONFIDENCE

256

COMPUTE WEIGHTED
AVERAGE OF HAND

257

CONTACT VELOCITIES
AUTOREGRESSIVELY UPDATE HAND
OFFSET ESTIMATES FROM MEASURED
OFFSETS AND VELOCITIES
UPDATE FINGER
OFFSET ESTIMATES

259

CONVERT ESTIMATED OFFSETS
TO ABSOLUTE POSITIONS
(

END

258

260

)

FIG. 17

APLNDC00030349

U.S. Patent

Oct. 12, 2010

Sheet 18 of 45

240

CURRENT
PROXIMITY
IMAGE
DIFFUSE
CURRENT IMAGE

262

267
FLATTENED
FINGERTIP
FEEDBACK

263
SMOOTHED
PROXIMITY
IMAGE
SEARCH FOR
SIGNIFICANT
LOCAL MAXIMA

US 7,812,828 B2

252
ESTIMATED HAND
POSITION OFFSETS

264
DEFINE SEGMENTATION

LOCAL

MAXIMUM

265

STRICTNESS REGIONS

PIXELS

266

268
CONSTRUCTELEtinODE
GROUPS AROUND LOCAL
MAXIMUM PIXELS

COMBINE
OVERLAPPING g 270
GROUPS

FIT ELLIPSES TO
COMBINED
, 272
GROUPS

PARAMETERIZED
ELECTRODE GROUPS

242

FIG. 18

ÛODV Drovided bv USÞTO from the Dina Iman. ¯

a.. naronienn

APLNDC00030350

U.S. Patent

Oct. 12, 2010

Sheet 19 of 45

US 7,812,828 B2

274

281

278
277

276
277

279

279

281

281

281

FIG. 19

i3

warrisiasi hu ifnDTi3 fram the DIRA imana

nn fidl9RI9n11

APLNDC00030351

a
,
+

5
15 -

15

U.S. Patent

a
+
,
+

.
252

Oct. 12, 2010

+,

252

Sheet 20 of 45

+,

US 7,812,828 B2

282
---

10 -

.

287
0-

287
284

----

285

-

285
I

-15

-10

-5

I

i

i

0

5

10

15

i

-20

i

i

i

FIG. 20A
282

m 10-

x

+

252

285

287

286

----

o-

+
-20

+

20

252-

---

5. +
1.6

.

252

285
i

i

i

i

O

5

10

15

20

-10
-5
0
5
10
15
HORIZONTAL SURFACE POSITION (cm)

20

-15

-10

-5

FIG. 20B
10

5
252

0

287

ß85
-20

-15

286

----

FIG. 20C

Copy provided by USPTO from the PIRS Image Database on 04/25/2011

APLNDC00030352

314 ''

U.S. Patent

Oct. 12, 2010

START)

Be i vv EEN

Sheet 21 of 45

US 7,812,828 B2

300
GET NEXT ELECTRODE IN
DIRECTION OF SEARCH

290

W PROXIMITY >
BACKGROUND?

N

REACHED BACKGROUND
LEVEL EDGE

Y

END )

304

292

N

306
N STRIC

SEGMENTATION
REGION?
N

Y

SEARCHING
HORIZONTAL ?

SEARCHING
HORIZONTAL ?

N

N

VERTICAL
MINIMUM ?
310

Y

Y
REACHED EDGE

294
312

Be i vvEEN
FINGERTIP AND
THUMB OR PALM

Y
296

HORIZ·
DIST. TO LOCAL MAX
> 2cm 7

298

TALL
HORIZONTAL
MINIMUM 7

N

"
HORIZ.
OR DIAGONAL
MINIMUM ?

( END )

308

Y

N

N

REACHED EDGE
FINGERS

Y

( END )

REACHED EDGE
BETWEEN PALM
HEELS
( END )

300
Fjg, 21

Conv nrovided hv um nu from the PIRS Imaae Database on 04/25/2011

APLNDC00030353

U.S. Patent

Oct. 12, 2010

Sheet 22 of 45

US 7,812,828 B2

( START )
PREDICT CURRENT POSITIONS y 320
OF EXISTING PATHS
FOR EACH GROUP y 322
FIND CLOSEST PATH

FOR EACH PATH, FIND
324
CLOSEST GROUP WITHIN "
TRACKING RADIUS

FORM GROUP-PATH PAIRS 1F g 326
GROUP & ACTIVE PATH ARE
CLOSEST TO ONE ANOTHER

ATTEMPT TO PAIR REMAINING
334
GROUPS WITH RECENTLY "
DEACTIVATED PATHS

ALLOCATE NEW PATHS FOR ANY g 336
REMAINING UNPAIRED GROUPS
DEACTIVATE ANY REMAINING c 344
UNPAIRED PATHS

UPDATE PATH g 346
PARAMETERS
( END )

FIG. 22

APLNDC00030354