Apple Inc. v. Samsung Electronics Co. Ltd. et al

Filing 462

Declaration of DEOK KEUN Matthew Ahn IN SUPPORT OF #461 APPLES OPENING CLAIM CONSTRUCTION BRIEF PURSUANT TO PATENT L.R. 4-5 filed by Apple Inc.(a California corporation). (Attachments: #1 Exhibit A, #2 Exhibit B Part 1, #3 Exhibit B Part 2, #4 Exhibit C Part 1, #5 Exhibit C Part 2, #6 Exhibit D Part 1, #7 Exhibit D Part 2, #8 Exhibit D Part 3, #9 Exhibit D Part 4, #10 Exhibit E Part 1, #11 Exhibit E Part 2, #12 Exhibit F, #13 Exhibit G, #14 Exhibit H, #15 Exhibit I, #16 Exhibit J, #17 Exhibit K, #18 Exhibit L, #19 Exhibit M Part 1, #20 Exhibit M Part 2, #21 Exhibit N, #22 Exhibit O, #23 Exhibit P, #24 Exhibit Q)(Jacobs, Michael) (Filed on 12/8/2011) Modified on 12/9/2011 linking entry to document #461 (dhm, COURT STAFF).

U.S. Patent

Oct. 12, 2010

Sheet 23 of 45

US 7,812,828 B2

(START)

DEFINE IDENTITY
ATTRACTORS AT DEFAULT 7- 350
CONTACT POSITIONS

TRANSLATE A I i MCTOR
352
TEMPLATE BY ESTIMATED C
HAND OFFSET
COMPUTE MATRIX OF
354
DISTANCES FROM EACH PATH "

TO EACH ATTRACTOR
COMPUTE ATTRACTOR
356
WEIGHTING FACTORS FROM "
FEATURES OF EACH PATH
FIND ASSIGNMENT B•-•-EEN
PATHS AND A I i MCTORS y 358

WHICH MINIMIZES SUM OF
WEIGHTED DISTANCES
*
360
HAND
N
ASSIGAMENTS

362
FINGE
ATTRACTORS

I

ENTÞTIVE

Y

E5D > 1
VERIFY THUMB
ASSIGNMENT

364
UPDATE FINGER

COUNTS AND

t

SUBSETS
END )

FIG. 23

APLNDC00030355

u. g .
2

373

U.S. Patent

12

Oct. 12, 2010

374

Sheet 24 of 45

US 7,812,828 B2

.

10 '

380

372,×
×- 375

z 6
O

O 4
CL
w

38O
×- 37 1

0

380
380

-4
-6

377
376

0

2

4

6
8
10
12
14
16
HORIZONTAL SURFACE POSITION (cm)

18

20

FIG. 24

APLNDC00030356

O

O
0

·

U.S. Patent

'

Oct. 12, 2010

'

Sheet 25 of 45

US 7,812,828 B2

z
O

O

50
100
150
CONTACT ORIENTATION (degrees)

FIG. 25A
LU

I-

O

1
2
3
4
5
6
CONTACT SIZE (NORMALIZED TOTAL PROXIMITY)

FIG. 25B
LU M
- LU

0
0

1
2
3
4
5
TOTAL PROXIMITY DIVIDED BY ECCENTRICITY

6

FIG. 25C

LU

0

2
4
6
8
DISTANCE TO NEAREST NEIGHBOR CONTACT (cm)

FIG. 25D

nnu nynvidad hu IIRDTf3 fram the DIRR Imana ¯

nn fla.. ......i

APLNDC00030357

U.S. Patent

Oct. 12, 2010

COMPUTE INTER-PATH
THUMB FACTORS

Sheet 26 of 45

US 7,812,828 B2

400

COMBINE WITH THUMB SIZE
& ORIENTATION FACTORS --402
OF INNERMOST AND NEXT
INNERMOST CONTACT
404

412
N

combined_thumb fact >
is thumb_thresh?

NNERMO

N

M

Y
406

SHIFT INNERMOST
PATH TO THUMB
A i i tmCTOR

N

combined_thumb_fact
not_thumb_thresh7

414Á

Y
408

INNERMOS
ASSIGNED TO
THUMB?

N
,,

Y

7413

EXISTING
ASSIGNMrsuse

SHIFT INNERMOST
PATHS AWAY FROM

OK
c 410

THUMB ATTRACTOR
I
I

( END )

FIG. 26

Conv nrovided hv uw us from the PIRS Imaae Database on 04/25/2011

APLNDC00030358

U.S. Patent

Oct. 12, 2010

Sheet 27 of 45

US 7,812,828 B2

(START)
GET ALL PATHS ASSIGNED
TO THE GIVEN TIAND

430

COMPUTE DISTANCES
FROM EACH PATH TO
OTHER PATHS

432

FIND SHORTEST RING
CONNECTING ALL PATHS
AND PASSING THROUGH
EACH ONCE

434

COMPUTE THUMB & PALM

436

WEIGHTING FACTORS FOR
EACH PATH
PICK INNERMOST , 438
PATH IN RING

440

INNERMOST
PATH THUMB 7
Y

PATHS ABOVE THIS
VERTICAL LEVEL ARE
FINGERTIPS, BELOW
ARE PALMS

442

N

INNERMOS
PATH A PALM
HEEL7
446

Y

PATHS AT THIS
VERTICAL LEVEL
ARE PALMS

N

448

PATHS AT THIS
VERTICAL LEVEL
ARE FINGERTIPS

444
( END )

FIG. 27

ÛODV DFOVided bv USPTO from the PIRS Imaae Database on 04/25/2011

APLNDC00030359

U.S. Patent

Oct. 12, 2010

Sheet 28 of 45

US 7,812,828 B2

('START)

& OUTNENR HAND485
ARTS TOUCHING

N

I

N

PREVIOUSLY
DETECTED ?

Y

I ( END )

Y

GET ESTIMATED FINGER &
PALM POSITIONS FOR c 487
LIFTED HAND PARTS

GET MEASURED POSITIONS &

SlZES OF TOUCHING FINGERS
& PALM HEELS

COMPUTE KNUCKLE FACTOR
FOR OUTER FINGERS

488

L 489

-

COMPUTE INDEX JUTTING
FACTOR FOR INNER FINGERS i490

495
SEND PARAMETERS OF
INNER FINGER PATHS TO
HANDWRITING RECOGNIZER
'
i

COMBINE FACTORS & FILTER
WITH OLD COMBINED FACTORS i 491

Y
492

FILTERED
FACTOR > PENGRIP
THRESH ?

493

i

INNER FINGERS
TOUCHING ?

N

N

494

SEND STYLUS LIFT SIGNAL TO
HANDWRITING RECOGN1ZER &
KNUCKLE/PALM MOTION TO CURSOR
END)

FIG. 28

APLNDC00030360

U.S. Patent

Oct. 12, 2010

Sheet 29 of 45

US 7,812,828 B2

(START)

450

CONTACT
PROXIMITIES
TABIUZED 9
N

,452

y

RETAIN PREVIOUS
IDENTlFICATIONS VIA
PATH EXTENSION

DEFINE & TRANSLATE LEFT &
( END )
RIGHT ATTRACTOR TEMPLATES L 453
y 456

PICK FIRST
CONTOUR '

GENERATE PARTITIONING
CONTOURS
L 454
TENTATIVELY DIVIDE HAND
1DENTITIES ACROSS CONTOUR

458

TENTATIVELY ASSIGN FINGER
IDENTITIES WITHIN EACH HAND L 460
EVALUATE BIOMECHANICAL
COHERENCE OF PARTITION
PICK NEXT
CONTOUR

470

LAST
CONTOUR

464
COST

N

LOWEST SO FAR
472
a

L 462
, 466

RECORD PARTITION
ÀS LOWEST COST

N

Y

473
y 474
CHOOSE
ASSIGN FINAL
C S
ND --> CONTACT IDENTITIES WITHIN EACH HAND
PARTITION

END )

FIG. 29

Copy provided by USPTO from the PIRS Image Database on 04/25/2011

APLNDC00030361

U.S. Patent

Oct. 12, 2010

10

475

o
5

Sheet 30 of 45

US 7,812,828 B2

ii

476

477

O~477

0-20

-15

-10

-5

0

FIG. 30A

5

10

15

20

E

z
O

·

-

!¯10

5

477

476
O---476

475

- 0-20

-15

-10

-5

0

FIG. 308

5

10

15

20

15

20

10

476~O R
5

476
O-~476

0

-20

477

-15

-475

-10
-5
0
5
10
HORIZONTAL SURFACE POSITION (cm)

FIG. 30C

Conv nrovided bv USPTO from the PIHS Image Database on 04/25/2011

APLNDC00030362

O

10

'

U.S. Patent

Oct. 12, 2010

'

Sheet 31 of 45

US 7,812,828 B2

Oz
LU

O
50
HORIZONTAL VELOCITY OF RIGHT HAND CLUSTER (mm/s)

FIG. 31A
m1
LLig

20

0

i

-12

i

1

i

i

I

-10
-8
-6
-4
-2
0
2
VERTICAL POSITION OF OUTERMOST FINGER
RELATIVE TO NEXT OUTERMOST (cm)

FIG. 318
z

CL

O
5
10
15
20
HORIZONTAL SEPARATION BL I vvEEN PALM CONTACTS (cm)

FIG. 31C

conv nmvidad hu USPTO from the PIRS Imaae Database on 04/25/2011

APLNDC00030363

O

'

U.S. Patent

Oct. 12, 2010

¯

Sheet 32 of 45

US 7,812,828 B2

1·
O

LU¡-

zo

O

-150

-100
-50
0
50
100
150
ANGLE BETWEEN INNERM· ~= AND
NEXT INNERMOST FINGER CONTACTS (degrees)

FIG. 32

O
I-

O

-6

-4

-2

0
2
4
6
8
10
EST1MATED HORIZONTAL
SEPARATION Be i vveEN THUMBS (cm)

12

FIG. 33

anv nrnvidad hu flRPTO from the PIRE Imann Datshana on 04/95/2011

APLNDC00030364

U.S. Patent

Oct. 12, 2010

Sheet 33 of 45

US 7,812,828 B2

( ST Mi )

GET HAND'S CURRENT
PATH PARAMETERS &
ID'S

500

SUPPRESSIVE
FINGER VELOCITY
FILTERING

502

MEASURE HAND'S
POLAR VELOCITY
COMPONENTS

504

MEASURE HAND'S
TRANSLATIONAL
VELOCITY COMPONENTS

MEASURE HAND'S
DIFFERENTIAL TILT
PRESSURE
COMPONENTS
DOWNS"' "

506

508

510

WEA-COMPONENTS
DEAD-ZONE FILTER ALL
COMPONENTS
y 512
BY FRACTION OF
FASTEST COMPONENT

( END')

FIG. 34

maar nemasiwinsi har i EODTTh Swann iha DIDO lamana

mm o

I El Add

APLNDC00030365

U.S. Patent

Oct. 12, 2010

Sheet 34 of 45

203
,
202

US 7,812,828 B2

204

'

205

20

/

207

206

FIG. 35

APLNDC00030366

U.S. Patent

Oct. 12, 2010

Sheet 35 of 45

US 7,812,828 B2

START)

522
AT LEAST
2 FINGERS
DOWN 7

N
I

524
SET RADIAL AND
ANGULAR VELOCITY
TO ZERO

Y
GET CURRENT AND PREVIOUS
POSITIONS OF INNERMOST AND
OUTERMOST TOUCHING FINGERS i 526

COMPUTE RADIAL VELOCITY FROM
CHANGE IN SEPARATION BETWEEN
INNERMOST AND Ou i tRMOST
COMPUTE ROTATIONAL VELOCITY
FROM SEPARATION AND CHANGE IN
ANGLE BETWEEN INNERMOST AND
Ou i eRMOST

( EÑD')

528

530

COMBINE WITH ROTATION AND

SCALING ABOUT A FIXED POINT
BETWEEN THUMB AND OTHER FINGERS U 531
534 ,
\

532

AVERAGE
PROXIMITY
DROPPING 7

N

CHECK FOR
RADIAL OR
I ROTATIONAL
DECELERATION

Y
( END )

FIG. 36

rano nravidad hu IIRPTO from the PIRS Imane Database on 04/25/2011

APLNDC00030367

U.S. Patent

Oct. 12, 2010

Sheet 36 of 45

US 7,812,828 B2

( START )
.
, 540
INIT TRANSLATION
WEIGHTINGS TO FINGER
PROXIMITIES

g 551
ACCEL RATIO =
CURRENT SPEEDI
PAST AvemoE

, 544
DECREASE TRANSLATION
WEIGHTING OF
RELATIVELY SLOW

552
ACCEL RAT1
> THRESH NEAR
ONE 7

FINGERS
y 546

Y

DECREASE TRANSLATION
WEIGHTING OF CENTRAL
FINGERS AS POLAR
COMPONENT SPEEDS
INCREASE

554
TRANSLATION
DIRECTION CLOSE TO
AST AVERAGE

, 548
COMPUTE TRANSLATION
VELOCITY AS WEIGHTED
AVERAGE OF FINGER
VELOCITIES

N
556

Y

550

AVERAGE
PROXIMITY
DROPPING ?

N

N

"
SE
TRANS ATION
DECELERATION
FLAG

,,
""
TRANSLATION , 558
DECELERATION

FLAG
Y
UPDATE MOVING
WINDOW AVERAGE OF g 560

TRANSLATION

,
( END )

VELOCITIES

FIG. 37

ramo nmuM.wi hu linDTr3 fmm tha ÞIRR Imann Datakaan on 04/25/2011

APLNDC00030368

U.S. Patent

Oct. 12, 2010

Sheet 37 of 45

US 7,812,828 B2

( START )

( END
562

HAND
FLATTENED 7

Y

† r 564
SET TILT & ROLL
COMPONENTS
TO ZERO

N
566

ALL PATH
PROXIMITIES
TABILIZED 9

N

¡
r 568
STORE CURRENT PATH
PROXIMITIES AS
REFERENCE PROXIMITIES

Y
COMPUTE UNWEIGiIiLu
AVERAGE OF PATH POSITIONS

570

COMPUTE RATIOS OF CURRENT , 572
PROXIMITY TO REFERENCE
PROXIMITY FOR EACH PATH

SET RATIOS LESS
THAN ONE TO ONE

y 574

COMPUTE AVERAGE OF
PATH POSITIONS
, 576
WEIGHTED BY PROXIMITY
RATIOS
COMPUTE TILT & ROLL
COMPONENTS FROM
y 578
DIFFERENCE VECTOR BETWEEN
WEIGHTED AND UNWEIGHTED

AVERAGES
( END )

FIG. 38

nano nrnvidad hv USPTO from the PIRS Imaae Database on 04/25/2011

APLNDC00030369

608

U.S. Patent

r 610

Oct. 12, 2010

624 p

Sheet 38 of 45

GET HAND'S CURRENT
PATH PARAMETERS & ID'S

US 7,812,828 B2

^
L 600

DE
ASSOCIATED
KEYPRESS
QUEUE
ELEMENTS

601

SEARCH FOR FINGER
SUBSETS PRESSED OR g 603
RELEASED
SIMULTANEOUSLY

N
604

YN

602

MARKER
PENDING
7

ANY

N

PRESSES
SYNCED
7

N

#FINGER
RELEASES
YNCED > 2

y

605

SET SYNC
TIME MARKER

# FINGER

N

606

PRESSES
YNCED > 2

A

Y

PAUSE SENDING OF
ASSOCIATED
KEYPRESS QUEUE
ELEMS

DELETE ASSOCIATED
KEY QUEUE ELEMS

CI

SYNC
MARKER

612OUCHIN
OR HALTED TOO

Y

LONG 7

QUEUE

ELEMENTS

614

N

615 -/

616
N

D
E
ASSOCIATED KEY

SYNCED
FINGERS
LIFTING 7

y

RESUME
N

KEY

QUEUE , 6U
SENDING
LIFTING
Y
SIMULTANEOUSLY
I B
7
618

FIG. 39A

ram,

has li DTr3 fram the DIRA Imana ¯

nn ftd19RI9tt11

APLNDC00030370

U.S. Patent

Oct. 12, 2010

Sheet 39 of 45

US 7,812,828 B2

DELETE ASSOCIATED g 620
KEYPRESS QUEUE
ELEMENTS
626
SYNCED
FINGERS DOWN
RIEFLY 7
Y

N

628

SIGNIFICANT
TERAL MOTION ?

I

C

N
LOOKUP CHORD
FROM SYNCED
FINGER ID'S

630

632
HORD HAS TA

N

EVENTS ?

, 634
RESTING CHORD:
NO --NTS
GENERATED

Y
APPEND CHORD g 636
TAP EVENTS TO
COMM QUEUE

FIG. 398

Pano nrnvidad hv IIRPTl3 from the PIRR Imana Ontahame on

.1

APLNDC00030371

N

U.S. Patent

Y

Oct. 12, 2010

GET HAND'S
EXTRACTED MOTION &
IDENTIFIED PATHS

Sheet 40 of 45

650
N

652

CSH RED
ONGOING 7

N

656

SYNCED
SUBSET OR
ALL FINGERS
LIDING

Y

660
, 658
DISABLE KEY &
| SELECT SLIDE CHORD
CHORD TAPS FOR +-- FROM SYNCED SUBSET
THIS HAND
OR COMBINATION OF
FINGERS TOUCHING

664

666

FINGER
SUBSET
LIFTED 7

y 668
LEAVE
CHORD
SLIDE MODE

RE-LIFTOF
DECELERATION
FLAG SET ?

N

SUBSET
PRESSED IN
SYNC ?

N

654

# FINGERS
TOUCHING > 2 ?

B

NEW

US 7,812,828 B2

N

, 667
SET CURRENT VELOCITY
COMPONENTS TO PRELIFTOFF AVERAGE

670

B

673
Y

HOLE SUBSE
SLIDING ?

N

SELECT NEW SUDE
B

A

674

CHORD FROM NEW
SYNCED SUBSET

FIG. 40A

Canu nenuidad lau liRDTT3 fram the DIRR imana

an fh&foCIOfH4

APLNDC00030372

U.S. Patent

Oct. 12, 2010

Sheet 41 of 45

US 7,812,828 B2

PICK FIRST SLICE
y 675
DEFINED FOR SELECTED
CHORD

APPLY SLICE'S VELOCITY , 676

694

PICK NEXT SUCE

GAIN FUNCTION TO
MOTION COMPONENTS

N
PROJECT
VELOCITY COMPONENTS g 677
INTO SLICE'S SPEED AND
DIRECTION RANGE

692
Y

ma l SUCE

A

FOR CHORD ?

INTEGRATE PROJECTED e 678
VELOCITY COMPONENTS
OVER TIME

i , 690
DISABLE FURTHER
EVENTS FROM ONESHOT SLICE

680

# UNITS OF
MOTION
>= 1 7
Y

N
N
ONE-S

Y

SUCE7
689

LOOKUP SLICE'S
KEYlMOUSEl3D , 682
EVENTS

RESSET OETHER , 688
INTEGRATORS /

684

APPEND EVENTS WI
# MOTION UNITS

TO COMM QUEUE

†

REMOVE INTEGER # p 686
a MOTION UNITS FROM

INTEGRATORS

FIG. 408

a..... ...mas:AaA hus I BODTA imm tha DIRA Imann Ontahnna on 04/25/2011

APLNDC00030373

U.S. Patent

Oct. 12, 2010

Sheet 42 of 45

US 7,812,828 B2

( START )

RETRIEVE KEY LAYOUT ç 700
REGIONS AND SYMBOLS

GET CURRENT IDENTIFIED
PATHS FOR BOTH HANDS

5

702

704

FINGERS
ON SAME HAND
PRESSED IN
SYNC7
N

, 706
TRANSLATE HAND'S

Y
>

KEY REGIONS BY
MEASURED HAND
OFFSETS

714
OTHING
TOUCHING
SURFACE FOR
AWHILE?
N

FINGERS
PARTIALLY
CLOSED 7

Y

708
y

N
,.
716
RESET KEY
LAYOUT
OFFSETS TO
ZERO

710
ADJUST REGIONS IN
EACH FINGER'S
COLUMN BY FINGER
OFFSETS

UPDATE DISPLAYED g 712
POSITIONS OF KEY

SYMBOLS
PROCESS
FINGER TAPS
,718
ON MORPHED KEY
LAYOUT

FIG. 41

.. -

, I luom imm tha Pins Imaae Database on 04/25/2011

APLNDC00030374