Apple Inc. v. Samsung Electronics Co. Ltd. et al
Filing
462
Declaration of DEOK KEUN Matthew Ahn IN SUPPORT OF #461 APPLES OPENING CLAIM CONSTRUCTION BRIEF PURSUANT TO PATENT L.R. 4-5 filed by Apple Inc.(a California corporation). (Attachments: #1 Exhibit A, #2 Exhibit B Part 1, #3 Exhibit B Part 2, #4 Exhibit C Part 1, #5 Exhibit C Part 2, #6 Exhibit D Part 1, #7 Exhibit D Part 2, #8 Exhibit D Part 3, #9 Exhibit D Part 4, #10 Exhibit E Part 1, #11 Exhibit E Part 2, #12 Exhibit F, #13 Exhibit G, #14 Exhibit H, #15 Exhibit I, #16 Exhibit J, #17 Exhibit K, #18 Exhibit L, #19 Exhibit M Part 1, #20 Exhibit M Part 2, #21 Exhibit N, #22 Exhibit O, #23 Exhibit P, #24 Exhibit Q)(Jacobs, Michael) (Filed on 12/8/2011) Modified on 12/9/2011 linking entry to document #461 (dhm, COURT STAFF).
U.S. Patent
Oct. 12, 2010
Sheet 43 of 45
US 7,812,828 B2
( START )
GET ANY PATH RECENTLY ,750
CREATED BY HAND PART
TOUCHDOWN
PATH
752
PROXIMITY
JUST CROSSED
KEYPRESS
HRESH
N
Y
N
y
PATH
754
IDENTIFIED
AS FINGER NOT
PALM7
Y
PATH'S
HAND SLIDINGI
WRITING 7
756
N
757
DEBOUNCE
OK ?
, 760
FIND CLOSEST
KEY REGION
Y
N
N
764
T 766
CREATE KEYPRESS
APPEND KEYPRESS
QUEUE ELEMENT TO ,
TAIL OF FIFO
KEYPRESS QUEUE
762
ANY KE
REGION
EARBY
y
'
QUEUE ELEMENT
CONTAINING PATH ID, e
CLOSEST KEY &
PRESS T1MESTAMP
FIG. 42
APLNDC00030375
N
U.S. Patent
e
N
Oct. 12, 2010
START )
H6
NN
Sheet 44 of 45
A
US 7,812,828 B2
DELETE CURRENT
ELEMENT EROM •
KEYPRESS QUEUE
PICK ELEMENT AT HEAD p-770
OF KEYPRESS QUEUE
B
778
Y
LEMEN
PATH STILL
IDENTIFIED AS
FINGER ?
y
PATH IN A
SYNCHRONIZED
SUBSET 7
772
N
774
782
B
780
N
i
FINGER SLID
TOO FAR 7
TIME
SINCE PRESS <
TAP TIMEOUT
Y
FINGER
Ll- -•· 7
N
784
APPEND PRECEDING
MODIFIERS &
ELEMENT'S KEY
REG10N SYMBOL TO
786
788
SKIP TO NEXT
ELEMENT IN
QUEUE
C
KEY REGION
A MODIFIER 7
HOST COMM QUEUE
792
Y
790
PATH
PROXIMITY
PROFILE
Y
IMPULSIVE
7
MOST
FINGERS
TOUCHING
?
FIG. 43A
sa-.... ---..:A-A km ISODTft tomaan ilma DIOC innersa Aminimman oft MË
APLNDC00030376
Y
U.S. Patent
805
800
Oct. 12, 2010
Y
802 ,
Sheet 45 of 45
NN
US 7,812,828 B2
D
D FOC 798N
lENT ?
REMCE
ANOTHER
ASYNCHRONOUS
Y
7
AP TI IIEOUT
FINGER
DOWN
HAND'S
OT ER FINGERS
Y
COMPUTE REPEAT
INTERVAL FROM
CURRENT
FINGERPROXIMITY
A
-.
C 806
INITIALIZE
TYPEMATIC
MODE FOR
ELEMENT
80
TIME
SINCE I &
SE
808
y
810
N
812
APPEND PRECEDING
O MODIFIERS & ELEMENT'S
KEY REGION SYMBOL TO
HOST COMM QUEUE
UPDATE vio i TYPEMATIC
SEND TIMESTAMP
FIG. 43B
t½nw nrnuidad hu URPTO from the PIRE Imane Datshame on ---11
APLNDC00030377
1
2
US 7,812,828 B2
ELLIPSE -· · · · G FOR MULTI-TOUCH
SURFACES
will always be a need in the art for multi-function manual
input devices which supplement voice input.
A generic manual input device which combines the typing,
CROSS-u a su -CE TO RELATED
pointing, scrolling, and handwriting capabilities of the stanAPPLICATIONS
s dard input device collectionmusthave ergonomic, economic,
andproductivityadvantages which outweighthe unavoidable
This application is a continuation of 11/015,434, entitled
sacrifices of abandoning device specialization. The generic
"Method and Apparatus for Integrating Manual Input," filed
device must tightly integrate yet clearly distinguish the difDec. 17, 2004 now U.S. Pat. No. 7,339,580, which is a conferent types of input. It should therefore appear modeless to
tinuation of 09/236,513 (now Pat. No. 6,323,846) filed Jan. 10 the user in the sense that the user should not need to provide
25, 1999 which claims the benefit of provisional application
explicit mode switch signals such as buttonpresses, arm relo60/072,509, filed Jan. 26, 1998, each of which is hereby
cations, or stylus pickups before switching from one mput
incorporated by reference in its entirety. This application is
activity to another. Epidemiological studies suggest that repalso related to Application Ser. No. 11/428,501, entitled
etition and force multiply m causmg repetitive strain injuries.
"Capacitive Sensing Arrangement," 11/428,503, entitled 15 Awkward postures, device activation force, wasted motion,
"Touch Surface," 11/428,506, entitled "User Interface Gesand repetition should be minimized to improve ergonomics.
tures," 11/428,515, entitled "User Interface Gestures",
Furthermore, the workload should be spread evenly over all
11/428,522, entitled "Identifying Contacts on a Touch Suravailable muscle groups to avoid repetitive strain.
face," 11/428,521, entitled "Identifying Contacts on a Touch
Repetition can be minimized by allocating to several
Surface", 11/559,736, entitled "Multi-Touch Contact Track- 20 graphical manipulation channels those tasks which require
ing Algorithm", 11/559,763, "Multi-Touch Contact Motion
complex mouse pointer motion sequences. Common graphiExtraction," 11/559,799, entitled "Multi-Touch Contact
cal userinterface operations such as fmding and manipulating
MotionExtraction,"11/559,822,entitled"Multi-TouchCona scroll bar or slider control are much less efficient than
tact Motion Extraction," 11/559,833, entitled Multi-Touch
specialized fmger motions which cause scrolling directly,
Hand Position Offset Computation, each of which is hereby 25 withoutthe step ofrepositioningthe cursor over an on-screen
incorporated by reference in its entirety,
control. Preferably the graphical manipulation channels
should be distributed amongst many finger and hand motion
- - GROUND OF THE INVENTION
combinations to spread the workload. Touchpads and mice
with auxilliary scrolling controls such as the Cirque®=
A. Field of the Invention
so Smartcat touchpad with edge scrolling, the IBM®= ScrollThe present invention relates generally to methods and
Pointm mouse with embedded pointing stick, and the Roller
apparatus for data input, and, more particularly, to a method
Mouse described in U.S. Pat. No. 5,530,455 to Gillick et al.
and apparatus for integrating manual input.
represent small improvements m this area, but still do not
B. Description of the Related Art
provide enough direct manipulation channels to eliminate
Many methods for manual input of data and commands to 35 manyoften-usedcursormotion sequences. Furthermore, as S.
computers are in use today, but each is most efficient and easy
Zhai et al. found in "Dual Stream Input for Pointing and
to use forparticulartypes ofdata input. For example, drawing
Scrolling," Proceedings of CHI '97 Extended Abstracts
tablets with pens or pucks excel at drafting, sketching, and
(1997), manipulationofmore than two degrees offreedom at
quick command gestures. Handwriting with a stylus is cona time is very difficult with these devices, preventing simulvenient for filling out forms which require signatures, special 40 taneous panning, zooming and rotating.
symbols, or small
of text, but handwriting is slow
Another common method for reducing excess motion and
compared to typing and voice input for long documents.
repetition is to automatically continue pointing or scrolling
Mice, finger-sticks and touchpads excel at cursor pointing
movement signals once the user has stopped moving or lifts
and graphical object manipulations such as drag and drop.
the finger. Related art methods can be distinguished by the
Rollers, thumbwheels and trackballs excel at panning and 45 conditions underwhich such motion continuation is enabled.
scrolling. The diversity of tasks that many computer users
In U.S. Pat. No. 4,734,685, Watanabe continues image panencounter in a single day call for all of these techniques, but
ningwhenthe distance and velocity ofpointing device movefew users will pay for a multitude of input devices, and the
ment exceed thresholds. Automatic panning is, stopped by
separate devices are often incompatible in a usability and an
moving the pointing device backin the opposite direction, so
ergonomic sense. For instance, drawingtablets are a must for so stopping requires additional precise movements. In U.S. Pat.
graphics professionals, but switching between drawing and
No. 5,543,591 to Gillespie et al., motion continuation occurs
typing is inconvenient because the pen must be put down or
when the fmger enters an edge border region around a small
held awkwardlybetweenthe fingers while typing. Thus, there
touchpad. Continued motion speed is fixed and the direction
is a long-felt need in the art for a manual input device which
correspondstothedirectionfromthecenterofthetouchpadto
is cheap yet offers --. Jent integration of---nmanual 55 the finger at the edge. Continuation mode ends when the
input techniques.
finger leaves the border region or lifts off the pad. DisadvanSpeech recognition is an exciting new technology which
tageously, users sometimes pause at the edge of the pad
promises to relieve some of the input burden on user hands.
without intending for cursor motion to continue, and the
However, voice is not appropriate for inputting all types of
unexpected motion continuation becomes annoymg. U.S.
data either. Currently, voice input is best-suited for dictation 60 Pat. No. 5,327,161 to Logan et al. describes motion continuof long text documents. Until natural language recognition
ation when the fmger enters a border area as well, but in an
matures sufficiently that very high level voice commands can
alternative trackball emulation mode, motion continuation
be understood by the computer, voice will have little advancanbeafonctionsolelyoflateralfingervelocity and direction
tage over keyboard hot-keys and mouse menus for command
at liftoff. Motion continuation decays due to a friction factor
and control. Furthermore, precise pointing, drawing, and 65 or can be stoppedby a subsequent touchdown on the surface.
manipulation ofgraphical objects is difficult with voice comDisadvantageously, touch velocity at liftoff is not a reliable
mands, no matter how well speech is understood. Thus, there
indicator of the user's desire for motion continuation since
I
a..... - ...:a-a a , ricanTra ham the DINA imana fantahnmaan 04/25/ŽÛ11
APLNDC00030378
US 7,812,828 B2
3
4
when approaching a large target on a display at high speeds
An ergonomic typing system should require minimal key
the user may not stop the pointer completely before liftoff.
tapping force, easily distinguish finger taps from resting
Thus it would be an advance in the art to provide a motion
hands, and cushion the fmgers from the jarring force of surcontinuation method which does not become activated unexface impact. Mechanical and membrane keyboards rely on
pectedly when the user really intended to stop pointer move- 5 the spring force in the keyswitches to prevent activation when
ment at a target but happens to be on a borderorhappens to be
the hands are resting on the keys. This causes an irreconcilmoving at significant speed during liftoff.
able tradeoff between the ergonomic desires to reduce the
Many attempts have been made to embed pointing devices
fatigue from key activating force and to relax the full weight
in a keyboard so thehands do not have to leavetypingposition
ofthehandsontothekeysduringrestperiods.Force---:--i to access the pointing device. These include the integrated 10 zation on touch surfaces is possible with capacitive or active
pointing key described in U.S. Pat. No. 5,189,403 to Franz et
optical sensing, which do not rely on fmger pressure, rather
al., the integrated pointing stick disclosed by J. Rutledge and
than resistive-membrane or surface-acoustic-wave sensing
T. Selker in "Force-to-Motion Functions for Pointing,"
techniques. The related art touch devices discussed below
Human-Computer Interaction- m i n run i '90, pp. 701-06
(1990), and the position sensing keys described in U.S. Pat. 15 will become confused if a whole hand including its four
fmgertips a thumb and possibly palm heels, rests on the surNo. 5,675,361 to Santilli. Nevertheless, the limited moveface. Thus, there exists a long felt need in the art for a multiment range and resolution of these devices, leads to poorer
touch surface typing system based on zero-force capacitive
pointing speed and accuracy than a mouse, and they add
sensing which can tolerate resting hands and a surface cushmechanical complexity to keyboard construction. Thus there
exists a need in the art for pointing methods with higher 20 Ion.
An ergonomic typing system should also adapt to indiresolution, larger movement range, and more degrees offreevidual hand sizes tolerate variations in typing style, and supdom yet which are easily --ble from typing hand posiport a range of healthy hand postures. Though many ergotions.
nomic keyboards have been proposed, mechanical
Touch .._ and touchpads often distinguish pointing
motions from emulated button clicks orkeypressesby assum- 25 keyswitches can only be repositioned at great cost. For
example, the keyboard with concave keywells described by
ing very little lateral fingertip motion will occur during taps
Hargreaves et al. in U.S. Pat. No. 5,689,253 fits most hands
on the touch surface which are intended as clicks. Inherent in
well but also tends to lock the arms in a single position. A
these methods is the assumption that tapping will usually be
touch surface key layout could easily be morphed, translated,
straightdownfromthe suspendedfingerposition, minimizing
those components of fmger motion tangential to the surface. 30 or arbitrarily reconfigured as long as the changes did not
confuse the user. However, touch surfaces may not provide as
This is a valid assumption if the surface is not fmely divided
much laterally orienting tactile feedback as the edges of
into distinct key areas or if the user does a slow, "hunt and
mechanical keyswitches. Thus, there exists a need in the art
peck"visual search for each key before striking. For example,
for a surface typing recognizer which can adapt a key layout
in U.S. Pat. No. 5,543,591 to Gillespie et al., a touchpadsends
all lateral motions to the host computer as cursor movements. 35 to fit individual hand postures and which can sustain typing
accuracy if the hands drift due to limited tactile feedback.
However, if the finger is lifted soon enough after touchdown
Handwriting on smooth touch surfaces using a stylus is
to count as a tap and ifthe accumulatedlateral motions are not
well-knownin the art, but it typically does not integrate well
---ve, any sent motions are undone and a mouse button
with typing andpointing becausethe stylus must be put down
click is sent instead. This method only works for mouse
commanaPipappa) if j=6
du / (Pimen_sizejaPimam_seppa) if Ì = 7
50
inner separationfactorinner_separation_fact is definedas the
ratio of the distance between the ......... st and next inner-
Mathematically the optimization can then be stated as finding
the permutation {ni, . . . , x,} of integer hand part identities
{1, . . . , 7} which
7
The separation between thumb and index finger is often
larger than the separations between fingertips, but all separationstendtogrowasthefingersareoutstretched.Thereforean
most finger contacts to the average of the distances between
55
other adjacent fingertip contacts, avg_separation: 12
innerseparationfact min
(54)
c;
FI - FNJ2 + (FI - FN)2
60
umerseparationfacta mit 1,
(55)
avgseparation
where ey is the weighted distance from contact i to attractorj,
and contact i and attractor j are considered assigned to one
The factor is clipped to be greater than one since an inneranotherwhennpj. This combinatorial optimizationproblem,
most separation less than the average can occur regardless of
known more specifically in mathematics as an assignment 65 whether thumb or index fmger is the innermost touching
problem, can be efficiently solved by a variety ofwell-known
finger. In case there are only two finger contacts, a default
mathematical techniques, such as branch and bound, local-
average separation of 2-3 cm is used. The factor tends to
Copy provided by USPTO from the PIRS Image Database on 04/25/2011
APLNDC00030393
Disclaimer: Justia Dockets & Filings provides public litigation records from the federal appellate and district courts. These filings and docket sheets should not be considered findings of fact or liability, nor do they necessarily reflect the view of Justia.
Why Is My Information Online?