Google Inc. v. Rockstar Consortium US LP et al

Filing 134

MOTION for Issuance of Letters Rogatory to the Superior Court of Justice of Ontario, Canada for Nortel Networks Corporation, Jean-Pierre Fortin, Angela de Wilton, Jaspreet Harit, Yee-Ning Chan, Brian Finlay Beaton, Bruce Dale Stalkie, Mitch A. Brisebois, Laura A. Mahan, Paul Michael Brennan, Brian Cruickshank, and John Eric Lumsden filed by Google Inc.. (Attachments: # 1 Exhibit A to Google's Notice of Unopposed Motion and Motion for Issuance of Letter Rogatory, # 2 Declaration of Kristin J. Madigan In Support of Google's Unopposed Motion for Issuance of Letter Rogatory, # 3 Exhibit 1, # 4 Exhibit 2, # 5 Exhibit 3, # 6 Exhibit 4, # 7 Exhibit 5, # 8 Exhibit 6, # 9 Exhibit 7, # 10 Exhibit 8, # 11 Exhibit 9, # 12 Exhibit 10, # 13 Exhibit 11, # 14 Exhibit 12, # 15 Exhibit 13, # 16 Exhibit 14, # 17 Exhibit 15, # 18 Exhibit 16, # 19 Exhibit 17, # 20 Exhibit 18, # 21 Exhibit 19, # 22 Exhibit 20, # 23 Exhibit 21, # 24 Proposed Order)(Curran, Patrick) (Filed on 9/29/2014) Modified on 9/30/2014 (cpS, COURT STAFF).

EXHIBIT 3

Case5:13-cv-05933-PSG Document1-2 Filed12/23/13 Page1 of 19

EXHIBIT B

Case5:13-cv-05933-PSG Document1-2 Filed12/23/13 Page2 of 19
US006037937A

Ulllted States Patent [19]

[11] Patent Number:

Beaton et al.

[45]

[54]

Date of Patent:

6,037,937
*Mar. 14, 2000

NAVIGATION TOOL FOR GRAPHICAL

5,745,716

4/1998 Tchao et al. .......................... .. 345/350

USER INTERFACE

5,748,926

5/1998 Fukuda et al. ..

5,760,773

6/1998 Berman et al.

[75] Inventors: Brian Finlay Beaton, Orleans; Colin
-

’

[73]

,

25521212165555: Dale

,

1e

9/1998 Moran et al. ......................... .. 345/358

5,812,134

’

9/1998

Pooser et al. ......................... .. 345/356

FOREIGN PATENT DOCUMENTS

This patent issued on a continued pros-

8

13;

European 5“?

-

uropean a‘

"

ecution application ?led under 37 CFR

1.53(d), and is subject to the tWenty year

0 684 543 11/1995

patent term provisions of 35 U.S.C.
154 a

2 .

( )
[21] AppL NO‘: 08/985,265
-

[22]

European Pat‘ Off’ '

OTHER PUBLICATIONS

IBM Technical Disclosure Bulletin, “Scroll Control Box,”
vol. 36, No. 4, Apr. 1993, pp. 399—403.

_

Flled'

.................................... ..

5,809,267

Assignee: Nortel Networks Corporation, CAX

[ * ] Notice:

345/347

g?ig?matsu 9t 91

_

223$?

. 345/358 X

Primary Examiner—Raymond J. Bayerl

Dec‘ 4’ 1997

Assistant Examiner—Thomas Nguyen

[51]

Int. Cl.7 ............................... .. G06F 3/14; G06F 3/02

Attorney, Agent, or Firm—Finnegan, Henderson, FaraboW,

[52]

US. Cl. ........................ .. 345/339; 345/352; 345/347;

Garrett & Dunner, L~L-P

345/341; 345/358
[58]

[57]

ABSTRACT

Field of Search ................................... .. 345/358, 352,

345/353, 341, 354, 350, 348, 347, 356

The system and method consistent With the present inven

tion provides improved graphical navigation tool for elec
[56l

References Cited
Us‘ PATENT DOCUMENTS

tronic devices. The graphical navigation tools may be used
to navigate through documents at any direction at varying
speeds. The navigation tool is preferably transparent to

4,954,967

9/1990 Takahashi ............................. .. 364/518

prevent obscuring the teXt of the underlying document.

5,297,253

3/1994 Meisel

395/348

Accordingly, the navigational tool does not take up addi

5a563a996 10/1996 Tchao

345/341

tional screen real estate. The GUI of the present invention

575707109 10/1996 Jens?“

345/146

5,590,257

12/1996

may be especially useful in portable electronic devices With

Forcler ....... ..

.. 707/530

Small dis 1a S

5,592,195

1/1997 Misono e161.

345/146

p y‘

5,600,765

2/1997 Ando et al. ....................... .. 345/326 X

5,745,116

4/1998 Pisutha-Armond ................... .. 345/358

24 Claims, 11 Drawing Sheets

MACROMEDI
SEARCH] BRAIN | lNDU
l l

MACROMEDIA
SHOCKWAVE

:EHAND

Case5:13-cv-05933-PSG Document1-2 Filed12/23/13 Page3 of 19

U.S. Patent

Mar. 14,2000

Sheet 1 0f 11

FIG. 1

FIG. 2

6,037,937

Case5:13-cv-05933-PSG Document1-2 Filed12/23/13 Page4 of 19

U.S. Patent

Mar. 14, 2000

_
:8
2a

.%ovD63aa
L

Sheet 2 0f 11

05.3%

2a3.5“ N-215m 8m2- 128

6,037,937

_
J
m
m

GI
mm.

Case5:13-cv-05933-PSG Document1-2 Filed12/23/13 Page5 of 19

Case5:13-cv-05933-PSG Document1-2 Filed12/23/13 Page6 of 19

U.S. Patent

Mar. 14, 2000

Sheet 4 0f 11

6,037,937

MEMORY 440

STATIC RAM 442

FLASH ROM 444

PROGRAM
510

TOUCH
SCREEN
PROGRAM
520

NAVIGATION
PROGRAM
530

FIG. 5

DRAWING
PROGRAM
540

Case5:13-cv-05933-PSG Document1-2 Filed12/23/13 Page7 of 19

U.S. Patent

Mar. 14, 2000

(430
FEATURE
PROCESSOR

Sheet 5 0f 11

[460
‘ ‘
‘

'

ANALOG
CONTROLLER

6,037,937

(474
H

SEQUECEHN
PANEL

FIG. 6

4}

06>
FIG. 7A

ACTIVATION
AREA

FIG. 7B

Case5:13-cv-05933-PSG Document1-2 Filed12/23/13 Page8 of 19

U.S. Patent

Mar. 14, 2000

'

Sheet 6 0f 11

MACROMEDI

SIJIEARCH BRAIN INDU
O

MACROMEDIA
SHOCKWAVE

FIG. 8

6,037,937

Case5:13-cv-05933-PSG Document1-2 Filed12/23/13 Page9 of 19

U.S. Patent

Mar. 14, 2000

Sheet 7 0f 11

6,037,937

WITHIN-PAGE
GATION
RCLE

0111

"U m lT'l S
PAG

_/E§€§E
FIG. 98
ADDITIONAL CONTROLS
FORWARD TO
T DOC

Bil

BAC K, HOME,

IIHI

FIG. 9C

.

NT,

Case5:13-cv-05933-PSG Document1-2 Filed12/23/13 Page10 of 19

U.S. Patent

Mar. 14, 2000

Sheet 8 0f 11

6,037,937

HDMEOU<
_O_DZI<NwZ_mw

‘mm
296

QZm/E8W
206
40
9.
Hz
I

EwmFZwO

HM

.3:QE

Case5:13-cv-05933-PSG Document1-2 Filed12/23/13 Page11 of 19

U.S. Patent

Mar. 14, 2000

Sheet 9 0f 11

6,037,937

VIEWING WINDOW DOES NOT

MOVE (ACTIVATION AREA)
VIEWING WINDOW
MOVES SLOWLY

VIEWING WINDOW
MOVES QUICKLY

FIG. 11A
VIEWING WINDOW MOVES
QUICKLY TO THE PREVIOUS PAGE

NEXT PAGE .2
ICON

vIEWING WINDOW MOVES

;

SLOWLY TO THE PREVIOUS PAGE

VIEWING WINDOW MOVES
_{

SLOWLY TO THE NEXT PAGE

PREVIOUS PAGE :“\
ICON

vIEWING WINDOW MOVES

QUICKLY TO THE NEXT PAGE

FIG. 11B

Case5:13-cv-05933-PSG Document1-2 Filed12/23/13 Page12 of 19

U.S. Patent

Mar. 14, 2000

Sheet 10 0f 11
TOUCH POINT
DISTRIBUTION

FIG. 12

1310
DETERMINE INDIVIDUAL POINTS
OF CONTACT MADE BY OBJECT
V

130

COMPUTE CENTROID OF
DETERMINED POINTS OF CONTACT
V

130

COMPUTE STANDARD
DEVIATION OF CENTROID

1340
DETERMINE POINTER OF THE
OBJECT MAKING CONTACT

FIG. 13

6,037,937

Case5:13-cv-05933-PSG Document1-2 Filed12/23/13 Page13 of 19

U.S. Patent

Mar. 14,2000

Sheet 11 0f 11

6,037,937

PRE‘SSURE

PEN

TIME

FIG. 14A

PRESSURE

A FINGER
TlME

FIG. 14B

Case5:13-cv-05933-PSG Document1-2 Filed12/23/13 Page14 of 19

6,037,937
1

2

NAVIGATION TOOL FOR GRAPHICAL
USER INTERFACE

tions in multiple layers, thus requiring a user to move deeply

into the hierarchy before reaching the desired function. This
is time consuming and renders the GUI cumbersome and
ineffective.
Therefore, it is desirable to provide navigation tools that
alloW small-siZe devices While maximiZing the use of avail

RELATED APPLICATIONS

This application is related to US. patent application, Ser.

No. 08/985,264 (Atty. Docket No. 3884.0116-00), entitled
INTELLIGENT TOUCH DISPLAY; and US. patent

able screen real estate.

application, Ser. No. 08/985,561 (Atty. Docket No.
3884.0117-00), entitled CONTEXTUAL GESTURE

It is also desirable to provide tools to navigate Within a
document at any direction at varying speeds.
It is further desirable to provide navigation tools that can
be activated Without requiring speci?c electronic devices.
In addition, it is further desirable to provide an improved

INTERFACE, both of Which are ?led concurrently hereWith,
and both of Which are hereby incorporated by reference.

10

BACKGROUND OF THE INVENTION

The present invention relates generally to graphical user
interfaces (GUI), and more particularly to a navigation tool
for graphical user interfaces.

15

SUMMARY OF THE INVENTION

Until relatively recently, softWare-based documents have
been primarily vieWed and manipulated on desktop or laptop

computers With relatively large displays, typically 640-480

GUI that simpli?es GUI by recogniZing various character
istics of the touch input.

Systems and methods consistent With the present inven
20

tion provide graphical control tools for ef?cient navigation
in display devices.

pixels or larger. These displays are often large enough to

Speci?cally, a method consistent With this invention of

display a full page of standard siZe page or at least a

controlling display of content information in a physical

signi?cant portion of the page. Hence, on-screen graphical
menus and controls displayed in WindoW of an application

did not greatly reduce the display area for the underlying
document. Computers also have peripheral devices such as

25

a keyboard or a mouse to control the display of content

information. Thus, vieWing and navigating around a single
page or multi-page document have not posed much dif?

culty.

30

Due to increasing focus on compactness of electronic

devices, hoWever, the displays especially in portable elec

means. The displaying means displays the content informa

electronic devices With a smaller display area include elec
35

tion in the physical vieWing area, and displays a represen
tation of a control tool over the display of content informa
tion. The receiving means receives a user input selecting the

graphical display-based telephones. Also available today are
communicators that facilitate various types of communica

control tool. Thereafter, the controlling means controls the
display of the content information according to the user

tion such as voice, faxes, SMS (Short Messaging Services)
messages, e-mail, and Internet-related applications. These
products can likeWise only contain a small display area.

area. The system also displays a representation of a control
tool over the display of content information. Thereafter, the
system receives a user input selecting the control tool, and
controls the display of content information according to the
user input.
A system consistent for this invention for controlling
display of content information in a physical vieWing area

includes displaying means, receiving means, and controlling

tronic devices are becoming smaller and smaller. Popular

tronic organiZers, PDA’s (personal digital assistants), and

vieWing area comprises several steps. Initially, the system
displays the content information in the physical vieWing

40

input.
BRIEF DESCRIPTION OF THE DRAWINGS

To enable users to navigate around a full page of content

information, these devices typically provide hard-keys for
arroWs as shoWn in FIG. 1. The hard-keys, hoWever, not only
increase the siZe but also add to the cost of the devices. Also,

hard-keys generally provide limited options for direction of

The accompanying draWings, Which are incorporated in
45

movement, e.g., vertical or horiZontal. They generally do not
provide the freedom to move in any direction.

In the draWings,
FIG. 1 shoWs conventional hard-key arroWs for naviga

Some displays of these devices also require a separate

tion control;

stylus having peripheral technology that requires transmis
sion of electromagnetic pulses or light to the display. These

FIG. 2 shoWs conventional on-screen graphical naviga

devices often require additional controllers such as buttons
on the body or the tip of the stylus for activation.
Furthermore, these styli require a poWer source, either

through Wire or battery, and their compatibility is generally

tion tool;
FIGS. 3A—3B are diagrams of an exemplary mobile

telephone consistent With the principles of the present inven
55

limited to a speci?c device.

mobile telephone of FIG. 3A;

With graphical on-screen arroWs or scroll bars that are

FIG. 5 is a block diagram shoWing the components of the
memory of FIG. 4;

typically used in full-siZe computer displays. The on-screen
60

compound the limitations of small displays. Similar to the
hard-keys, the on-screen arroWs also generally restrict the
navigational movement to horiZontal or vertical direction.
In other forms of on-screen GUIs, e.g., pop-up menus,
also take up valuable screen space, further reducing the
available display area for content information. Additionally,
on-screen pop-up menus typically provide available func

tion;
FIG. 4 is a block diagram shoWing the elements of the

As shoWn in FIG. 2, other devices substitute hard-keys
scroll bars, hoWever, occupy valuable screen real estate and

and constitute a part of this speci?cation, illustrate the
invention and together With the description, serve to explain
the principles of the invention.

FIG. 6 is a block diagram of touch screen functionalities;
FIGS. 7A—7B shoW an exemplary inactive and active

graphical navigation tool, respectively;
FIG. 8 is a sample screen shoWing an active navigation
65

tool;
FIGS. 9A—9C shoW exemplary features of the navigation

tool;

Case5:13-cv-05933-PSG Document1-2 Filed12/23/13 Page15 of 19

6,037,937
4

3

nals and supporting GSM protocols. BOCK 425 supplies the
digital audio signals to VEGA 423 for digital-to-analog
conversion. VEGA 423 applies the analog audio signals to

FIGS. 10A—10C are sample screens showing the naviga

tion tool performing various navigation functions;
FIGS. 11A—11B shoW exemplary features of the naviga
tion tool relating to speed of navigation;

audio transducers 427. Audio transducers 427 include
speaker 428 and microphone 429 to facilitate audio com
munication by the user.
Feature processor 430 provides GUI features and a Java
Virtual Machine (JVM). Feature processor 430 communi
cates With BOCK 425 using high level messaging over an

FIG. 12 is a diagram illustrating a touch point distribution;
FIG. 13 is a ?owchart illustrating the process of deter

mining the siZe of the object making contact With the
vieWing area; and
FIGS. 14A and 14B are graphs shoWing the touch char
acteristics of a pen and a ?nger, respectively.

asynchronous (UART) data link. Feature processor 430
contains additional system circuitry, such as a liquid crystal

display (LCD) controller, timers, UART and bus interfaces,

DESCRIPTION OF THE PREFERRED
EMBODIMENT

Reference Will noW be made in detail to the present

and real time clock and system clock generators (not
15

preferred embodiment of the invention, an example of Which
is illustrated in the accompanying draWings. Where

Memory 440 stores data and program code used by
feature processor 430. Memory 440 includes static RAM
442 and ?ash ROM 444. Static RAM 442 is a volatile

appropriate, the same reference numerals refer to the same

memory that stores data and other information used by
feature processor 430. Flash ROM 444, on the other hand,

or similar elements. The appended claims de?ne the scope of

the invention; the folloWing description does not limit that

is a non-volatile memory that stores the program code

scope.

executed by feature processor 430.
Sliding keypad 450 enables the user to dial a telephone
number, access remote databases, and manipulate the GUI

The graphical navigation tool of the present invention
may be implemented in a Wide range of electronic devices
mentioned above such as electronic organiZers, PDA’s, and

graphical display-based telephones. Although the need to

shoWn).

25

maximiZe the use of screen real estate is most critical in

portable electronic devices With small displays, the present
invention can also be implemented in full-siZe computers or

features. Sliding keypad 450 preferably includes a mylar
resistive key matrix that generates analog resistive voltage in
response to actions by the user. Sliding keypad 450 prefer
ably connects to main housing 210 (FIG. 3A) of mobile
telephone 310 through tWo mechanical “push pin”-type

electronic devices. For purposes of illustration, hoWever, the
present invention Will be explained in detail in a mobile

contacts (FIG. 4).

telephone environment.

device that acts as an interface betWeen feature processor

Analog controller 460 is preferably a Phillips UCB1100

Speci?cally, FIG. 3A shoWs a mobile telephone 310 and

430 and sliding keypad 450. Analog controller 460 converts

FIG. 3B shoWs an exemplary Wireline telephone preferably
having the graphical navigation tool consistent With the
present invention. Mobile telephone 310 includes main

digital signals for presentation to feature processor 430.
Display module 470 preferably includes a 160x320 pixel

the analog resistive voltage from sliding keypad 450 to
35

housing 210, antenna 320, keypad 330, and display 340.

LCD 472 With an analog touch screen panel 474 and an

FIG. 4 shoWs the hardWare elements in mobile telephone

electroluminescent backlight. LCD 472 operates in conjunc

310 including antenna 410, communications module 420,

tion With feature processor 430 to display the GUI features.
Analog controller 460 scans touch screen overlay 474 While
feature processor 430 refreshes LCD 472.

feature processor 430, memory 440, sliding keypad 450,

analog controller 460, display module 470, battery pack 480,
and sWitching poWer supply 490.

Battery pack 480 is preferably a single lithium-ion battery
With active protection circuitry. SWitching poWer supply 490

Antenna 410 transmits and receives radio frequency infor
mation for mobile telephone 310. Antenna 410 preferably
comprises a planar inverted F antenna (PIFA)-type or a short
stub (2 to 4 cm) custom helix antenna. Antenna 410 com
municates over a GSM (Global System for Mobile

ensures highly ef?cient use of the lithium-ion battery poWer
45

by converting the voltage of the lithium-ion battery into
stable voltages used by the other hardWare elements of
mobile telephone 310.

Communications) sWitching fabric using a conventional

FIG. 5 is a block diagram illustrating the components of

voice B-channel, data B-channel, or GSM signaling channel
connection.

memory 440. Static RAM 442 stores data and other infor
mation used by feature processor 430. Flash ROM 444
contains various programs including a program 510, a touch
screen program 520, a navigation program 530, and a

Communications module 420 connects to antenna 410

and provides the GSM radio, baseband, and audio function
ality for mobile telephone 310. Communications module
420 includes GSM radio 421, VEGA 423, BOCK 425, and
audio transducers 427.
GSM radio 421 converts the radio frequency information
to/from the antenna into analog baseband information for
presentation to VEGA 423. VEGA 423 is preferably a Texas

draWing program 540. Program 520, preferably Written in
languages such as Java, C, or C++ for Macintosh, is a main
55

program overseeing the operation of mobile telephone 310.
Touch screen program 520 facilitates processing of touch

input on touch screen panel 474 using a typical touch input

algorithm. Navigation program 530 handles navigation of

Instruments VEGA device, containing analog-to-digital
(A/D)/digital-to-analog (D/A) conversion units 424. VEGA

the content information display. DraWing program 540 is a

423 converts the analog baseband information from GSM
radio 421 to digital information for presentation to BOCK
425.
BOCK 425 is preferably a Texas Instruments BOCK
device containing a conventional ARM microprocessor and
a conventional LEAD DSP device. BOCK 425 performs

may be one of any commercially available packages or a

GSM baseband processing for generating digital audio sig

graphical draWing package. Programs 520, 530, and 540
user-de?ned feature program or macro.

The present invention provides various features through
tactile GUI. Initially, LCD 472 displays various GUI fea
65

tures. Referring to FIG. 6, a user touches touch screen panel

474 to provide user input, for example, to navigate around
a document or invoke a desired function. Analog controller

Case5:13-cv-05933-PSG Document1-2 Filed12/23/13 Page16 of 19

6,037,937
5

6

460 scans touch screen panel 474 and reads the correspond

not display the respective previous or next page icons. This

ing analog voltage of touch screen panel 474. Analog

Would apply to one-page documents, or When the user is at
the beginning or end of a multi-page document. In one

controller 460 then converts the analog values into corre

sponding digital values representing the Cartesian
coordinates, Which are transmitted to feature processor 430

embodiment consistent With the present invention, a
momentary touch of the next page icon causes navigation

for processing. The resolution of the touch input depends on
the ability of analog controller 460 to discern among mul

program 530 to jump to the next page While a continuous
touch on the next page icon causes navigation program 530

tiple levels of analog values, generally de?ned in bits.

to continue scrolling through succeeding pages of the under
lying document. The previous page icon may embody simi

FIGS. 7A—7B shoW an exemplary graphical navigation
tool preferably used to navigate around documents that are
too large to vieW Within a single screen of a physical display

10

lar characteristics.
The user may also control the speed of the navigation. As

shoWn in FIG. 11A, the speed of the navigation accelerates

(hereinafter referred as “vieWing area”). The navigation tool
may be used to vieW any kind of document including faxes,

as the user touch moves from the center of the circle toWard

Web pages, or e-mail. In one embodiment consistent With

the circumference of the circle, i.e., tip of the arroW. Hence,

the present invention, an inactive navigation tool is dis
played and accessible to the user at all times (FIG. 7A). The
user may activate the navigation tool by touching and
holding the center of the navigation tool for a predetermined
time period, for example, one to tWo seconds (FIG. 7B). An
activated navigation tool is preferably transparent to avoid
hindering the display of content information in the vieWing

15

toWards the tip of the arroW. The speed of navigation,
therefore, is determined by the distance of the touch relative
to the center of the circle. LikeWise, similar principles apply
to previous or next page/document icons Where a touch
closer to the outer edge of the previous or next page/

area as shoWn in FIG. 8. Alternatively, the navigation star
may change colors or other features of its appearance to

indicate its active status. A solid line image, for example,
may be used in greyscale displays that do not support

the vieWing WindoW moves sloWly When the user touches
the blunt end of the arroW located at the center of the circle
While the speed accelerates as the user moves the ?nger

document icons accelerates navigation through the docu
ment as shoWn in FIG. 11B.
25

transparency.

Although the exemplary transparent tool discussed above
is for navigation, transparent control tools may be imple

The present invention may be designed such that feature
processor 430 ignores any touch input on the navigation tool
unless the navigation tool has been activated. Instead, the
touch input may be interpreted as input to access control
buttons in the underlying document, Write on the underlying
document, or invoke other functions related to the underly

mented for a variety of functions. Atransparent tool may, for
example, be used for a Web broWser application Where the
controls may be used for appropriate functions such as
moving forWards or backWards through different Web pages
or returning to home page. One skilled in the art may easily
vary the design or the functionality of the graphical navi

ing document. This Will prevent against unintentional navi

gation tools described above Without departing from the

gation in the vieWing WindoW in case the user inadvertently

scope of the present invention.
In an exemplary embodiment of a navigation tool
described above, a ?nger touch invokes navigational func

touches touch screen panel 474. In an alternative 35

embodiment, the present invention may accept stylus input
to access the underlying document While a ?nger or non

tions based on the feature selected and the location of the

electromagnetic touch on any part of the navigation tool
invokes the navigation function.
Referring to FIGS. 9A—9C, once the navigation tool is
activated, the user may navigate through the document by

user touch. Alternatively, other objects making contact With
touch screen panel 474 may invoke other tools or functions.

Apointy stylus touch, for example, may invoke a menu With

cardinal points representing multiple line Widths, colors, or
patterns.

selecting the graphical arroWs, e.g., up, right, left, and doWn
arroWs (FIG. 9A), or graphical page icons, e.g., previous or
next page (FIG. 9B). One skilled in the art may vary the type

and number of graphical tools signi?cantly. For example, the

45

navigation tool may provide graphical representations for
forWard, next document, back, or home functions (FIG. 9C).
FIGS. 10A—10C shoW exemplary screen displays While
the user is touching the navigation tool. Upon touching the
right arroW of the navigation tool, for example, the right

application normally navigable by a ?nger touch, use of a
sharp stylus may automatically invoke a document annota

arroW is highlighted and navigation program 530 moves the

tion application for marking up the underlying document.
As described above, the touch-responsive GUI of the

display to the right (FIG. 10A). Similarly, touching the doWn
arroW moves the display doWn (FIG. 10B). Although the
four arroWs are presented to guide the users, navigation
program 530 supports navigational movement at any direc

55

equidistant betWeen the up and right arroWs, for example,

a particular document does not have a page corresponding to
a previous or next page icon, navigation program 530 Will

present invention are facilitated though various components

including touch screen panel 474, analog controller 460, and
feature processor 430. Speci?cally, analog controller 460

tion. If the user touches an area of the navigation tool

navigation program 530 Will move the display toWards the
upper-right portion of the underlying document at a
45-degree angle. Touching the arroWs or any area in
betWeen, moves the display in the selected direction until
navigation program 530 reaches the edge of the page.
Touching the next page icon moves the vieWing WindoW
to the next page of the underlying document (FIG. 10C). If

In another embodiment consistent With the present
invention, tools or application programs may be stored in
?ash ROM 444 to provide related interfaces to the user. The
use of a ?nger may, for example, invoke tools or dialogues
that are ?nger-touchable and large Whereas the use of a sharp
stylus may invoke a modi?ed GUI With smaller touch
targets. In a yet another embodiment, in a document vieWing

scans touch screen panel 474 to read the corresponding
analog voltage of touch screen panel 474 activated by a user

touch. Analog controller 460 then converts the analog values
into a digital value representing the Cartesian coordinates,
Which is transmitted to feature processor 430 for processing
according to the functionalities of the present invention.
65

When a user touches touch screen panel 474, program 510
initiates touch screen program 520 to determine the pointer

siZe of the object making contact With touch screen panel
474 based on a touch point distribution or pointer siZe of the

Case5:13-cv-05933-PSG Document1-2 Filed12/23/13 Page17 of 19

6,037,937
7

8

touch input. As shown in FIG. 12, touch screen program 520
can, for example, determine Whether the pointer size of the
object is a ?nger or a sharp object.
FIG. 13 is a ?owchart illustrating the process of deter

quick drag to the right may invoke a draWing tool to draW

from the starting point to the ending point of the touch
points. In a document vieWing application, the same touch
may invoke a navigation tool to move the vieW of the

mining the siZe of the object making contact With the
vieWing area. Touch point program 520 ?rst determines the

document in the direction of the ?nger drag.
All of the above functions and features described above
focuses on providing intuitive GUIs and minimiZe the need

individual points of contact made by the object (step 1310).
It computes a centroid, or other average point, of the

for users to memoriZe complicated, hierarchical menus or

determined points of contact (step 1320). Touch program
520 then computes a standard deviation of the centroid as

procedures. Additionally, the present invention maximiZe
10

Well as the variance (step 1330), and determines the pointer
siZe based on the centroid and the standard deviation (step
1340). These computations are preferably performed on a
real-time basis to provide immediate system response to the
touch input. In order to achieve optimum results and

accuracy, analog touch controller 460 preferably generates

15

150 points per second or more. Touch program 520 may also
use the amount of pressure imposed on touch screen panel
474 as a function of time to determine the siZe of object. As

shoWn in FIG. 14A, for example, if the amount of pressure
increases or decreases sharply at a particular instant in time,
touch point program 520 may determine that the touch
corresponds to a pen. A ?nger touch, on the other hand,
results in a gradual increase and decrease in pressure as
illustrated by a smoother curve in FIG. 14B.

the present invention and in construction of this system
Without departing from the scope or spirit of the invention.
Other embodiments of the invention Will be apparent to
those skilled in the art from consideration of the speci?ca
tion and practice of the invention disclosed herein. The
speci?cation and examples should be considered as exem
plary only, With the true scope and spirit of the invention

indicated by the folloWing claims.
What is claimed is:
1. A method of activating functions responsive to a user

input, comprising:
25

Program 510 can also be programmed to correlate certain

pointer siZe to certain objects and invoke corresponding
functions or tools. Such GUI provides a richer, yet simpli?ed
interaction betWeen the user and mobile telephone 310. If
program 510 determines that the pointer siZe of the object
corresponds to the siZe of a ?nger, program 510 may initiate
a navigation tool. If the pointer siZe corresponds to the siZe
of several ?ngers, program 510 may invoke a drag function
of the navigation tool. On the other hand, if program 510
determines that the pointer siZe of the object corresponds to

available screen real estate While providing a Wide array of
GUI and tools.
It Will be apparent to those skilled in the art that various
modi?cations and variations can be made in the system of

providing a manipulable area portion in a physical vieW
ing area, said manipulable area portion having at least
one manipulation function associated thereWith;
displaying a representation of a control tool overlaying
the manipulable area portion, said control tool having
at least one control tool function associated thereWith;
receiving a user input to the physical vieWing area cor

responding to the manipulable area portion and the

representation of the control tool;
35

siZe of a sharp point or pen, program 510 may initiate a

draWing tool supported by draWing program 540. Similarly,

determining if the user input selects the control tool; and
activating the at least one manipulation function When the
user input does not select the control tool or permitting
the at least one control tool function to be activated
When the user input does select the control tool.
2. The method of claim 1, Wherein the at least one control
tool function includes a function of changing a display of

if program 510 determines that the pointer siZe of the object
corresponds to siZe of a pencil eraser, program 510 may
initiate an erase function of the draWing tool. One skilled in
the art may easily vary the functions or tools initiated by
program 510. Additionally, the functions or tools may be

information in the manipulable area portion, the method

commercial softWare packages, predetermined functions, or

further including:

user-de?ned macros.

In addition to using the pointer siZe to determine the
desired GUI, program 510 can also incorporate other char

after at least one control tool function is permitted to be
45

activated, receiving a subsequent user input selecting a

predetermined portion of the control tool; and
activating the function of changing a display of informa

acteristics of the user touch, e.g., gestures or movements, to
simplify GUI and maximiZe screen real estate. A gesture

track gestures and movement of user input based on vector
direction and magnitude, all in the context of active user

tion in response to the subsequent user input.
3. The method of claim 2, Wherein the function of
changing a display of information includes shifting infor
mation in the manipulable area portion.
4. The method of claim 3, Wherein the shifting includes

application. This type of contextual gesture interface can

scrolling.

recogniZing interface extends the ability of the present
invention to distinguish betWeen different siZed pointers to

infer by context, the implement, and the gesture chosen by

5. The method of claim 3, Wherein the shifting includes

the user What functions the user Wishes to invoke. 55

jumping.

Accordingly, all these functions are available Without menus
or scroll bars and do not require additional screen areas to

6. The method of claim 2, Wherein the function of
changing a display of information includes control of the
speed of the change of the display of information based on
the subsequent user input.
7. The method of claim 2, Wherein the changing includes

display the functions.
Program 510 recogniZes other characteristics of the touch
input including the context of the input, namely the task or
sub-task applications running When the GUI is invoked. If a
user is in a document navigation application, for example,
program 510 interprets a quick drag to the right as a next
page function. If the underlying task is an editing
application, program 510 may interpret the same gesture as
a highlight function and highlight a portion of the document
touched by the user. Similarly, in graphics application, a

displaying a different one of a plurality of pages of infor
mation.
8. The method of claim 1, Wherein the displaying a
representation of the control tool includes displaying arroWs
65

indicating direction.
9. The method of claim 1, Wherein the information
displayed in the manipulable area portion includes a portion

Case5:13-cv-05933-PSG Document1-2 Filed12/23/13 Page18 of 19

6,037,937
9

10

of a document and wherein the at least one control tool
function includes a function related to the document.

means for activating the function of changing a display of
information in response to the subsequent user input.
15. The apparatus of claim 14, Wherein the function of
changing a display of information includes shifting infor
mation in the manipulable area portion.

10. The method of claim 1, Wherein the displaying the
representation of the control tool includes displaying a

virtually transparent representation of the control tool.
11. The method of claim 1, Wherein the displaying the
representation of the control tool includes displaying an
opaque representation of the control tool.
12. The method according to claim 1, Wherein the at least
one manipulation function includes an editing function for a
document.

16. The apparatus of claim 15, Wherein the shifting

includes scrolling.
17. The apparatus of claim 15, Wherein the shifting
10

18. The apparatus of claim 14, Wherein the function of
changing a display of information includes control of the
speed of the change of the display of information based on
the subsequent user input.
19. The apparatus of claim 14, Wherein the changing

13. An apparatus for activating functions responsive to a

user input, comprising:
means for providing a manipulable area portion in a

physical vieWing area, said manipulable area portion
having at least one manipulation function associated

thereWith;

includes displaying a different one of a plurality of pages of
information.
20. The apparatus of claim 13, Wherein the means for
displaying a representation of the control tool includes

means for displaying a representation of a control tool

overlaying the manipulable area portion, said control
tool having at least one control tool function associated

thereWith;

displaying arroWs indicating direction.
21. The apparatus of claim 13, Wherein the information
displayed in the manipulable area portion includes a portion

means for receiving a user input to the physical vieWing
area corresponding to the manipulable area portion and

the representation of the control tool;
means for determining if the user input selects the control

tool; and

25

means for activating the at least one manipulation func
tion When the user input does not select the control tool
or permitting the at least one control tool function to be
activated When the user input does select the control
tool.
14. The apparatus of claim 13, Wherein the at least one
control tool function includes a function of changing a

predetermined portion of the control tool, after at least
one control tool function is permitted to be activated,
and

of a document, and Wherein the at least one control tool
function includes a function related to the document.

22. The apparatus of claim 13, Wherein the means for

displaying the representation of the control tool includes
displaying a virtually transparent representation of the con
trol tool.
23. The apparatus of claim 13, Wherein the means for

displaying the representation of the control tool includes
displaying an opaque representation of the control tool.
24. The apparatus according to claim 13, Wherein the at

display of information in the manipulable area portion, the
apparatus further including:
means for receiving a subsequent user input selecting a

includes jumping.

35

least one manipulation function includes an editing function
for a document.

Case5:13-cv-05933-PSG Document1-2 Filed12/23/13 Page19 of 19

UNITED STATES PATENT AND TRADEMARK OFFICE

CERTIFICATE OF CORRECTION
PATENT NO. : 6,037,937

DATED

= March 14, 2000

INVENTOR(S) :

.

Bnan Beaton, et 211.
It is certified that error appears in the above-identified patent and that said Letters Patent is hereby
corrected as shown below:

Title page, col. 1, item [73], after "Assignee: Nortel Networks Corporation,"
"CAX" should be --CANADA--.

Signed and Sealed this

Nineteenth Day of December, 2000

Q. TODD DICKINSON

Arresting O?‘icer

Commissioner of Palenrs and Trademarks