Apple Inc. v. Samsung Electronics Co. Ltd. et al

Filing 559

Declaration of Deok Keun Matthew Ahn in Support of #558 Response ( Non Motion ), filed byApple Inc.. (Attachments: #1 Exhibit 1.01, #2 Exhibit 1.02, #3 Exhibit 1.03, #4 Exhibit 1.04, #5 Exhibit 1.05, #6 Exhibit 1.06, #7 Exhibit 1.07, #8 Exhibit 1.08, #9 Exhibit 1.09, #10 Exhibit 1.10, #11 Exhibit 1.11, #12 Exhibit 1.12, #13 Exhibit 1.13)(Related document(s) #558 ) (Jacobs, Michael) (Filed on 12/29/2011)

EXHIBIT 1.02

Battery Monitor
The power adapter is
connected and recharging
one of the batteries.

IIIII

II i (

APLNDC00028362

et No.:
Serial No.:
Filed:
Art Unit:
Sheet No.:
Express Mail:
Inventor(s):

04860.P1365

5 of 17
TB377895617US
Steven W,. Christenr i r:OD AND APPARATUS FOR DISPLAYING
AND ACCESSING CONTROL AND STATUS
INFORMATION IN A CO- = «R SY -

APLNDC00028363

IIIn

II

F/ (7URE

III

2E

APLNDC00028364

Doiet No.:
Serial No.:
Filed:
Art Unit:
Sheet No.:
Express Mail:
Inventor(s):
14e

a..

04860.P1365

If of 17
TB377895617US
Steven W,. Christensen
METHOD Aarn APPARATUS FOR DISPLAYING
AND ACCESSING COi, mOL AND STATUS
INFORM^TION IN A COMPUTER SYSTEM

APLNDC00028365

2/l

0000Dimlli
IIIII

II i
Fawee 2 F

APLNDC00028366

i

Docket No.:
Serial No.:
Filed:
Art Unit:
Sheet No.:
Express Mail:
Inventor(s):

04860.P1365

Titler

Ma · OD ANrn APPARATUS FOR DISPLAYING
arn ACCESSING CONTROL AND STATUS
imuRMATION IN A COMPUTER SYSTEM

'7 of 17
TB377895617US
Steven W,. Christensen

APLNDC00028367

Roy

08/M6289

Control Strip
Show Control Strip
O Hide Control Strip

APLNDC00028368

Docket No.:
04860.P1365
Serial No.:
Filed:
Art Unit:
Sheet No.:
Ñ of 17
Express Mail: TB377895617US
Inventor(s): Steven W,. Christensen

Title:

METHOD AND APPARATUS FOR DISPLAYING
AND AC= ---ING COis mOL AND STATUS
INFORMATION IN A COMPUTER SYS -

APLNDC00028369

MAIN LOOP
called by operating system)

*

save previous context,
setup strip's context

secondary
initialization
mplete

no

run secondary
initialization

got

yes

determine the user
event type

- 404

null event (idle)?

yes

no

40s

mouse click?

run idle tasks

/4c k

yes

process mouse click

no
Y

restore previous
context

EXIT

Fa

4

APLNDC00028370

Docket No.:
Serial No.:
Filed:
Art Unit:
Sheet No.:
Express Mail:
Inventor(s):

..

04860.P1365

9 of 17
TB377895617US
Steven W,. Christensen

METHOD AND APPARATUS FOR DISPLAYING
AND ACCESSING COis 1xOL AND STATUS
INFORM^TION IN A COMPUTER SYSTEM

APLNDC00028371

Control Strsp Main Loop
Irun secondary in tla zat'on
SECONDARY

INITIALIZATION

has the Finder
started up?

m -

yes
T
create the strip's
window

5 1

successful?

no -

yes
T
initialize

's font,

open, initialize extemal
modules
So

-

set strip's default
screen location, size

?
So

load user configuration

?

Fog

dnsw e

(

s

EXIT

APLNDC00028372

Bapket No.:
Serial No.:
Filed:
Art Unit:
Sheet No.:
Express Mail:
Inventor(s):
·

·

04860.P1365

10 of 17
TB377895617US
Steven W,. Christensen
METHOD AND APP^ ^TUS FOR DISPLAYING
AND AC. --,ING CONTROL AND STATUS
INFORMATION IN A CO- · · - SYS> rx

APLNDC00028373

Control Strsp Masn Loop
lopen. nitial ze external
modu es
OPEN, INITIALIZE
TERNAL MODULE

more module
files?

m

601

yes
T
open module file

load module code into
memory
I cal module lo Inhialize
itself
call module to get
module's features

†

~605

cal module to get its
display width

EXIT

APLNDC00028374

E

f Ni>.:
04860.P1365
Serial No.:
Filed:
Art Unit:
Sheet No.:
U of 17
Express Mail: TB377895617US
Inventor(s): Steven W,. Christensen
Title:

METHOD AND APP^*^TUS FOR DISPLAYING
AND ACCESSING COis mOL AND STATUS
INFORMATION IN A COmv eK SYSTEM

APLNDC00028375

Çontrei Strsy Ma n Leop
/draw the str'p's contents
DRAW THE STRIP'S
CONTENTS

isthestrip
visible?

i

i

no

no -

modules to
dmw7

no

need to be

button?

draw button
background

no

status-only
background

EXIT

APLNDC00028376

ckÈNo.:
Serial No.:
Filed:
Art Unit:
Sheet No.:
Express Mail:
Inventor(s):
tle

04860.P1365

i ½f 17
TB377895617US
Steven W,. Christensen
a:LIIIOD AND APPARATUS FOR DISPLAYING
AND ACCESSING COiN 1xOL AND STATUS
INFORMATION IN A COmu : R S~ -M

APLNDC00028377

Contro Strio Ma n Loop
Irun die tasks
RUN IDLE TABKS

2 01
window update

drawthe smilis

pending?

oontents

fin7.

no
haa sti is

yee

R amieto save

yes

ca

to

So3

+

no

4

i

'

cai cunant idle nodule
tenmitsidletask
post-process the idle

update iD ci next
nodule to idle

does module

k sale to save

save undule's

need to save

now?

sonings to d

no

'

I update strip a help

EXIT

APLNDC00028378

Däälket No.:
Serial No.:
Filed:
Art Unit:
Sheet No.:
Express Mail:
Inventor(s):

e

o

04860.P1365

1 3 of 17
TB377895617US
Steven W,. Christensen

METHOD AND APPARATUS FOR DISPLAYING
AND AC -ING CONTROL AND STATUS
INFO
^TION IN A COmisv i eK Svar nivi

APLNDC00028379

Çqntrol Strip Masn Lopp
Iprocess mouse c ck
PROCESS MOUSE
CUCK

mousecuck
imide strip?

9of
detemsne which
module was clicked on

½ot
move operation?

cickable
module?

yes

move module's display,
or move entire strip

md

yes

track the - ciok

mousestilin
modulefs

md

unds

I

I
caithemoduleto
process the click

†

post process the click

EXIT

APLNDC00028380

Ðócket No.:
04860.P1365
Serial No.:
Filed:
Art Unit:
Sheet No.:
Sof 17
Express Mail: TB377895617US
Inventor(s): Steven W,. Christensen

Tige

o

METHOD AND APPARATUS FOR DISPLAYING
AND ACCESSING CONTROL AND STATUS
INFORMATION IN A CO-r-o ots SYSTEM

APLNDC00028381

Çontrol Strip Main Lopp
Ipost process the click
(

MODULE POST
PROCESSING

module wants to
update settings?

flag that this module
has a save pending

no

module needs
to resize

cal module to update
its width

no

b a

/005

to

yes e

close the module

a
DD

no

module help
state change?

/009

†

yes r Invalidate old help state

¡nc

DD

EXIT

APLNDC00028382

Docket No.:
Serial No.:
Filed:
Art Unit:
Sheet No.:
Express Mail:
Inventor(s):
Ti

04860.P1365

\¾ of 17
TB377895617US
Steven W,. Christensen
METHOD AND APPARATUS FOR DISPLAYING
AND ACCESSING CONTROL AND STATUS
INFORMATION IN A COMPUTER SYSTEM

APLNDC00028383

barGraphTopLeft.h

barGraphTopLeft.v-

Illilill

SBGetBarGraphWidth(8)

APLNDC00028384

Docket No.:
04860.P1365
Gerial No.:
Filed:
Art Unit:
Sheet No.:
I teof 17
Express Mail: TB377895617US
Inventor(s): Steven W,. Christensen

Title:
,-

METHOD AND APPARATUS FOR DISPLAYING
AND ACCESSING CONTROL AND STATUS
INFORMATION IN A COMPUTER SYaltal

APLNDC00028385

BarGraphSlopeLeft

BarGraphFiatRight

BarGraphSlopeRight

APLNDC00028386

Docket No.:
Serial No.: O o
Filed:
o
Art Unit:
Sheet No.:
la o
Express Mail: TB377895617US
Inventor(s): Steven W,. Christensen
Title:

METHOD AND APPARATUS FOR DISPLAYING
AND ACCESSING COis mOL AND STATUS
INFORMATION IN A COMPUTER SYSTEM

APLNDC00028387

ra errace.

owse:wers.y

to Pogµ7 r

s .a

vannee-

CLK

C

IF

'IO

/1M.5

NR^½T jeg

P

/03

vm +
PtoDE.m
log
7..f o

Cstu

yv.r lo ;

SLAT

Loe,se. avr too

g.7

IAI TERFAt

lot

t/Nr T
IMAD M

Ct.n c.K.

Durc.E - et•ý

GipERATot

it.o
D AI --

punay

OLATIL.G

I2f
326

go

CHI P

IW

Q)elyyor]

mF-.

iO
R.ofyy 1)ICV-

DiME

/WE NG'/

COWsoid£ft

TEytP
SErJ om.

I

A

1

p..,

«

/2C
oC¾

aygong
/27

Cux1•x6
-

/G

I 3

F/GUKE /

APLNDC00028388

File

Edit

Ulem

Label

Special

Entensions
Name

Size

Vabbit

ftpplications
10 items

12.6 MB in disk

AppleLink PowerBook Edition

MacTerminal

MoviePlayerm
Fln Of

E×press Modem

Remote Access Cient Folder

ResEdit

313.6 MB available

SimpleTe×t

Tools

BPO

PrintMonitor

66K

QuickTime
Serial Tool

831K
33K

Text vooi

12 6 MB in disk

I

2eK

i

2 items

313.6 MB avai

61K

Trash

APLNDC00028389

v

PRINT OF DRAWINGS
AS'ORIGINALLY FILED

2e

close b0N

module display area

left scroll azzow
2o4

drag/size inb

zight scroll axxow
goy

icruge 28

APLNDC00028390

PRINT OF DRAWINGS
AS ORIGINALLY FILED

Hide Battery Level

Hide Battery Consum: ion
Hide Time Remainin

APLNDC00028391

PRINT OF DRAWINGS
AS ORIGINALLY FILED

Battery Monitor
The power adapter is
connected and recharging
one of the batteries.

APLNDC00028392

PRINT OF DRAWINGS
AS ORIGINALLY FILED

Fl frv E

2E

APLNDC00028393

PRINT OF DRAWINGS
M ORIGINALLY FILED

2/ /

llllluill

zt 1.
Fi one 2 F

APLNDC00028394

PRINT OF DRAWlNGS
AS ORIGINALLY FILED

:d i

Control Strip

-

Show Control Strip
Hide Control Strip

APLNDC00028395

PRINT OF DRAWINGS
AS ORIGINALLY FILED

MAIN LOOP

†

called by operating system)

save previous context,
setup strip's context

secondary
initialization
mplete

null event (idle)?

gol-

no

run secondary
initialization

yes >

run idle tasks

yes >

process mouse click

no

mouse click?

no

Y

restore previous
context

*

EXIT

Fic

4

APLNDC00028396

PRINT OF DRAWING3
AS ORIGINALLY FILED

Control Strip Main Loop

Irun secondarV Initialization
SECONDARY
INITIALIZATION

has the Finder
started up?

03
no

yes
oreate the strip's

So

window

successful?

no -

yes

T
initialize strip a font,

t

open, initialize amtemal
modules
,

set strip a default
screen location, size

?
So
60 $

load user configuration

†

draw the s

s

†
EXIT

Fys

APLNDC00028397

PRINT OF DRAWINGS
AS ORIGINALLY FILED

Control Strio Main Loop
lopen. Initialize external
modules

(EXO

N

n

more Imdule
files?
yee
Y

m

(.01

open module file

load imdule code into

marrory
†

ob 3

cal module to Inidalize
itself
I

callmodulatoget

rmdule's features

605
cal module to get Its
display width
close module file

(

EXIT

APLNDC00028398

PRINT OF DRAWINGS
AS ORIGINALLY FILED

*/doP

R 398287

Control Strip Main .oop
/draw the strp's co itents
DRAW THE STRIP'S

CONTENTS

isthestdp
visible?

no -

draw button
background

status-crgy
background

graphics

grapNm

'

mirreduletodraw

I

ksel

I

' kiL

EXIT

APLNDC00028399

PRINT OF DRAWINGS
AS ORIGINALLY FILED

Control Strio Main Loop
Irun idle tasks

RUN IDLE TASKS

3t>
windoeupdate

MN

pending?

s

contents

ne

yn

yes

tion to

ne

+

osicurrentdiermdule
a nat its Mie task

pest·process the Wie

*

updata10 d non

meddetadle

does module
need W save
d

ne

"

a sale save
nour?
i

save imdule's
seeings to disk
I

+

update strips help
stessages

?
EXIT

APLNDC00028400

PRINT OF DRAWINGS
AS ORIGINALLY FILED

Control Sttso Main Loop
Iprocess nouse click
PROCESS MOUSE
CLICK

Imuse click

imide strip?

9 Ol
detemine which
module was clicked on

½ot
move operation?

yee jnne module's display,

er rmve entim strip

yes
I

track the mouse click

' 10 6
Imuse still in
rmdule's

unds

I

no N

I
cai the module to

EXIT

APLNDC00028401

PRINT OF DRAWINGS
AS ORIGINALLY FILED

Control Strip Main Loop
/post process the click
MODULE POST
PROCESSING

ntxiule wants to
update settings?

Bag that this module
hasa save pending

yes

to

yes r

close the tradule

yes : invalidate old help state

lo
EX1T

Fe- lo

APLNDC00028402

PRINT OF DRAWINGS
AS ORIGINALLY FILED

barGmphTopLeft.h

barGmphTopLeft.vSBG etBarGmphWidth(8)

APLNDC00028403

PRINT OF DRAWINGS
M ORIGINALLY FILED

BarGraphSlopeLeft

BarGraphFiatRight

BarGraphSlopeR1ght

Cr u R.E

APLNDC00028404

UNITED STA̤ DEPARTMENT OF COMMERCE
. Patent and Trademark Office

y

APPUCATION NUMBER

FIUNG DATE

/

Address: COMMISSIONER OF PATENTS AND TRADEMARKS
Washington, O C. 20231

FIRST NAMED APPUCANT

/

ATTY. DOCKET NO./TFTLE

DATEMAILED:

/

11/08/94

NOTICE TO FILE MISSING PARTS OF APPLICATION
FILING DATE G·<- ED
An ApplicationNumber and Filing Date have been assigned to this application. However, the items indicated
below are missing. The required items and fees identified below must be im ls byted ALONG waan
THE
NT OF A SURCr anwE for items 1 and 3-6 only of i
'iar large entities or
$
or small entities who have filed a verified statement claiming such status. The surcharge is set forth in
37 CFR l.16(e).
If all required items on this form are filed within the pe
entity, O small entity (verified statement filed), is i

w, the total amount owed by applicant as a garge
.

Applicant is given ONE MONTH FROM THE DATE OF THIS LETTER, OR TWO MONTHS FROM THE
FILING DATE of this application, WHICHEVER IS LATER, within which to file all required items and pay any fees
required above to avoid abandonment. Extensions of time may be obtained by filing a petition accompanied by the
extension fee under the provisions of 37 CFR 1.136(a).
1. O The statutory basic filing fee is: O missing O insufficient. Applicant as a O large entity O small
entity, must submit $
to complete the basic filing fee.
2. O Additional claim fees of $
as a O large entity, O small entity, including any
required multiple dependent claim fee, are required. Applicant must submit the additional claim
fees or cancel the additional claims for which fees are due.
3. O The oath or declaration:
O is missing.
O does not cover items omitted at time of execution.
An oath or declaration in compliance with 37 CFR 1.63, identifying the application by the above
Application Number and Filing Date is required.
4. O The oath or declaration does not identify the application to which it applies. An oath or declaration
in compliance with 37 CFR 1.63, identifying the application by the above Application Number and
Filing Date, is required.
5.4 The signature to the oath or declaration is: Mssing; O a reproduction; O by a person other than
the inventor or a person qualified under 37 CFR 1.42, 1.43, or 1.47. A properly signed oath or
declaration in compliance with 37 CFR 1.63, identifying the application by the above Application
Number and Filing Date, is required.

6. O The signature of the following joint inventor(s) is missing from the oath or declaration:
An oath or declaration listing the names of all inventors and signed by
the omitted inventor(s), identifying this application by the above Application Number and Filing
Date, is required.
7. O The application was filed in a language other than English. Applicant must file a verified English
translation of the application and a fee of A
under 37 CFR 1.17(k), unless this fee has
already been paid.

8. O A $

processing fee is required for returned checks. (37 CFR 1.21(m)).

9. O Your filing receipt was mailed in error because check was returned without payment.
10. O The application does not comply with the Sequence Rules. See attached Notice to Comply with
Sequence Rules 37 CFR 1.821-1.825.

11. O Other.

En cuLo o punac oal - «- - sL ullu novice-w
i N gl i
Division, Special.Processing and Correspondence Branch (103) 308-120 .

plication Processing

A copy of this notice MUST be returned with the response.
Foax P10-1588aux.s-as)

OFFICE COPY

APLNDC00028405

4860.P1365

Patent

IN THE UNITED STATES PATENT AND TRADEMARK OFFICE
Application of:
DEght

Steven W. Christensen
Serial No.:
Filed:

For'

08/316,237

)

September 30, 1994

)

- · -OD AND APPARATUS FOR
DISPLAYING AND ACCESSING CONTROL
AND STATUS INFORMATION IN A
COMPUTER SYSTEM

)
)
)
)

Commissioner of
Patents and Trademarks
Washington, D.C. 20231
Attention:

Application Division
Special Handling Unit

RESPONSE TO NOTICE TO FILE MISSING PARTS OF APPLICATION
(FILIN(I DATE GRANTED)
Sir:

In response to the Notice to File Missing parts of Application (Filing Date Granted)
mailed November 8, please find enclosed:
(1)

a duly executed Declaration and Power of Attorney with respect to the

above-referenced patent application;
(2)

a check in the amount of $130.00 in payment of the surcharge of 37 C.F.R.

§ 1.16(e);
(3)

a copy of the Notice to File Missing Parts of Application; and

If any additional fee is required, please charge Deposit Account No. 02-2666. A
duplicate of this Response is enclosed for deposit account charging purposes.
Respectfully submitted,
BLAKELY, SOKOLOFF, TAYLOR & ZAFMAN

Dated: Ë

l

L--; 1994

12400 Wilshire Blvd.

Seventh Floor
Los Angeles, CA 90025-1026

[phone]

Michael Mallie
Reg. N . 36,591
I hereby certify that this correspondence is being deposited with the

United States Postai Serviœ an; Mass mail with suffic½nt po'ege
in an envelope addressed !c tM Commissioner of Paten: A

Trademarks, Washington, D.G. 20231

x

Date of Deposit

Name of Person Mainng Ucriespondence
signature

Dato

APLNDC00028406

o

y's Docket No.:

04860.P1365

pagen,

DECLARATION AND POWER OF ATTORNEY FOR PATENT APPLICATION

a below named inventor, I hereby declare that:
My residence, post office address and citizenship are as stated below, next to my name.
I believe I am the original, first, and sole inventor (if only one name is listed below) or an

original, first, and joint inventor (if plural names are listed below) of the subject matter
which is claimed and for which a patent is sought on the invention entitled

the specification of which
YY

is attached hereto.
was filed on
September 30, 1994
Application Serial No. _08/316,237
and was amended on

as

(if applicable)

I hereby state that I have reviewed and understand the contents of the above-identified
specification, including the claim(s), as amended by any amendment referred to above. I do not
know and do not believe that the claimed invention was ever known or used in the United States
of America before my invention thereof, or patented or described in any printed publication in
any country before my invention thereof or more than one year prior to this application, that
the same was not In public use or on sale in the United States of America more than one year
prior to this application, and that the invention has not been patented or made the subject of an
inventor's certificate issued before the date of this application in any country foreign to the
United States of America on an application filed by me or my legal representatives or assigns
more than twelve months (for a utility patent application) or six months (for a design patent
application) prior to this application.

I acknowledge the dut to disclose all information known to me to be material to patentability as
defined in Title 37, ocae or reaeral Regulations, Section 1.56.
I hereby claim foreign priority benefits under Title 35, United States Code, Section 119, of any
foreign application(s) for patent or inventor's certificate listed below and have also identified
below any foreign application for patent or inventor's certificate having a filing date before that
of the application on which priority is claimed:
Prior Foreinn ApDiication(s)

Priority
Claimed

(Number)

(Country)

(Day/Month/Year Filed)

i i

(Number)

(Country)

(Day/Month/Year Filed)

i i

(Number)

(Country)

(Day/Month/Year Filed)

i i

I hereby claim the benefit under Title 35, United States Code, Section 120 of any United States
application(s) listed below and, insofar as the subject matter of each of the claims of this
application is not disclosed in the prior United States application in the manner provided by the
first paragraph of Title 35, United States Code, Section 112, I acknowledge the duty to disclose

all information known to me to be material to patentability as defined in Title 37, Code of
Federal Regulations, Section 1.56 which became available between the filing date of the prior
application and the national or PCT international filing date of this application:
(Application Serial No.)

Filing Date

(Status -- patented,
pending, abandoned)

(Application Serial No.)

Filing Date

(Status -- patented,
pending, abandoned)

APLNDC00028407

I

I hereby appoint Keith G. Aske f, Reg. No. 33,828; Aloysius T. C. AuYeung, Reg. No. 35,432·
Bradley J. Bereznak, Reg. No.-33A74; Micnael A. Bernadicou, Reg. No. 35,234; Roger W.
Blakely, Jr., Reg. No. 25,831 Timothy R. Croll, Reg. No. 36.771; Daniel M. De Vos, Reg.
P37,813; Stephen D. Gross, Reg. No. 31.020; David R. Halvorson, Reg. No13,395; James P.
nao, neg. No._3624; Michael D. Hartogs, Reg. No. 36,547· Iian Don Hickman, Reg. No.
35,894; George W Hoover II, Reg. No. 32,992; Pau . Horstmann, Reg. No. 36,167: Tracy L
g. No. 34,188; Eric S. Hyman, Rég¯ Ño. 30,139 Dag H. Johansen, Reg No. 36,172
Stephen L. Kirig,¯Reg. No. 19,18 · aniel C. M
No. 33.532: Michael J. Malli
No. 36,591; James D. Mc ar and, Reg. No. 32,544; Anthony C. Murabito, Reg. No. 35,295;
RokeagIn, Reg. Noe20,340;Aames R¯ŠaÏter, Reg.-Nad5 66_S; Robert A. Saltzoerg, neg.
No. 36 910; James C. Scheller, Reg. No. 31,195; Edward W. Scott, IV, Reg. No. 36.000;
Nic olas Joseph Skarlatos, Reg. P37,941; Iviaria ivicCormack Sobrino, Reg. N^¾ =¯6 ¯
Stanley W. Sokoloff, Reg. No. NJohn C. Stattler, Reg. No. 36,285; Edwin H. Taylor
Reg. No. 25,129; Lester J. Vincent, Reg. No. 31,460; Ben J. Yorks, Reg. No. " "l° ©^rnian
Zafman, Reg. ivo. žð,250: Mark Aaker,+leg No
effrey J. Brooks, Reg. No. 35,834;
Paul D. Carmichael, Reg. No._ 18.879; Vernon Randall Gard, Reg. No. 33.886: Richard
Reg. No.
ft T.1Vlartin; TTëë.No.12,426; Helene S. Plotka, Reg. No. 35,981; and
Nancy R. Simon Reg No. 36,930; my attorneÿ¯sÏTrid Thomas X. Li, Reg Un- 3 ,, wsu cuwin
A. Sloane, Reg. No. 34 po) my patent agents, of i ^=I v SOKOLOFF, TAYLO
AFMAN with
offices located at l Š400 Wilshire Bosulevard, 7th Floor, Los Anaeles, CalÏfornia 50025,
telephone (310) 20r-dovu, with full power of iubstifuiion and revoöatlän iõ prosecute this
application and to transact all business in the Patent and Trademark Office connected herewith.
I hereby declare that all statements made herein of my own knowledge are true and that all
statements made on information and belief are believed to be true; and further that these
statements were made with the knowledge that willful false statements and the like so made are
punishable by fine or imprisonment, or both, under Section 1001 of Title 18 of the United
States Code and that such willful false statements may jeopardize the validity of the application
or any patent issued thereon.
Full Name of Sole/First Inventor

Steven W Christensen

Inventor's Signature

-

Residence Milpitas. California
(City, State)
Post Office Address

Date I
Citizenship

(

IIR A
(Country)

1514 Mount Diablo Avenue
Milpitas. California 95035

Fulf Name of Second/Joint Inventor

inventor's Signature
Residence

Date
Citizenship

(City, State)

(Country)

Post Office Address

Full Name of Third/Joint Inventor

Inventor's Signature
Residence

Date
Citizenship

(City, State)

(Country)

Post Office Address
Rev. 12/15/93 (D1) cak

2

APLNDC00028408

UNITED STN
Patent and Ýrataáark Office

COMMER

Address: COMMISSIONER OF PATENTS AND TRADEMARKS
Washington. D.C. 20231
APPUCATION NU

MF:

ATE

37

FIRST NAMED APPUCANT

09 30/94

ATTY. DOCKET NO./TITLE

CHRISlTNSEN

S

04860.F'1365

03/41/1108
LN LV 501 LOFF . Tío'T LOR & l/4F MAN
12400 WH.SHIRE BLVía
SEVEidTH FLOOR
LOS sãNGELE.S,

Clo 90025

Ogg
DATEMAILED:

11/08/94

NOTICE TO FILE MISSING PARTS OF APPLICATION
FILING DATE GRANTED
AnApplication Number and Filing Date have been assigned to this application. However, the items indicatád
below are missing. The required items and fees identified below must be tim-1 submitted ALONG WÊ'H
THE P
IT OF A SURCHARGE for items 1 and 3-6 only of A
or large egiŠes or
$
or small entities who have filed a verified statement claiming such status. The suzcharge is på forth in
37 CFR 1.16(e).

If all required items on this form are filed within the perio,d set below, the total amount owed by appliŠtas a glarge
entity, O small entity (verified statement filed), is A
Applicant is given ONE MONTH FROM THE DATE OF THIS LETTER, OR TWO MONTHS FROM THE
FILING DATE of this application, WHICHEVER IS LATER, within which to file all required items and pay any fees
required above to avoid abandonment. Extensions of time may be obtained by filing a petition accompanied by the
extension fee under the provisions of 37 CFR l.136(a).
1. O The statutory basic filing fee is: O missing O insufficient. Applicant as a O large entity O small
entity, must submit $
to complete the basic filing fee.
2. O Additional claim fees of $
= a O large entity, O small entity, including any
required multiple dependent claim fee, are required. Applicant must submit the additional claim
fees or cancel the additional claims for which fees are due.
3. O The oath or declaration:
O is missing.
O does not cover items omitted at time of execution.
An oath or declaration in compliance with 37 CFR 1.63, identifying the application by the above
Application Number and Filing Date is required.

4. O The oath or declaration does not identify the application to which it applies. An oath or declaration
m compliance with 37 CFR 1.63, identifying the application by the above Application Number and
Filing Date, is required.
5.4 The signature to the oath or declaration is:

ssing; O a reproduction; O by a person other than

the inventor or a person qualified under 37 CFR 1.42, 1.43, or 1.47. A properly signed oath or
declaration m compliance with 37 CFR 1.63, identifying the application by the above Application
Number and Filing Date, is required.
6. O The signature of the following joint inventor(s) is missing from the oath or declaration:
An oath or declaration listing the names of all inventors and signed by
the omitted inventor(s), identifying this application by the above Application Number and Filing
Date, is required.
7. O The application was filed in a language other than English. Applicant must file a verified English
translation of the application and a fee of $
under 37 CFR 1.17(k) unless this fee has

alr

e

8. O A $

/94 08316237

1 105

130.06 CK

processing fee is required for returned checks. (37 CFR 1.21(m)).

9. O Your filing receipt was mailed in error because check was returned without payment.
10. O The application does not comply with the Sequence Rules. See attached Notice to Comply with

Sequence Rules 37 CFR 1.821-1.825.
11. O Other.

IA

Division, Special Processing and Correspondence Branch [phone].

ài

,

plication Processing

A copy of this notice MUST be returned with the response.
BUMPW-Isaagtsv.s.es)

COPY TO BE RETURNED WITH RESPONSE

APLNDC00028409

\\-

. 1365

IN THE =
-o

1

e

'47RADY

STATES PAins i AND TRAD "^* OFFICE

lication of:

)

even W. Christensen

Examiner:

Serial No.: 08/316,237

Art U ° : 2104

Filed: September 30, 1994

-

)

For: Method And Apparatus For Displaying And )
Accessing Control And Status Information In )
A Computer System
)

g$

ggO

INFORMATION DISCLOSURE STATEMENT
Commissioner of
Patents and Trademarks
Washington, D.C. 20231

Pursuant to 37 C.F.R. 1.97, Applicant hereby provides a listing on PTO Foim
1449 of references and encloses a copy of the references cited on the above referen

c

form.
The Applicant certifies that the information contained in the information disclosure
statement was cited in a communication from a foreign patent office not more than three
months prior to this date as required for submission of Information Disclosure Statements
under 37 C.F.R. § 1.97(e).
While this statement is being submitted as a means to comply with 37 CFR 1.97, it

is not to be construed as an admission that the material identified above represents material
prior art to Applicant's invention.
Very truly yours,

BLAKELY, SOKOLOFF, TAYLOR & ZAFMAN
Date'

1996
Ja s C. Scheller, Jr.
R . No. 31,195

12400 Wilshire Blvd.

Seventh Floor
Los Angeles, CA 90025

[phone]

I hereby certify that this correspondence is being deposited with the United States Postal Service as first class
mail with sufficient postage in an envelope addressed to the Commissioner of Patents and Trademarks,
Washington, D.C. 20231
Date of Deposit

Cindy Mumhy
Name of Per n Mailing Correspondence
ature

/

Date

APLNDC00028410

Sheet
. F(‡.
(RE\/

TO-8

U.S. DEPARTMENT OF COMMERCE
PATENT AND TRADEMARK OFFICE

DOCKET NO.

of

i

SERIAL NO.

04860.P1365

08

,

APPLICANT

INFORMATION DISCLOSURE CITATION

Steven W. Christensen
FILING DATE

(Use several sheets if necessary)

GROUP

09/30/94

2896 2 5

U.S. PATENT DOCUMENTS
'E

MIANLER

DOCUMENTNUMBER

DATE

NAME

CLASS

St'

Ace
"

FlUNGDATE
IF APPROPRIATE

FOREIGN PATENT DOCUMENTS
DOCUMENTNUMBER

DATE

COUNTRY

CLASS

SUSCLASS

7 A 05/06/92 EPO

-

-

5 8 4 3 9 2 A 03/02/94 EPO

--

TRANSLATION

--

YES

4

3

NO

OTHER DOCUMENTS (including Author, Title, Date, Pertinent Pages, Etc.)
Steven Harris, et al.: "Inside WordPerfect 6 for Windows", 1994, USA, pp.1104-1108

Mark A. Benge, Matt Smith: "Designing Custom Controls", Spring 1993, USA, pp. 72-85

International Search Rip

EXAMINER

DATE CONSIDERED

--

EXAMINER: Initial if citation considered, whether or not citation is in conformance with MPEP 609; Draw line
througah citation if not in conformance and not considered. Include copy of this form with next communication to

APLNDC00028411

Europäische

:tentamt

European Patent Office

Office européen des brevets

Publication number:

0 483 777 A2

EUROPEAN PATENT APPLICATION
Application number: 91118455.4

Int. CIA GO6F 3/033

Date of filing: 29.10.91
Priority: 31.10.90 US 606227
Date of publication of application:
06.05.92 Bulletin 92/19
Designated Contracting States:
DE ES FR GB IT SE

Applicant: Hewlett-Packard Company

3000 Hanover Street
Palo Alto, California 94304(US)
Inventor: Mathis, Barry R.
1433 McBain Avenue
Campbell, California 95008(US)
Inventor: Kobara, Shizunori S.
301 Spinnaker Street
Foster City, California 94404(US)
Inventor: Anderson, Stephen R.

620 Chesley Avenue
Mountain VIew, California 94040(US)

Representative: Liesegang, Roland, Dr.-Ing. et
al
FORRESTER & BOEHMERT

Franz-Joseph-Strasse 38
W-8000 München 40(DE)

Three dimensional graphic interface.
A method of graphically depicting the operation
of a computer. The method includes depicting a

functional aspect of a task by means of a threedimensional abstraction (84, 86, 88) such as an icon
and depicting the operation of the computer by

varying the appearance of the abstraction over time.
In one aspect of the invention, a window (82) displays scaled-down versions of files to indicate that

the files are open but not active. In another aspect, a
linear array of three-dimensional abstractions (84,
86, 88) is provided along an edge of the video
display, some of the abstractions (84, 86) suggestive
of a function accessible without respect to any par-

ticular application and others (88) suggestive of a
group of applications. In another aspect, a threedimensional abstraction is used to suggest a file

cabinet (90) having containers (92, 94) such as drawers, notebooks and the like for files and other objects. In another aspect, the display includes a plurality of abstractions (120) suggestive of controls of a
device being emulated by the computer. In another

aspect, a transporter is depicted by a frame (123) on
a background (125) which encloses an abstraction
(133) suggestive of data to be transmitted between
the computer and a remote location; alternately highlighting opposite sides of the frame, sequentially

highlighting different ones of a set of nested frames
(137, 139) and fading the abstraction between a
distinct image (133) and an indistinct image (135)
provide a visual representation of transmitting the
data to, and receiving the data from, the remote
location.

APLNDC00028412

EP 0 483 777 A2

90

92

Notica how some of these root imel windows

-enel to get themselses buried under their

children. It as apperent that some techniques
wall be recurrecl to alake these ervenal windows

reedily accessible.
Ci hites

C3 44¥

*

I. K

er

sAnagit

Toiser
p.

pgy

C2 Aken.shmsµts

y

um

102

98

Certain windor types might want to take on helpfla

visual chorectoristics, such as this Folder with a
Tab at the top, which makes it immediately

. Lots ofother Container eknects are

le, like cessettes 1br video or midte Ales, or
notebooks, shakes, slicle trigs; etc.

96

The various contamer objects might carry with

100

them color eccles to help the user track their
relationships

at of the
i:e the
e::posure of the document cor.tento. When
utivated the wmdow could tiew its headerback
agin. To tuo"e such a ?'infcw wu me tid just
edel the mouse near the ley lo «tivate it.end
then slide up to the titlebo= to greb it end ar.o re
it to its new location.

CopyrigM
Hovisit Packerg Company Juns 1988

FIG 7

2

APLNDC00028413

1

EP 0 483 777 A2

This application is related to EP 89 113 701.0 (=

sheet ,of paper or file relating to a current task
would lie on top of papers or files on a desktop

FB 2715) the disclosure of which is incorporated

herein by reference.
Background of the Invention

5

The present invention is directed to an interface between a user of a computer and the program or programs being executed by _ the computer. More particularly, the invention is directed to
a graphic interface which presents a cohesive
three-dimensional appearance to thereby provide a
realistic and comprehensible view of the tasks being carried out by the computer.
To execute a program loaded into a computer,
it is necessary for the user to enter commands
which are associated with the particular program
being executed. Users who frequently run a program are typically familiar with the commands for
the program, and can therefore enter them without
the need to refer to documentation or to be prompted by the program itself in order to determine the
appropriate commands. However, with the wide-

To

15

20

proliferation of software written for those computers, many users are not intimately familiar with all
of the necessary commands and protocols for the
various programs they may desire to run. As a
result, many programs are written in a "userfriendly" format, by which the user is apprised of
the functions capable of being performed by the
program and prompted for appropriate commands
through the use of menus and/or symbols which
appear on the video screen of the user's computer.
When a user selects a particular function to be
performed, a display of the choices available to the
user within the context of that function may appear

25

30

35

on the screen in one of a number of possible
additional selections that enable the function to be

40

interface which further increases the user's capabilities in this regard. More particularly, the visual
metaphors that have been employed in the past
have been largely two-dimensional in , nature. In
other words, the windows associated with various
programs and tasks can be arranged anywhere
within the width and height of the video screen. If a
number of windows are presented at one time, it
can be appreciated that the screen can begin to
appear cluttered, in much the same manner as a
desk with a large number of papers on it. For
example, various types of multi-tasking software
enable two or more applications to be run on a
computer simultaneously. If each application has a
number of individual windows associated with it,
the video screen will quickly become filled with
portions of windows.
In a real world situation, when large numbers of
papers or files begin to accumulate on a desk, they
are often stacked upon one another, to maintain
organization. However, when windows are stacked
upon one another in a two-dimensional desktop
metaphor, the user loses sight of all but the topmost window. Accordingly, it is desirable to give
the impression of depth, as well as height and
width to the visual metaphor, in order to provide
the user with a more realistic, and hence more

easily manageable, representation of his work load.

carried out according to his or her desires.
As a further step toward the objective of facili-

tating the user's comprehension and access to
program functionality, some programs have employed a visual metaphor of a desktop to serve as
an interface between the user and the program. In
this visual metaphor, information relating to each
program that is being executed, as well as each
task being performed within the program, is dis-

relating to other active tasks. This approach to
presenting the information to the user on the video
screen enables the user to focus upon a task
appearing in a particular window, but to be conscious of other tasks to be performed as represented by the visible portions of other windows
appearing on the screen.
While the use of the visual metaphor has greatly enhanced the user's ability to manage multiple

ongoing tasks in the e×ecution of a program, it is
desirable to provide an improved form of graphic

spread popularity of personal computers and the

formats. When a particular item is chosen from the
displayed options, the user may be prompted for

2

45

Further in this regard, it is desirable to be able to
provide each of the elements used in the metaphor
with the attributes of physical objects, to thereby
provide more realistic feedback to the user as he

or she selects and operates upon individual elements to control the program.

Brief Statement of the invention
50

played on a preselected area of the screen, often

referred to as a "window". The various windows on
a screen can be likened to sheets of paper lying on

in accordance with the present invention, a
comprehensive approach is used in the display of
graphical elements to provide increased visual appeal to the user as well as increased clarity of

a desktop. The window pertaining to the program

currently being executed, or the task within that
program that is currently being performed, is typically represented as overlaying the other windows
on the screen, in much the same manner as the

55

information. A characteristic depth, as well as
height and width, is given to each of the elements
that is employed to provide information to the user,
to thereby present a cohesive three-dimensional

3

APLNDC00028414

3

EP O 483 777 A2

4

appearance to the entire visual metaphor which
makes up the interface. In one aspect of the invention, every graphic element, such as a window
border, an icon, or a box, is represented with the
use of at least three shades of color. The body of
the element is represented by a main, or base
color. One edge of the element, or preferably two

providing a visual metaphor of different types of file
containers can be provided. These containers are
more readily recognizable to the user and can be
accessed with greater ease than a conventional
5
textual file directory, to enable the user to ascertain
and retrieve the files stored in the computer. To
keep the area of the screen as Clutter-free as
adjacent edges (for rectangular elements), are
possible, the window for the reduced files and the
highlighted with a lighter shade of color, and the
file space can be on sliding panels that are selecopposite edge or adjacent edges are shaded.with a
to tively removed from view and retrieved as needed.
darker shade of color. This approach causes each
In another aspect of the invention, various
element to appear as if it is raised or recessed
hardware items and software functions are simurelative to a background surface, in dependence
lated within windows on the video screen to faciliupon the respective edges which are chosen for
tate the user's comprehension of operations being
the highlighting and the shading.
ya
performed by the computer. For example, the conin a further aspect of the invention, this aptrol panel of an item such as a tape player or a
proach is also applied to individual features of
television is displayed along one border of a winsymbols shown within elements as well as to the
dow when the computer is simulating the operation
borders of the elements themselves. Thus, the
of that equipment. To control this simulated operaimage of an object represented within an icon
2o
tion of the item, the user accesses control panel
appears to be raised above or recessed below a
elements such as buttons to provide a more realisbase surface, to provide the three dimensional eftic and familiar type of control.
fect. In a further application of this approach, eleSoftware functions are also graphically simuments such as boxes are appropriately highlighted
lated according to the teachings of the invention.
and shaded to provide the appearance of a solid
25 Transmission of data between two computers is
object having mass within a window, and the comdepicted by the gradual disappearance of an icon
ponents typically contained within such boxes,
representing the data from the screen of the sendsuch as selection buttons or bars, are highlighted
Ing computer and the gradual appearance of a
and shaded to appear as objects mounted on this
similar icon on the screen of the receiving comsolid object.
so
puter. This metaphor is further enhanced by simuin another aspect of the invention, shadows are
lating movement of the icon away from the user at
employed to provide the impression that multiple
the sending computer and towards the user at the
windows are spaced from one another in a depth
receiving computer, for example by sequentially
direction. To further enhance the realistic impresemphasizing spaces between nested frames that
sion, the edges of the shadows are contoured in
as
surround the icon on the screen.
accordance with the topology of the underlying
A further e×planation of these and other feaobjects upon which they are cast. To insure that
tures, as well as the manner in which the invention
the shadows do not obliterate underlying windows,
achieves a comprehensive three-dimensional apthey can be made transparent so that the user can
pearance to the visual metaphor, is set forth
determine the contents of windows lying beneath
no
hereinafter with reference to specific examples and
the shadow.
embodiments illustrated in the accompanying drawFurther features of the invention facilitate the
Ings.
user's recognition of an active task or session
along with comprehension of other sessions that
Brief Description of the Drawings
are open but not presently active. These features
45
include highlighting an active window, and removFigures 1A-1C are illustrations contrasting coning currently non-functional components from inacventional two dimensional representations of
tive windows. To enable the user to distinguish
icons with the three-dimensional approach embetween similar files representing inactive sesployed in the present invention;
sions, the files can be represented by scaled-down
so
Figure 1D is an enlarged view of a three-dimenversions of inactivated sessions, and appear in a
sional icon similar to that shown in Figure 1A,
window or special location on the screen which is
illustrating the highlighting and shading of the
dedicated to files. Color can be employed to enborder of the icon in greater detail;
hance the user's comprehension of transitions from
Figure 2 is an illustration of a window containing
one work environment to another55
vanous components that are presented in accorTo enable the user to more easily recall all of
dance with the principles of the present inventhe files stored th the computer, including those
tion;
which are not open, a visually oriented file space
4

APLNDC00028415

5

EP 0 483 777 A2

Figure 2A is an enlarged view of a portion of the
dialog box panel illustrated in Figure 2;
Figure 3 is an illustration of a video screen
containing multiple winòows;
Figure 4 is another representation of a video
display containing several windows, illustrating
the use of shadows;
Figure 5 is another depiction of a screen containing multiple windows, illustrating the use of a
pinup board:
Figure 6 is a depiction of a screen containing an
icon dashboard;
Figure 7 is a depiction of a screen containing a

e

file space window;

Figure 8 is a depiction of a video screen having
the pinup board and file space windows appearing on sliding panels, and using one-point perspective to provide the impression of depth;
Figure 9 is a view similar to Figure 8, with the

is

sliding panels slid out of view and further illustrating the use of one-point perspective;
Figure 10 is a view of a video replay window
which simulates the controls of a machine being

20

along the right hand and lower edges of an object,
such as an icon. This dark line has a constant
width, typically one or two pixels. Even when drop
shadows are employed on selective objects, however, the overall image appearing on the screen
still appears to be only two-dimensional in nature.
In contrast, through the comprehensive application
of three-dimensional imaging effects to all of the
elements displayed in an image, the graphic interface of the present invention provides a cohesive
appearance which gives a realistic three-dimensional impression.
The imaging effects which are employed in the
conte×t of the present invention will first be explained with reference to individual components
that can appear in a displayed interface. In the
illustrated embodiments, the particular components
which are used to Íllustrate these effects are icons.
Figures 1A, 18 and 1C respectively illustrate images of icons for a folder, a calculator and a clock.
The three-dimensional images of these icons, generated in accordance with one aspect of the
present invention, are shown at the top of each

emulated on the computer;
Figure 11 is a view of a transporter that simu-

25

lates the transmission of data from a computer

to a remote location;
Figure 12 a a view of an icon image within a
transporter during a transmission operation; and
Figure 13 is a block diagram illustrating the
architecture of a system for implementing the
present invention.

ao

Description of the illustrated Embodiments

as

As noted above, an underlying concept of the
present invention is to employ a comprehensive
approach to the three-dimensional depiction of ele-

ments on a screen, to provide the user with the
impression that the entire image being viewed has
depth and that individual objects within the image
have solidity and mass. To facilitate an understanding of the invention, this concept will be described
with reference to some features of a conventional
windowing program, such as Microsoft WindowsyM
sold by Microsoft Corporation. Implementation of
the concepts underlying the invention does not
affect the functionality of the windowing program,
and therefore such aspects of the program will not
be described in detail herein. Rather, the discussion of the invention will concentrate on the graphical features and visual effects provided by the
invention.
It was known in the past to apply drop shadows
to individual elements of a graphical display in an
attempt to impart a three-dimensional characteristic
to those particular elements. Basically, a _drop
shadow consists of an opaque dark line appearing

6

40

45

50

55

figure, and contrasted with conventional two-dimensional depictions of the icons shown at the bottom
of each figure.
Referring to Figure 1A, the body 10 of the
element, i.e., the icon of a folder, is represented on
the video screen by means of a base, or main
color. In addition to the main color that is used for
the body 10 of the element, a lighter tint 12 and a
darker tint 14 of the main color are employed to
mdicate the edges of the element in a manner
which creates a three-dimensional effect. For each
of the elements depicted in Figures 1A, 18 and 1C,
the upper and left-hand edges of the element are
indicated by a lighter tint 12, and the opposite
edges, i.e., the lower and right-hand edges, are
indicated by a tint 14 which is darker than the base
COlor. By means of this arrangement, a three-dimensional impression is created in which it appears that a light from an imaginary source is
shining over the user's left shoulder onto the element. If desired, it is possible to highlight and
shade only two opposite edges of the icon, rather

than all four edges. For example, the top edge
could be highlighted and the bottom edge shaded,
while the two side edges are defined by the main
color. In this example, a light would appear to be
shining from a location directly above the user.
Throughout the following description of the invention, the term "color" is to be interpreted in a
general sense to refer to video images displayed
on monochrome monitors as well as those appearmg on so-called color monitors which present images . having red, green and blue components.
Thus, reference in the text to "lighter" and "darker"

5

APLNDC00028416

7

EP 0 483 777 A2

colors or tints of colors refers not only to different
hues or tones of the colors, but also to different

gray scale values for a monochrome color.
The use of highlighting and shading on opposite edges of the component is illustrated with
greater clarity in Figure 1D, which comprises an
enlarged view of an icon similar to that shown in
Figure 1A. This icon differs slightly from that shown
in Figure 1A, however, in that it is provided with an
outer border or frame 13. The outer frame.is also
represented by the main color, and its edges are
appropriately highlighted, at 15, and shaded, at 17,
in a manner consistent with the highlighting and
shading of the edges of the main body of the icon
itself.
The use of the lighter and darker colors is
employed not only for the border of the component
itself, but also to indicate the individual features of
any object or symbol displayed within the component. Thus, with reference to Figure 1A, the upper
and left hand edges of the layers of the folder 18
are highlighted by means of the lighter tint, and the
lower and right-hand edges of the folder are
shaded by the darker tint. Thus, the folder itself
appears as a three-dimensional object which is
raised above a surface defined by the body 10 of
the icon.
In general, the three-dimensional effect is attained with the use of three color tints, a base
color, a lighter highlight tint and a darker shading
tint. In a preferred embodiment of the invention, the
highlight tint is about 35-45% lighter than the base
color, most preferably 40% lighter, and the darker
tint is also about 35-45% darker, and most preferably 40% darker, than the base color. If desirable,
more than one basic color can be employed. For
example, the highlighting and shading at the edges
of the folder 18 could be different colors from that
used to indicate the borders of the icon itself.

In the folder icon depicted n Figure 1A, all of
the upper and left-band edges of the icon border
and the individual folder features are highlighted

whereas the lower and right-hand edges of each
feature are shaded, to create the impression of
raised elements. Referring to Figure 1B, a similar
approach is employed for the body 20 of the
calculator and each of its buttons 22, to again
create the impression that the calculator rises
above the surface of the icon and the buttons
project upward from that body. However, an opposite arrangement is employed for the display 24
of the calculator, to create the impression that it is
recessed within the body 20. More particularly, the

of the clock are shaded while the lower and righthand edges are highlighted, to create the impression that the face 26 of the clock is recessed into
the surface of the icon. A consistent approach is
5
used for the highlighting and shading throughout
the image, in accordance with the location of the
imaginary light source noted previously.
Figure 2 illustrates the manner in which these
imaging techniques are applied to the total display
70
to provide a comprehensive three-dimensional impression. This figure depicts a single window 30
that may be associated with a word processing
system. The window 30 contains depictions of folders 32 relating to different files that can be opened
ye
by the user, and a dialog box 34, which enables
users to configure the format of documents.
As is conventional in a windowing system, the
border of the window 30 is defined by a frame 35.
In accordance with the principles of the present
2o
invention, the individual elements which make up
the frame are provided with highlighting and shading, as described in reference to the previous figures. Thus, the upper and left hand edge of each
elongated member 36 of the frame is highlighted,
25
and the lower and right hand edges of these members are shaded. If the frame is provided with
corner elements 38, for example to assist in the
sizing of the window on the video screen, the
edges of these elements are similarly highlighted
so
and shaded.
In addition to the elongated member 36, the
top portion of the window is provided with a control
menu box 40, a title bar 42, and various sizing
boxes 44. These elements can be employed in a
35
conventional fashion to enable the user to call up a
menu relating to basic functions associated with
the window, relocate the window in the video display, and change the size of the window on the
screen or iconize the window, for example. Again,
no each of these components is provided with the
highlighting and shading to maintain the three-dimensional appearance of the displayed structure.
Similarly, symbols illustrated within each element,
such as the space bars on the control menu box 40
45
and the arrowheads on the sizing boxes 44, are
provided with highlighting and shading, to create
the impression of a non-planar topology for the

so

upper and left-hand edges of the display are

shaded with the darker tint, whereas the lower and
right-hand edges are highlighted with the lighter
tint. A similar approach is used in the clock icon o_f
Figure 1C, in which the upper and left-hand edges

8

se

illustrated structure.
When multiple windows are presented on the
video screen it is often helpful to provide an indication to the user of the currently active window. In
the context of the present invention, this purpose
can be accomplished by changing color or shading
along the elements 36, 38, 40, 42 and 44 of the
active window. This makes the active window more
promine.nt and thus readily recognizable by the
user.

APLNDC00028417

9

EP 0 483 777 A2

To further facilitate a recognition of the active
window among multiple windows, the user's workspace within the border of the windows can be
highlighted, as shown in Figure 3. To this end, the
workspace areas 31 of the other windows can be
made darker, so that the workspace 33 of the
active window is the brightest part of the screen,
and thereby easily recognizable. This approach
focusses the user's attention on the active document, and away from other features on the screen
which may serve as distractions.
Further along these lines, components of nonactive windows which are not currently usable can
be removed, to eliminate possible distractions. For
example, when a window is inactive, its menu bar
cannot be used to perform any functions. Therefore, it can be removed from the display, as shown
with respect to the two left-hand windows in Figure
3. With this approach, the portion of the screen
available for the display of usable information is
maximized.

5

component adjacent each listing to enable the user
to

selected from the available options, as shown, for
example, in the upper left hand subpanel relating to
the location of the text on the page, the selection

components comprise mutually exclusive operators
15

2o

25

a plane positioned above the window frame, thereby resembling a piece of paper on top of a clipboard, or the like. If desired, the scroll bar 37 that

is conventionally employed to shift the contents

ao

within the window can be incorporated within the
shadow 35. As shown in Figure 3, the width of the
scroll bar can normally be quite narrow, since the
user need only be aware of its presence. However,
when a cursor (not shown) is brought close to the

35

scroll bar, its width can increase to make it easier
for the user to grab and move it as desired. Again,

this feature maximizes workspace area in favor of
the depiction of functional components that are not

impression that the box represents a panel 45. If
desired, the panel can be provided with a title bar
46 to assist in moving it within the confines of the
window 30. The edges of this title bar are also
appropriately highlighted and shaded in a manner
consistent with the highlighting and shading of the
panel itself.
The particular example of a dialog box illustrated in Figure 2 contains three main subjects.
which respectively relate to the location of the text
on the page, the spacing of lines, and the attributes
of characters. These subjects are separated within

the pa.nel by means of subpanels 48 which are
appropriately highlighted and shaded to indicate
that they are raised above the surface of the main
panel 45 defining the box. Each subpanel 48 contains a listing of the selections available for the
subject of interest, and an appropriate selection
to make a selection. If only one choice can be

Another feature which can be employed to
enhance the recognition of the workspace is to
create a dark shadow effect 35 along two edges of
the workspace, such as the bottom and right
edges. This effect makes the workspace appear as

being currently employed.
In accordance with another aspect of the invention, the dialog box 34 (Figure 2) is presented to
the user as a physical structure which has mass
and solidity. To this end, the body of the box 34 is
illustrated with a main color, and its edges are
appropriately highlighted and shaded to create the

10

4,

45

so

known as radio buttons 50. Within this panel, the
top two radio buttons are depicted as being raised
above the surface of the subpanel and the lowest
button is provided with an opposite type of shading
and highlighting, to create the impression that it is
recessed below the surface of the subpanel. In
other words, the lowest button appears to have
been pushed by the user, to indicate that he has
selected the "Right Justify" option. Preferably, this
reversing of the highlighting and shading occurs as
the user makes the selection, for example by striking an "Enter" key or clicking a mouse button. This
type of effect provides the user with the impression
that he is interacting with a dynamic element, i.e., a
physically moving button, through graphic animation.
If more than one choice can be selected within
a particular subject area, the selection components
are illustrated as check buttons 52, as shown in the
other two subpanels on the menu 34. Again, these
check buttons 52 are depicted as being raised
above the surface of the subpanel if they have not

been selected, and depressed below its surface if
the user has selected the options associated with
them, to provide the dynamic feedback.
When a button is depicted as having been
depressed, the main part of the button can remain
the same color, as shown with respect to the
lowest one of the radio buttons 50. Alternatively,
the center of the button can be inverted, e.g., made
dark, as shown with respect to the check button 52
associated with the "Underline" option
In addition to the subpanels, the main panel 45
also includes three raised buttons 54 which can be
selected, or pushed, by the user to respectively
indicate whether he or she would like to accept the

format options illustrated by the depressed selection components, cancel those selections, or request additional help regarding the selections.

As can be seen from the illustration of Figure
55

2, the highlighting and shading of each of the
individual components, as well as the panel, cre-

ates the overall impression that the dialog box
comprises a separate integrated structure having
mass, rather than a series of items drawn on a
sheet of paper. Figure 2A comprises an enlarged

APLNDC00028418

11

EP 0 483 777 A2

' portion of the dialog box panel 45 and subpanels
48 shown in Figure 2, to more clearly illustrate the
use of the highlighting and shading. This figure
also illustrates the manner in which more than two
colors or tints can be used for the highlighting and
shading. Referring to each of the radio buttons, it
can be seen that the highlighted edges are defined
by a light color 49 and a very light color 51.
Similarly, the shaded edges are represented by a

window 73 shown in Figure 4 illustrates an example

of a window which has been proportionately reduced in size to create the impression that it is
5

dark color 53 and a very dark color 55. Preferably,
the very light color 51 and the very dark color 55
are on opposite edges.
In addition to individual components and integrated structures, multiple windows are presented
on the video screen in a fashion which presents the
impression that the image has overall depth, to
thereby complete the impression that it is truly
three-dimensional in nature. This approach is carried out with the use of shadows and appropriate
shading for the individual windows. Figure 4 illustrates an example of a portion of the video screen
having six windows and a number of various icons

yo

75

20

rently working, is displayed as the topmost window

25

64 which overlies all the other windows. If desired,
these other windows can be successively displayed
as being beneath the active window in order of
their usage, i.e., the next most recently used window 66 could appear immediately below the active
window 64, the second most recently used window
68 could appear immediately below that, and so
on.
To enhance the impression that the windows

30

are spaced from one another in the depth direction

35

ao

45

can be 10% darker than the workspace area of the
window lying immediately above it. Alternatively,
the amount by which each successive window's

workspace area becomes darker can be made dependent upon the number of displayed windows,

so that the difference in darkness of adjacent windows decreases as the number of windows increases.
To further enhance the impression of depth,
the size of a window, including all of its component

pology of the structures upon which they are cast,
to further enhance the impression that these struc-

tures have a three-dimensional characteristic.
For example, with reference to the right hand
edge of the shadow 74 cast by the active window
64, it can be seen its top portion is defined by a
relatively straight line 76 where the edge of the
shadow lies on the flat workspace for the window
68. At the bottom portion of the frame of the
window 68, the edge of the shadow 74 is recessed
to the left at 78, to indicate that the frame pro-

trudes above the surface of the workspace for the
window. Similarly, below the frame the edge of the
shadow 74 e×tends farther to the right at 80, to
indicate that the workspace of the ne×t succeeding
window is located at a deeper distance from the
active window 64.
In practice, the shadows are simulated by generating additional windows, with these additional

windows having a uniform color throughout in ac-

other windows become successively darker as one
proceeds in the depth direction. By way of example, the workspace area of each successive window

located very deep within the displayed image.
Each window is also provided with a shadow to
illustrate its displacement from other structures below it. The location of the shadow is consistent with
the highlighting and shading applied to individual
structures. Thus, for the examples illustrated in the
drawings, the shadows are shown to the right and
bottom of the associated windows. The widths of
the shadows vary from one window to the next, to
create the impression that the windows are at varying depths from one another. Further, edges of the
shadows are not represented as straight lines, as
was conventionally done with drop shadows. Rath-

er, they are contoured in accordance withsthe to-

depicted thereon. As is conventional, the active
window, i.e., the one within which the user is cur-

the workspace area of the windows are represented
with successively darker shades of color. Referring
to Figure 4, the workspace area 70 of the active
window 64 is illustrated with the brightest color,
e.g., white, as described previously. The workspace
area 72 of the ne×t lowest window 66 is slightly
darker in color, and the workspace areas of the

12

so

cordance with the desired color of the shadow. In
order that the shadows not obliterate too large a
portion of the screen, they are preferably made
transparent. For example, the shadow can be generated as a see-through window that is represented
by an intermediate gray-scale value or color tone
and that enables the highlighting and shading of
underlying elements to be seen by the user. Thus,

the user can readily ascertain the contents of the
lower windows even if they have a shadow cast
upon them. As a window is moved, the edges of
the shadow dynamically shift in accordance with
the topology of the lower structures over which the

window is being moved.
The use of shadows is not confined to windows. Rather, as shown in Figure 2. dialog box

55

features and the information contained therein, can

panels and menu panels can also cast shadows, to
further enhance the impression that they are physical structures separate from the window work-

be made proportionately smaller as the window
becomes located further back in the display. The

space areas. Again, the edges of the shadows are
preferably contoured in accordance with the posi8

APLNDC00028419

13

EP 0 483 777 A2

tion of any underlying objects, as shown with respect to the icon 81 illustrated in Figure 2, to
enforce the impression of depth in the displayed
image.
In addition to the foregoing features which contribute to a comprehensive three-dimensional effect

5

in the user interface, other approaches can be
employed to facilitate the user's comprehension
and recognition of the information contained on the
screen. One such approach is with respect to the
use of icons. In conventional windowing programs,
each application that is being run has its own
document icon In many cases, the icons used to
designate files tend to be similar, and in some
cases identical. For example, if a user is running a
spreadsheet program, two different worksheets created within that program may be resembled by
identical icons which are generated as the user

15

completes each worksheet and moves on to an20

25

30

35

invention, standardized icons are not used for applications. Instead, a scaled-down version of the
contents of a window is created when the user
exits the window and issues a command to mini-

screen in which a number of documents can be
held until they are needed again. Although the
scaled down versions of the documents may not

40

45

be readable, they are recognizable by the user.
This approach utilizes a person's ability to instantaneously sort and classify items on a visual basis.

Thus, with the use of the pinup board containing
scaled-own versions of window contents, the user

so

can find a document such as an organization chart,

for example, much faster than if he were required
to read icon titles.
There are certain types of basic functions per-

formed with personal computers that readily lend
themselves to the use of icons. For example, some

tories, ,time management and calendars, address
files, etc. Each of these functions which are sometimes referred to as "desktop accessories", can be
represented by an icon 84, 86 having a symbol
related to the function, as shown in Figure 6. To
provide the user with ready and familiar access to
each of these functions, the icons are preferably
always accessible, regardless of the work the user
is doing. In accordance with one aspect of the

10

other. When the user desires to retrieve one of the
iconized worksheets, it is not possible for him to
tell which icon pertains to which worksheet. As a
result, he must access one of the icons, for example by clicking a button on a mouse when the
cursor is located over the icon. Such an action
causes the title of the icon to appear on the screen,
and from this the user may be able to determine
whether the icon pertains to the worksheet of interest. A difficulty arises, however, if the user has not
seen the worksheets for a period of time, and
therefore cannot .remember their titles, or if they
were created by another person, in which case the
current user may not understand the meaning behind the titles.
In accordance with one aspect of the present

mize the file. Referring to Figure 5, the various
scaled-own versions of the files appear in a special
window 82, which is labeled a pinup board. This
arrangement provides an orderly, small area on the

14

55

invention, they are located at a fi×ed place within
the visual display and arranged in a fixed order, so
that the user becomes familiar with their placement
and can easily access them. In the example illustrated in Figure 6, all the icons pertaining to standard functions are located in a line along the bottom of the display in a predetermined order, providing a visual metaphor similar to the dashboard
in an automobile. Specifically, the functions available on the dashboard are always present at a
fi×ed location for the user, regardless of the view

that appears on the main part of the screen. When
the user selects one of the icons, for example the
mail icon 84, a mail window or menu will be generated. This window or menu provides the user
with the means for selecting various tasks associated with the mail operation, such as reading new
mail, sending mail to another person, or accessing
and reading old mail files. When the user has
completed the mail-related operations, the window
is put away and the icon 84 reappears on the
dashboard at the bottom of the screen.
As shown in Figure 6, the icons pertaining to
standard functions are present at the left half of the
dashboard. The right portion of the dashboard is
similar in nature, but contains functions that are

tailored to the specific needs of the user. Within
this portion of the dashboard, the user can organize
other applications into individual drawers 88. Applications having related functions can be grouped
Ín a Single drawer. For example, one drawer could
be labeled "desk top publishing" and include applications such as an outliner, a text processor, a
drawing program, a scanner interface, and a page
layout program. In essence, the various drawers on
the dashboard become work spaces in which related functions are grouped, similar to a real life
situation in which belongings are grouped according to various relational patterns. By accessing a
particular drawer 88, a visual metaphor is created
in which the drawer is pulled up out of the dashboard, and icons of the various applications contained within that drawer are displayed.
Another visual metaphor that can be employed
to facilitate use of the computer pertains to the
area of files. Information is stored within computers
according to various file and file structures that
bear little resemblance to everyday physical exper-

of these functions may include mail, phone direc-

iences. As a result, many users of computers have
9

APLNDC00028420

15

EP 0 483 777 A2

difficulty conceptualizing the files and their organization. In accordance with a feature of the present

invention, the filing structure can be made much
more visual, as well as spatial in concept. To this
end, a visually oriented file space is provided in a

5

background window 90, as shown in Figure 7. This
space is provided with shelves and drawers, to
utilize a person's greater ability to recall the locations of objects when they are stored spatially and
visually, rather than merely by name.
_
Within the file space 90, different types of

yo

containers for documents and files can be displayed. For example, these containers can be
drawers 92 of different sizes and shapes, notebooks 94 and other storage media, such as slide
trays, video disks, books, etc. When a particular

is

drawer or notebook is opened, or other type of
storage media is accessed, a window 96 is created
which can have a shape that represents a distinguishing characteristic of the accessed object. For
2o
example, a window 100 representative of a drawer
that has been opened can illustrate a plurality of
folders 102 which are arranged in a particular hierarchy recognizable to the user. When one of these
folders is accessed, a new window 96 is created
25
which has a tab 98, characteristic of the appearance of a folder.
As discussed previously, it is desirable to minimize the display of function-related information that
is not currently being used, to thereby maximize
30
the area available for the main work space. However, for functionally related displays such as the
file space 90 and the pinup board 82, the user
should be continuously provided with quick access
to them. To this end, these two windows can be
as
provided on sliding panels that always remain
open, but are not always in the background of the
screen. Figure 8 illustrates a video screen having a
main workspace window 104 and two sliding panels
respectively relating to the pinup board 82 and the
40
file space 90. When the user does not require
access to the functions provided by each of these
latter windows, they can be slid to the left, to
effectively hide them behind the left edge of the
screen. The resulting display would be as shown in
45
Figure 9, in which only the main workspace window
104 is visible. The two sliding panels are represented by a pair of symbols 106 and 108 which
appear on a narrow strip 110 at the left edge of the
screen. By selecting either of these symbols, the
so
user can cause the appropriate functional window
to slide to the right and thereby become visible, to
access the files contained therein.
Referring again to Figure 8, a further aspect of
the pinup board 82 is illustrated Specifically, the
55
pinup board is divided into upper and lower sections. The upper section 112 can contain reduced
representations of core functions that are universal

16

in their ,applicability. For example, some of these
functions might comprise a listing of things to do
for the day, a reminder of a meeting to attend, or
the like. These functions would remain displayed
on the upper section of the board during the day,
regardless of the particular work space the user is
operating in.
In the lower half 114 of the pinup board, the
reduced files appearing therein relate to the particular work space that is presently active. Thus, if

the user should change work spaces, for example
by going from one window to another, the documents that appear in the lower portion 114 of the
pinup board will change in accordance with the
new work space. Thus, for example, as the user
goes from a word processing program to a spreadsheet program, the items appearing on the lower
portion 114 of the pinup board could change from
files created within the word processing system to
previously created worksheets pertaining to the
spreadsheet program.
A similar concept can be applied to the file

space as well. Particularly, as the user changes
from one work space to another, the configuration
of the file space can change, to reflect the different
storage needs associated with different types of
work spaces.
In many instances computers are being used
to emulate the functions of other types of machines. For example, animation can be stored in
the memory of the computer and displayed an the
video screen to provide video action, similar to the
playing of a tape on a video tape recorder. During
such an operation, a window which simulates the
controls on the video recorder can be provided on
the video screen, as illustrated in Figure 10. Referring thereto, the stored animation is displayed within a workspace 116 formed by a standard window
118. At the bottom of the window, a control panel

of a conventional video cassette recorder is displayed. This control panel includes buttons 120
corresponding to the typical functions provided by
such a recorder, for example play, fast forward,
rewind, etc. By individually accessing these but-

tons, for example by means of a mouse, the user
can cause the animation displayed within the work
space 116 to behave in the same manner as the
results of playing a tape on a video cassette recorder.
Other types of hardware which can be emulated within a computer can similarly have accompanying windows which simulate the control panels
appearing on such hardware. For example, if the
computer is used to emulate a television, the normal controis of a television can be illustrated at the
bottom or the side of a suitable window.

10

APLNDC00028421

17

EP 0 483 777 A2

in addition to simulating hardware elements,
the present invention provides a dynamic metaphor
for depicting a software function as it is being
carried out by the computer. One such function is
the transmission of data between a first computer
and a remote location such as a second computer
remotely located from the first. The visual metaphor that is used to represent this function is
referred to as a transporter.
As shown in FIG. 11. a method of depicting a
transmission of data according to the invention
includes displaying a frame 123 against a background 125 (indicated in FIG. 11 by diagonal lines)
on a video screen, the frame 123 enclosing a data
transport region 127 of the screen; highlighting a
portion 129 of the frame 123 by displaying such
portion in a relatively lighter color than that of the
background and shading a portion 131 of the frame
123 opposite the highlighted portion 129 by displaying it in a relatively darker color than that of the
background; and displaying, in the data transport
region 127, at least a portion of an image 133 of an
icon representative of data to be transmitted between the computer and the remote location. The
icon preferably resembles a document such as a
sheet of paper or the like. Preferably the entire
image is in the transport region, but a portion of the
image may overflow out of the transport region if
desired.
Transmission of data represented by the icon
image 133 is depicted by varying the image between a relatively distinct image, as shown in FIG.
11, and a relatively indistinct image 135, as shown
in FIG. 12. As the image becomes less distinct
over a time interval of a few seconds, the visual
impression is that the image is fading away, and in
a preferred embodiment the image entirely disappears as the data is transmitted from the user to
the remote location. As the image becomes more
distinct, the visual impression is that the image is
materializing, and in a preferred embodiment the
display starts with no icon image at all and the
image gradually materializes over a period of several seconds until it forms a sharp image of the
icon, thereby indicating the reception of data from
the remote location. Thus, the transmission of data
from one computer to another is depicted by the
gradual disappearance of an icon representing the
data on the screen of the sending computer and
the gradual appearance of a similar icon on the
screen of the receiving computer.
When the user wishes to initiate a transmission
of data from the computer to a remote location, the
user places an icon image representing the data in
the data transport region 127, tor example by dragging it there with a mouse or other pointing device.
Of course, depending upon the mode of operation

5

to

15

20

25

30

35

18

of the computer, the document could be placed in
the transport region in response to an appropriate
keyboard command.
After the icon has been placed in the transport
region, transmission begins, for example in response to an appropriate transmit command from
the user, and the image gradually becomes less
distinct, preferably fading away entirely as the data
is transmitted. If the user is receiving data, an icon
image gradually appears in the transport region to
indicate that data is being received. The user
moves the icon out of the transport region to place
the data in a local file in the computer or to make
other use of it, as desired.
Making the image more or less distinct is accomplished in software by adding or deleting pixels
from among a group all of which make,up a complete, distinct image.
In a preferred embodiment the transmission of
data is depicted by displaying not just the frame
123 but a plurality of frames of successively larger
sizes such as, in addition to the frame 123, a frame
137 and a frame 139. Each frame except the
largest is nested within the frame of next larger
size. Thus the frame 123 is nested within the next
larger frame 137, and the frame 137 in turn is
nested within the largest frame 139. Portions of the
background are visible in spaces between the
frames; specifically, a portion 141 of the background is visible between the frames 123 and 137
and a portion 143 of the background is visible
between the frames 137 and 139. A portion of each
frame is highlighted by displaying that portion in a
relatively lighter color than that of the background;
specifically, portions 145 and 147 of the frames
137 and 139 are so highlighted. Similarly, portions
149 and 151 opposite the highlighted portions 145
and 147, respectively, are shaded by means of a

relatively darker color than that of the background.
4o

45

The frames are oriented such that their highlighted
portions are adjacent one another and their shaded
portions are adjacent one another.
Preferably, the spaces between the frames are
sequentially emphasized, for example by changing

color, to convey a direction of transmission of the
data in the transport region. For example, to visually depict transmission from the computer to the

50

55

remote location, the sequential emphasizing proceeds from the space 143 between the two largest
frames toward the space 141 between the two
smallest frames, thereby simulating motion away
from the user. The background outside the largest
frame and the background in the transport region
may be included in this sequential emphasizing if
desired, for example by emphasizing first the backgrouod around the largest frame, then the space
143, then the space 141, then the background in
the transport region. Conversely, to visually depict

APLNDC00028422

19

EP 0 483 777 A2

reception from the remote location, the sequential
emphasizing proceeds from the space between the

two smallest frames toward the space between the
two largest frames, thereby simulating motion toward the user·
A functional block diagram of a system in
which the foregoing principles of the present invention can be implemented is illustrated in Figure 13.
The system operates in conjunction with a conventional windowing system 122, such as the X WINDOW SYSTEM developed and distributed by the
Massachusetts institute of Technology (MIT). Basically, the windowing system comprises a set of
graphics routines which enable an application to
create and manipulate communication regions, i.e.,
windows, on the screen of a video terminal 124.
These routines are contained in windowing libraries
126. When an application program 128 needs to
place information on the screen, it requests a window from the windowing system 122. The appropriate routines are called from the windowing library and the necessary graphics are displayed on
the screen, along with the data from the applications program, through a display server 130 which
accesses the display through the computer's operating system.
A window manager 132 and user interface libraries 134 are also provided in the system. The
user interface libraries 134 contain subroutines for
the generation of components that are associated
with specific features of the application program.
Thus, for example, when an application program
136 requests a window to display a dialog box or a
menu, the appropriate subroutines are called from
the user interface library to generate the images of
the dialog box panel and the buttons on that panel,
in a conventional manner.
The window manager 132 provides for the generation of graphic effects which might be considered to be more generic to the overall display
rather than specific to an application program.
These effects include the shading and highlighting
of the frames for individual windows, the highlighting and shading of icons, and the generation of
shadows. Again, these effects are created by
means of subroutines which may be stored within
the window manager itself or in a separate user
interface library 138.
In operation, the window manager provides information pertaining to the generation of all of the
three-dimensional effects in response to user entered commands that are not specific to an application program. To this end, the window manager,
when it is first placed into operation, requests the
windowing system to inform it of events which may
have an effect upon the displayed image, such as
actuation of a mouse button. Whenever such an
event occurs, the windowing system notifies the

20

window manager of it through the display server
130. In response, the window manager examines

the particular event and determines whether it is
5

10

is

one which affects any portion of the display over
which it has control, for example actuation of a
system menu button. If so, the window manager
takes the appropriate action, e.g., calling the user
interface subroutine to inverse the highlighting and
shading of the button to indicate that it has been
pushed. If, on the other hand, the event is one over
which the window manager does not exercise control, the display server passes the event on to the
application program for service thereby.
It will be appreciated by those of ordinary skill
in the art that the present invention can be embodied in other specific forms without departing
from the spirit or essential characteristics ,thereof.

The presently disclosed embodiments are therefore
2o

25

considered in all respects to be illustrative and not
restrictive. The scope of the invention is indicated
by the appended claims rather than the foregoing
description, and all changes that come within the
meaning and range of equivalents thereof are in-

tended to be embraced therein.

Claims
1.

as

A method of graphically depicting the operation of a computer in a computer video display
of the kind which provides a plurality of representations of windows (64, 66, 68), the windows being associated with tasks performed
by the computer to provide a visual interface
between a user and the computer, comprising
the steps of:

40

depicting a functional aspect of a task by
means of a three-dimensional abstraction (84,
86, 88) suggestive of that functional aspect;
and

so

depicting the operation of the computer by

varying the appearance of the abstraction over

time.

45
2.

A method according to claim 1 wherein:
depicting the functional aspect comprises

so

generating a three-dimensional abstraction (33)
suggestive of a workspace in which a portion
of an open file is visible, and
depicting the operation comprises provid-

se

ing a window (82) for displaying inactive files,
displaying a scaled-down version of the abstraction of the workspace area and the file in
said window, and removing any display of the

12

APLNDC00028423

21

EP O 483 777 A2

file from any other visible window, thereby
indicating any files which are open for use but

7.

not currently active.
3.

A method as in claim 2 and further comprising

a

the step of enabling the user selectively to
remove the inactive file window from the dis-

8.

play when access to inactive files is not needed and to retrieve the inactive file window
when access to an inactive file is desired.

4.

5.

yo

A method as in claim 3 wherein removing the
inactive file window comprises causing the
window to appear to slide behind a portion
(110) of a border (35) of the display.

15

A method as in claim 2 wherein depicting the
operation comprises grouping any abstractions
suggestive of files that are associated with an
active workspace into a first portion (114) of

20

the inactive file window (82) and changing the
display of abstractions in the first portion according to any change from one active workspace to another, whereby the abstractions
that are being displayed in the first portion at
any given time are associated with that workspace which is then active.
6.

25

30

depicting the functional aspect comprises

generating a linear array of three-dimensional
as

to any particular application being executed by
the computer, and a second portion of the
array having a plurality of abstractions (88)

each suggestive of a group of applications, and
depicting the operation comprises expanding an abstraction of the second portion of the
array to provide a plurality of choices each
associated with an application in the group,
whereby the user can access a function suggested by an abstraction in the first portion of
the array by selecting that abstraction and the
user can access an application in a group
suggested by an abstraction in the second
portion of the array by first expanding that
abstraction and then selecting one of the resuiting choices.

A,method as in claim 6 wherein expanding an
abstraction is depicted by a visual simulation
of opening a drawer which contains a plurality
of abstractions each indicative of one of the
choices.
A method as in claim 1 wherein depicting the

functional aspect comprises generating a
three-dimensional abstraction suggestive of a
file cabinet (90) having containers (92, 94) for
files and other objects and a plurality of threedimensional abstractions suggestive of objects
stored in the containers.
9.

A method as in claim 8 wherein depicting the
operation comprises displaying a window (100,
96) having three-dimensional features suggestive of an object (92, 102) in the cabinet (90) in
response to a selection of that object by a
user, thereby suggesting removal of the object
from the cabinet and opening Of the object for
use.

A method according to claim 1 wherein:

abstractions (84, 86, 88) along an edge of the
video display, a first portion of the array having
a plurality of abstractions (84, 86) each suggestive of a function accessible without respect

22

40

45

so

10. A method as in claim 1 wherein a task comprises emulating a device, depicting the functional aspect comprises displaying a plurality
of abstractions (120) suggestive of controls of
the device being emulated, and depicting the
operation comprises manipulating a display associated with the emulation according to user
selection of various ones of the displayed controls.
11. A method as in claim 1 wherein:
depicting the functional aspect comprises
displaying a frame (123) against a background
(125) and placing in the frame an abstraction
(133) suggestive of data to be transmitted between the computer and a remote location, and
depicting the operation comprises varying
the display of the frame and the data abstraction over time in a manner suggestive of transmission of the data between the computer and
the remote location.
12. A method as in claim 11 wherein varying the
display comprises varying the data abstraction
between a relatively distinct image (133) and a
relatively indistinct image (135) to represent

materialization and dematerialisation of the
data.
55

13. A method as in claim 11 wherein varying the

display comprises alternately highlighting first
and second portions of the frame by displaying
the highlighted portion (129) in a relatively

APLNDC00028424

23

EP O 483 777 A2

24

i

lighter color than that of the background and
shading a portion (131) of the frame Opposite
the highlighted portion by displaying it in a
relatively darker color than that of the background to represent motion of the data.

5

14. A method as in claim 11 wherein:

depicting the functional aspect comprises
displaying a plurality of frames (137, 139) of

to

successively larger sizes, each frame except
the largest being nested within the frame of
next larger size, and
depicting the operation comprises sequentially highlighting portions of successive ones
of the frames from the largest to the smallest
and varying the data abstraction from a relatively distinct image to a relatively indistinct

15

image to simulate motion of the data away

2o

from the user and thereby to represent transmission of a message from the user to the
remote location.
15. A method as in claim 11 wherein:

depicting the functional aspect comprises
displaying a plurality of frames (137, 139) of
successively larger sizes, each frame except
the largest being nested within the frame of
next larger size, and
depicting the operation comprises sequentially highlighting portions of successive ones
of the frames from the smallest to the largest
and varying the data abstraction from a relatively indistinct image to a relatively distinct
image to simulate motion of the data toward
the user and thereby to represent reception of
a message from the remote location.

25

30

as

4o

45

SO

55

14

APLNDC00028425

EP 0 483 777 A2

12

16
O

FIG 1A

14

FIG 18

FIG iC

APLNDC00028426

EP 0 483 7T7 A2

12

13

18

15

17

10

17

FIG 1D

APLNDC00028427

EP 0 483 777 A2

36
!!

•

32

Letiers

2

Bing

46

44

Forrnat Setup

35

l_eft Justif9
Right Justify

Double Space

Centered

50 -

Single Space

Rulers On/Off

48

34
36

36
Bold

Yes

italics
52

Underline

No

Double Underline

36

46

54

f1aybe

45

More Bills

81
38

FIG 2

APLNDC00028428

EP 0 483 777 A2

Fo r ma t
Left Justify

Center Justify
Right Justify
Bold

Ita lics

FIG 2A

APLNDC00028429

O---.= a--, Sample Folder
Name

19Pe

site

SpecielityexamPles

Folder

254K

ORAS©leehBackground

PixelPaint

120K

O Ivlinirnel Window Concept

P¡ygs¡nt

gy

Iconio Desh TOP Conapt

Felder

siso r

Color Scheme PlePen!

Folder

Ty, i.

Notepad·
Perhaps when windows are inactive some of the
overhead items could disappear to ma:dmize the
exposure of the document contenic. When
activated the window could.grow its header back
again. To move such a window you would just
click the mouse near the top to activate it, and then slide up to the titlebox to gab it and move
it to its new location.

FIG 3

The main principle is to make the User's area
the screen the dominant Visual elements, end to
minimize the System Overhead elements like th
window borders, scrollbars, menubars, etc.
This window uses highlighting the document
background as the means of identifying the ac
window. Also, menubers are eliminated from th
inactive windows, where they cennot be used
anywegr.

The minimal scrollbar below will grow to an essi
to-use size when the ctirsor gets into its region
stars visually small most of the time. The brigh't
portion of the shadow indicates the portion of th
screen being displayed.

Copyright
Hewlett Packard Company June 1988

APLNDC00028430

EP 0 483 777 A2

ikspace
Terminal

Fife Funotfons
Pt

t

.

ndow
66

Lyr
rimi

N:avi

UIG
78

80
64

FIG

4

APLNDC00028431

02

Sample Folder
Speelelityerariples
Fullsormensackgreemd
MoimalWii.dowem.eepr.
shage..vedoi÷logeo:
IconioDeskToo¢onosor
Colorscherna ProposaJ

Folder
PhaiPaint
riste,¡,2
realPsint
Foider
Felder

25•11..
1201<
egg
law
4501::
/:4 A

The object here is to minimize the impact of the
"system" elements like the window borders and
buttons. The User's Documents should be the
dominant elements on the screen. In a screen
layout as complex as this one in MS Windotvs/
IkTew Wexa. the screen real estate devoted to the
system elements approaches 40%. Some individual
smell windoivs may have less than 10% of their
content actually showing.
Some questions which arise:

Do the menubers really hwe to be showing on inactive windows?

Should the scrollters always be there?

The small windosv above, called the Pin-up
Board, shows an idea that is an alternative to
iconifying documents. By showing a scaleddown version of the document, the user can
easily tell which document is which, reducing
the need to displey the ican's title.

Shouldn't the more accomplished and familiar
user be allowed to make some of the features
imrisible. or visible only when needed?

o

Does one really need to see the name of the
document when it is displayed fulbsized on the
screen?

M>/opar
Perhaps when windows are inactive some of the
overhead itemo could disappear to maximice the
exposure of the document contents. When
activated the window could grow its hee.derback
again. To move such a window you would just
click the mouse near the top to activate it, and
then slide up to the titlebox to grab it and move
il to its new location.

.

:

N: +:I i:¾:¾:

(

.

N:M:¾: :¾:

:

N:¾:¾:¾:¾: :y:N

Copyright
Hewlett Packard Company June 1988

FIG

5

APLNDC00028432

sample Folder
Name

ty,

Specielityemmples
Full sormen Background

Moimi window cenoeor
Shadeve<I Dieloa Bo×
Iconie DeskTop concept
colorscherne Poposa!

g¡

Folder
PlaalPeint

254K
120::

The area scross the bottom of the screen is where

Pi>alPaint os y
PimiPaint 134 K
volder
esa it
Fol.kr
me.it

1

July
2

3

the cr,man applications, the desktop necessities,
could reside. Nestled together they form sort of
a "dashboard" for the computer, with a wide variety
of options available. On the left side are the core
applications. On the right side the user could add
and organize his own applications.

a

6

7

8

9

10

11

12

13

14

16

16

17

18

Til

19

20
27

86 i

Below is a "drawer" Ibll of applications the user
might have on his system. These drawers are like
pull-up menus, but they hold iconio representations
of the applications. These little icons should look
much richer then the typical B&W ones of todey.
After elL they might cost you several thousand
dollars. They should look like little jewels in a
drawer of a jewehybox.

Notepa<r
Perhaps when windows are inactive some of the
overhead items could disappear to maximize the
exposure of the document contents. When
activated the window could grow its header back
again. To move such a window you would just
click the mouse near the top to activate it. and
then slide up to the titlebox to grab it and move
it to its new location.

O

These button-like objects could
also be the "doors" to entirely
different workspaces, with different background colors. pinup boards, documents, etc.

64:if

FI G 6

'

Copyright
®Hewlett Packard Company June 1988

APLNDC00028433

s

me

e

-90

92

Notice hco some of these root level windows
tend to get themselves buried under their
children. It is apparent that some techniques
will be required to make these crucial windows
reedily accessible.

--

MNutoshn½gets

UK

Fol¾r

mee

Certain wind w types might Amt to teke on helpful
Visual characteristics, such as this Folder with a
Tab at the top, which makes it immediately
recognizable. Lots of other Container objects are
possible, like cassettes fbr video or audio files, or
notebooks, shelves, shde trays, etc,
The various container objects might carry with
them color cocles to help the user track their
relationships

102

171

W

CO

't 00
96

le of the
y--a
're the
e::posure of the document contents. When
activated the window could pow its hee.der back

again. To move such a window you wonid just
chek the mouse near the top to activate it, and
then slide up to the titlebox to grab it e.nd move
it to its new location.

Copyright
Hewlett Packard Company June 1988

FIG 7

APLNDC00028434

I

FG 8

Copyrigit
Hewielt Packard Company June 1980

APLNDC00028435

S

-!E
-J

1.

FlG 9

APLNDC00028436