Motorola Mobility, Inc. v. Apple, Inc.

Filing 282

NOTICE by Apple, Inc. Notice of Filing of Declaration of Jill J. Ho in Support of Apple Inc.'s Motion to Compel Production of Documents and Testimony Regarding Set-Top Box Components and Exhibits Thereto (Attachments: # 1 Affidavit Declaration of Jill J. Ho in Support of Apple Inc.'s Motion to Compel Production of Documents and Testimony Regarding Set-Top Box Components, # 2 Exhibit B, # 3 Exhibit C, # 4 Exhibit D, # 5 Exhibit G, # 6 Exhibit H)(Pace, Christopher)

EXHIBIT C

III 11111111 III II II II II II II II II
US005594509A
United States Patent [191
Florin et al.

[11[ Patent Number:
I45] Date of Patent:

OTHER PUBLICATIONS

[543 METHOD AND APPARATUS FOR
AUDIO-VISUAL INTERFACE FOR THE
DISPLAY OF MULTIPLE LEVELS OF
INFORMATION ON A DISPLAY
[75] Inventors: Fabrice Florin, Mill Valley; Michael

Buettner, Budingame; Glenn Corey,
San Rafael; Janey Fritsche, Mill
Valley; Peter Maresca, Palo Alto;
Peter Miller, Los Altos Hills; Bill
Purdy, San Anselmo; Stuart Sharpe;
Nick West, both of San Francisco, all
of Calif.
[73] Assignee: Apple Computer, Inc., Cupertino,
Calif.

5,594,509
Jan. 14, 1997

Alexander, Visualizing cleared-off desktops, Computer
World, May 6, 1991, p. 20.
Itiroshi Ishii, Kazuho Arita, Clearface: Translucent Multiuser Interface for Team Work Station, ECSCW, Sop., 1991,
pp. 6-10.
Hiroshi Ishii, Naomi Miyaka, Toward an Open Shared
Workspace, Communication of the ACM, Dec., 1991, vol.
34, No. 12, pp. 37-50.
(List continued on next page.)
Primary Examiner James J. Groody
Assistant Examiner--Jeffrey S. Murrell
Attorney, Agent, or Firm--Blakely, Sokoloff, Taylor & Zafman

[57]

ABSTRACT

[21] Appl. No.: 81,931

An interactive audio-visual (A/V) transceiver is advantageously coupled to a television and/or telephone (T/T) cable,
[22] Filed:
Jun. 22, 1993
a TV, a video recorder (VCR), and other A/V devices. The
A/V transceiver switches data between a program/service
(Under 37 CFR 1.47)
provider and the connected A/V devices. In one embodiment, the transceiver includes three primary modules, a
[511 Int. CI.6 ....................................................... 04N 5/45
H
a CPU, a system bus, system
U.S. CI ...........................
348H31; 348/705; 348/734; main module including control unit, an audio-visuai bus,
memory, an infra-red (Ill)
348/505; 348/906 an A/V decoder, an A/V processor, and an A/V encoder, an
[58] Field of Search .....................................
348/731,734, A/V connect module including a number of tuner/demodu348/484, 706, 906, 467, 564, 705, 569, lators and a switch, and an optional CD ROM module. The
462, 570, 468, 7, 563, 10, 565, 552; 358/335; A/V transceiver hardware is complemented with an operatH04N 5/445, 5/45 ing system and software program which supports the functions provided in the A/V user interface. Additionally, a
remote control device is provided to communicate with the
[56]
References Cited
A/V transccivcr to interactively manage selection of proU.S. PATENT DOCUMENTS
gram and service sources, selection program and service
offerings from any selected source, viewing of selected
Re. 32,632 1211985 Atkinson ................................. 340/709
program offerings, and interaction with selected service
Re. 34,340 8/1993 Freeman .................................... 358/86
offerings. The remote control device is advantageously
4,290,142 9/1981 Schnee et al ............................... 455/3
provided with a basic A/V control button group, an interactive control button group, an auxiliary control button group
(List continued on next page.)
and a numeric key pad to facilitate control of the transceiver.
The interactive control button group includes an info button,
FOREIGN PATENT DOCUMENTS
a list button, a categories button, a pix button, a mark button,
0239884 10/1987 European Pat. Off..
a jump button, and a pointing device consisting of up, down,
393555 1011990 European Pat. Off. ......... H04N 7/08
left, and right arrow buttons, and a center select button.
420123 4/1991 European Pat. Off. ....... H04N 5/702
(List continued on next page.)

71 Claims, 50 Drawing Sheets

FL-Apple0000709

5,594,509
Page 2

U.S. PATENT DOCUMENTS
4,381,522
4,533,910
4,536,791
4,555,775
4,573,072
4,622,545
4,6z11,205
4,706,121
4,748,618
4,750,036
4,772,882
4,785,408
4,812,834
4,829,558
4,847,604
4,847,700
4,873,623
4,884,223
4,890,320
4,899,136
4,914,517
4,914,732
4,931,783
4,935,865
4,937,821
4,939,507
4,959,720
4,977,455
4,987,486
4,995,078
5,008,853
5,014,125
5,047,867
5,062,060
5,072,412
5,148,154
5,151,782
5,151,789
5,155,806
5,157,768
5,177,604
5,195,092
5,206,722
5,220,420
5,223,924
5,236,199
5,239,540
5,247,347
5,253,066
5,253,067
5,283,819
5,353,121
5,357,276
5,404,393
5,410,326
B 1 4,977,455

4/1983 Lambert .................................... 358186
8/1985 Sukonick et al ........................ 340/721
8/1985 Campbell et al ....................... 358/122
11/t985 Pike ........................................ 364/900
2/1986 Freeman .................................... 358/86
11/1986 Atkinson ................................. 340/747
211987 Beyers, Jr. .............................. 358/335
11/1987 Young ..................................... 348/906
5/1988 Braun et al ............................... 370/94
6/1988 Martinez ................................. 358/147
9/1988 Mical ...................................... 340/709
11/1988 Britton et al ........................ 364/513.5
3/1989 Wells ...................................... 3401721
5/1989 Welsh ........................................ 358/84
7/1989 Doyle ...................................... 340/706
6/1989 Freeman .................................. 358/343
10/1989 Lane et al ............................... 364/188
11/1989 Ingle et al ............................... 364/550
12/1989 Monslow et al .......................... 380/10
2/1990 Beard et al ............................. 340/706
4/1990 Dulfield ............................... 358/191.1
4/1990 Henderson et al ..................... 3401825
6/1990 Atldnson ................................. 340/710
6/1990 Rowe et al ............................. 364/188
6/1990 Boulton ..................................... 358/86
7/1990 Beard et aJ ............................. 340/706
9/1990 Duffield et al .......................... 348/734
12/1990 Young ..................................... 358/455
1/1991 Johnson et al ............................ 358/86
2/1991 Monslow et al .......................... 380/10
4/1991 Bly et al ................................. 364/900
5/1991 Pocock et al ............................. 358186
9/1991 Steubbe et at .......................... 348/569
10/!991 Kohaic.
12/1991 Henderson et aI ..................... 395/159
9/1992 Mackay et al .......................... 340/712
9/1992 Ferraro ...................................... 358/86
9/1992 Young .................................. 358/194.1
10/1992 Hoeber et al ........................... 395/157
10/1992 Hoeber et al ........................... 395/157
1/1993 Martinez ................................... 358/86
3/1993 Wilson et al .......................... 370/94.2
4/1993 Kwan ........................................ 358/86
611993 Hoarty et al .............................. 358/86
611993 Strabbe ..................................... 358/86
811993 Thompson, Jr. ........................ 273/439
8/1993 Rovira et al .............................. 370/77
9/1993 Litteral et al ............................. 358/85
10/1993 Vogel ...................................... 348/906
10/1993 ~ Chaney et al ........................ 358/191.1
2/1994 Glick et al ................................ 379/90
10/1994 Young et al ............................ 348/563
10/1994 Banker et al ............................... 348/7
411995 Remillard .................................. 379/96
4/1995 Goldstein ................................ 348/134
4/1993 Young ..................................... 358/142

FOREIGNPATENT DOCUMENTS
6174476 4/1986 Japan .............................

H04N 51445

48879 2/1990
8601962 3/1986
8912370 12/1989
9001243 2/1990
396062 11/1990
9118476 11/1991
9311639 611993
9311640 611993
9311638 611993

Japan ............................. H04N 51445
WIPO .............................. H04N 7/16
WIPO ............................ H04N 7/167
WlPO .............................. H04N 7/18
WIPO ............................ H04N 71173
WIPO .............................. H04N 7/10
WIPO .............................. H04N 7/16
WIPO .............................. H04N 7116
WIPO .............................. H04N 7/16

OTHER PUBLICATIONS
Article entitled: Learning Considerations In User Interface
Design: The Room Model; author: Patrick R Chan; publication of the Software Portability Laboratory, University of
Waterloo, Waterloo, Ontario, Canada, Jul., 1984.
Article entitled: Creation/Modification of the Audio Signal
Processor Setup For A PC Audio Editor; publication International Business Machines, IBM Technical Disclosure Bulletin, vol. 30, No. 10, Mar. 1988.
Article entitled: Browsing Within Time-Driven Multimedia
Documents; authors: Stavms Christodoulakis and Steven
Graham; publication of the Institute for Computer Research,
University of Waterloo, Waterloo, Ontario, Canada, Jul.,
1988.
Article entitled: Impact: An Interactive
Natural-Motion-Picture Dedicated Multi-Media Authoring
System; authors: Hirotada Ueda, Takafumi Miyatake, and
Satoshi Yoshizawa; Conmmnications of the ACM, Mar.,
1991, pp. 343-350.
Instructional manual entitled: Sonic The Hedgehog; authors:
Sega of America, 1991.
42nd Annual Convention and Exposition of the National
Cable Television Association, Jun. 6, 1993, SE CA, pp.
82-89, Mack Daily "Addressable Decoder with Downloadable Operation".
18th International Television Symposium and Technical
Exhibition, May 10, !993, Montreux, Switzerland, pp.
555-567, Hoarty "Multimedia a on Cable Television Systems".
IBM Technical Disclosure Bulletin, vol. 34, No. 7A, Dec.
1991, pp. 375-377, "Interactive Computer Conference
Server".
IBM Technical Disclosure Bulletin, vol. 36, No. 7, Jul. 1993,
pp. 53-54, "Interactive Device for Conventional TVs to
Improve Functionality".
Elektor, No. 4, Apr. 1990, pp. 10-12 "D2B-Homebus FOr
Audio and Video".
42rid Annual Convention and Exposition of the NCTA, Jun.
6, 1993, SF, CA, pp. 223-236, Bestler "Flexible Data
Structures and Interface Rituals for Rapid Development of
OSD Applications".

FL-Apple0000710

U.S. Patent

Jan. 14, 1997

Sheet 1 of 50

5,594,509

©
u

FL-Apple0000711

U.S. Patent

Jan. 14, 1997

Sheet 2 of 50

5,594,509

FL-Apple0000712

U.S. Patent

Jan. 14, 1997

Sheet 3 of 50

5,594,509

FL-Apple0000713

U.S. Patent

5,594,509

Sheet 4 of 50

jan. 14, 1997

FIG. 4b

FIG.
120

125
115
,!22

142

1
155,

142

1
134

134

140
.140
138

146

1

168
172

1
1

.,177

176---....,_

-I
I

FL-Apple0000714

Sheet S of SO

Jan. 14, 1997

U.S. Patent

FIG. 5a

82

5,594,509

15 122 115

¯

MUTE

POWER
MARK

JUMP

81

138
144
155

1

,150
34
140

136
146~

72

MENU

162
RECC

~8

PLAY/PAUSE
FFWD

17

®Q®
Q
MIC.

9

FL-AppleO000715

U.S. Patent

j+,~. 14, 1997

Sheet 6 of 50

5,594,509

FL-Apple0000716

U.S. Patent

Jan. 14, 1997

Sheet 7 of 50

5,594,509

E

FL-Apple0000717

U.S. Patent

Jan. 14~ 1997

5,594,509

Sheet 8 of 50

Z

0

FL-Apple0000718

U.S. Patent

Jan. 14, 1997

Sheet 9 of 50

5,594,509

FL-Apple0000719

U.S. Patent

Jan. 14, 1997

5,594,509

Sheet 10 of 50

o

FL-Apple0000720

U.S. Patent

Jan. 14, 1997

Sheet 11 of 50

5,594,509

FL-Apple0000721

-IG. 12

250

<]Thursday 10/15 8PMD

220--

~ ALL PROGRAMS
Personal Story
9 A&B
10 NTM
Rock Today <1> Party Tim~
11 DBS _"Say, Nay, B,ill¥~
12 BMT
Manager’s Forum
Windsuding <!> Basketball
13 TC
14 DCV
Planet Life
15 NKK
Wild Joe <1> Fun Times
16 TSN
Top StoryNews
17 HMX
Top Movie
18 STM
,Best Movie
19 WRR Comedy Program
Popular Program
20, NNC

iv

~251

:IG. 13

<]Thursday 10/15 8PMD
ALL P R OG RAMS

9
10
11
12
13
14
15
16
17
18
19
,20

A&B
NTV®
DBS
BMT
TC
DCV
NKK
TSN
HMX
STM
WRR
NNC

Personal Story
~ock’Today <i> Pady Tim,
"Say Nay" Billy
Manager’s Forum
Windsuding <1> Basketball
Planet Ufe
Wild Joe <1> Fun Times
Top StoP/News
Top Movie
Best Movie
Comedy Program
Popular Program

.251

8:06:30

FIG. 14
<]Thursday 10/15 8PMD
L~
180

7
8
9
10
11
12
13
14
15
16
17
18

ALL PROG R A M S

XYZ
CMM
A&B
NTV
DBS
BMT
TC
DCV
NKK
TSN
HMX
STM

St. Louis at SF (6:30)
ShowbitsThis Month
Personal Story
Rock Today <i> Party Time (8:10)~
"Say Nay" Billy
Manager’s Forum
Windsurfing <1> Basketball
Planet Life
Wild Joe <!> Fun Times
Top Story News
Top Movie
Best Movie

FIG. 15

<]Thursday 10/15 8PMD

180

~ ALL
7 XYZ
8 CMM
9 A&B
10 NTV
11 DBS
12 BMT
13 TC
14 DCV
15 NKK
16 TSN
17 HMX
18 STM

PROGRAMS

St. Louis at SF (6:3(J)
Showbits This Month
Personal Story
Rock Today <!>Party Time (8.;i0)1~., (~
"Say Nay" Billy
Manager’s Forum
Windsurfing <1> Basketball
Planet Life
Wild Joe <1> Fun Times
Top Story News
Top Movie
Best Movie
~

°252

FIG. 16

ALL PROGRAMS
1 BILL.... BILLBOARD ~
The Lights
2 FXX
Center Stage Sports
3 SPN
Day Court
4 NJJ
Lonesome Bird (8:00)
5 CFF
This Generation
6 VT1
7. XYZ
St. Louis at SF (6:30)
News <1> Politics Today
8 CMM
Personal Story (8:00)
9 A&B
Bluff
10 NTV
Intrigue
11 DBS
Shop Talk
12 BMT

FIG.

LL P R O G RA MS

1
2
3
180

BILL.
FXX
SPN

4 NJJ
5 CFF
6 VT1
7 XYZ
8 CMM
9 A&B
10 NTV
11 DBS
12 BMT

BILLBOARD ~) "J
Strongmen
College Football
Specials Tonight
News <!> New MTGH
This Generation
News <1> Early Peek
K. Johnson <!> Resources
The Movie Special (5:00)
Rock This Week <t> Overview
Jan Can Cook <!> Intrigue
Personal Interviews

.FIG. 18

,~Sa~:~lrday 10/17 6PM
ALL PROGRAMS
180

.=
I
2

RILL
FXX
SPN

4 NJJ

RILLBOARD
STRONGHEN
COLLEGE FOOTBALL

Specials Tonight,

RECORD~SPEClALS TONIGHT - 60 ,,,ln

,~259
’VCR

~

ONCE

FIG. 19

Saturday 10/17 6PMD
180

ALL P R"OG RAMS
1 BILL B I LLBOARD
2 FXX
Strongmen
3 SPN
College Football
4 NJJ
..Specials Tonight
5 CFF
News <!> New MTGH
6 VT1
This Generation
7 XYZ
News <1> Early Peek
8 CMM
K. Johnson <!> Resources
9 A&B
The Movie Special (5:00)
10 NTV
Rock This Week <!> Overview
11 DBS
Jan Can Cook <!> Intrigue
12 BMT
Personal Interviews

U.S. Patent

Jan. 14, 1997

Sheet 20 of SO

5,594,509

FL-Apple0000730

U.S. Patent

Jan. 14, 1997

Sheet 21 of 50

5,594,509

FL-Apple0000731

-IG. 22

CATEGORIES: ALL PROGRAMS
30O
302

305

310

315

312

313

FIG. 2 5

CATEGORIES: FAVORITES @

302

305

310

315

312

313

U.S. Patent

ja~. 14, 1997

Sheet 24 of 50

5,594,509

FL-Apple0000734

U.S. Patent

J~. I4, 1997

Sheet 25 of 50

5,594,509

FL-Apple0000735

U.S. Patent

j+~. 14, 1997

Sheet 26 of 50

5,594,509

FL-Apple0000736

U.S. Patent

Jan. 14, 1997

Sheet 27 of 50

5,594,509

FL-Apple0000737

U.S. Patent

Jan. 14, 1997

5,594,509

Sheet 28 of 50

0
0

FL-Apple0000738

U.S. Patent

Jan. 14, 1997

Sheet 29 of 50

5,594,509

FL-Apple0000739

--IG. 30

325

Thursday 10115 8PM

320,

FAVORITES

7
8
9
10
11
12
13

VT1
XYZ
CMM
A&B
NTV
DBS
BMT
ST

Sax Cymbals
St. Louis at SF (6:30)’~_/’
Showbits This Month
Personal Story
Rock Today <l> Party Time~’
"Say Nay" Billy
Manager’s Fo.rum
Windsurfing~’.l> Basketball

CATEGORIES: FAVORITES ®

3O5

:23:30

FIG. 31

362

6
7
8
9
10
11
12

VT1
XYZ
CMM
A&B
NTV
DBS
BMT

Sax Cymbals
St. Louis at SF (6:30)~/
Showbits This Month
Personal Story
Rock Today <1> Party TimeV/
"Say Nay" Billy
Manager.’s’ Fo~m

FAVORITES: FREQUENT
-300
318

U.S. Patent

Jan. 14, 1997

Sheet 32 of 50

5,594,509

FL-Apple0000742

U.S. Patent

Jan. 14, 1997

Sheet 33 of 50

5,594,509

FL-Apple0000743

U.S. Patent

Jan. 14, 1997

Sheet 34 of 50

5,594,509

FL-Apple0000744

U.S. Patent

Jan. 14, 1997

Sheet 35 of 50

5,594,509

FL-Apple0000745

U.S. Patent

Jan. 14, 1997

Sheet 36 of 50

5,594,509

FL-Apple0000746

U.S. Patent

Jan. 14, 1997

Sheet 37 of 50

5,594,509

FL-Apple0000747

FIG. 38

180
390--..,

I PREVII~W]
CREDITS
Length
382

TICKET
388

STARRING
FABRICE FLORIN

180

WRITTEN BY
JEFF BLA’I-r
390,,

DIRECTED BY
FABRICE FLORIN
PRODUCED BY
JOHN SCULLY
~::!<~

CREDITS

INFO

TICKET

FIG. 40

180

Jeff’s World
Rated R
Apple Pictures
90 rain.
When a sleazy TV exec offers
Jeff and John a fat contract
to tape their Cable-access show,
they can’t believe their luck.
But they soon discover the road from
basement to big time Is a gnarly one.

CREDITS

382

384

INFO

TICKET

386

388

FIG. 41

TO SEE
JEFF’S WORLD
PLEASE ENTER YOUR MOVIEPASS NUMBER
...-425

YOU WILL BE BILLED $3.00

CREDITS

INFO

TICKET

[3 [3

388

U.S. Patent

jan. 14, 1997

Sheet 42 of 50

5,594,509

~ZW

I rT"~

u-o~

W cO

FL-Apple0000752

:IG. 43
<]Thursday 10115 8PMD Je~s--~~World
FAVORITES

180~

12 BMT
13 ST
47 PXZ
48 SHP
49 TRL
50 TCT
1 BILL
6 VT1
7 XYZ
8 CMM
9 A&B
10 NTV

Manager’s Forum
Windsurfing <t> Basketball
Jeff’s World

TV S~o C’ac’~’s e .........
Sunny City Vacationtime
Technology Today
BILLBOARD
Sax Cymbals
St. Louis at SF (6:30)
Showbits This Month

Personal Story
Rock Today <1> Party Time

U.S. Patent

J~n. 14, 1997

Sheet 44 of 50

5,594,509

FL-Apple0000754

U.S. Patent

Jan. 14, 1997

Sheet 45 of 50

5,594,509

FL-Apple0000755

FIG. 46

~t
the
attic cacy’s
180

I I~uralite Cookware I @
Lucks Mini Espresso
Lucks Brewmaster
Regatta Home Bakery
Juice Tiger Extractor

406

FIG. 47

~
the
attic cacy’s
Duralite Cookware
! Lu&ks Mini Esp,r.esso ] @
Lucks Brewmaster
Regatta Home Bakery
Juice Tiger Extractor

4O6

FIG. 4 8

Lucks Mini Espr
Espresso and Cappuccito
maker from the world’s
leader in coffee and
expresso makers. W’dh
Perfect Froth nozzle for
perfect cappuccino.
LIMITED OFFER:
Was $99.99 Now $69.991

408

FIG. 49

Lucks Mini Espresso

180

-4 cup 10 oz

Shop Card NUMBER
420-

I****1
OR CALL 1-555-SHOPSpp

Price
Cal. Sales Tax
Shipping

$69.99
5.61
3.00

Total Cost

$78.60

409

FIG. 50

Lucks Mini Espresso
180

4 cup 10 oz
Your order
will be delivered
by RST tomorrow
before 5PM.

Price
Cal. Sales Tax
Shipping

$69.99
5.61
3.00

Total Cost

$78.60

409

5,594,509
1

2

In the computer industry, a variety of graphic user interfaces have been developed to facilitate human interaction
with computer systems. Many display systems utilize metaphors in the design of the interface as a way of maximizing
5 human familiarity, and conveying information between the
user and the computer. It is well known that designing
RELATED APPLICATIONS
around a familiar metaphor helps reduce human learning
time (See for example, Patrick Chart, "Learning ConsiderThe present application is related to the following conations In User Interface Design: The Room Moder’, Report
temporaneously filed applications:
10 CS-84-16, University of Waterloo Computer Science
(a) Ser. No. 081082,081, entitled "Method And Apparatus
Department, Ontario, Canada, July, 1984, and the references
For Controlling An Audio-Visual System For Selectively
cited therein). Research in interface design using metaphors
Viewing And Interacting With Programs And Services From
in man-machine interfaces may be applied to multi-media
A Number of Program/Service Sources";
systems, and in particular, to interactive television systems.
(b) Ser. No. 081082,056, entitled "Methods And Appara-. 15 The marriage of video and television technology with
tus For Managing Selection Of Audio-Visual Program And
computer interface technology provides consumers with
Service Offerings";
maximum flexibility in storing, retrieving and viewing tele(c) Ser. No. 081081,471, entitled "Methods And Apparatus
vision and other audio-visual programming. As wil! be
For Managing Viewing Of And Interaction With Audiodescribed, the present invention provides methods and appaVisual Program And Service Offerings".
2o ratus tbr presenting an improved audio-visual user interface,
which includes various user-selectable features for viewing
and controlling a television, video tape recorder (VCR) and
BACKGROUND OF THE INWENTION
other audio-visual devices. As will be disclosed, the present
1. Field of the Invention
invention’s user interface provides a user-friendly mechaThe present invention relates to the field of audio-visual 25 nism for consumers to view, record, and play back TV and
A/V programs, as well as control other A/V home entertainsystems. More specifically, the present invention relates to a
ment devices using a remote control device. Furthermore,
system for selectively viewing and interacting with proinformation such as TV program listings and additional
grams and services from a number of program/service
information related to programs as well as selecting and
sources, a control device for controlling the system, and the
methods and apparatus incorporated in the system for man- 3o controlling categories of iuteractive programs and services
may be provided through the user interface of the present
aging selection, viewing, and interacting with the program/
invention.
service offerings.
2. Art Background
SUMMARY OF THE INVENTION
Over the past 40 years, television and motion pictures 35
have become an important aspect of everyday life for people
The present invention provides methods and apparatus for
in the industrialized world. The development of advanced
presenting an improved audio-visual user interface for
technology in the areas of digital and high definition teleselecting and displaying cable television or other audiovision (HDTV), video recording systems, laser disks and
visual programs, as well as controlling various audio-visual
compact disc (CD) entertainment systems, coupled with 40 devices and interactive services. The present invention
satellite, cable television and telephone services, have proassumes a service provider provides cable television and/or
vided opportunities for viewers to store, retrieve and selectelephone (T/T) service to users via a T/T cable, including a
tively display a variety of television and audio-visual or
digital channel of program/service listings, at least one
interactive programming on home entertainment systems.
digital back channel (from the user’s home to a central file
Over the past decade, improvements in personal computing 45 server), a number of analog TV channels, a number of digital
systems have provided a variety of powerful miniaturized
pay-per-view channels, and other interactive services transpersonal computers which permit the storage of data and
mitted from remote storage devices such as digital file
contro! of home appliances, such as entertainment systems,
servers. Under the present invention an interactive audiothrough the use of microprocessors. Additionally, a variety
visual (A/V) transceiver is advantageously coupled to a
of graphic user interfaces have been developed to ease 5o user’s television, video tape recorder, and the T/T cable. In
human interaction with these new personal computer sysone embodiment, the transceiver includes three primary
tems.
modules. A main module includes a central processing unit
The combination of computer technology with television
(CPU) coupled over a digital system bus to system memory
and, preferably, an infrared (IR) or similar wireless control
(TV) and audio-visual (A/V) systems, has fostered the
development of multi-media interactive entertainment sys- 55 unit. The main module is further coupled over an audiotems. The combination of computer hardware and software
visual bus to an A/V decoder, an A/V processor, an A/V
with audio-visual systems has brought new forms of art and
encoder, a TV, an A/V connect module connected to the T/T
entertainment into being. CD read only memory (ROM) cable, and to one or more VCRs and other A/V devices. The
main module may also be coupled to an optional CD ROM
systems coupled to personal computers permit interactive
video computer simulations, unique self-paced learning 6o module, to permit the playback of multi-media CD ROM
environments and interactive "movies", not possible in
titles, audio CDs, and the like. Other optional modules or
earlier television systems (See, "The Revolution Starts
external devices (such as additional VCRs, CDs, hard disks,
Here", Newsweek, page 42 (Jan. 18, 1993)). It is known that telephone, fax and answering machines) may be connected
to this transceiver and controlled through the use of the
consumers desire interactive television and multi-media
applications for home use. One of the biggest challenges for 65 audio-visuai user interface of the present invention. The A/V
an interactive television service is the design of an interface
transceiver hardware is complemented with an operating
that is easy and fun to use by average consumers.
system software program which supports the functions proMETHOD AND APPARATUS FOR
AUDIO-VISUAL INTERFACE FOR THE
DISPLAY OF MULTIPLE LEVELS OF
INFORMATION ON A DISPLAY

FL-Apple0000761

5,594,509
3

4

vided in the present invention’s audio-visual user interface.
on connected A!V playback devices, a numeric keypad for
dialing channel numbers, confirming financial transactions
In operation, the cable or telephone service provider
with personal identification numbers, an optional microtransmits an interleaved data stream preferably including a
phone with talk button for supplementing the present user
television program listing, together with program titles,
program times, categories, channel numbers and the like, interface with direct voice commands through the use of
from a central file server on the digital channel of program voice recognition technology, a pay-per view interface, and
listings to the transceiver. The data is interleaved such that
a home shopping interface, as well as other features
the current day’s data is followed by other weekly data.
described herein.
A remote control device is preferably provided for comBRIEF DESCRIPTION OF THE DRAWINGS
mu.nicating with the transceiver, and includes a number of 10
user selectable interactive functions such as: an info button,
FIG. 1 is a functional block diagram of the audio-visual
a list button, a categories button, a pix button, a mark button,
(A/V) system of the present invention.
a jump button, a select button, and a pointing device
FIG. 2 is a functional block diagram illustrating the major
consisting of a set of cross-hair arrow buttons (up, down,
components of the audio-visual transceiver of FIG. 1 in
left, right) or other pointing devices capable of providing the 15 further detail.
same functions (such as a mouse, thumb stick or touch pad).
FIG. 3a diagramatlcally illustrates the electronic spectrum
While viewing the TV, a user may obtain additional inforutilized by the present invention.
mation on a current program by depressing the info button,
FIG. 3b conceptually illustrates the present invention’s
and obtaining more detailed information using the pointing
device. By depressing the list button on the remote control 20 use of an interleaving data stream to download program!
device, the transceiver displays a program listing of the
service listings, and other information, to the present invencurrent programs available for viewing. Through the use of
tion’s A/V transceiver.
the pointing device, viewers can scroll up and down the
FIG. 4a illustrates one embodiment of the remote control
program listing or view a highlighted program in full screen device of the present invention with the sliding panel in a
by pressing the select button. By pushing the right or left
closed position.
arrow buttons on the pointing device, program listings may
FIG. 4b illustrates the first embodiment of the remote
also be viewed for different hours, days and up to several
control device of the present invention shown in FIG. 4a
weeks in advance through the present invention’s use of data
with the sliding panel in an open position.
interleaving. The depression of the categories button on the
FIGS. Sa and Sb illustrate second embodiment of the
remote control device results in the display of a categories 30 remote control device of theapresent invention with no
menu bar on the TV screen, which includes categories such
sliding panel and additional features.
as "all", "sports", "news" and "favorites". The seIection of
FIG. ~i illustrates the present invention’s full screen viewone of these categories results in the display of only proing function as viewed by a user with a graphic overlay.
grams within that category. Moreover, through the depressions of both the list button and the categories button, a 35 panel which includes the current channel number as well as
the channel logo and identifier.
program listing is displayed of the currently available proFIG. 7 illustrates the present invention’s info function
grams within the selected category. Through the use of the
arrow buttons on the pointing device, category listings for
showing the first page of the basic information track ("infodifferent times and dates up to several weeks in advance may
track").
also be viewed. The selection of the favorites category 40 FIG. 8 illustrates the present invention’s info function
results in the display of currently available favorite prowith a second page of basic information, obtained by using
grams, including programs that are frequently watched by a
the right arrow button on the remot~ control device.
particular TV household, marked programs, or programs
FIG. 9 is an additional illustration of the info function of
that match preselected user preferences.
the present invention in which the user has requested a third
The depression of the pix button results in the display of 45 page of information using the remote control device.
a "visual menu" of multiple picture-in-picture (PIP) winFIG. 10 illustrates the present invention’s use of the info
dows along the perimeter of the TV screen. Each of the PIP function providing the user with additional tracks of inforwindows displays one of the programs currently being
mation (such as the "Tid Bits" information track), by using
received by the A/V transceiver in the last category selected
the down arrow button of the remote control device.
by the viewer. The PIP windows are displayed at a variable 50
FIG. 11 is a further illustration of additional information
frame rate (N) (depending on the number of windows), and
tracks ("info-tracks"), such as this "Info Mart", accessible to
are captured by one or more tuners in the transceiver. A
the user of the present invention using the down arrow
center PIP window displays a larger video image (along with
button of the remote control device.
audio) of the currently highlighted PIP window. The other
FIG. 12 illustrates the present invention’s use of the list
PIP windows can be similarly viewed by using the pointing 55
function to display current program/service listings and to
device on the remote control. The depression of the mark
highlight a particular program. Viewers may scroll up or
button allows users to "bookmark" a particular program for
down that list by using the up or down arrow buttons on the
later viewing. The depression of the jump button allows a
remote control device.
user to jump between previously marked programs.
Additional features, functions, and interface screens are 6o FIG. 13 illustrates the list function of the present invention in which the NTV programming during the hour of 8:00
available to the viewer using the teachings of the present
to 9:00 p.m. includes two programs, one starting at 8:00
invention, including a menu button for controlling other
p.m., the other starting after the previous program has ended.
devices connected to the A/V transceiver (such as CD, VCR,
etc.), a record button for making copies of programs onto
FIG. 14 illustrates the list function of the present invertconnected A/V recording devices (such as VCRs, hard disks, 65 tion which allows the user to highlight the second program,
etc.), a plurality of control buttons for playing back, stopwhich starts at 8:10 p.m., using the right arrow button of the
ping, rewinding, or fast-forwarding audio-visual programs
remote control device.

FL-Apple0000762

5,594,509
5
FIG. 15 is an additional feature of the list function of the
programs", is highlighted displaying a list of previously
present invention where the user marks the highlighted
marked programs.
program with the mark button of the remote control device.
FIG. 33 conceptually illustrates the pix function of the
FIG. 16 illustrates the selection of an alternate hourly time present invention wherein multiple PIP windows are disslot beginning at 9 p.m. using the right arrow button of the 5 played by pressing the pix button on the remote control
remote control device.
device.
FIG. 17 illustrates the selection of an alternate day for the
FIG. 34 illustrates the pix function of the present invenprogram/service listing by depressing the right arrow button
tion in which a new PIP window has been highlighted and
of the remote control device.
is displayed in a larger format in the center of the display
FIG. 18 illustrates displaying of a record panel confirming to screen.
the title and length of the program to be recorded, along with
FIG. 35 is a further illustration of the present invention’s
a confirmation of which VCR and tape to record on, by
pix function in which, through the use of the remote control
depressing a select button in the remote control device.
device, a PIP window showing a pay-per-view program has
FIG. 19 illustrates the present invention’s use of the select
been highlighted and is displayed in the center portion of the
button on the remote control device while the program/ 15 screen.
service listing is displayed to instruct the selected VCR or
FIG. 36 illustrates the menu function which permits users
MV recording device to make a copy-of the highlighted
to press a menu button to switch the television display to
program.
other audio-visual devices coupled to the transceiver, such
FIG. 20 illustrates the present invention’s "reminder"
as VCR, CD, on-line services, telephone, etc. For example,
feature, which automatically displays a live picture-in-pic- 2o once VCR is selected, with the right arrow button, the TV
ture (PIP) window of a previously marked program when the
displays an image from the videotape currently in the VCR
program is received by the A/V transceiver.
and the user can press the play button on the remote to
FIG. 21 illustrates the present invention’s jump function,
playback a previously recorded program on the video tape.
wherein pressing a jump button one the remote control
FIG. 37 illustrates one embodiment of the present invendevice permits a user to jump directly to a full screen view 25
tion for selecting a pay-per-view channel offering entitled
of a previously marked program, and wherein pressing the
"Jeff’s World" through the use of the select button of the
jump button again results in the display of another marked
remote control device.
program.
FIG. 38 illustrates a preview feature of pay-per-view
FIG. 22 i~lustrates the categories function of the present
invention which is displayed by pressing the categories 30 under the above illustrated embodiment.
button on the remote control device.
FIG. 39 illustrates a credits feature of pay-per-view under
the above illustrated embodiment.
FIG. 23 further illustrates the categories function wherein
the "favorites" category is highlighted by pressing the right
FIG. 40 illustrates an info feature of pay-per-view under
arrow button on the remote control device.
the above illustrated embodiment.
FIG. 24 further illustrates the categories function wherein35 FIG. 41 illustrates a confirmation feature for ordering a
the "sports" category is highlighted by depressing the right
pay-per-view offering through the use of the numeric keypad
arrow button on the remote control device.
on the remote control device under the above illustrated
embodiment.
FIG. 25 illustrates a representative television image in the
FIG. 42 illustrates the reminder feature advising the user
case where the "sports" category has been selected by 40
depressing the select button on the remote control device.
that the selected offering of pay-per-view will begin in three
FIG. 26 is a further illustration of a representative tele- minutes under the above illustrated embodiment.
vision image in which the "sports" category has been
FIG. 43 illustrates one embodiment of the present invention for selecting and interacting with a home shopping
selected, and an alternate sports channel has been further
selected by a user through the use of the up arrow button on 45 service (TV Shop) offering.
the remote control device.
FIG. 44 illustrates a sample commercial shown to the user
FIG. 27 iIlustrates a sports program listing in which the
if the home shopping service is seIected through the use of
list button has been depressed after the "sports" category has
the select button on the remote control device under the
been selected.
above illustrated embodiment.
FIG. 28 illustrates the categories function in conjunction 50 FIG. 45 illustrates a menu permitting the user to select
with the list function.
various shops available on the home shopping service using
the select button on the remote control device under the
FIG. 29 further illustrates the categories and list functions
above illustrated embodiment.
when the down arrow button is depressed on the remote
control device, breaking down the "sports" category into 55 service in46 is a a cookware product is highlighted under the
FIG. which further illustration of the home shopping
sub-categories such as baseball or football.
above illustrated embodiment.
FIG. 30 is a further illustration of the categories and list
functions in which the category "favorites" is highlighted
FIG. 47 is a further illustration of the home shopping
and can now be selected by depressing the select button or
service in which a mini-espresso machine is highlighted, and
broken down into sub-categories by depressing the down 60 can be selected for ordering with the select button on the
arrow button on the remote control device.
remote control device under the above illustrated embodiment.
FIG. 31 further illustrates the categories and list functions,
where the "favorites" category has been broken down into a
FIG. 48 is an information screen provided to the user after
sub-category of "frequent" programs which are currently
selecting a mini-espresso machine for ordering under the
listed.
65 above illustrated embodiment.
FIG. 32 is a further illustration of the categories and list
FIG. 49 is an order screen in which the user, using the
functions, where another "favorites" sub-category, "marked
numeric keypad on the remote control device, enters a

FL-Apple0000763

5,594,509
7

8

info, list, record, mark, jump, category, pix and menu
personal identification number to order the mini-espresso
macbAne under the above illustrated embodiment.
functions, as well as the overall structure and operation of
FIG. 50 is a sample order confirmation provided to the the present invention’s user interface.
In addition, in the following description, numerous speuser subsequent to ordcring a product in the home shopping
5 cific details are set forth such as functional blocks represervice under the above illustrated embodiment.
senting data processing devices, and metaphors such as
NOTATION AND NOMENCLATURE
screen, menu and other configurations to assist the user in
navigating through the user interface, etc., to provide a
The detailed descriptions which follow are presented
thorough understanding of the present invention. However,
largely in terms of interface display images, algorithms, and to it will be apparent to one skilled in the art that the present
symbolic representations of operations of data bits within a
invention may be practiced without these specific details. In
computer memory. These algorithmic descriptions and repother instances, well known circuits and structures are not
resentations are the means used by those skilled in the data
described in detail so as not to obscure the present invention
processing arts to most effectively convey the substance of
unnecessarily. For the purpose of this application, the word
their work to others skilled in the art.
15 "audio-visual" and its abbreviation, "AW", will be used
An algorithm is here, and generally, conceived to be a self
interchangeably and will have the same meaning.
consistent sequence of steps leading to a desired result.
General System Configuration
These steps are those requiring physical manipulations of
physical quantifies. Usually, though not necessarily, these
The multi-media interactive television system of the
quantifies take the form of electrical or magnetic signals20 present invention is illustrated in schematic form in FIG. 1.
capable of being stored, transferred, combined, compared,
As shown in FIG. 1, a cable or telephone service provider 50
displayed and otherwise manipulated. It proves convenient
provides cable television or telephone (T/T) services over a
at times, principally for reasons of common usage, to refer
T/T cable 52 to a plurality of users coupled to the cable or
to these signals as bits, values, elements, symbols, charactelephone system, as is known. Although in this Specificaters, images, terms, numbers, or the like. It should be borne 25 tion reference is made to a cable television or telephone
in mind, however, that all of these and similar terms are to system, it will be appreciated by one skilled in the an that
be associated with the appropriate physical quantities and
the present invention may be used in conjunction with a
are merely convenient labels applied to these quantities.
variety of other electronic transmission systems including
In the present case, the operations are machine operations
satellite service systems, microwave systems, fiber optic,
performed in conjunction with a human operator. Useful30 and radio frequency (RF) systems.
machines for performing the operations of the present invenAs illustrated in FIG. 1, the T/T cable 52 is coupled to an
tion include general purpose digital computers, digitally
audio-visual transceiver 54 which comprises a number of
controlled displays or other similar devices. In all cases,
separately identifiable modules. The transceiver 54 is
there should be borne in mind the distinction between the
proximity
a VCR
35 intended to be located in58, as wellto and coupled tooptional
method operations of operating a computer and/or display
511, a television (TV)
as one or more
system, and the method of computation itself. The present
audio-visual devices 57 such as additional VCRs, laser disc
invention relates to method steps for operating a computer
players, camcorders, stereos, various storage devices, teleand interactive display system, and processing electrical or
phones, faxes and answering machines, as shown. Moreother physical signals to generate other desired physical 40 .over, it is contemplated that transceiver 54 may be directly
signals.
incorporated into the VCR 56, the TV $8, or one the other
The present invention also relates to apparatus for perA/V devices 57. As will be described below, a remote
forming these operations. This apparatus may be specially
control device 60 communicates with the transceiver 54
constructed for the required purposes or it may comprise a
preferably through a wireless transmission signal (for
general purpose computer selectively activated or reconfig45 example, an infrared 0R) signal), or other mechanisms
ured by a computer program stored in the computer. The
known in the art. Additionally, as illustrated in FIG. 1, the
method steps presented herein are not inherently related to
transceiver 54 is further capable of communicating with the
any particular computer or other apparatus. In particular,
television 58, the VCR 56 and the other A/V devices 57
various general purpose machines may be used with prothrough infrared or other means.
grams in accordance with the teachings herein, or it may 50 Referring now to FIGS. 1 and 2, the transceiver 54
prove more convenient to construct more specialized appacomprises three primary modules: a main module 62, an A/V
ratus to perform the required method steps. The required
66 and
structure for a variety of these machines will appear from the connect module ROM) an optional compact disc read only 62
memory (CD
module 70. The main module
description given below. Machines which may perform the
includes a central processing unit (CPU) 63 coupled over a
functions of the present invention include those manufacan infra-red (IR)
tured by the Assignee, Apple Computer, Inc., as well as other 55 system bus 114 to a system memory 65 andwireless control
control unit 82, which sends and receives
manufacturers of computer and computer controlled mutlsignals to and from the remote control device 60. The CPU
media systems.
63 is further coupled through the system bus 64 to a memory
and bus controller 80, which is itself coupled through an A/V
DETAILED DESCRJP’I~ON OF THE
60 decoder 74 and an A/V encoder 78 to the A/V connect
INVENTION
module 66, as well as to an optional CD ROM module 70.
The CPU 63 is also coupled through the system bus 64, the
The following detailed description will be divided into
memory and bus controller 80 and an A/V bus 73 to an A/V
several sections. The first of these will describe a general
processor 77 and an optional A/V memory 75.
system arrangement for receiving and generating the audiovisual user interface of the present invention. Subsequent 65
The A/V connect module 66 switches and receives analog
sections will deal with the functional aspects such as the
audio-visual signals and digital data from a plurality of
present invention’s interactive functions, which includes the
audio-visual sources inciuding the T/T cable 52, the video

FL-Apple0000764

5,594,509
9

10

cassette recorder (VCR) 56, or the other A/V devices 57, and such as order information sent by the CPU 63 through the
couples those signals and data to the main module 62
memory and bus controller 80 and the system bus 64 and
through the video decoder 74. The A/V connect module
transmit them to the service provider 50 through the A/V
further switches audio-visual signals and data received from
connect module 66 and a back channel 102 described below.
the main module 62 through the video encoder 78, and sends 5 The CPU 63 is also coupled over the system bus 64 to a
them back out to the T/T cable 52, the VCR 56, the other system memory 65 including both volatile and non-volatile
A/V devices 57 and/or the TV 58. The A/V connect module
components. The non-volatile part
66 includes a switcher 67 and one or more programmable memory 65 includes read-only memory (suchof system
memory
as ROM),
tuners/demodulators 69, wherein one tuner/demodulator
reads and displays a current program from one of the lo which is used to store an operating system and playback
software, fonts, sounds and the like used in the present
channels received over the T/T cable 52, and additional
invention. The non-volatile part of system memory 65 also
tuners/demodulators (or the same tuner/demodulator, used
includes rewritable memory (such as SRAM), which is used
in alternation) are used to read and display data from the
for persistent storage of mark or record indicators, listings of
side-band channeIs in picture-in-picture (pip) windows. In
programs viewed or taped, and other user preferences.
addition, the A/V connect module 66 may include dcscrambling circuitry (not shown) to descramble premium pro-15 Additionally, the volatile part of system memory 65 includes
sufl]cient random access memory (such as RAM or DRAM)
grams received over the T/T cable 52. It is further contemfor the temporary storage of data received over the T/T cable
plated that the A/V connect module 66 provides a graphic
52 or from the other devices 56-58 connected to the transoverlay function that superimposes an A/V signal from the
video encoder 78 against another A/V signal from the T/T 2o ceiver 54.
The transceiver 54 can also include an optional CD ROM
cable 52, the VCR 56, or the other A/V devices 57 allowing
both signals to be simultaneously displayed on the TV 58,
module 70 coupled through the memory and bus controller
the VCR 56 or the other A/V devices 57. Finally, the A/V 80 to the system bus 64 and the CPU 63. This optional CD
connect module 66 can be used to transmit data such as order
ROM module 70 constitutes a general purpose storage
information to the cable (T/T) service provider 50 over a
device to permit playback of optical compact discs (CDs),
25 including multi-media CD ROM titles, audio CDs, photo
back channel 102 described below.
CDs, and motion picture CDs on the TV 58. It is contemThe A/V decoder 74 is used to decode data encoded in the
plated that those transceivers which do not include an
vertical blanking interval or special side-band cable chaninternal CD ROM module could instead include a CD ROM
nels into digital data and couple them to the CPU 63 through
interface allowing users
an external CD ROM
the memory and bus controller 80 and the system bus 64. 30 drive to the transceiver 54to connectdate. It is also contemat a later
Furthermore, the A/V decoder 74 is used to convert analog
plated that this CD ROM module 70 may in the future use
audio-visuaI signals from the A/V connect module 66 into
different electronic, magnetic, optical or storage technolodigital A/V data and couple them to the A/V processor 77
gies other than the current CD ROM embodiment.
through the memory and bus controller 80 and the A/V bus
Additional modules may be added to the transceiver 54
73. Moreover, the A/V decoder 74 may be used to resize 35
audio-visuai signals from the A/V connect module 66 in including, for example, such options as a small computer
system interface (SCSI) for accessing digital storage devices
order to create picture-in-picture windows and the like. It is
also contemplated that the A/V decoder 74 may be used to
such as hard disks, a modem for exchanging digital data over
decompress certain analog or digital signals (such as MPEG
telephone lines, or a serial port for controlling other devices
motion video and the like) and couple them to the A/V
over a wired connection. Another module presently being
processor 77 through the memory and bus controller 80 and
contemplated is a remote beeper button for locating the
rcmote contro! device 60. Many consumers often misplace
the A/V bus 73.
their remote control devices, and it is contemplated that the
The memory and bus controller 80 is used to route data
present invention may provide a function wherein a speaker
and control signais between the system bus 64, the A/V bus
73, the A/V decoder 74, the AJ V encoder 78 and the optional 45 on remote control 60 would beep while the beeper button is
CD ROM module 70. The A/V bus 73 is a high-speed digital pressed until the user locates the remote control.
Referring now to FIG. 3a, there is shown one possible
bus used to free up the system bus 64 during the transmission of large amount of audio-visual data between the A/V
electronic spectrum of signals provided by the cable T/T
processor 77, the optional A/V memory 75 and, through the
service provider 50 over the T/T cane 52 to the transceiver
memory and bus controller 80, the A/V decoder 74, the A/V5o 54. As illustrated in FIG. 3a, the spectrum includes at least
encoder 78, and the optional CD ROM module 70. The A/V one digital program listing channel 100, a plurality of back
channels 102, and a plurality of standard analog’IN channels
processor 77 coupled to the A/V bus 73 is used to manipulate, process, render, mix and otherwise re-arrange digital 103. In addition, and as will be described more fully below,
data into coherent audio-visual displays. An optional A/V
the spectrum includes pay-per-view digital channels 106,
memory module 75 also coupled to the A/V bus 73 can be55 and further additional definable digital channels 10g offering
used to store A/V data before or after processing by the A/V
a variety of interactive services, in addition to the general
processor 77 (using components such as video random
purpose or special channels or services particular to the
access memory (VRAM) to hold, for example, a frame specific application of the present invention.
buffer equivalent to one 640x480x8-bit color image in
As illustrated in FIG. 3b, the digital program listing
memory).
6o channel 100 provides data representing daily and weekly
program listings and related information from the service
The A/V encoder 78, coupled to the A/V bus 73 through
provider 50 to the transceiver 54. As shown in FIG. 3b, the
the memory and bus controller 80, is used to convert digital
A/V data from the A/V processor 77 or the optional A/V
listing channel 100 includes a repetitive data stream having
memory 75 into analog audio-visual signals which are
data representing today’ s listing 110 interleaved with weekly
routed to the TV 58, the VCR 56 or the other A/V devices65 listings. For example, there is shown today’s data 110,
through the A/V connect module 66. It is also contemplated
followed by Monday’s data 112, and then once again followed by today’s data 110. As seen in FIG. 3b, Tuesday’s
that the A/V encoder 78 may be used to encode other data

FL-Apple0000765

5,594,509
11

12

data 114 is followed once again by today’s data 110, and
ment, when TV is selected, the list button 138 permits the
thereafter by Wednesday’s data 116. Wednesday’s data 116
user to view current and future TV listings up to one week
is followed by today’s data 110, and thereafter by Thursin advance. The categories button 140 permits the viewer to
day’s data 118. Thursday’s data 118 is followed once again select programs/services by category. For example, and as
by today’s data 110, and thereafter by Friday’s data 119. 5 will be described below, the user may desire to watch only
Finally, following today’s data 110, Monday’s data 112 is sports programs, only news programs, or the like. As will be
once again transmitted along the data stream provided by the
disclosed, the categories button 140, and the list button 138
operate in concert to provide listings of selected categories,
service provider 50. For programming purposes, the data
preferably will include rifles of programs, show times, such as sports or news programs, on the TV 58. The mark
special captions, length information, categories, and key lo button 142 permits the user to mark programs for reminders,
words, as well as channel numbers provided from the service
later recall, or switching between programs which have been
provider 50 over the TFF cable 52, and received by the
selected using the mark button 142. A jump button 132 lets
the user switch to the program that was last marked, then to
transceiver 54. The data stream is an interleaved data stream
which repeats on a weekly basis as shown in FIG. 3b. It is
the program that was marked before that, and so on, until all
presently contemplated that there would be at least 100
marked programs have been shown, looping back to the last
different channels of programs and/or services. A weekly 15 marked program again. As shown in FIGS. 5a and 5b, the
program/service listing with information relating to 100 jump button 132 and the mark button 142 are placed side by
channels of programs/services could require over 2 megaside together on the dividing line between the first and
bytes of information. This data stream of programs/services
second control button groups 115 and 134 to emphasize their
listing information (illustrated in FIG. 3b) is received by the interrelation.
transceiver’s main CPU module 62, whereafter the sections20
Also provided with the interactive control button group
that are most relevant to the users are stored in the system
134 is a pointing device consisting of an up arrow button
memory 65. By interleaving the current day’s data with the
145, a down arrow button 146, a left arrow button 148, and
weekly data, the daily data, the CPU module 62 can update
right arrow button 150. For example, if the user is viewing
the system memory 65 periodically and still provides quick a program/service listing through the use of list button 138,
25 a
access to the viewer, without having to store all the received
and desires to scroll up or down within the listing, the up
program/service information, thereby reducing the amount
arrow button 145 and the down arrow button 146 are
of system memory 65 required and the associated hardware
utilized. In addition, a center select button 155 is provided
cost.
for permitting the user to select one of several choices,
Referring once again to FIG. 3a, the back channels 1112 30 represented by words or icons in graphic menus or lists
are used to engage in a variety of transactions, such as
displayed on the TV 58, for example a "bull’ s eye" as shown
ordering products, home banking services and pay-per-view
in FIG. 5a. As will be described, the interactive control
movies, as will be described more fully below. In operation, button group 134 may be used in a variety of applications,
any request by a user to view a pay-per-view movie or order
including selection of pay-per-view channels, home shopa product is transmitted to the service provider 50 over at
ping services, and the like, as well as to display and control
least one back channel 102 through the T/T cable 52. It is 35 programs from other A/V devices connected to the transcontemplated that as channel capacity increases, so will the
ceiver 54.
number of back channels, allowing for broader band twoThe third control button group is the auxiliary control
way communications such as telephone conversations or
button group 160 comprising various A/V control buttons
interaction with on-line networks.
40 for controlling the VCR 56 or the other AFV devices 57
Referring now to FIGS. 4a, 4b, and FIGS. 5a and Sb, two connected to the transceiver 54. As illustrated in FIG. 5a, the
alternative embodiments of the remote control device 60 are
auxiliary control button group 160 includes a record button
shown in further detail. It is expected that the model shown
162, a rewind button 164, a stop arrow button 166, a
in FIGS. 5a and 5b would be more commonly used. As play/pause button 168, a fast forward button 170, and a
illustrated in the figures, both embodiments of the remote 45 menu button 172. In accordance with the teachings of the
control device 60 are comprised of four control button
present invention, the user depresses the record button 1t12
groups 115, 134, 160 and 176. These control button groups
while watching a television program on TV 58, which brings
115, 134, 160 and 176 will first be described briefly here, and up a record panel confirming the current program’ s title and
their usage will be described in further detail below.
length, along with a highlighted select icon and confirmation
The first control button group is the basic control button 50 of which A/V device and type to use for recording. Once the
group 115 which includes standard television control butcenter select button 155 is depressed, the VCR 56 or the
tons such as power on/off button 120, volume control button
other selected A/V device 57 begins to record the program
122 and a mute button 125. Moreover, channel up/down
currently being viewed by the user. Alternatively, if the user
buttons 130 are provided to the embodiment illustrated in
has pressed list button 138, and a program listing is highFIGS. 4a and 4b to increment or decrement the currently55 lighted, pressing the record button 162, and then the center
viewed channel on television 58, as is well known. For the
select button 155 results in the VCR 56 being programmed
embodiment illustrated in FIGS. 5a and 5b, the channel
to record the particular show highlighted in the program
control function is effectuated using the up and down arrow
listing. Pressing the menu button 172 displays icons for all
buttons 145 and 146.
the A/V devices 57 currently connected to the transceiver 54
The second control button group is the interactive control 60 and allows users to select another device with the arrow
buttons. Pressing the center select button 155 displays the
button group 134 which includes an info button 136, a list
output of that device and lets the user control that device.
button 138, a categories button 140, and a mark button 142.
The info button 136 permits the user to obtain multiple
The fourth control button group is the numeric keypad
levels of information on programs while the user is watching
panel 176, which permits the user to directly enter channel
one of the program offerings on the TV 58 (or other A/V 65 numbers to be viewed on the TV 58 by pressing a single,
devices). The list button 138 permits the user to view listings
double or triple digit number, followed by the ok button 178
for programs/services. In the presently preferred embodi(FIG. 5a). For the remote control device 60 shown in in FIG.

FL-Apple0000766

5,594,509
13

14

4b, the user would depress the select button 155 after
be displayed if the up arrow button 145 is depressed.
entering values using the keypad 176. In addition, the
Referring now to FIG. 8, which illustrates a new graphic
numeric keypad 176 may be used by the user to enter a
overlay panel 190 displayed in response to the user having
personal identification number ("PIN"), to be transmitted to
depressed the right arrow button 150 on the remote control
the service provider 511 through the A/V connect module 66 5 device 60 while using the information function. The transfor payment of pay-per-view movies, products and the like.
ceiver 54 provides additional information on the currently
viewed program. As illustrated, an additional page of inforIn addition to the control button groups, both embodiments comprise a transmitter (not shown) for transmitting
mation on the program currently viewed is displayed. For
signals to the audio-visual system. Additionally, the model
example, additional guests on the show "Showbits This
illustrated in FIGS. 5a-Sb is further provided with a talk10 Month" include "Madeline, Harold Black and Harmer".
button 181, a microphone 179, a speaker 182 and a receiver Referring now to FIG. 9, which illustrates another new
(not shown). The talk button 181 when depressed allows the
graphic overlay panel 190 displayed in response to the user
user to speak into the microphone 179, thereby providing
having depressed the right arrow button 150 again. A third
input to the A/V system through voice, and receive voice
page of information is displayed on the screen 180 relating
feedback from the A/V system through the speaker 182. 15 to the currently viewed program "Showbits This Month". It
will be noted that the currently viewed television program
("Showbits This Month" in this example) continues to be
Full Screen Viewing Function
viewed in the main portion of the screen 180, while different
Referring now to FIG. 6, there is shown a representative
graphic overlay panels 190 are superimposed over the lower
full screen audio-visual program displayed on the screen 180 20 third portion of the screen 180.
of the TV 58. By depressing the channel upldown buttons
In the currently preferred embodiment, a user may selec130 on the remote in FIGS. 4a and 4b, (or by depressing the tively move between pages of inl0rmation retating to the
up and down arrow buttons 145 and 146 on the remote
currently viewed program by pressing the right arrow button
control device in FIGS. 5a-Sb), users can switch channels as
150 or the left arrow button 148 on the remote control device
is commonly done with regular TV remotes. As the user 25 li0, or alternatively, the main module 62 will automatically
selects a channel to view using the remote control device 60,
scroll through the various pages of information with a
a graphic overlay panel 185 is displayed in the lower right
predetermined amount of display time for each page of
hand portion of screen 180. As illustrated, the graphic
information.
overlay panel 185 includes the channel number currently
a currently
being viewed 186 along with the channel logo and identifier 3o Referring now to FIG. 10, there is shownat San Frangame
188. Additionally, a marking identifier 189, such as a check viewed baseball on the entitled "Saint Louis
cisco" displayed
screen 180. Assume now the user
mark, is also displayed, as will be described, to indicate that
also depresses the info button 136 on the remote control
the program has been marked through the use of the mark
device 60. As previously described, information relating to
button 142 previously discussed with reference to the remote 35 the currently viewed program is displayed within the graphic
control device of FIGS. 4a, 4b, and FIGS. 5a and Sb.
overlay panel 190. (n the present example, information
relating to the baseball player, "John Smith", is provided
Information Function
within the overlay 190. In the event the down arrow 210 is
Referring now to FIG. 7, there is shown the screen 180 in highlighted, an additional information track is available to
which the information (info) function 136 has been40 the user. This additional information track may be displayed
through the use of down arrow button 146 on the remote
requested by the user. As the user is watching a particular
contro! device 60, and is referred to here as "Tid Bits". As
television program on the TV 58, he may depress the info
shown in FIG. 10, the "Tid Bits" information track gives
button 136 to obtain information in the form of a graphic
additional information relating to the particular subject
overlay panel 190 on the screen 180. In operation, the
transceiver 54 coupled with the main module 62 through the 45 currently being displayed on the screen 180. In the example
of FIG. 10, the first baseman John Smith is currently being
A/V connect module 66 superimposes the graphic overlay
viewed on the screen 180, and information relating to John
panel 190 against the currently displayed video image. This
Smith is provided in the graphic overlay panel 190. It will
process is also used in other functions, such as the full screen
also be noted that in FIG. 10, the down arrow 210 is
viewing function, where the graphic overlay panel 185 is
superimposed on the screen 180. The overlay graphic panel 50 highlighted, indicating that yet another information track is
190 contains an ’T’ logo 192 indicating that the information available.
Still referring to FIG. 10, it will be noted that a down
button 136 has been depressed, and also displays the title of
arrow 210 is shown. By depressing the down arrow button
the currently viewed program 194 ("Showbits This Month"
in this example), a category name 196 ("News" in this 146 on the remote control device 611 once again, an Info Mart
example), and the length of the overall program ("60 rain" track information track as shown in FIG. 11 is displayed by
55
CPU 63 in the graphic overlay panel 190. As illustrated, the
in this example). In addition, the channel number and
Info Mart information track provides the viewer with the
network logo ("8" and "CMM" in this example) are also
opportunity to purchase products, services, programs and the
provided to the user. Also, as illustrated in FIG. 7, the overall
like related to the program which is currently being viewed
length of the currently viewed program is graphically illustrated, by a time bar 20tl showing how much time remains6o (in the present example, the baseball game "Saint Louis at
before the program ends. A left arrow 201, a right arrow 202, San Francisco"). The information track disappears when the
info button 136 is pressed again.
and a down arrow 210 are displayed and highlighted to
indicate that the left arrow button 148, the right arrow button
While the information has been described with the left
150, and the down arrow button 146, of the remote control
and arrow arrow buttons 148 and 1511 causing different pages
device 60 can be depressed to cause additional information 65 of an information track to be displayed, and the up and down
arrow buttons 145 and 146 causing different information
to be displayed. An up arrow 205 is also displayed, but is not
highlighted, to indicate that no additional information will
tracks to be displayed, it is contemplated that the informa-

FL-Apple0000767

5,594,509
15

16

tion displayed may be related to each other in other applidown arrow button 146 on the remote control device 60 to
move the highlighting over to alternate program/service
cation dependent manners. It is further contemplated that
offerings. For example, if the user presses the down arrow
tiffs information function can also be used with audio-visual
button 146, as shown in FIG. 13, the NTV listing is then
programs or services other than TV shows, in particular
when the menu button 172 is used to display the output of 5 highiighted. Similarly, as shown in FIG. 14, by continuing to
the VCR 56, the CD 70, a hard disk, a telephone or another depress the down arrow button 146, the highlighting continues to move upward towards the lower channel numbers
other A/V device 57 connected to the transceiver 54.
and once the highlighting reaches Channel 9, the entire
The data comprising the information of the various inforscrolls to display consecutively
mation tracks shown in FIGS. 7-11 are downloaded using program/service listing previously not displayed. At the
channel numbers
10 lowertime, the higher channel numbers previously displayed
the method illustrated in FIG. 3b, i.e. the interleaved data same
stream transmitted from the service provider 50, to the
at the bottom part of the screen 180 will consecutively
transceiver 54. The data are stored in the system memory 65
disappear. As illustrated in FIG. 14, Channels 7 and 8 are
and updated as new data are received over the digital
now displayed, and Channels 19 and 20 are no longer
program listing channel 100. Alternatively, data displayed in
displayed to the user. If the up arrow button 145 is now
the information tracks such as the exemplary Tid Bits or15 pressed instead, the highlighting will move down towards
Info-Mart information tracks may be transmitted by the
the higher channel numbers as illustrated in FIG. 14. Notaservice provider 50 to the transceiver 54 using the vertical
bly, movement of the program/service listing and the highlighting in the present invention are specifically calcnlated to
blanking intervals, which are inherently part of the analog
conform with an average person’s expectations in using a
TV charmel 103 used to display the currently viewed program on the TV 58. In another embodiment, data comprising 20 television remote control. While the list function has been
described with the down arrow button 146 causing highthe information tracks may be transmitted and downloaded
lighting to move towards the lower channel numbers, and
through the use of separate info-track channels comprising
the up arrow button 145 causing highlighting to move
additional channels in the electronic spectrum illustrated in
towards the higher channel numbers, it will be appreciated
FIG. 3a.
25 that the correspondence between the arrow buttons 145 and
146 and the highlighting movement may be implemented in
List Function
other ways as best suits the anticipated user of the interface
Referring now to FIG. 12, the list function of the present system.
invention will be described. While viewing a television
Skipping now to FIG. 16, a further feature of the list
program displayed on the TV 58, the user may depress the 30 function of the present invention is illustrated. In the event
the user depresses the right arrow button 150, the CPU 63
list button 138 on the remote control device 60 to obtain a
displays the next hourly time slot listing on the screen 180.
program/service listing for the current date and time during
In the example of FIG. 16, the time slot for 9:00-10:00 p.m.
which the user is watching television. As shown in FIG. 12,
is illustrated. It will be appreciated that, had the user
upon depressing the list button 138, a program/service
depressed the left arrow button 148
times,
listing 220 is displayed for the current date and time at which 35 program/service listing for 6:00-7:00three would then the
p.m.
have been
the viewer has depressed the list button 138. As illustrated,
displayed. Additionally, as illustrated in FIG. 17, the conprograms/services currently available for viewing and intertinued depression of the right arrow button 150 for a
action are identified by their channel numbers, channel name
time
present embodiment
identifiers (for example, Channel 10, NTV), and by ti0es of predetermined2amount ofresults(in the display of program/
for more than seconds)
in the
the programs/services. In the event that the program/service 40 service listings for subsequent days. In this case, the conlisting includes two programs for a given time interval, then
tinued depression of the right arrow button 150 has resulted
the names of both programs are displayed. For example, in
in a scanning of the program/service listings through to
HG. 12, Channel 10 is NTV which beginning at 8:00 p.m. Saturday, October 17. It will be noted that in the currently
is showing a program entitled "Rock Today", and beginning
preferred embodiment, a scanning of subsequent days
at 8:10 p.m. is showing a program called "Party Time". In results in the display of the program/service listing from the
45
addition, as shown in FIG. 12, a picture-in-picture window
beginning of prime time, namely, 6:00 p.m. (or 7:00 p.m., at
250 continues to display the currently viewed program
the service provider’s option). It will also be noted that the
which the user was last viewing. Accordingly, the user may
info function can be used in conjunction with the list
continue to view the currently selected program, and the
function to display program/service information about the
current program/service listing simultaneously. By pressing 50 currently highlighted program/service. It will further be
the select button 155 in the current time slot, the user can
noted that the listing display will disappear if the user
view the currently highlighted program in full screen, as
depresses the list button 138 once again. It is also contemlong as that program is playing live at the current time. For
plated that this list function can be used with audio-visual
example, in the case of FIG. 13, the pressing of the select programs or services other than TV programs, in particular
button 155 would display NTV’s "Rock Today" program in 55 when the menu button 172 is used to display the output of
full screen. It will be noted that although the current impleother A/V devices 57. For example, if the VCR 56 is
mentation of the list function displays a list of programs/
displayed on the TV 58, pressing the list button 138 on the
services on single lines for one hour time slots, the invenremote control device 60 would display a listing of all
tion’s user interface also supports other styles of display, for
programs recorded by the user on the VCR 56, highlighting
example, a grid showing programs on two lines, each line 60 the program now displayed from the VCR 56. Similarly, the
representing a two hour time slot.
list function would display CD ROM or telephone listings
It will also be noted that the currently selected program
when these A/V devices are selected through the menu
being displayed in the picture-in-picture window 250 is
function.
initially highlighted in the program/service listing to assist
Record Function
the user. In the example of FIG. 12, Channel 11, PBS, a show
65
entitled "Say Nay Billy" is highlighted. Moreover, channels
Referring now to FIG. 18, which illustrates a program/
may be selected by pressing the up arrow button 145 or the
service listing displayed in response to the user having

FL-Apple0000768

5,594,509
17

18

continuously depressed the right arrow button 150 on the
audio-visual programs other than TV shows, allowing a user
remote control device 60. The exemplary program/service
to mark and subsequently jump between, for example, a
listing displayed is for Saturday, October 17 beginning at
VCR program, a CD ROM program, an on-line service or an
prime time (6:130 p.m.). The up and down arrow buttons 145 answering machine message, in addition to the TV programs
previously marked while viewing television channels.
and 146 on the remote control 60 are then used to highlight
the program "Specials Tonight". Assume now the user
desires to record the exemplary program "Specials Tonight".
Categories l~unction
By pressing the record button 162 on the remote control
device 60, a record panel 259 is displayed by the CPU 63 on
Referring now to FIGS. 22-32, the categories function of
the screen 180, confirming the title and length of the 10 the present invention will be described. To initiate the
program to be recorded, along with a highlighted select
categories function of the present invention, the categories
("bull’s eye") icon as shown in FIG. 18. It is contemplated button 140 is depressed on the remote control device 60. As
that other functions can be included in the record panel 259,
illustrated in FIG. 22, the CPU 63 displays a categories
such as a function for selecting different VCRs or other A/V
graphic overlay panel 300 which is superimposed over the
recording devices, a function for selecting different tapes or 15 currently viewed program on screen 180. A plurality of
reading media, and/or a function for selecting how often the
categories are identified by representative icons. The first
recording is to take place (once, weekly, daily). Once the
category which is selected by default is the "all" category
select button 155 is pressed, the program listing is redis302, and is highlighted around the border. The "all" category
played without the record panel 259. A record icon 260 is
302 refers to all currently available programs/services being
displayed on the screen 180, as shown in FIG. 19. The 20 transmitted by the service provider 50. By depressing the
"Specials Tonight" program which will begin on Saturday,
right arrow button 150 or left arrow button 148 on the
October 17, will then be recorded automatically. In the event
remote control device 60, other categories may be highthe viewer changes his mind and does not desire to record
lighted. For example, by depressing the right arrow button
the program "Specials Tonight", by pressing the record
150, the next category "favorites", as indicated by a star icon
button 162 again and then pressing the select button 155 25 305, is highlighted (See FIG. 23). Similarly, by depressing
once more, the program listing will be redisplayed without
the right arrow button 150 again, a movies icon 310 is
the record icon 260 and no recording will take place.
highlighted. By depressing the right arrow button 150
another time, a sports icon 315 is highlighted (see FIG. 24),
Mark and Jump Functions
and so on. Additional categories may include a news icon
3o 312 and a music video icon 313. However, it will be
Skipping now to FIG. 15, the mark function of the present
appreciated that many other categories can be displayed. By
invention and its operation will be described. Shown in FIG.
15 is a program listing reflecting Channe! 10 (NTV) having repeatedly pressing the right arrow button 150, the user can
make category icons appear, scrolling from right to left
been selected using the up and down arrow buttons 145 and
146 on the remote control device 60. Assume now the 35 against the category overlay panel 300.
In FIG. 24, the sports category has been highlighted by the
current time is 8:08:30 p.m. In accordance with the teachuser as indicated by the highlighted sports icon 315. To
ings of the present invention as described earlier, the proselect a category that is currently highlighted, the user can
gram "Rock Today" would be highlighted on the screen 180.
press the select button 155 and the category overlay panel
The program "Party Time" may be highlighted by the viewer
300 disappears while the currently viewed program disby pressing the right arrow button 150. The mark function
played on the screen 180 is replaced by the nearest available
may then be initiated by pressing the mark button 142 on the
remote control device 160. As illustrated in FIG. 15, a mark program in the category just selected. The selection of the
sports category results in the display of only sports related
identifier 252 is displayed adjacent to the selected program
(in the present example, "Party Time"). Skipping again to programs on the TV 58. FIG. 25 illustrates an exemplary
FIG. 20, a reminder box 254 is displayed on the screen 180 baseball game which is displayed after the user presses the
45
setect button 155. As illustrated, the sports icon 330 is also
at the time the program "Party Time" begins (in the present
example, 8:10 p.m.), regardless of what channel is currently displayed in the graphic overlay panel 185 along with the
channel identifier 188.
being viewed then by the user on TV 58. It should also be
noted that programs playing live in a current time slot czn be
Assume now after the nearest sports program has been
marked while displayed in full screen by depressing the 50 displayed on the screen 180, the user desires to view
mark button 142. This causes a mark identifier 189 to be additional sports category programs on the TV 58. By
displayed in the graphic overlay panel 185, as shown in
depressing the up channel button 130 or the up arrow button
FIGS. 6 and 21.
145 on the remote control device 60, the transceiver 54
Referring now to FIGS. 20 and 21, depressing the jump displays the next higher number channel which is currendy
button 132 while the reminder PIP window 254 is displayed showing a sports program. In the example of FIG. 26, the
55
sports program relates to a windsurfing event. As shown, the
switches the program displayed in the main viewing area of
sports icon 330 continues to be displayed along with the
the screen 180 to the program which the user has previously
marked (see FIG. 21). If more than one program has been numerical channel indicator and the channel identifier 188
(in the example of FIG. 26 "Sportstime") in the graphic
marked, by repetitively depressing the jump button 132,
users can jump to the other previously marked programs 60 overlay panel 185.
displaying them in full screen in the main display area of the
Referring now to FIG. 27, if the user has selected the
screen 180 one at a time. It will be appreciated that in an
category of sports (icon 315), and depresses the list button
alternate embodiment, the picture-in-picture window 254
138, a program/service listing 360 of currently available
illustrated in FIG. 20 may be preceded by a reminder icon sports programs is displayed on screen 180. It will also be
displayed automaticalIy in the PIP window 254 a few 65 noted that the currently viewed sports program will continue
minutes prior to the actual start of the program. It will aiso
to be displayed in a picture-in-picture window 365. As
be noted that the mark and jump functions may be used with
previously discussed, with respect to the list function, pro-

FL-Apple0000769

5,594,509
19

2O

grams which are shown during half hour intervals (See FIG.
listing 363 of only programs that have been previously
27 "Windsurfing" and "Basketball") are identified and may
marked. Other favorite sub-categories include a record subbe individually selected using the remote control device 60.
category of all programs selected for recording, and personReferring now to FIG. 28, which illustrates a program alized favorites sub-categories such as "Morn" 321 and
listing with the sports icon 330 in the category overlay panel 5 "Joe" 322. It is contemplated that new preferences can be
highlighted, displayed in response to the user having pressed
defined by individual users by selecting a special favorites
the list button 138, and then presses the categories button
sub-category, represented by a "set-up" icon 323, then
140. Assume now the user desires to break down the main
marking preferred items from a list of programs and/or
sports category into finer sub-categories such as baseball,
categories, as well as specifying an icon and/or a name for
football, basketball, hockey, etc. By depressing the down
this new preference set. By first defining and subsequently
arrow button 146 on the remote control device 60, the user
selecting, for example, the "Morn" sub-category icon 321, a
causes the CPU 63 to select a default sport sub-category
user can view only programs that match the interests defined
(baseball in this example) and display a plurality of sports
earlier by the mother of the household.
sub-category icons against the categories overlay panel 300
It is also contemplated that this category function can be
as illustrated in FIG. 29. The default sports sub-category
icon 315 is also highlighted, and the program listing 361 15 used in combination with A/V programs or services other
than TV programs, in particular when the menu button 172
comprises only programs of that sports sub-category, i.e.
is used to select the VCR 56, or one of the other A/V devices
baseball in this example. By pressing the right arrow button
57. For example, pressing the categories button 140 while
150 on the remote control device 60, the user can highlight
the football sub-category icon 317, then press the select 20 viewing a VCR listing (FIG. 36, below) would enable users
to find all previously recorded comedy programs. Similarly,
button 155 to cause only football programs to be listed.
the categories function could enable users to find phone
Alternatively, the user can press the up arrow button 145 to
listings in the "gardening" category, according to telephone
display the main categories again, with the sports icon 315
d~rectory information accessed from the provider 50 via the
highlighted as illustrated in FIG. 28. The user can also press
T/T cable 52.
the categories button 140 once more to make the categories
25 While the present invention has been described with the
overlay panel 300 disappear.
Referring now to FIG. 30, which illustrates a program categories function having different effects on the displayed
image when used during full screen display and when used
listing 320 with the favorite icon 305 in the category overlay
in conjunction with list function, it will be appreciated that
panel highlighted, displayed in response to the use having
the present invention also may be practiced with the catselected the "favorites" category using the left arrow button 30 egories function having consistent effect on the displayed
148 on the remote control device 60. The program listing
320 comprises favorite programs for the current time slot image in either situation, as desired.
including frequently viewed programs, marked programs,
Pix Function
and programs to be recorded, as well as programs that match
specified user preferences. Assume now that the user desires 35 Referring now to FIGS. 33 and 34, the present invention’ s
to resolve the "favorites" category into finer sub-categories.
picture-in-picture (or pix function) will now be described.
By pressing the down arrow button 146 on the remote
The depression of the pix button 144 on the remote control
control device 60, the user causes the CPU 63 to display a
device 60 results in the display of a pix dispIay 381 as
plurality of favorites sub-category icons against the categoillustrated in FIG. 33. As shown in the present embodiment,
ries overlay panel 300, as illustrated in FIG. 31. The CPU 63 12 picture-in-picture video windows are arranged within the
40
highlights the "frequent" programs sub-category icon 318
screen 180, along the perimeter of the pix display 381, with
and displays a program listing 362 of programs most frea central picture-in-picture window 375 also displayed.
quently watched by the user as determined by the CPU 113
Depressing the pix button 144 on the remote control device
from previous monitoring of the user’s viewing habits. In the
60 results in the simultaneous display of multiple programs
normal course of operation, the CPU 63 monitors programs 45 currently transmitted over the TFF cable 52, the VCR 56, the
which are most frequently viewed and compiles a priodtized
CD ROM 70 and/or the other A/V devices 57. Moreover, one
list of those frequent programs for each time slot, which list
of the picture-in-picture windows displaying the last prois stored in the rewritable non-volatile part of the system
gram viewed in full screen, such as, for example, picturerandom access memory (such as SRAM) 65.
in-picture window 380, is highlighted. As shown in FIG. 33,
In the presently preferred embodiment, if a viewer views50 the highlighted smaller screen is also displayed in larger size
a television program for more than 10 minutes, the current in the central picture-in-picture window 375, with the audio
program and channel identifiers are compared to frequent
track of the program also audible to the user. Accordingly, a
programs already stored in system memory 65 for that
user may watch a motion picture or television program,
particular time slot, and an updated list of frequent programs
including the audio track, in window 375, while viewing
is created by CPU 63 and stored in the system memory 65. 55 multiple other programs simultaneously. In the presently
In the event there are no frequent programs stored in the
preferred embodiment, the A/V connect module 66 samples
system memory 65 for any particular time slot, then a
(or "grabs") a video frame from each of the 12 programs
general list of most frequently watched programs is retrieved
every pre-determined (N) number of frames. Accordingly, a
from the system memory 65. It is anticipated that the
user viewing the pix display 381 perceives each of the 12
majority of frequent programs will be stored in time slots60 small picture-in-picture windows as sequential representacorresponding to prime time, or during daytime television
tive images of every N frames per window. However, the
hours.
picture-in-picture window 375 represents a full motion
display of the currently selected program, provided that a
By pressing the right arrow button 150, the user can move
second TV tuner 67 is included within the A/V connect
the highlighting from the "frequent" programs sub-category
icon 318 to the "marked programs" sub-category icon 319, 65 module 66.
as shown on FIG. 32. Accordingly, the CPU 63 highlights
Additionally, in the preferred embodiment of the present
the "marked programs" icon 319 and displays a program
invention, it is contemplated that the user is able to view the

FL-Apple0000770

5,59~,509
21

22

pix display 381 by category by pressing the categories
viewed A/V source icon is automatically highlighted, typibutton 140 and selecting a particular category, such as
cally the TV icon 421. By depressing the right arrow button
favorites or movies. For example, the pix display 381 150, or left arrow button 148 on the remote control 60, other
illustrated in FIG. 33 is comprised of only favorite pro- A/V source icons may be highlighted and previewed. For
grams, as the user had previously selected the favorites 5 example, by depressing the right arrow button 150, the next
category. Had the user selected the "marked" subcategory,
A! V source icon, VCR 422, is highlighted, as shown in FIG.
only marked programs would be displayed on the pix
36. Simultaneously, CPU 63 displays an audio-visual prodisplay 381, allowing users to easily add or delete programs
gram derived from the highlighted source icon, in this case
in the pix display 381 by using the mark button 142. If no a videotape playing in the VCR 56 onto the screen 180, as
categories have been selected (which is equivalent to select- 10 illustrated in FIG. 36. To select and control any A/V source
ing the "all programs" category), then the pix display 381
whose icon is currendy highlighted, in this case the VCR 56,
displays the 12 programs nearest to the last selected prothe user can press the select button 155, whereafter the
gram. Similarly, pressing the info button 136 would display graphic overlay panel 420 disappears while the currently
the graphic overlay panel 190 shown in FIG. 7 for program viewed program sourced from the VCR 56 is displayed in
information.
15 full screen. Thereafter, the user can press the play button 168
Referring now to FIG. 34, which illustrates the selection on the remote control device li0 to cause the CPU 63 to
of a different pix program to be viewed in the central instruct the VCR 56 through the A/V connect module 66 or
picture-in-picture window 375. By depressing the left arrow the IR control 82 to playback a previously recorded program
on the VCR 56. Similarly, the user can press the stop arrow
button 148 on the remote control device 60, the highlighting
20
380 is moved one window to the left. In the example of FIG. button 166, the rewind button 164, and/or the fast-forward
button 170 on the remote control device 60 to further control
34, a "Discover It" program is now highlighted and shown
simultaneously in the central picture-in-picture window 375. the program sourced from VCR 56. It is contemplated that
the CD ROM module 70 and some of the other A/V devices
Similarly, through the use of the left arrow button 148 and
57 including other CDs, VCRs, laser disc players, hard
the right arrow button 150 along with the up arrow button
145 and the down arrow button 146 on the remote control 25 disks, telephones, answering machines, as well as transmitdevice 6D, each of the pix programs in the pix display 381 ting devices connected to voice mail, electronic mall and
various on-line setwices can be similarly selected and conmay be viewed within the central picture-in-picture window
trolled through the use of the menu function, as illustrated in
375, as desired. As in the list function, pressing the select
FIG. 36.
button 155 displays the currently highlighted program in full
screen. Alternatively, pressing the pix button 144 a second 3O
Other Functions
time causes the pix display 381 to disappear and the last
highlighted program appears in full screen.
Other functions may be realized with the present invenReferring to FIG. 35, selection of pay-per-view movie tion, including numeric data entry. As shown in FIGS. 41
currently shown in one of the picture-in-picture window 380 and 49, by pressing one or more buttons on the numeric
of the pix display 381 for viewing is illustrated. As described35 keypad 176 of the remote control device 60, a user can
confirm a financial transaction by entering his or her perearlier, by depressing the left arrow button 148 on the remote
control device 60, the PIP window 38[} becomes highlighted.
sonal identification number (PIN) in a confirmation display
In the example of FIG. 32, the pay-per-view channel shows field 425. The numeric keypad 176 may also be used for
a preview trailer for the motion picture "Jeff’s World", dialing TV channel numbers, telephone numbers and other
which is now displayed in the center PIP window 375 and40 program source identifiers, as desired.
the perimeter PIP window 38D simultaneously. To select the
Yet another function which may be optionally realized by
pay-per-view movie, the select button 155 is depressed on
the present invention is a talk function, whereby a user can
the remote control device 60. It is contemplated that pressing
depress the talk button 181 on the remote control device 60,
a single, double or triple digit number on the numeric
45 and thereafter issue spoken commands into the microphone
keypad 176, followed by the ok button 178 of the remote 179 located on the remote control device 60 in lieu of or in
control 60, will display the corresponding pay-per-view
addition to pressing buttons on the remote control device 60.
channel, along with the 12 nearest channels. It is also
Through the use of voice recognition hardware and software
contemplated that this pix function can be used with audiooperating in conjunction with the CPU 63, the present
visual programs and services other than TV shows, in
invention can be made to interpret the spoken commands
particular when the menu button 172 is used to display the 50 requested by the user, and invoke the corresponding funcoutput of the VCR 56, the CD 70, a hard disk or one of the tions. For example, a user may speak the word "sports"
other A/V devices 57 connected to the A/V transceiver 54.
while pressing talk button 181, which is equivalent to
invoking the category function and highlighting the sports
Menu Function
55 category, as if the user had pressed the corresponding
sequence
earlier. It
Referring now to FIG. 36, the menu function will now be templated of buttons, as described device 60 is further conthat the remote control
in conjunction
described. The menu function permits users to switch the
with the TV 58 and a camera can be used as a video
television display to any A/V source connected to the
transceiver 54, such as the VCR 56, the CD ROM module telephone for video conferencing and the like, wherein
70, or one of the other A/V devices 57, and lets users control 60 having engaged a telephone line (via the menu function) the
user speaks directly into the remote control device 60 as if
those A/V sources directly. As shown in FIG. 36, upon
it were a standard telephone.
pressing the menu button 172 on the remote control device
60, the CPU 63 displays on the screen 180 a graphic overlay
Pay-Per-View Interface
panel 420 which is superimposed over the currently viewed
audio-visual program. As further shown in FIG. 36, a65
The present invention includes one possible pay-per-view
plurality of audio-visual sources connected to the transceiver
(PPV) interface which will now be described. Assume that
the user has selected the TV source icon 421 using the menu
54 are identified by representative icons. The currently

FL-Apple0000771

5,594,509
23

24

function displayed in the graphic overlay panel 420, and
service results in a continuous full-motion video display of
further selects a pay-per-view program using either the list
various paid-for commercials or advertising messages. As
or the pix function. This selection results in an initial title
shown in the representative screen of FIG. 44, a TV Shop
screen to be displayed, as illustrated in FIG. 37, which may
icon is displayed along with a blinking select icon (or right
be static or animated. As shown in FIG. 38, upon the 5 arrow icon) in a graphic overlay panel 400, at the start and
selection of the PPV program and subsequent to the display
end of each advertisement, prompting users to press the
of the title screen of FIG. 37, a preview icon 382, a credit select button 155 (or fight arrow 150).
icon 384, an info icon 386, and a ticket icon 388 are
Referring now to FIG. 45, by pressing the select button
displayed, along with a preview trailer which is continu155 (or the right arrow button 150), the user can view
ously looping and is displayed in a half-screen picture-in- 1o additional information relating to the advertisements shown
picture window 390. It will be noted that in FIG. 38, the in the full motion video section of the TV Shop. In the
preview icon 382 has been highlighted by default, thereby
example of FIG. 45, it will be noted that a listing of the
corresponding to the trailer being shown in the window 390.
advertisers is provided, in which the advertisers whose
Referring now to FIG. 39, by depressing the right arrow
commercial was playing last is highlighted. Through the use
button 1S0 of the remote control device 60, the credit icon 15 of the up arrow button 145 or the down arrow button 146,
384 may be highlighted. The credits for the pay-per-view
the user may scroll through the advertisers, sequentially
movie are displayed on the screen 180 within the window
highlighting each of their advertising identifiers, along with
390. By depressing the right arrow button 150 once again,
a select icon 403 prompting a user to select that advertiser.
the info icon 386 is highlighted. As shown in FIG. 40, Moreover, as shown in FIG. 45, an "exit" icon 402, a
information related to the pay-per-view movie is then pro-20 "stores" icon 404, a "specials" icon 406, an info icon (or
vided to the user, such as the title, the production company,
"compare" icon) 408, an "order" icon 409 and other icons
a brief description of the movie, the rating and the total
may be presented to the viewer. As illustrated in FIG. 46 the
length. If the user desires to watch the pay-per-view movie,
selection of the "Attic at Casey’s" advertiser with the select
he/she depresses the right arrow button 150 once again to
button 155 (or the right arrow button 150) results in a display
highlight the ticket icon 388, as illustrated in FIG. 41. 25 of the various specials available from that advertiser. In FIG.
46, there is shown cookware highlighted in conjunction with
As shown in FIG. 41, upon the selection of the highlighted
a still image of the cookware for sale. By depressing the
"ticket" icon 388, the CPU 63 generates and displays on the
down arrow button 146 on remote control device 60, the
screen 180 a request for the user to input a "movie pass"
highlighted section on the display can be moved down such
password using the numeric keypad 176 of the remote
control device ~i0. After entering the personal identification 30 that the mini-espresso (see FIG. 47) is selected. Simultaneously with the highlighting of the mini-espresso selection,
number (PIN) using the numeric keypad 176, causing a
a still image of the mini-espresso machine for sale is shown.
confirmation sound to be played and confirmation symbols
such as asterisks to fill in a blank confirmation display 425,
Referring to FIG. 48, to obtain further information regardthe user depresses the ok button 178 or the select button 155
ing the mini-espresso machine illustrated in FIG. 47, the
on the remote control device 60 to order and view the 35 user depresses the select button 155 (or the right arrow
pay-per-view movie. As shown in FIG. 42, transceiver 54 button 150) on the remote control device 60 to highlight the
then displays the ordered program if it is available, or
info icon 408. As illustrated in FIG. 48, information relating
displays a message informing the viewer that the pay-perto the particular product (i.e. espresso machine) is shown on
view movie will begin screening within X number of the screen 180. To order the mini-espresso machine illusminutes. In operation, depressing the ok button 178 or the 40 trated in FIGS. 47 and 48, the user depresses the select
select button 155 subsequent to the entering of the movie
button 155 (or the right arrow button 150) to highlight the
order icon 409. As illustrated in FIG. 49, a personal idenpass personal identification number (PIN) results in the CPU
63 providing a signal to the cable service provider 50, over
tification order number window 420 is displayed in which
the T/I" cable 52 using one of the back channels 102 (See 45 identification number (PIN). After176) inputs a personal
the user (using the numeric keypad entering the personal
FIG. 3a). It will further be appreciated that during the
waiting period prior to the transmission of the pay-per-view
identification number (PIN), the ok button 178 or select
movie, additional trailers and other previews may be disbutton 155 is depressed to order the product. In the presently
played to the user on the screen 180. It will also be noted that
preferred embodiment, once the personal identification numif the user incorrectly enters his/her movie pass PIN number,
ber (PIN) of the user is entered into the system of the present
pressing the clear button 177 on the numeric keypad 176 50 invention and the ok button 178 or the select button 155 is
allows them to clear the confirmation display 425 and start
depressed, a signal is provided by the CPU 63 to the service
over again. It will also be appreciated that other pay-perprovider 50, using one of the back channels 102 (See FIG.
view interfaces may be designed and implemented using the
3)a. As illustrated in FIG. 50, a confirmation of the order,
teachings of the present invention on general purpose audioalong with a delivery time is displayed to the user. It will
55 also be appreciated that other home shopping interface
visual user interface.
variants may be designed and implemented using the teachHome Shopping Interface
ings of the present invention.
The present invention includes one possible home shopping interface, which will now be described. In FIG. 43, a 6o
channel identified as "SHP" or "TV Shop" represents a
dedicated home shopping service within the electronic spectrum illustrated in FIG. 3a. It will be noted that although in
its present embodiment, TV Shop is presented as a television
charmel, it could also be presented as an on-line service 65
through a transmitting device cormeeted to the T/T cable 52.
In the current embodiment, the selection of the TV Shop

Summary

The present invention as described provides methods and
apparatus for a unique audio-visual interface for television
and other A/V programs and services. The present invention’s functions and features provide a user interface which
helps users find and view programs of interest, access related
information, control various audio-visual devices and order
products or services from their own homes. While the

FL-Apple0000772

5,594,509
25

26

present invention has been described with references to the
wherein a plurality of direction cues arc displayed on said
A/V display, wherein one or more of said plurality of
various figures, it will be appreciated that the figures are for
illustration only, and do not limit the spirit and scope of the
direction cues are available direction cues, said availinvention. For example, although the figures have, by necesable direction cues being highlighted on said A/V
sity, used example screens having certain attributes, icons, 5
display;
criteria or functions, it will be appreciated that the invention
wherein said control means includes direction arrow
is not limited by the specific examples provided. Rather, the
buttons, said user activating one of said direction arrow
present invention has application in any television and/or
buttons corresponding to the direction identified by one
audio-visual display system. In addition, it will be noted that
of said available direction cues resulting in said transthe present invention may be realized using a variety of 10
ceiver causing said A/V display to display an additional
computer hardware and computer software, and is not limlevel of information related to said currently viewed
ited to any particular hardware or software systems.
program.
Moreover, it will be appreciated that many of the func4. The audio-visual system as defined by claim 3 wherein
tions and features of the present invention, such as graphic
activation of a left and a right direction button on said
overlay panels, icons, and selection methods, are not static 15 control means results in movement between pages of inferevents and are perceived by the user as having motion, color,
mation relating to said currently viewed program.
sound and/or animation. Specifically, it wilt be appreciated
5. The audio-visual system as defined by claim 3 wherein
that the present audio-visual user interface also includes an
activation of an up and a down button on said control means
audio interface using a plurality of sound icons and backresults in movement between additional levels of informagrounds, which compliment the visual interface and are
tion, wherein said additional levels of information provide
played almost every time a function of the present invention ~0 data on a plurality of currently viewed subjects in said
is used, and which cannot be illustrated graphically here, but
currently viewed program.
are part of the present invention. Due to the limitations of a
6. The audio-visual system as defined by claim 3 wherein
said additional levels of information are selectively diswritten specification, the reader is referred to a videotape
entitled "EZTV" submitted by the Applicants concurrent
played in an overlay region that comprises a horizontal bar
with the filing of the application on which this patent is 25 on a lower portion of said A!V display.
7. An interactive user interface and audio-visual (A/V)
based.
system, comprising:
We claim:
a transceiver coupled to an audio-visual (A/V) display for
1. An interactive user interface and audio-visual (A/V)
receiving audio-visual signals in an electronic signal
system comprising:
spectrum from a signal source, said transceiver includa transceiver coupled to art audio-visual (A/V) display for
ing interface generation means for causing said A/V
receiving audio-visual signals in an electronic signal
display to display an interface, said interface including
spectrum from a signal source, said transceiver includselectively displayed multiple levels of information
ing interface generation means for causing said A/V
related to an audio-visual program viewed on said A/V
display to display an interface, said interface including
display;
selectively displayed multiple levels of information 35 control means in communication with said transceiver for
related to an audio-visual program viewed on said A/V
permitting a user viewing said A/V display to selecdisplay, wherein said multiple levels of information
tively display said multiple levels Of information on
include information received by said transceiver over a
said A/V display;
plurality of information tracks;
40 wherein said transceiver further includes a switcher and at
control means in communication with said transceiver for
least one tuner/demodulator coupled to said signal
permitting a user viewing said A!V display to selecsource; said switcher coupling said at least one tuner/
tively display said multiple levels of information on
demodulator to said A/V display and a storage means,
said A/V display;
for switching digital and analog A/V data between said
whcrcin said interface is displayed in an overlay region 45
signal source, said A/V display and said storage means,
that includes at least one direction cue to identify at
said transceiver further including
least one direction on said A/V display corresponding
at least one processor unit,
to at least one additional level of information.
at least one system memory, unit, and
2. The audio-visual system as defined by claim 1 wherein
a bus controller cooperatively coupled together for receiva plurality of direction cues are displayed on said A/V 50
ing input control and data signals and in response
display, wherein one or more of said plurality of direction
generating output control and data signals for said
cues are available direction cues said available direction
signal source, said A/V display, and said storage means.
cues being highlighted on said A/V display.
8. The audio-visual system as defined by claim 7 wherein
3. An interactive user interface an audio-visuai (A/V)
said electronic signal spectrum includes at least one digital
system comprising:
55 program listing channel for downloading program listing
a transceiver coupled to an audio-visual (A/V) display for
data from said signal source to said system memory.
receiving audio-visual signals in an electronic signal
9. The audio-visual system as defined by claim 8 wherein
spectrum from a signal source, Said transceiver includsaid program listing data is downloaded to said system
ing interface generation means for causing said A/V
memory such that said data is interleaved, and wherein said
display to display an interface, said interface including 60 program listing data comprises data for a current date
selectively displayed multiple levels of information
followed by data for another day of the week.
related to a currently viewed program viewed on said
10. The audio-visual system as defined by claim 8
A/V display:
wherein said program listing data includes program title
control means in cornmuhication with said transceiver for
data.
permitting a user viewing said A/V display to selec- 65 11. The audio-visual system as defined by claim 8 wherein
said program listing data includes show time data and
tively display said multiple levels of information on
said A/V display;
channel number data.

FL-Apple0000773

5,594,509
27

28

12. The audio-visual system as defined by claim 8
wherein said program listing data includes length data and
caption data for audio-visual programs.
13. The audio-visual system as defined by claim 7
wherein said electronic signal spectrum comprises analog 5
and digital MV channels.
14. The audio-visual system as defined by claim 7
wherein said control means includes a hand held remote
control device.
15. An interactive user interface and audio-visual (A/V) 10
system, comprising:

a transceiver coupled to an audio-visual (A/V) display for
receiving audio-visual signals in an electronic signal
spectrum from a signal source, said transceiver including interface generation means for causing said A/V
display to display art interface, said interface including
selectively displayed multiple levels of information
related to an audio-visual program viewed on said A/V
display, wherein said multiple levels of information
include information received by said transceiver over a
pIurality of information tracks;
control means in communication with said transceiver for
a transceiver coupled to an audio-visual (A/V) display for
permitting a user viewing said A/V display to selecreceiving audio-visual signals in an electxonie signal
tively display said multiple levels of information on
spectrum from a signal source, said transceiver includsaid A/V display;
ing interface generation means for causing said MV 15 wherein said interface generation means is further
display to display an interface, said interface including
includes listing means for causing said A/V display to
selectively displayed multiple levels of information
selectively display a program listing that contains Iistrelated to an audio-visual program viewed on said A/V
ing information related to A/V programs viewable on
display;
said A/V display;
control means in communication with said transceiver for20 wherein said interface generation means includes marking
permitting a user viewing said A/V display to selecmeans for marking programs on said program listing.
tively display said multiple levels of information on
24. The audio-visual system as defined by claim 23
said A/V display;
wherein said control means includes a remote control device
operated by said user having direction arrow buttons, includwherein said interface generation means further includes
listing means for causing said A/V display to selec- 25 ing up, down, left and right buttons.
25. The audio-visual system as defined by claim 22
tively display a program listing that contains listing
wherein said marking means is activated by said user
information related to A/V programs viewable on said
choosing a selected program from said program listing using
A/V display;
said up and down buttons and then activating a mark button
wherein said program listing includes a first picture-ina-picture (PIP) window in which the currently viewed30 on said remote control device.
211. An interactive User interface and audio-visual (A/V)
program viewed by said user prior to said activation of
system, comprising:
said listing means is displayed on said A/V display.
a transceiver coupled to an audio-visual (A/V) display for
16. The audio-visual system as defined by claim 15
receiving audio-visual signals in an electronic signal
wherein said program listing includes a program listing icon
spectrum from a signal source, said transceiver includto identify to said user that said listing means has been 35
ing interface generation means for causing said A/V
activated.
display to display an interface, said interface including
17. The audio-visual system as defined by claim 15
selectively displayed multiple levels of information
wherein a listing for the currently viewed program that is
related to an audio-visual program viewed on said A/V
displayed in said first picture-in-picture (PIP) window is
display;
highlighted when said program listing is activated.
40
18. The audio-visual system as defined by claim 17
control means in communication with said transceiver for
wherein said control means includes a remote control device
permitting a user viewing said A/V display to selecoperated by said user having direction arrow buttons, includtively display said multiple levels of information on
ing up, down, left and right buttons.
said A/V display
19. The audio-visual system as defined by claim 18 45 wherein said interface generation means further includes
wherein said program listing scrolls up and down to display
listing means for causing said A/V display to selecadditional program listings for said current date and time
tively display a program listing that contains listi.ng
when said user continues to depress respectively, said up and
information related to A/V programs viewable on said
down buttons on said remote control device.
A/V display:
20. The audio-visual system as defined by claim 18 50 wherein said interface generation means includes marking
wherein depressing said up and down buttons causes a
means for marking programs on said program listing,
particular program listing to be selected, said particular
each of said programs on said program Iisting correprogram listing being thereby highlighted on said A/V
sponding to one of said audio-visual signals;
display.
wherein said marking means is activated by said user
21. The audio-visual system as defined by claim 1855
choosing a selected program from said program listing.
wherein said interface generation means causes said A/V
Using up and down buttons on said control means and
display to display listings for future time slots if said user
then activating a mark button on said control means;
depresses a predefined button on said remote control device
wherein when said user activates said mark button on said
and in which the speed of display increases if the user
60
control means, said listing means causes said A/V
depresses the predefined button longer than N seconds.
display to display a reminder mark adjacent to said
22. The audio-visual system as defined by claim 15
marked program
wherein said program listings are displayed in hourly time
slots, programs of less than one hour in duration being
wherein said interface generation means causes said A/V
displayed adjacent to one another separated by a half hour
display to display a reminder indication on a date and
delineation mark.
65
at a time based on the date and time that said selected
23. An interactive user interface and audio-visual (A/V)
program is carried over the audio-visual signal that
system, comprising:
corresponds to said marked program.

FL-Apple0000774

5,594,509
29

3O

27. The audio-visual system as defined by claim 26
receiving audio-visual (A/V) signals in an electronic
wherein said interface generation means causes said A/V
signal spectrum from a signal source using a transdisplay to display a reminder icon at a predefincd Iocation on
ceiver coupled to an A/V display, said transceiver
said A/V display on the date and at the time that said selected
including interface generation means;
program is carried by one of said audio-visual signals.
5
said interface generation means causing said A/V display
28. An interactive user interface and audio-visuai (A/V)
to display an interface, said interface including a cursystem, comprising:
rently viewed level of information that has been
selected by a user from multiple levels of information
a transceiver coupled to an audio-visual (A/V) display for
related to an audio-visual program viewed on said A/V
receiving audio-visual signals in an electronic signal
display;
spectrum from a signal source, said transceiver includ- 10
said user selectively displaying said multiple levels of
ing interface generation means for causing said g
information on said A!V display using control means in
display to display an interface, said interface including
communication with said transceiver;
selectively displayed multiple levels of information
said user pressing one or more direction buttons on said
related to an audio-visual program viewed on said A/V
control means to cause said currently viewed leve! of
display;
information about said audio-visual program to be
control means in communication with said transceiver for
replaced in said interface with a next level of informapermitting a user viewing said A/V display to selection about said audio-visual program.
tively display said multiple levels of information on
36. The method as defined by claim 35 further including
said A/V display;
the step of displaying a numeric level indicator identifying
wherein said interface generation means further includes 2o the currently viewed level of information.
listing means for causing said A/V display to selec37. The method as defined by claim 35 wherein said
tively display a program listing, said program listing
interface is displayed in an overlay region that includes at
including information related to A/V programs view- least one direction cue to identify at least one direction on
able on said A/V display;
said A/V display corresponding to at least one additional
wherein said interface generation means includes marking 25 level of information.
38. The method as defined by claim 37 wherein a plurality
means for marking a selected program on said program
of direction cues are displayed on said A/V display, wherein
listing;
said plurality of direction cues include one or more available
wherein said interface generation means causes said A/V
direction cues, said available direction cues being highdisplay to display a reminder icon at a predefined 30 lighted on said A/V display.
location on said A/V display on the date and at the time
39. The method as defined by claim 35 wherein said
that said selected program is carried by one of said
control means includes a remote control device having
audio-visuai signals;
direction arrow buttons, said user activating one of said
wherein said reminder icon comprises a picture-in-a- direction arrow buttons corresponding to the direction idenpicture (PIP) window in which said selected program is 35 tiffed by one of said available direction cues resulting in said
displayed.
transceiver causing said A/V display to display an additional
29. The andio-visuai system as defined by claim 28 level of information related to a currently viewed program.
wherein said control means further includes a jump button,
40. The method as defined by claim 39 wherein activation
the activation of said jump button by a user while said PIP
of a left and a right direction button on said control means
window is displayed results in said A/V display displaying 4o results in movement between pages of information relating
said selected program in a primary portion of said A!V to said currently viewed program.
display, and said program originally displayed in said pri41. The method as defined by claim 39 wherein activation
mary portion displayed in said PIP window.
of an up and a down button on said control means results in
30. The audio-visuai system as defined by claim 29 movement between additional information levels, said addiwherein further repetitive activation of the jump button 45 tional information levels providing data on a plurality of
results in the display in said primary portion of said display
currently viewed subjects in said currently viewed program.
of other selected programs which said user has marked using
42. The method as defined by claim 29 wherein said
said marking means.
interface is displayed in an overlay region that comprises a
31. The audio-visual system as defined by claim 28 horizontal bar on a lower portion of said A/V display.
wherein said interface generation means further includes 5O 43. A method for generating and displaying an interface
record means for causing programs to be recorded by a
on an audio-visual (A/V) system comprising the steps of:
storage recorder coupled to said transceiver.
receiving audio-visual (A/V) signals in an electronic
32. The audio-visuai system as defined by claim 31
signal spectrum from a signal source using a transwherein said record means is activated by said user choosing
ceiver coupled to an A/V display, said transceiver
said selected program on said listing using up and down 55
including interface generation means for causing said
buttons on said control means and then activating a record
A/V display to display an interface, said interface
button on said control means.
including selectively displayed multiple levels Of
33. The audio-visual system as defined by claim 32
information related to an audio-visual program viewed
whereupon said user activating said record button, said
on said A/V display;
listing means causes said A/V display to display a record 6o a user selectively displaying said multiple levels of inforicon adjacent to said selected program.
mation on said A/V display using control means in
34. The audio-visual system as defined by claim 31
communication with said transceiver;
wherein said record means causes said storage recorder to
wherein said transceiver further includes:
record said selected program on the date and at the time said
a switcher and
selected program is received by said transceiver.
65
35. A method for generating and displaying an interface
at least one tuner/demodulator coupled to said signal
on an audio-visual (A/V) system comprising the steps of:
source,

FL-Apple0000775

5,594,509
31

32

said switcher coupling said at least one tuner/demodulapicture-in-picture (PIP) window in which the currently
tor, said A/V display, and a storage means, for switchviewed program viewed by said user prior to said activation
ing digital and analog A/V data between said signal of said listing means is displayed on said A/V display.
source, said A/V display and said storage means, said
58. The method as defined by claim $7 wherein said
transceiver further comprising at least one processor
5 listing means causes said AJV display to display a program
unit, at least one system memory unit, at least one data
listing icon to identify to said user that said program listing
bus, an A/V decoder, an A/V encoder, a control interhas been activated.
face unit, and a bus controller cooperatively coupled
59. The method as defined by claim 58 wherein said
together for receiving input control and data signals and
listing means further causes said A/V display to display the
in response generating output control and data signals
for said signal source, said A/V display, and said 10 current date and time.
storage means.
60. The method as defined by claim 57 wherein A/V
44. The method as defined by claim 43 wherein said
program listings in said program listing are displayed in
electronic signal spectrum includes at least one digital
hourly time slots, programs of less than one hour in duration
program listing channel, and further including the step of
t5 being displayed adjacent to one another separated by a half
downloading program listing data from said signal source to
hour delineation mark.
said system memory.
61. The method as defined by claim 57 wherein upon
45. The method as defined by claim 44 wherein said
activation of said listing means, the listing for the currently
program listing data is downloaded, such that said data is
viewed program displayed in said first PIP window is
interleaved and wherein said program listing data comprises 20 highlighted.
data for a current date followed by data for another day of
62. The method as defined by claim 61 wherein said
the week.
program listing is selectively scrolled up and down by said
46. The method as defined by claim 44 wherein said
user to view additional program listings for said current date
program listing data includes program title data.
and time by continuing to depress, respectively, said up and
47. The method as defined by claim 44 wherein said 25 down buttons on said [remote]control means.
program listing data includes show time data and channel
63. The method as defined by claim 61 wherein depressnumber data.
ing said up and down buttons selects a particular program
48. The method as defined by claim 44 wherein said
listing, said particular program listing being thereby highprogram listing data includes length data and caption data
30 lighted on said A/V display.
for A/V progranls.
64. The method as defined by claim 61 wherein said
49. The method as defined by claim 43 further wherein
interface generation means causes said A/V display to dissaid interface generation means includes marking means for play listings for future time slots if said user depresses a
marking A/V programs in a program listing.
button on said control means where the speed of display
50. The method as defined by claim 49 wherein said 35 increases if the user depresses said button longer than N
marking means is activated by said user choosing a selected
seconds.
A/V program from said program listing using up and down
65. A method for generating and displaying an interface
buttons and then activating a mark button on said control
on an audio-visual (A/V) system comprising the steps of."
means,
receiving audio-visual (A/V) signals in an electronic
4o
$1. The method as defined by claim 43 wherein when said
signal spectrum from a signal source using a transuser activates a mark button on said control means, a listing
ceiver coupled to an AN display, said transceiver
means causes said A/V display to display a reminder mark
including interface generation means for causing said
adjacent to a selected program in a program listing.
A/V display to display an interface, said interface
including selectively displayed multiple levels of infor52. The method as defined by claim $1 wherein said 45
mation related to an audio-visual program viewed on
interface generation means causes said A/V display to dissaid MV display;
play a reminder icon at a predefined location on said A/V
a user selectively displaying said multiple levels of infordisplay on the date and at the time that said selected program
mation on said A/V display using control means in
is carried on one of said audio-visual signals.
communication with said transceiver;
53. The method as defined by claim 43 wherein said 5o
wherein said transceiver further includes listing means for
electronic signal spectrum comprises analog and digital
causing said A/V display to selectively display a prochannels.
gram listing that contains information related to A/V
54. The method as defined by claim 43 wherein said
programs viewable on said A/V display;
transceiver further includes listing means for causing said55
wherein said interface generation means includes marking
A/V display to selectively display listing information related
means for marking a selected program on said program
to AiV programs viewable on said A/V display.
listing;
$5. The method as defined by claim $4 wherein the
wherein said interface generation means causes said AN
activation of said listing means results in a listing interface 6o
display to display a reminder icon at a predefined
being displayed on said A/V display.
location on said A/V display on the date and at the time
56. The method as defined by claim 55 wherein said
that said selected program is carried on one of said
listing means causes said A/V display to display a program
audio-visual signals;
listing of currently available A/V programs available to the
wherein said reminder icon includes a picture-in-picture
user for viewing on said A/V display.
window in which said selected program is displayed.
65
57. The method as defined by claim 56 wherein said
66. The method as defined by claim 65 further including
listing means causes said A/V display to display a first
the step of activating a jump button on said control means,

FL-Apple0000776

5,594,509
33

34

lighted program on said program listing using said up and
the activation of said jump button results in said A/V display
displaying said selected program in a primary portion of said
down buttons and then activating a record button on said
display, said program originally displayed in said primary
control means.
portion displayed in a PIP window.
67. The method as defined by claim litl further including 5 70. The method as defined by claim 68 whereupon said
user activates said record button, said listing means causes
the step of said user repetitively activating said jump button
resulting in the display in said primary portion of said A/V
said A/V display to display a record icon adjacent to said
display of other programs which said user has marked using
highlighted program.
a marking means.
71. The method as defined by claim ~19 wherein said
68. The method as defined by claim ~15 wherein said
record means causes said storage recorder to record said
interface generation means further includes record means for
highlighted program on the date and at the time said highcausing programs to be recorded by a storage recorder
lighted program is received by said transceiver.
coupled to said transceiver.
69. The method as defined by claim
record means is activated by said user choosing a high-

FL-Apple0000777