Motorola Mobility, Inc. v. Apple, Inc.

Filing 94

NOTICE by Motorola Mobility, Inc. of Filing Brief on Claim Construction (Attachments: # 1 Exhibit, # 2 Exhibit, # 3 Exhibit, # 4 Exhibit, # 5 Exhibit, # 6 Exhibit, # 7 Exhibit, # 8 Exhibit, # 9 Exhibit, # 10 Exhibit, # 11 Exhibit, # 12 Exhibit, # 13 Exhibit, # 14 Exhibit, # 15 Exhibit, # 16 Exhibit, # 17 Exhibit, # 18 Exhibit, # 19 Exhibit, # 20 Exhibit, # 21 Exhibit, # 22 Exhibit, # 23 Exhibit, # 24 Exhibit, # 25 Exhibit, # 26 Exhibit, # 27 Exhibit, # 28 Exhibit, # 29 Exhibit, # 30 Exhibit, # 31 Affidavit)(Giuliano, Douglas)

Exhibit 10
to Motorola’s Opening Claim Construction Brief
July 28, 2011

111111111111111111111111111111111111111111111111111111111111111111111111111
US005583560A

United States Patent

[11]

Florin et ale
[54]

Date of Patent:

5,583,560

Patent Number:

[45]

[19]

48879
8601962

METHOD AND APPARATUS FOR
AUDIO-VISUAL INTERFACE FOR THE
SELECTIVE DISPLAY OF LISTING
INFORMATION ON A DISPLAY

2/1990
3/1986

Dec. 10, 1996

Japan ............................. H04N 5/445
WlPO.

(List continued on next page.)
OTHER PUBLICATIONS

[75]

[73]

Inventors: Fabrice Florin, Mill Valley; Michael
Buettner, Burlingame; Glenn Corey,
San Rafael; Janey Fritsche, Mill
Valley; Peter Maresca, Palo Alto;
Peter Miller, Los Altos Hills; Bill
Purdy, San Anselmo; Stuart Sharpe;
Nick West, both of San Francisco, all
of Calif.
Assignee: Apple Computer, Inc., Cupertino,
Calif.

42nd Annual Convention and Exposition of the National
Cable Television Association, Jun. 6, 1993, SF, CA, pp.
82-89, Mack Daily "Addressable Decoder With Downloadable Operation".
(List continued on next page.)

Primary Examiner-John K. Peng
Assistant Examiner-Chris Grant
Attorney, Agent, or Firm-Blakely, Sokoloff, Taylor & Zaf- .
man
[57]

[21]

Filed:

An interactive audio-visual (AN) transceiver is advantageously coupled to a television andlor telephone (TIT) cable,
a TV, a video recorder (VCR), and other AN devices. The
AN transceiver switches data between a program/service
provider and the connected AN devices. In one embodiment, the transceiver includes three primary modules, a
main module including a CPU, a system bus, system
memory, an infra-red (IR) control unit, an audio-visual bus,
an AN decoder, an AN processor, and an AN encoder, an
AN connect module including a number of tuner/demodulators and a switch, and an optional CD ROM module. The
AN transceiver hardware is complemented with an operating system and software program which supports the functions provided in the A/V user interface. Additionally, a
remote control device is provided to communicate with the
AN transceiver to interactively manage selection of program and service sources, selection program and service
offerings from any selected source, viewing of selected
program offerings, and interaction with selected service
offerings. The remote control device is advantageously
provided with a basic AN control button group, an interactive control button group, an auxiliary control button group
and a numeric key pad to facilitate control of the transceiver.
The interactive control button group includes an info button,
a list button, a categories button, a pix button, a mark button,
a jump button, and a pointing device consisting of up, down,
left, and right arrow buttons, and a center select button.

Appl. No.: 82,081

[22]

Jun. 22, 1993
(Under 37 CFR 1.47)

[51]
[52]
[58]

6

Int. CI. ..................................................... H04N 71173
U.S. Cl. ................................... 34817; 348112; 348113;
455/5.1; 45514.2
Field of Search .................. ,................. 34817, 10, 12,
348/734,906,563,564,565,566,567;
455/4.2; H04N 7116, 7/173, 5/44, 5/445,
5/00

[56]

References Cited
U.S. PATENT DOCUMENTS

Re.32,632
Re.34,340
4,290,142

3/1988 Atkinson ................................. 3401709
8/1993 Freeman .................................... 358/86
9/1981 Schnee et al ............................... 455/3

(List continued on next page.)
FOREIGN PATENT DOCUMENTS
0239884 10/1987
393555 10/1990
420123 4/1991
74476 4/1986

European Pat. Off..
European Pat. Off........ H04N 7/087
European Pat. Off. ....... H04N 51782
Japan ............................. H04N 5/445

50

ABSTRACT

16 Claims, 50 Drawing Sheets

54

~

58 TELEVISION SET

56
OTHER AUDIO-VISUAL
DEVICES

180 SCREEN

57

000

EXHIBIT 10
PAGE 1

5,583,560
Page 2

U.S. PATENT DOCUMENTS
4,381,522
4,533,910
4,536,791
4,555,775
4,573,072
4,622,545
4,641,205
4,706,121
4,748,618
4,750,036
4,772,882
4,785,408
4,812,834
4,829,558
4,847,604
4,847,700
4,873,623
4,884,223
4,890,320
4,899,136
4,914,517
4,914,732
4,931,783
4,935,865
4,939,507
4,987,486
4,995,078
5,008,853
5,014,125
5,047,867
5,062,060
5,072,412
5,148,154
5,151,782
5,151,789
5,155,806
5,157,768
5,177,604
5,195,092
5,206,722
5,220,420
5,223,924
5,236,199
5,239,540
5,247,347
5,253,066
5,253,067
5,283,819
5,353,121
5,357,276
5,404,393
5,410,326
Bl 4,977,455

4/1983
8/1985
8/1985
11/1985
2/1986
11/1986
2/1987
11/1987
511988
611988
911988
1111988
3/1989
5/1989
7/1989
7/1989
5/1987
1111989
12/1989
2/1990
4/1990
4/1990
611990
611990
711990
111991
211991
4/1991
511991
911991
10/1991
12/1991
911992
911992
911992
10/1992
10/1992
111993
3/1993
4/1993
6/1993
6/1993
8/1993
8/1993
9/1993
10/1993
10/1993
211994
1011994
1011994
4/1995
411995
4/1993

FOREIGN PATENT DOCUMENTS

Lambert .................................... 358/86
Sokonick ................................ 340/721
Campbell et al. ...................... 358/122
Pike ........................................ 364/900
Freeman .................................... 358/86
Atkinson ................................. 340n47
Beyers, Jr. .............................. 358/335
Young ..................................... 358/142
Brown ....................................... 370/94
Martinez ................................. 358/147
Mical ...................................... 340n09
Britton et al. .......................... 364/513
Wells ...................................... 340n21
Welsh ........................................ 372/92
Doyle ...................................... 340n06
Freeman .................................. 358/343
Lane et al. .............................. 364/188
Ingle et al ............................... 364/550
Monslow et al. ......................... 380/10
Beard et al . ............................ 340n06
Duffield ............................... 358/191.1
Henderson et al. .................... 340/825
Atkinson ................................. 3401710
Rowe et al. ............................ 364/188
Beard et al. ............................ 340n06
Johnson et al. ........................... 358/86
Monslow et al. ......................... 380110
Bly et al. ................................ 364/900
Pocock et al. ............................ 358/86
Strubbe et al. ......................... 358/335
Kolnick ................................... 364/521
Henderson, Jr. et al. .............. 395/159
MacKay et al. ........................ 3401712
Ferraro ...................................... 358/86
Young .................................. 3581194.1
Hoeber .................................... 395/157
Hoeber et al . .......................... 395/157
Martinez ................................... 358/86
Wilson et al. ......................... 370/94.2
Kwan ........................................ 358/86
Hoarty et al. .. ........................... 358/86
Strubbe ..................................... 358/86
Thompson, Jr. ........................ 273/439
Rovira et al. ............................. 370n7
Litteral et al. ............................ 358/85
Vogel ...................................... 348/906
Chaney et al. .. ..................... 3581191.1
Glick et al. ............................... 379/90
Young et al. ........................... 348/563
Banker et al. ............................ 348/12
Remillard .................................. 379/96
Goldstein .................................... 34817
Young ..................................... 358/142

4829558 5/1989
8912370 12/1989
9001243 2/1990
396062 1111990
9118476 1111991
9311640 6/1993
9311639 6/1993
9311638 6/1993

WIPO.
WIPO.
WlPO.
WlPO.
WlPO.
WIPO.
WIPO .
WIPO .

OTHER PUBLICATIONS
18th International Television Symposium and Technical
Exhibition, Jun. 10, 1993, Montreux, Switzerland, p.
555-567, Hoarty "Multimedia on Cable Television Systems".
IBM Technical Disclosure Bulletin, vol. 34, No.7 A, Dec.
1991, pp. 375-377, "Interactive Computer Conference
Server".
IBM Technical Disclosure Bulletin, vol. 36, No. 07, Jul.
1993, pp. 53-54, "Interactive Device for Conventional TVs
to Improve Functionality".
Elektor, No.4, Apr. 1990, pp. 10-12, "D2B-Homebus Flir
Audio and Video".
42nd Annual Convention and Exposition of the NCTA, 6/6,
1993, SF, CA, pp. 223-236, Bestler "Flexible D~ta Structures and Interface Rituals for Rapid Development of OSD
Applications" .
Alexander, Visualizing cleared-off desktops, Computer
World, May 6, 1991, p. 20.
Hiroshi Ishi, Kazuho Arita, Cleaiface: Translucent Multiuser Inteifacefor Team WorkStation, ECSCW, Sep., 1991,
pp.6-lO .
Hiroshi Ishii, Naomi Miyaka, Toward an Open Shared
Workspace, Communications Of The ACM, Dec., 1991, vol.
34, No. 12, pp. 37-50 .
Article entitled: Learning Considerations In User Inteiface
Design: The Room Model; author: Patrick P. Chan; publication of the Software Portability Laboratory, University of
Waterloo, Waterloo, Ontario, Canada, Jul. 1984.
Article entitled; CreationIModification of the Audio Signal
Processor Setup For A PC Audio Editor; publication of
International Business Machines, IBM Technical Disclosure
Bulletin, vol. 30, No. 10, Mar. 1988.
Article entitled: Browsing Within Time-Driven Multimedia
Documents; authors: Stavros Christodou1akis and Steven
Graham; publication of the Institute for Computer Research,
University of Waterloo, Waterloo, Ontario, Canada, Jul.,
1988.
Article
entitled:
Impact:
An
Interactive

Natural-Motion-Picture Dedicated Multi-Media Authoring
System; authors: Hirotada Ueda, Takafumi Mityatake, and
Satoshi Yoshizawa; Communications of the ACM, Mar.,
1991, pp. 343-350.

EXHIBIT 10
PAGE 2

SERVICE
PROVIDER

54

50

- --------,
f-------- "

62

FIG. I

~
•
•

rJl
~

~

=

70

t""'I'-

52
AjV
MAIN
CONN[C
0 CD-ROM
MODULE MODULE (opiionalj

~

I'D

r

i--'
",Q
i--'

\C
\C

="

r:I:l

::r
I'D

K:R

a

000
000

58 TELEVISION SET

i--'

~
til

Q

56
OTHER AlJDIO-VISlJAL
DEVICES

57

60

180 SCREEN

~NOTE

u-.
u-.

CONTROL

'Jt

000

00
W
'Jt

u-.

=

="
EXHIBIT 10
PAGE 3

Transceiver

L54

Television
(or Telehone)
Cable
52

..

,-------------- ----------------,
~---------------. -- ----------------------,
~D-ROM
... Module
Tuner/
,
70
, Demodulator ~
,
67
,
------------_.
I
I
I
I
I
I
I

I

t

I

:

VCR

I

,
56

Other

NY

,

Devices
57

TV
58

L

Main
Module
62

I
1
I
1
I
1
1
I

L--

~ID=erl
:

Memory

74·

Switcher
NY
69
~ Encoder
78

I

I

I

~

I
I

•
,
•
•

I
I
1
1

I

-=

....::::::::::----

I

I
I
I

IR
IControl I

A/V Connect.
Module 66

82

I

<:==>

Controller
80

~

NY
Bus
73

"===+~=1

...

1
1

•
I
•
•
•
•
•

&8us

~

~[M~~I

d
•

rJl
•
~

~

ft

=
t""'I'-

~

('I)

~

....
....
\C

",Q

\C

="

System

/~s

I c:u I

_

~

00

::r
('I)
~
N

S,
til

Q

System
Memory
65

- - - - - - - - - - - - - - - - - - - - - . - - - - - - - - - - - - - - - - - - - - - - - ______ 1

1 ______ - - - - - - - - - - -

u-.
u-.

'Jt

FIG. 2

Remote
Control
60

EXHIBIT 10
PAGE 4

00
W
'Jt

u-.
~

~

PROCRAJ,(
I

•

••

\

TOO4Y : MON.
; DATA
I
D4TA II
I

I

1OO4Y ; TUES.
1 DArA
I
I
DATA ,
I

•
rJl

•

USTTNC ll4 TA STRF.AJ.I
I

rODAY : WEO.
DATA
OOA II

I
I

I

I

CA7fGORIES,/XlYWORDS

TOll4 Y ; MON.
; DATA
041;4 ,
I

,

I

\

•••

I

I

~

=
t""'I'-

~

I'D

r

ELECTRONIC
SPECTRUN

CAPTIONS
LENGTH

~

~

I

rOlM Y : THUffS. TODAY: FRI.
; D4rA
I DATA
I
I
l14TA II
Il4TA II

o TITlES OF PROGRAMS
o SHOW TIMES

o
o
o
o

I

i-o'

",Q
i-o'

t

FIG. 3b

\C
\C

="

CHANNEL NUMBERS

DIGlrAL PROGRAM LISTING a«NN£1. - 100
BACK CHANN!l.{S) - 102

00

::r

m.
sa,
~

EXISTING ANALOG 1V
CHANNaS - 10J

FIG. 30

til

Q

PAY PfR VIEW
DIGITAL CfWiNELS - 106

ADDmON.JJ. lXCITAL

I

f

CIWtN£J.S - 108

FOR OTHER INTERACTIW: SERVICES

u-.
-.

u-.
00
w
-.
u-.

=

="

EXHIBIT 10
PAGE 5

u.s. Patent

Dec. 10, 1996

FIG. 4b

FIG. 40

60

:J

132

<G)

(0

130 •
144

5,583,560

Sheet 4 of 50

..

~

<CD

0>

~

((!)

(@>

136
155

138

146

162

176__

~ ~ "'-"C~

<@~

<CD

<CD> <®> @>
(D@).~

<CD<@)

<0>
I
J

I

I
I
I
J

I

I _________ J
I
L

EXHIBIT 10
PAGE 6

u.s. Patent

Dec. 10, 1996

5,583,560

Sheet 5 of 50

FI G. 50

FIG. 5b

160166

176
c::::=

=

---

® ®
@ ® ®
(J) ® ®
®

(D

177

EXHIBIT 10
PAGE 7

176-

------

u.s. Patent

Dec. 10, 1996

5,583,560

Sheet 6 of 50

0>

CO
,..

o

co
,...

\

EXHIBIT 10
PAGE 8

co

CO
,..

u.s. Patent

Dec. 10, 1996

5,583,560

Sheet 7 of 50

o

0)
'!-

r
'\.
---~---

o

CO
'!-

\

CD
0>
'!-

1:
--

E

o

E

c."'" 0

0>

0

(\/

CO

•

..J

«
o

c
o

-:
0
w

l"-

e>
~

~

__________________

~--------4-------~

(\/

0>

,..-

EXHIBIT 10
PAGE 9

u.s. Patent

Dec. 10, 1996

Sheet 8 of 50

5,583,560

o

co

~\

.

(.?

~

L--------------------------------------

EXHIBIT 10
PAGE 10

d

•
rJ).
•

/180

FIG. 9

~

~

ft

=
t""'I'-

~
~

i--'

"'c
i--'

\C
\C

="

00

::r
I'D
~

\C

o
....,

•
201
210

-

~

!Jt
C

SHOWBITS THIS MONTH
205

1
#3_
fA\.

Also: Merry H1IIs 67898

~[J~ and this Week's Top 10 videos

l l DAILY PROGRAM - Repeated at 12am
202

190

u-.
-.
u-.

00
~
-.

u-.

~
EXHIBIT 10
PAGE 11

u.s. Patent

Dec. 10, 1996

Sheet 10 of 50

5,583,560

o

.

C)

-

~L-______________-------------,~-r--~

EXHIBIT 10
PAGE 12

u.s. Patent

Dec. 10, 1996

5,583,560

Sheet 11 of 50

0
0

Z

(/)

0

••

C1)

E

...•

CO

tn

<t

->,

LL

(1)

"C

0:

~

CI)

0
0
~

CO

0

<t ...
0

.....
a:

E

~

Z

~

en

'f-

~

I- m
L..

<t

en

-

::>
0

...J

•
I-

tn
~lt)

0
Z

U.

-•

0.0') 0
• 0

co
•

- ~~
::::--~
m {I}

o

~

c Ctl
o ()

C\J

0
C\J

en.
t.r'
o~8~~
~ do t{)~..-

.

~o
-.".,..:
C\J

(!)

u.

\.....

I
co

.........
0
0)
,...

0

,...

EXHIBIT 10
PAGE 13

0

0""
C\I C\J

,I

FI G. 12

Cj
•
rJJ.
•

250

~
~

a.

<]Thursday 10/15 8PM

220

ALL PROGRAMS
9 A&B
10 NTV
11 DBS
180~

12 8MT
13
14
15
16

TC
DeV
NKK
TSN

17 HMX
18 STM
19 WAR
20, NNe

rv

~
r!:I

~

Personal Story
Rock Today <I> Party Tim
"Say Nay" Billye
Managers Forum
Windsurfing <1> Basketball
Planet Life
Wild Joe <I> Fun Times
Top Story"News
Top Movie
Best Movie
Comedy Program
Popular Program

.. -

~

"'c
~

-=
-=
="
251
rJ'l

Q'"
r!:I
r!:I

.....

~

~
Q

'til
"'"
C

-~

01
-.
01

00
W
-.
til

=

="

EXHIBIT 10
PAGE 14

Cj
•

rJ).

FI G. 13

•

~
~

~

~
t""'I'-

<]Thursday 10/15 8PM

6

180,

9
10
11
12

13
14
15
16
17

18
19
20
,

ALL

PROGRAMS

A&8
Personal Story
NTV ® Rock Today <I> Party Tim
I
DBS
"Say Nay" Billy
BMT
Managers Forum
Windsurfing <b Basketball
TC
DeV
Planet Life
Wild Joe <I> Fun Times
NKK
.
Top Story News
TSN
Top Movie
HMX
Best Movie
STM
Comedy Program
WAR
Popular Program
NNe

~
!'tI

~

i--'
",Q

118:06:30 I
lfa:06:S0U

251

i--'

\,Q
\,Q

="

r:I:l

::r
!'tI
~

i--'

~

9,
til
Q

u-.
-.

til
00
~
-.

(SZ

u-.

=

~

EXHIBIT 10
PAGE 15

01
-.
01

[SZ

00

~
-.
01
0\

=

EXHIBIT 10
PAGE 16

Cj
•
00
•

FIG. 15

~

a.
(0

=
~

<]Thursday 10/15 8PM

6

ALL

7

XYZ

8

CMM
A&8
NTV
DBS
BMT
TC

9
180

"

10
11
12
13

PROGRAMS
St. Louis at SF (6:3d)

~

~
~

~
~

~Q

Showbits This Month

~

Personal Story
118:08:30
Rock Today <I> Party Time (8:1 O)R]® • .
:

"Say Nay" Billy
Manager's Forum
Windsurfing <1> Basketball
14 DeV Planet ute
15 NKK Wild Joe <I> Fun Times
16 TSN Top Story News
17 HMX Top Movie
18 STM Best Movie

rsz

252

II

\Cl
\Cl

="

rJ'J.

=~
~

.....

~

!.Jl

c
.....,
!.Jl
Q

01
-.
01
00

~
-.

01
0\

=

EXHIBIT 10
PAGE 17

d

•
00
•

FIG. 16

'"'C

a
~

=

<:]11imHYJrr@@]@w ~ @#~ ® @WJ[M1\i)

6
180~

I

ALL

BILL

1
2
3
4

SPN
NJJ

5

CFF

6
7
8

VT1

9
10
11
12

A&8
NTV

FXX_

[SZ~

Xyz
CMM

DBS
8MT

~

PROGRA.MS
~

BILLBOARD 0 1
The Lights
Center Stage Sports
Day Court
Lonesome Bird (8:00)
This Generation
st. Louis at SF (6:30)
News <I> Politics Today
Personal Story (B:OO)
Bluff
lntrigue
Shop Talk

- -

~

~

- - -- -

~

....

F

18:09:30]

....

-=
-=
Q\

~
a
....
Q\

~
o
(It

-~-J

til
til

~

oe

~

~

til
0\

=

EXHIBIT 10
PAGE 18

0
•

00
•

FIG. 17

~

~

~

(0

=
~

-.t

(J1@@RMU'@1@W 11 @b'1I fl ®~rMll[)

6
1
2
3
4
180

'"

5
6
7
8

ALL

~

PROGRAMS

BILL
FXX
SPN
NJJ

Strongmen
College Football
Specials Tonight
.
News <I> New MTGH
This Generation
News <I> Early Peek
K. Johnson <I> Resources
The Movie Special (5:00)
Rock This Week <I> Overview
Jan Can Cook <I> Intrigue
Personal Interviews

~

~

BILLBOARD G>

CFF
VT1

Xyz

CMM
9 A&8
10 NTV
11 DBS
12 8MT

~

~Q
~

118:1.0:3011

I,C
I,C

="

rJ'J.

=~
~

.....

~

......:J

0
.....,
Ul
Q

(J]

-.

LSZ

(J]

~
-.

(J]

0\

=
EXHIBIT 10
PAGE 19

Cj
•
r.n
•

FIG. 18

~

~

1fi

\]J <]Saturday 10/17 6PM\> C;C
'
180

"

6

ALL

PRO G RAM'S

1

BILL

BILLBOAlUl

2
3

FXX
SPN

STRONGMEN
COLLEGE FOOTBALL

4

NJJ

5

CFF

6

VTl

7

XYZ

8
9
10
11

CMM
A&B
NTV

DBS

a

trrttirje

~

~

~

~

~Q

\1 8 :11 :30 11

~

-=
~

Specials Tonight
News
New KTGH
This Generation
News
Early Peek
K. Johnson
Resources
The Movie Special (5:00)
Rock This Week
Overview
Jan Can Cook
Intrigue

[7.)

l:l'"
~

~
~

00

~

~

RECORD:SPECIALS TONIGHT - 60 min

~~EJOOB
"'V"

/

259

til
-.

til

~
~

-.

til
0\

=

EXHIBIT 10
PAGE 20

Lj

•
00
•

FI G. 19

~

a.
(0

~

<J Saturday 10/17 6PM
180

'"

6:.

ALL

PROG"RAMS

C:Zmmv:

1

BILL

BILLBOARD

2
3
4
5
6

FXX

Strongmen
College Football
Specials Tonight
NeWs <I> New MTGH
This Generation
News <I> Early Peek
K. Johnson <I> Resources
The Movie Special (5:00)
Rock This Week <I> Overview
Jan Can Cook <I> Intrigue
Personal Interviews

7

SPN

NJJ
CFF
VT1
XYZ

CMM
A&8
10 N1V
11 DBS
12
, BMT

8
9

=

:fi
0

~

r!)

!"l
~

~Q
~

\Cl
\Cl

="

260
rJ1

=m.

~

\Cl

S,
!.Jl
Q

01
-.

[SZ

01
00

~
-.

01
0\

=

EXHIBIT 10
PAGE 21

u.s. Patent

Dec. 10, 1996

Sheet 20 of 50

5,583,560

o

C\J

.

(!)

-

~L-

______________________________

\

o
co
~

EXHIBIT 10
PAGE 22

~~~

u.s. Patent

Dec. 10, 1996

5,583,560

Sheet 21 of 50

(J)

OJ
T""

C\J'

.

t9

~L-________________~~~L-__-i~______~

I

o

00
T""

EXHIBIT 10
PAGE 23

u.s. Patent

Dec. 10, 1996

Sheet 22 of 50

EXHIBIT 10
PAGE 24

5,583,560

u.s. Patent

Dec. 10, 1996

Sheet 23 of 50

5,583,560

§}~
@)

en

w
~
a:

fiO}~

- §~
0
>
e:!
11.

0

••

('I')

en

w
a:

0
Cl

W
le:!
0

tI)

C\I

.

<.9
LL

,
0

co

~

EXHIBIT 10
PAGE 25

~

u.s. Patent

Dec. 10, 1996

Sheet 24 of 50

5,583,560

o

or-

C')

v
C\J

.

C)

~ ~------------------~

\

o

CO
or-

EXHIBIT 10
PAGE 26

u.s. Patent

Dec. 10, 1996

5,583,560

Sheet 25 of 50

to

co
,...

CO

co
,...

l()

C\J
(!)
~

L-________________l l - - -_ _

\

o
co
,...

EXHIBIT 10
PAGE 27

~

_ _ _ _ _ __===~

u.s. Patent

Dec. 10, 1996

Sheet 26 of 50

5,583,560

CD
C\J

.

(!)

~L=============

_________________

\

o

CO
,..

EXHIBIT 10
PAGE 28

d

•
00
•

FIG. 27

~

~
(0

=
~

Il~] <]Thursday 10/15 8PM[>
i

360

1~

I

6

SPORTS

t... 3 SPN
7
11
13
36

,
Baseball Tonight (7:00)

I

--l§r'

-

-=

~--

XYZ
St. Louis at SF (6:30)
DBS
"Say Nay" Billy'
ST
® WlndSUrfing~> Basketball
INS
College Football (7:00)

-

=:b"~

)18:20:30 U

~

~

'"
9"'
'~
""'
0\

,

ga

a
~

s,
~

[sz-n~-

~_m

___

~

-~

--

01
01
00

~

~

~

01
0\

=
EXHIBIT 10
PAGE 29

FIG. 28

o
•

365

00

•

~
~

]1
360

SPORTS

SPN
XYZ
11 DBS
13 ST
36 INS
3
7

180~
\,;

Thursday 10/15 8PM
Baseball TOni~(7:00)
St. Louis at S d:30)
"Say Nay" ~
Windsurfing • > Basket
College Football (7:00)

-----

=

0

~

--~

~

~

118:21 :30

~

~=

I,

~

I.C
I.C

wr

="

rJ'J.

l:l'"
to

~
N

QC

CATEGORIES: SPORTS

g,

@

=

Ul

A..

~[lK] lb1 ~[§]~
Y

315

[1l]

300
(J]
~

(J]

00
~

~

(J]

0\

=
EXHIBIT 10
PAGE 30

u.s. Patent

..--

O\~!I

- I'-- ~

\

I

I~I

"--

I!

I

5,583,560

Sheet 29 of 50

Dec. 10, 1996

::::1
'c

@

~
~

ro....:Cf)

Cl)XO
(1')f'...or-

CJ)

(j)

..c

I-

~

Q)
CJ)

al

N

C>
lJ....

~

<!

m

(n

W

>.

as·

=

••

(/)

<:

a:

a..>-ro

al

~

n. ~

0r-

t=;:::;:::.

",

.

.9

CD (f)

R-

m

-

..J
...J

o ....
I- CU =
- en CO
10 .- >.
.o:Jz

...J
...J

en w
~

:J

en

-C

~
0

~

0

.·0
,.....C")

r.n

CO
'U

en
....

O
0'-"

CO

r-

[I
en ~
«
m

ZNCI)

~
D-

O~~

LzS-.J

@]
"7

~

-,J
I

\

o

CX)

,-

EXHIBIT 10
PAGE 31

~~
(\')

FtG. 30

d
•

325

00

•

~

~

320~11

IIIIWJIII

Thursday 10/15 8PM
FAVORITES

6
7

180/

I\

Vf1
XYZ

8
9
10
11

CMM
A&B

NTV

DBS
12 BMT
13 ST

o
-----

ft

=
~

-

Sax Cymbals
st. Louis at SF (6:30)~
Showbits This Month
Personal Story
Rock Today <I> Party Time~
"Say Nay" Billy
Manager's Forum
WindsUrfing~1> Basketball

(18:23:301]

CATEGORIES: FAVORITES (!)

[gQ)[d§QIJ
305

eI

('D

~

....
~=
....
\C
\C
Q\

C'-l

=a

=
o
....,
=
~

Ul

01
-.
01
00

~
-.

01

~
EXHIBIT 10
PAGE 32

d
•

FI G. 31

00

•

~

~

ft

Thursday 10/15 8PM
FREQUENT PROGRAMS

362

BILL
6
7
8

VT1

XYZ
CMM
9 A&B
10 NTV
11 DBS
12 BMT

180~

---

BILLBOARD J®
Sax Cymbals
St. Louis at SF (6:30)-1
Showbits This Month
Personal Story
Rock Today <I> Party Time~
"Say Nay" Billy
Manager's Forum

FAVORITES: FREQUENT

0

318

~

::::::~

ttl

~

.---..

I-"

, (1 8 : 2 4: 30

~=

II

I-"

I"C
I"C

Q\

rJ).

=a
~

I-"

~

@

<J1\l~3:~11~MARKeDl LJ [MOM) ~ [ SJ:
~
Il
L::J
/
9
JII

=

=
til

1

_ ......,-+-300
,
Ul
-..

Ul
00

~
-..

Ul

~
EXHIBIT 10
PAGE 33

FIG. 32

363

~

325

I

Thursday 10/15 8PM
MARKED PROGRAMS

7
10
13
36

Lj
•
rJJ
•

st. Louis at SF

XYZ

._

-~~ ;-===

- db--I II

----

(6:30
Rock Today <I> Party Tim~ev..,..-------~
Windsurfing~ Basketball
~18:25:30' ,
College Football (7:00) ~

NTV
ST
INS

il

180~

~

~

ft

=
~

eI

rD

~

....
~=
....
\C
\C

e'I

00

=a
id
s,

I'--._F_A_V.-;;.O_R;.;;....;IT~E_S_:__
M_A...;.....R_K..;;;;..,ED_---=G:::=..--_ _ _ ___
L::::,.

[~~~~ Hj[MARKED

)\18 [MOM) B [Sj; 1

)7
319

Ut

=

Ul
-.
Ul
00

~
-.
Ul

~

EXHIBIT 10
PAGE 34

u.s. Patent

Dec. 10, 1996

Sheet 33 of 50

5,583,560

,...
o

CO
C")

CO
('I)

~

r(')

.

(!)
~

L-__________________________________

~
,...

EXHIBIT 10
PAGE 35

~

u.s. Patent

Dec. 10, 1996

Sheet 34 of 50

5,583,560

,...
co

('I')

V

g/IL-.--_ _ _----'"
EXHIBIT 10
PAGE 36

u.s. Patent

Dec. 10, 1996

Sheet 35 of 50

5,583,560

I~

.

(!)
~

L-~--

________________________

EXHIBIT 10
PAGE 37

~==~

u.s. Patent

Dec. 10, 1996

5,583,560

Sheet 36 of 50

IT]
[]]
o

o

to
rr>

.

C>

~~====~========~~------------

EXHIBIT 10
PAGE 38

u.s. Patent

Dec. 10, 1996

Sheet 37 of 50

5,583,560

C

...I lb)

a:

0

s:

~

U)

OJ

...

U.

LL (!;J

W
...,

f'-

lQ

[d

[;J

g

[J;J

rt')

.

C)

ll..

EXHIBIT 10
PAGE 39

u.s. Patent

Dec. 10, 1996

5,583,560

Sheet 38 of 50

Den
tO

0

W

o a:

o

(X)

r<>0

C\I

g.

C1J
('I)

.

(!)

LLL--J~~__+-____--~------------~---L-----

o

0)

(I)

EXHIBIT 10
PAGE 40

u.s. Patent

Dec. 10, 1996

Sheet 39 of 50

5,583,560

o

~\

~
lid·
[d

Ud

z
-

.a:

Q
[d

CJO
Z-'
-LL
O:w
0: 0
<l::-

J-a:

C/)OJ

<l:

[d

lL

ful:

z:s
Wm
l=u..
-LL

O:w
3:-,

>-~
OJ a:
00
w-'
~u..

OW

wQ

0: a:
-co
0<l:

u..

m

>->-

[0-,

0-'

w=>
OU
::>Cf)
Oz
OI

0:0

Q.-:l

r<>

\1

d

[bJ

LLL-~~~________--~----------~--~-----

EXHIBIT 10
PAGE 41

o
•

00

FIG. 40

•
~

ITJ [j ILl lLl

[j

ILl

[j

lLl

lLJ

=
=

;-

ro

~

180

/

Jeff's World
Rated R
Apple Pictures
90 min.
When a sleazy TV exec offers
Jeff and John a fat contract
to tape their cable-access show,
they can't believe their luck.
But they soon dIscover the road from
basement to bIg time Is a gnarly one.
o

0

~
~

~
~

\C

-:,
C"Il

=i
.&;;.

o

g,
~

a

d

~

rD

[]

[>{j]
382

384

386

388

Ul
-..

Ul
~

~
-..

Ul

~

EXHIBIT 10
PAGE 42

u.s. Patent

5,583,560

Sheet 41 of 50

Dec. 10, 1996

.ad
a:
w

co
co

m

::

('t)

::l
Z

0
0

U)
(J)

<C

oD.
.... !!!

wo:>

wOO

(J);::E

oena:
~

- :>
lLO

•

('I")

(fl-

0

I

....
....
-

-f(

UJ

iC

..J

10

I

m

m

I-

::
::>

W

>-

It>""')a:
UJ

Z

ltd
~

Q

w

....I

0

(J)

c(

....
D.
W

.

<..?
lL.

EXHIBIT 10
PAGE 43

0

~

0 0
ill
0 0:

u

0
0
0

U.S. Patent

Dec. 10, 1996

5,583,560

Sheet 42 of 50

[d'

Ud

Q

p;]

Q

Q

l1d
[[J

Ud
lQ

0
--1

ZOO

CC, W
OooJSf-:::>
0: 2

[b1
-

[J

lQ

~«~

lQ

tb(f)(1)

l!d

LLJJ

g

[g

[g

Q

lQ

EXHIBIT 10
PAGE 44

FIG. 43

d

•

rJJ

•

~

~

:mr-.~ <1Thursday 10/15 8PM(>
6
180~

FAVORrTES

12
13
47
48

BMT
ST
PXZ
SHP

49
50
1
6

TAL

7

TCT
BilL

VT1
XYZ
CMM

8
9 A&8
110 NlV

Manager's Forum
Windsurfing <I> Basketball
Jeff's World
TV Shop: Cacy's 0
Sunny City Vacationtime .
Technology Today

;-

=

~

~

Jeff's World

\0

~

rD

~

~

.P
~

1[8:35:30 )

\C
\C
~

C'-l

=-

BILLBOARD

m.

Sax Cymbals
St. louis at SF (6:30)
Showbits This Month
Personal Story
Rock Today <I> Party Time

(\7

~
~

=
til

I

til
til
00

'4

~

'4

til

="
=
EXHIBIT 10
PAGE 45

u.s. Patent

Dec. 10, 1996

5,583,560

Sheet 44 of 50

o
o

~

v

V

c:J

~~==~====================~

EXHIBIT 10
PAGE 46

FIG. 45

•

,

d
•

rJ'J.
•

I

~

SHOP
I

~

v

~

=

STORES

~

CD

o

180

/

~

o
tI)

r

~

Q

The World Company
IThe Attic at Cacy's
Williams-Napa
John Brewer
TV Shop Directory
'\J

vo

~

I

-...t....-403

\C
\C
~

rJl

=-

tI)

~

~
S,
01
Q

~I>
OJ

'"
OJ

402

404

406

408

409

00
tN
'"
OJ

=
Q\

EXHIBIT 10
PAGE 47

FIG. 46

d

•
•

rJ'J.

the

~

at

~

;-

a

attic cacy's

~

180

~

IDuralite Cookware -I @

tI)

~

/

'""'

...
Q

'""'
~

Lucks Mini Espresso
Lucks Brewmaster
Regatta Home Bakery
Juice Tiger Extractor

Q\

~

&
.,.

Q\

'V

S,
01

<J I

EXIT

I

~ST:ESQ

'j) FI

Q

[>
OJ
OJ
00
tN
"OJ
"-

406

=
~

EXHIBIT 10
PAGE 48

d

•
rJl
•

FIG. 47

~

at
the
attic cacy's

~

~

=
~

180

/

6

Duralite Cookware
[Lucks Mini Espresso'
Lucks Brewmaster
Regatta Home Bakery
Juice Tiger Extractor

F.XIT,

I ro1 Sf;" 61

....
....
\C
Q

vo

\C

®

\l

<J I

~

~

=-..

r.n

=Ii

tl

s,
01
Q

C>

OJ
OJ
00
tN
'"
OJ
",.

406

=
Q\

EXHIBIT 10
PAGE 49

d
•

FIG. 48

rJJ.
•
~

e;

~

=

Lucks Mini Espresso

~

180

/

~

tI)

~

Espresso and Cappuccito
maker from the worfd's
leader in coffee and
expresso makers. Wrth
Perfect Froth nozzle for
perfect cappuccino.
LIMITED OFFER:
Was $99.99 Now $69.99!

~

Q

....

'"

I.C
I.C

=-..

00

=-

tI)

~

"..,

®

oe

S,
CIt

Q

<lIEXlr!

~
~

f>
M
408

EXHIBIT 10
PAGE 50

01

'"
01

00
tN
'"
01

~

d
•

FIG. 49

rJ).

•

~

=
~

=
~

180

/
4 cup 10 oz

~

('D

r
'""'
",Q
\C
'\C
""'
~

Shop Card

NUMBER

Price
Cal. Sales Tax
Shipping

$69.99
5.61
3.00

~

Ii

_____________________ M _____________ _

1****1
OR CAll l-SSS-SHOPSPP

Total Cost

"..,

$78.60

\C

sa,
01
Q

<J IExrr I .~ Q
STOa';ES

'}JIM

[>
II

OJ

'til

00

~

409

EXHIBIT 10
PAGE 51

0-.

=

Q\

d

•

rJJ.
•

FIG. 50

~

[

~

Lucks Mini Espresso
180

,/'

~

tI)

~
~

Q

vo

~
~
~

4 cup 10 oz

Your order
will be delivered
by RST tomorrow
before 5PM.

="

$69.99

Price
Cal. Sales Tax
Shipping

5.61
3.00

--------------------------_.-------$78.60
Total Cost

r.n

=Ii
!.It

Q

o
.....,
!.It
Q

<J I

EXIT•• \

rJ STO~ES Q

l:b tJl
(

409
EXHIBIT 10
PAGE 52

r>

01
-..
01

00
~
-..

01

Q\
Q

5,583,560
1

2

METHOD AND APPARATUS FOR
AUDIO· VISUAL INTERFACE FOR THE
SELECTIVE DISPLAY OF LISTING
INFORMATION ON A DISPLAY

maximum flexibility in storing, retrieving and viewing television and other audio-visual programming. As will be"
described, the present invention provides methods and apparatus for presenting an improved audio-visual user interface,
which includes various user-selectable features for viewing
and controlling a television, video tape recorder (VCR) and
other audio-visual devices. As will be disclosed, the present
invention's user interface provides a user-friendly mechanism for consumers to view, record, and play back TV and
A/V programs, as well as control other A/V home entertainment devices using a remote control device. Furthermore,
information such as TV program listings and additional
information related to programs as well as selecting and
controlling categories of interactive programs and services
may be provided through the user interface of the present
invention.

5

BACKGROUND OF THE INVENTION

l. Field of the Invention:
The present invention relates to the field of audio-visual
systems. More specifically, the present invention relates to a 10
system for selectively viewing and interacting with programs and services from a number of prograrnJservice
sources, a control device for controlling the system, and the
methods and apparatus incorporated in the system for managing selection, viewing, and interacting with the program/ 15
service offerings.
2. Art Background:
SUMMARY OF THE INVENTION
Over the past 40 years, television and motion pictures
have become an important aspect of everyday life for people
The present invention provides methods and apparatus for
in the industrialized world. The development of advanced 20 presenting an improved audio-visual user interface for
technology in the areas of digital and high definition teleselecting and displaying cable television or other audiovision (HDTV), video recording systems, laser disks and
visual programs, as well as controlling various audio-visual
compact disc (CD) entertainment systems, coupled with
devices and interactive services. The present invention
satellite, cable television and telephone services, have proassumes a service provider provides cable television and/or
vided opportunities for viewers to store, retrieve and selec- 25 telephone (TfT) service to users via a TfT cable, including a
tively display a variety of television and audio-visual or
digital channel of program/service listings, at least one
interactive programming on home entertainment systems.
digital back channel (from the user's home to a central file
Over the past decade, improvements in personal computing
server), a number of analog TV channels, a number of digital
systems have provided a variety of powerful miniaturized
pay-per-view channels, and other interactive services transpersonal computers which permit the storage of data and 30 mitted from remote storage devices such as digital file
control of home appliances, such as entertainment systems,
servers. Under the present invention an interactive audiothrough the use of microprocessors. Additionally, a variety
visual (A/V) transceiver is advantageously coupled to a
of graphic user interfaces have been developed to ease
user's television, video tape recorder, and the TfT cable. In
human interaction with these new personal computer sysone embodiment, the transceiver includes three primary
35 modules. A main module includes a central processing unit
tems.
(CPU) coupled over a digital system bus to system memory
The combination of computer technology with television
(TV) and audio-visual (A/V) systems, has fostered the
and, preferably, an infra-red (IR) or similar wireless control
development of multi-media interactive entertainment sysunit. The main module is further coupled over an audiotems. The combination of computer hardware and software 40 visual bus to an A/V decoder, an A/V processor, an A/V
with audio-visual systems has brought new forms of art and
encoder, a TV, anA/V connect module connected to the TfT
entertainment into being. CD read only memory (ROM)
cable, and to one or more VCRs and other A/V devices. The
main module may also be coupled to an optional CD ROM
systems coupled to personal computers permit interactive
module, to permit the playback of multi-media CD ROM
video computer simulations, unique self-paced learning
environments and interactive "movies", not possible in 45 titles, audio CDs, and the like. Other optional modules or
earlier television systems (See, "The Revolution Starts
external devices (such as additional VCRs, CDs, hard disks,
telephone, fax and answering machines) may be connected
Here", Newsweek, page 42 (Jan. 18, 1993)). It is known that
consumers desire interactive television and multi-media
to this transceiver and controlled through the use of the
applications for home use. One of the biggest challenges for
audio-visual user interface of the present invention. The A/V
an interactive television service is the design of an interface 50 transceiver hardware is complemented with an operating
that is easy and fun to use by average consumers.
system software program which supports the functions provided in the present invention's audio-visual user interface.
In the computer industry, a variety of graphic user interIn operation, the cable or telephone service provider
faces have been developed to facilitate human interaction
transmits an interleaved data stream preferably including a
with computer systems. Many display systems utilize metaphors in the design of the interface as a way of maximizing 55 television program listing, together with program titles,
program times, categories, channel numbers and the like,
human familiarity, and conveying information between the
from a central file server on the digital channel of program
user and the computer. It is well known that designing
around a familiar metaphor helps reduce human learning
listings to the transceiver. The data is interleaved such that
time (See for example, Patrick Chan, "Learning Considerthe current day's data is followed by other weekly data.
ations In User Interface Design: The Room Moder', Report 60
A remote control device is preferably provided for comCS-84-16, University of Waterloo Computer Science
municating with the transceiver, and includes a number of
Department, Ontario, Canada, July, 1984, and the references
user selectable interactive functions such as: an info button,
cited therein). Research in interface design using metaphors
a list button, a categories button, a pix button, a mark button,
in man-machine interfaces may be applied to multi-media
a jump button, a select button, and a pointing device
systems, and in particular, to interactive television systems. 65 consisting of a set of cross-hair arrow buttons (up, down,
left, right) or other pointing devices capable of providing the
The marriage of video and television technology with
same functions (such as a mouse, thumb stick or touch pad).
computer interface technology provides consumers with

EXHIBIT 10
PAGE 53

5,583,560
4

3

FIG. 3a diagramatically illustrates the electronic spectrum
While viewing the TV, a user may obtain additional inforutilized by the present invention.
mation on a current program by depressing the info button,
and obtaining more detailed information using the pointing
FIG. 3b conceptually illustrates the present invention's
use of an interleaving data stream to download program/
device. By depressing the list button on the remote control
device, the transceiver displays a program listing of the 5 service listings, and other information, to the present invention's AIV transceiver.
current programs available for viewing. Through the use of
the pointing device, viewers can scroll up and down the
FIG. 4a illustrates one embodiment of the remote control
device of the present invention with the sliding panel in a
program listing or view a highlighted program in full screen
closed position.
by pressing the select button. By pushing the right or left
arrow buttons on the pointing device, program listings may 10
FIG. 4b illustrates the first embodiment of the remote
also be viewed for different hours, days and up to several
control device of the present invention shown in FIG. 4a
weeks in advance through the present invention's use of data
with the sliding panel in an open position.
interleaving. The depression of the categories button on the
FIGS. Sa and Sb illustrate a second embodiment of the
remote control device results in the display of a categories
remote control device of the present invention with no
menu bar on the TV screen, which includes categories such 15 sliding panel and additional features.
as "all", "sports", "news" and "favorites". The selection of
FIG. 6 illustrates the present invention's full screen viewone of these categories results in the display of only proing function as viewed by a user with a graphic overlay
grams within that category. Moreover, through the deprespanel which includes the current channel number as well as
sions of both the list button and the categories button, a
the channel logo and identifier.
program listing is displayed of the currently available pro- 20
FIG. 7 illustrates the present invention's info function
grams within the selected category. Through the use of the
showing the first page of the basic information track C"infoarrow buttons on the pointing device, category listings for
track").
different times and dates up to several weeks in advance may
FIG. 8 illustrates the present invention's info function
also be viewed. The selection of the favorites category
results in the display of currently available favorite pro- 25 with a second page of basic information, obtained by using
the right arrow button on the remote control device.
grams, including programs that are frequently watched by a
FIG. 9 is an additional illustration of the info function of
particular TV household, marked programs, or programs
the present invention in which the user has requested a third
that match preselected user preferences.
page of information using the remote control device.
The. depression of the pix button results in the display of
FIG. 10 illustrates the present invention's use of the info
a "visual menu" of multiple picture-in-picture (PIP) win- 30
function providing the user with additional tracks of infordows along the perimeter of the TV screen. Each of the PIP
mation (such as the "Tid Bits" information track), by using
windows displays one of the programs currently being
the down arrow button of the remote control device.
received by the AIV transceiver in the last category selected
by the viewer. The PIP windows are displayed at a variable
FIG. 11 is a further illustration of additional information
frame rate (N) (depending on the number of windows), and 35 tracks ("info-tracks"), such as this "Info Mart", accessible to
are captured by one or more tuners in the transceiver. A
the user of the present invention using the down arrow
center PIP window displays a larger video image (along with
button of the remote control device.
audio) of the currently highlighted PIP window. The other
FIG. 12 illustrates the present invention's use of the list
PIP windows can be similarly viewed by using the pointing
function to display current program/service listings and to
device on the remote control. The depression of the mark 40 highlight a particular program. Viewers may scroll up or
button allows users to "bookmark" a particular program for
down that list by using the up or down arrow buttons on the
later viewing. The depression of the jump button allows a
remote control device.
user to jump between previously marked programs.
FIG. 13 illustrates the list function of the present invenAdditional features, functions, and interface screens are 45 tion in which the NTV programming during the hour of 8:00
available to the viewer using the teachings of the present
to 9:00 p.m. includes two programs, one starting at 8:00
invention, including a menu button for controlling other
p.m., the other starting after the previous program has ended.
devices connected to the AIV transceiver (such as CD, VCR,
FIG. 14 illustrates the list function of the present invenetc.), a record button for making copies of programs onto
tion which allows the user to highlight the second program,
connected AIV recording devices (such as VCRs, hard disks,
etc.), a plurality of control buttons for playing back, stop- 50 which starts at 8: 10 p.m., using the right arrow button of the
remote control device.
ping, rewinding, or fast-forwarding audio-visual programs
FIG. 15 is an additional feature of the list function of the
on connected AIV playback devices, a numeric keypad for
present invention where the user marks the highlighted
dialing channel numbers, confirming financial transactions
program with the mark button of the remote control device.
with personal identification numbers, an optional microphone with talk button for supplementing the present user 55
FIG. 16 illustrates the selection of an alternate hourly time
slot beginning at 9 p.m. using the right arrow button of the
interface with direct voice commands through the use of
voice recognition technology, a pay-per view interface, and
remote control device.
a home shopping interface, as well as other features
FIG. 17 illustrates the selection of an alternate day for the
described herein.
60 program/service listing by depressing the right arrow button
of the remote control device.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 18 illustrates displaying of a record panel confirniing
the title and length of the program to be recorded, along with
FIG. 1 is a functional block diagram of the audio-visual
a confirmation of which VCR and tape to record on, by
(AIV) system of the present invention.
FIG. 2 is a functional block diagram illustrating the major 65 depressing a select button in the remote control device.
FIG. 19 illustrates the present invention's use of the select
components of the audio-visual transceiver of FIG. 1 in
further detail.
button on the remote control device while the program!

EXHIBIT 10
PAGE 54

5,583,560
5

6

service listing is displayed to instruct the selected VCR or
FIG. 36 illustrates the menu function which permits users
AIV recording device to make a copy of the highlighted
to press a menu button to switch the television display to
program.
other audio-visual devices coupled to the transceiver, such
FIG. 20 illustrates the present invention's "reminder"
as VCR, CD, on-line services, telephone, etc. For example,
feature, which automatically displays a live picture-in-pic- 5 once VCR is selected, with the right arrow button, the TV
ture (PIP) window of a previously marked program when the
displays an image from the videotape currently in the VCR
program is received by the AIV transceiver.
and the user can press the play button on the remote to
playback a previously recorded program on the video tape.
FIG. 21 illustrates the present invention's jump function,
wherein pressing a jump button one the remote control
FIG. 37 illustrates one embodiment of the present invendevice permits a user to jump directly to a full screen view 10 tion for selecting a pay-per-view channel offering entitled
of a previously marked program, and wherein pressing the
"Jeff's World" through the use of the select button of the
jump button again results in the display of another marked
remote control device.
program.
FIG. 38 illustrates a preview feature of pay-per-view
FIG. 22 illustrates the categories function of the present
under the above illustrated embodiment.
invention which is displayed by pressing the categories 15
FIG. 39 illustrates a credits feature of pay-per-view under
button on the remote control device.
the above illustrated embodiment:.
FIG. 23 further illustrates the categories function wherein
FIG. 40 illustrates an info feature of pay-per-view under
the "favorites" category is highlighted by pressing the right
the above illustrated embodiment.
arrow button on the remote control device.
FIG. 41 illustrates a confirmation feature for ordering a
FIG. 24 further illustrates the categories function wherein 20
the "sports" category is highlighted by depressing the right
pay-per-view offering through the use of the numeric keypad
arrow button on the remote control device.
on the remote control device under the above illustrated
embodiment.
FIG. 25 illustrates a representative television image in the
case where the "sports" category has been selected by
FIG. 42 illustrates the reminder feature advising the user
depressing the select button on the remote control device. 25 that the selected offering of pay-per-view will begin in three
FIG. 26 is a further illustration of a representative teleminutes under the above illustrated embodiment.
vision image in which the "sports" category has been
FIG. 43 illustrates one embodiment of the present invenselected, and an alternate sports channel has been further
tion for selecting and interacting with a home shopping
selected by a user through the use of the up arrow button on
service (TV Shop) offering.
30
the remote control device.
FIG. 44 illustrates a sample commercial shown to the user
FIG. 27 illustrates a sports program listing in which the
if the home shopping service is selected through the use of
list button has been depressed after the "sports" category has
the select button on the remote control device under the
been selected.
above illustrated embodiment.
FIG. 28 illustrates the categories function in conjunction 35
FIG. 45 illustrates a menu permitting the user to select
with the list function.
various shops available on the home shopping service using
FIG. 29 further illustrates the categories and list functions
the select button on the remote control device under the
when the down arrow button is depressed on the remote
above illustrated embodiment.
control device, breaking down the "sports" category into
FIG. 46 is a further illustration of the home shopping
sub-categories such as baseball or football.
40 service in which a cookware product is highlighted under the
FIG. 30 is a further illustration of the categoiies and list
above illustrated embodiment.
functions in which the category "favorites" is highlighted
FIG. 47 is a further illustration of the home shopping
and can now be selected by depressing the select button 01:
service in which a mini-espresso machine is highlighted, and
broken down into sub-categories by depressing the down
can be selected for ordering with the select button on the
arrow button on the remote control device.
45 remote control device under the above illustrated embodiment.
FIG. 31 further illustrates the categories and list functions,
where the "favorites" category has been broken down into a
FIG. 48 is an information screen provided to the user after
sub-category of "frequent" programs which are currently
selecting a mini-espresso machine for ordering under the
listed.
above illustrated embodiment.
FIG. 32 is a further illustration of the categories and list 50
FIG. 49 is an order screen in which the user, using the
functions, where another "favorites" sub-category, "marked
numeric keypad on the remote control device, enters a
programs", is highlighted displaying a list of previously
personal identification number to order the mini-espresso
marked programs.
machine under the above illustrated embodiment.
FIG. 33 conceptually illustrates the pix function of the 55
FIG. 50 is a sample order confirmation provided to the
present invention wherein multiple PIP windows are disuser subsequent to ordering a product in the home shopping
played by pressing the pix button on the remote control
service under the above illustrated embodiment.
device.
FIG. 34 illustrates the pix function of the present invenNOTATION AND NOMENCLATURE
tion in which a new PIP window has been highlighted and 60
is displayed in a larger format in the center of the display
The detailed descriptions which follow are presented
screen.
largely in terms of interface display images, algorithms, and
FIG. 35 is a further illustration of the present invention's
symbolic representations of operations of data bits within a
pix function in which, through the use of the remote control
computer memory. These algorithmic descriptions and repdevice, a PIP window showing a pay-per-view program has 65 resentations are the means used by those skilled in the data
processing arts to most effectively convey the substance of
been highlighted and is displayed in the center portion of the
their work to others skilled in the art.
screen.

EXHIBIT 10
PAGE 55

5,583,560
7

8

An algorithm is here, and generally, conceived to be a self
consistent sequence of steps leading to a desired result.
These steps are those requiring physical manipulations of
physical quantities. Usually, though not necessarily, these
quantities take the form of electrical or magnetic signals
capable of being stored, transferred, combined, compared,
displayed and otherwise manipulated. It proves convenient
at times, principally for reasons of common usage, to refer
to these signals as bits, values, elements, symbols, characters, images, terms, numbers, or the like. It should be borne
in mind, however, that all of these and similar terms are to
be associated with the appropriate physical quantities and
are merely convenient labels applied to these quantities.
In the present case, the operations are machine operations
performed in conjunction with a human operator. Useful
machines for performing the operations of the present invention include general purpose digital computers, digitally
controlled displays or other similar devices. In all cases,
there should be borne in mind the distinction between the
method operations of operating a computer andlor display
system, and the method of computation itself. The present
invention relates to method steps for operating a computer
and interactive display system, and processing electrical or
other physical signals to generate other desired physical
signals.
The present invention also relates to apparatus for performing these operations. This apparatus may be specially
constructed for the required purposes or it may comprise a
general purpose computer selectively activated or reconfigured by a computer program stored in the computer. The
method steps presented herein are not inherently related to
any particular computer or other apparatus. In particular,
various general purpose machines maybe used with programs in accordance with the teachings herein, or it may
prove more convenient to construct more specialized apparatus to perform the required method steps. The required
structure for a variety of these machines will appear from the
description given below. Machines which may perform the
functions of the present invention include those manufactured by the Assignee, Apple Computer, Inc., as well as other
manufacturers of computer and computer controlIed mutimedia systems.

"audio-visual" and its abbreviation, "AN", will be used
interchangeably and will have the same meaning.

10

15

20

25

30

35

40

45

DETAILED DESCRIPTION OF THE
INVENTION
The following detailed description will be divided into
several sections. The first of these will describe a general
system arrangement for receiving and generating the audiovisual user interface of the present invention. Subsequent
sections will deal with the functional aspects such as the
present invention's interactive functions, which includes the
info, list, record, mark, jump, category, pix and menu
functions, as well as the overall structure and operation of
the present invention's user interface.
In addition, in the following description, numerous specific details are set forth such as functional blocks representing data processing devices, and metaphors such as
screen, menu and other configurations to assist the user in
navigating through the user interface, etc., to provide a
thorough understanding of the present invention. However,
it will be apparent to one skilled in the art that the present
invention may be practiced without these specific details. In
other instances, well known circuits and structures are not
described in detail so as not to obscure the present invention
unnecessarily. For the purpose of this application, the word

General System Configuration

5

50

55

60

65

The multi-media interactive television system of the
present invention is illustrated in schematic form in FIG. 1.
As shown in FIG. 1, a cable or telephone service provider 50
provides cable television or telephone (Tff) services over a
Tff cable 52 to a plurality of users coupled to the cable or
telephone system, as is known. Although in this Specification reference is made to a cable television or telephone
system, it will be appreciated by one skilled in the art that
the present invention may be used in conjunction with a
variety of other electronic transmission systems including
satellite service systems, microwave systems, fiber optic,
and radio frequency (RF) systems.
As illustrated in FIG. 1, the Tff cable 52 is coupled to an
audio-visual transceiver 54 which comprises a number of
separately identifiable modules. The transceiver 54 is
intended to be located in proximity to and coupled to a VCR
56, a television (TV) 58, as well as one or more optional
audio-visual devices 57 such as additional VCRs, laser disc
players, camcorders, stereos, various storage devices, telephones, faxes and answering machines, as shown. Moreover, it is contemplated that transceiver 54 may be directly
incorporated into the VCR 56, the TV 58, or one the other
AN devices 57. As will be described below, a remote
control device 60 communicates with the transceiver 54
preferably through a wireless transmission signal (for
example, an infrared (IR) signal), or other mechanisms
known in the art. Additionally, as illustrated in FIG. 1, the
transceiver 54 is further capable of communicating with the
television 58, the VCR 56 and the other AN devices 57
through infrared or other means.
Referring now to FIGS. 1 and 2, the transceiver 54
comprises three primary modules: a main module 62, an AN
connect module 66 and an optional compact disc read only
memory (CD ROM) module 70. The main module 62
includes a central processing unit (CPU) 63 coupled over a
system bus 64 to a system memory 65 and an infra-red (IR)
control unit 82, which sends and receives wireless control
signals to and from the remote control device 60. The CPU
63 is further coupled through the system bus 64 to a memory
and bus controller 80, which is itself coupled through an AN
decoder 74 and an AN encoder 78 to the AN connect
module 66, as well as to an optional CD ROM module 70.
The CPU 63 is also coupled through the system bus 64, the
memory and bus controller 80 and an AN bus 73 to an AN
processor 77 and an optional AN memory 75.
The AN connect module 66 switches and receives analog
audio-visual signals and digital data from a plurality of
audio-visual sources including the Tff cable 52, the video
cassette recorder (VCR) 56, or the other AN devices 57, and
couples those signals and data to the main module 62
through the video decoder 74. The AN connect module
further switches audio-visual signals and data received from
the main module 62 through the video encoder 78, and sends
them back out to the Tff cable 52, the VCR 56, the other
AN devices 57 andlor the TV 58. The AN connect module
66 includes a switcher 69 and one or more programmable
tuners/demodulators 67, wherein one tuner/demodulator
reads and displays a current program from one of the
channels received over the Tff cable 52, and additional
tuners/demodulators (or the same tuner/demodulator, used
in alternation) are used to read and display data from the

EXHIBIT 10
PAGE 56

5,583,560

9

10

side-band channels in picture-in-picture (pip) windows. In
addition, the A/V connect module 66 may include descrambling circuitry (not shown) to descramble premium programs received over the Tff cable 52. It is further contemplated that the A/V connect module 66 provides a graphic
overlay function that superimposes an A/V signal from the
video encoder 78 against another A/V signal from the Tff
cable 52, the VCR 56, or the other A/V devices 57 allowing
both signals to be simultaneously displayed on the TV 58,
the VCR 56 or the other A/V devices 57. Finally, the A/V
connect module 66 can be used to transmit data such as order
information to the cable (Tff) service provider 50 over a
back channel 102 described below.
The A/V decoder 74 is used to decode data encoded in the
vertical blanking interval or special side-band cable channels into digital data and couple them to the CPU 63 through
the memory and bus controller 80 and the system bus 64.
Furthermore, the A/V decoder 74 is used to convert analog
audio-visual signals from the A/V connect module 66 into
digital A/V data and couple them to the A/V processor 77
through the memory and bus controller 80 and the A/V bus
73. Moreover, the A/V decoder 74 may be used to resize
audio-visual signals from the A/V connect module 66 in
order to create picture-in-picture windows and the like. It is
also contemplated that the A/V decoder 74 may be used to
decompress certain analog or digital signals (such as MPEG
motion video and the like) and couple them to the A/V
processor 77 through the memory and bus controller 80 and
the A/V bus 73.
The memory and bus controller 80 is used to route data
and control signals between the system bus 64, the A/V bus
73, the A/V decoder 74, the A/V encoder 78 and the optional
CD ROM module 70. The A/V bus 73 is a high-speed digital
bus used to free up the system bus 64 during the transmission of large amount of audio-visual data between the A/V
processor 77, the optional A/V memory 75 and, through the
memory and bus controller 80, the A/V decoder 74, the A/V
encoder 78, and the optional CD ROM module 70. The A/V
processor 77 coupled to the A/V bus 73 is used to manipulate, process, render, mix and otherwise re-arrange digital
data into coherent audio-visual displays. An optional A/V
memory module 75 also coupled to the A/V bus 73 can be
used to store A/V data before or after processing by the A/V
processor 77 (using components such as video random
access memory (VRAM) to hold, for example, a frame
buffer equivalent to one 640x480x8-bit color image in
memory).
The A/V encoder 78, coupled to the A/V bus 73 through
the memory and bus controller 80, is used to convert digital
A/V data from the A/V processor 77 or the optional A/V
memory 75 into analog audio-visual signals which are
routed to the TV 58, the VCR 56 or the other A/V devices
through the A/V connect module 66. It is also contemplated
that the AIV encoder 78 may be used to encode other data
such as order information sent by the CPU 63 through the
memory and bus controller 80 and the system bus 64 and
transmit them to the service provider 50 through the A/V
connect module 66 and a back channel 102 described below.
The CPU 63 is also coupled over the system bus 64 to a
system memory 65 including both volatile and non-volatile
memory components. The non-volatile part of system
memory 65 includes read-only memory (such as ROM),
which is used to store an operating system and playback
software, fonts, sounds and the like used in the present
invention. The non-volatile part of system memory 65 also
includes rewritable memory (such as SRAM), which is used
for persistent storage of mark or record indicators, listings of

programs viewed or taped, and other user preferences.
Additionally, the volatile part of system memory 65 includes
sufficient random access memory (such as RAM or DRAM)
for the temporary storage of data received over the Tff cable
52 or from the other devices 56-58 connected to the transceiver 54.
The transceiver 54 can also include an optional CD ROM
module 70 coupled through the memory and bus controller
80 to the system bus 64 and the CPU 63. This optional CD
ROM module 70 constitutes a general purpose storage
device to permit playback of optical compact discs (CDs),
including multi-media CD ROM titles, audio CDs, photo
CDs, and motion picture CDs on the TV 58. It is contemplated that those transceivers which do not include an
internal CD ROM module could instead include a CD ROM
interface allowing users to connect an external CD ROM
drive to the transceiver 54 at a later date. It is also contemplated that this CD ROM module 70 may in the future use
different electronic, magnetic, optical or storage technologies other than the current CD ROM embodiment.
Additional modules may be added to the transceiver 54
including, for example, such options as a small computer
system interface (SCSI) for accessing digital storage devices
such as hard disks, a modem for exchanging digital data over
telephone lines, or a serial port for controlling other devices
over a wired connection. Another module presently being
contemplated is a remote beeper button for locating the
remote control device 60. Many consumers often misplace
their remote control devices, and it is contemplated that the
present invention may provide a function wherein a speaker
on remote control 60 would beep while the beeper button is
pressed until the user locates the remote control.
Referring now to FIG. 3a, there is shown one possible
electronic spectrum of signals provided by the cable Tff
service provider 50 over the Tff cable 52 to the transceiver
54. As illustrated in FIG. 3a, the spectrum includes at least
one digital program listing channel 100, a plurality of back
channels 102, and a plurality of standard analog TV channels
103. In addition, and as will be described more fully below,
the spectrum includes pay-per-view digital channels 106,
and further additional definable digital channels 108 offering
a variety of interactive services, in addition to the general
purpose or special channels or services particular to the
specific application of the present invention.
As illustrated in FIG. 3b, the digital program listing
channel 100 provides data representing daily and weekly
program listings and related information from the service
provider 50 to the transceiver 54. As shown in FIG. 3b, the
listing channel 100 includes a repetitive data stream having
data representing today' s listing 110 interleaved with weekly
listings. For example, there is shown today's data 110,
followed by Monday's data 112, and then once again followed by today's data 110. As seen in FIG. 3b, Tuesday's
data 114 is followed once again by today's data 110, and
thereafter by Wednesday's data 116. Wednesday's data 116
is followed by today's data 110, and thereafter by Thursday's data 118. Thursday's data 118 is followed once again
by today's data 110, and thereafter by Friday's data 119.
Finally, following today's data 110, Monday's data 112 is
once again transmitted along the data stream provided by the
service provider 50. For programming purposes, the data
preferably will include titles of programs, show times,
special captions, length information, categories, and key
words, as well as channel numbers provided from the service
provider 50 over the Tff cable 52, and received by the
transceiver 54. The data stream is an interleaved data stream
which repeats on a weekly basis as shown in FIG. 3b. It is

5

10

15

20

25

30

35

40

45

50

55

60

65

EXHIBIT 10
PAGE 57

5,583,560

12

11

marked programs have been shown, looping back to the last
presently contemplated that there would be at least 100
marked program again. As shown in FIGS. Sa and 5b, the
different channels of programs andlor services. A weekly
jump button 132 and the mark button 142 are placed side by
program/service listing with information relating to 100
side together on the dividing line between the first and
channels of programs/services could require over 2 megabytes of information. This data stream of programs/services 5 second control button groups 115 and 134 to emphasize their
interrelation.
listing information (illustrated in FIG. 3b) is received by the
transceiver's main CPU module 62, whereafter the sections
Also provided with the interactive control button group
that are most relevant to the users are stored in the system
134 is a pointing device consisting of an up arrow button
memory 65. By interleaving the current day's data with the
145, a down arrow button 146, a left arrow button 148, and
weekly data, the daily data, the CPU module 62 can update 10 a right arrow button 150. For example, if the user is viewing
the system memory 65 periodically and still provides quick
a program/service listing through the use of list button 138,
access to the viewer, without having to store all the received
and desires to scroll up or down within the listing, the up
program/service information, thereby reducing the amount
arrow button 145 and the down arrow button 146 are
of system memory 65 required and the associated hardware
utilized. In addition, a center select button 155 is provided
cost.
for permitting the user to select one of several choices,
Referring once again to FIG. 3a, the back channels 102 15 represented by words or icons in graphic menus or lists
displayed on the TV 58, for example a "bull's eye" as shown
are used to engage in a variety of transactions, such as
in FIG. Sa. As will be described, the interactive control
ordering products, home banking services and pay-per-view
button group 134 may be used in a variety of applications,
movies, as will be described more fully below. In operation,
including selection of pay-per-view channels, home shopany request by a user to view a pay-per-view movie or order
a product is transmitted to the service provider 50 over at 20 ping services, and the like, as well as to display and control
programs from other AN devices connected to the transleast one back channel 102 through the Tff cable 52. It is
ceiver 54.
contemplated that as channel capacity increases, so will the
The third control button group is the auxiliary control
number of back channels, allowing for broader band twobutton group 160 comprising various AN control buttons
way communications such as telephone conversations or
25 for controlling the VCR 56 or the other AN devices 57
interaction with on-line networks.
connected to the transceiver 54. As illustrated in FIG. Sa, the
Referring now to FIGS. 4a, 4b, and FIGS. Sa and 5b, two
auxiliary control button group 160 includes a record button
alternative embodiments of the remote control device 60 are
162, a rewind button 164, a stop arrow button 166, a
shown in further detail. It is expected that the model shown
play/pause button 168, a fast forward button 170, and a
in FIGS. Sa and 5b would be more commonly used. As 30 menu button 172. In accordance with the teachings of the
illustrated in the figures, both embodiments of the remote
present invention, the user depresses the record button 162
control device 60 are comprised of four control button
while watching a television program on TV 58, which brings
groups 115, 134, 160 and 176. These control button groups
up a record panel confirming the current program's title and
115, 134, 160 and 176 wiII first be described briefly here, and
length, along with a highlighted select icon and confirmation
their usage wiII be described in further detail below.
35 of which AN device and type to use for recording. Once the
center select button 155 is depressed, the VCR 56 or the
The first control button group is the basic control button
other selected AN device 57 begins to record the program
group 115 which includes standard television control butcurrently being viewed by the user. Alternatively, if the user
tons such as power on/off button 120, volume control button
has pressed list button 138, and a program listing is high122 and a mute button 125. Moreover, channel up/down
buttons 130 are provided to the embodiment illustrated in 40 lighted, pressing the record button 162, and then the center
select button 155 results in the VCR 56 being programmed
FIGS. 4a and 4b to increment or decrement the currently
to record the particular show highlighted in the program
viewed channel on television 58, as is well known. For the
listing. Pressing the menu button 172 displays icons for all
embodiment illustrated in FIGS. Sa and 5b, the channel
the AI V devices 57 currently connected to the transceiver 54
control function is effectuated using the up and down arrow
45 and allows users to select another device with the arrow
buttons 145 and 146.
buttons. Pressing the center select button 155 displays the
The second control button group is the interactive control
output of that device and lets the user control that device.
button group 134 which includes an info button 136, a list
The fourth control button group is the numeric keypad
button 138, a categories button 140, and a mark button 142.
panel 176, which permits the user to directly enter channel
The info button 136 permits the user to obtain mUltiple
levels of information on programs while the user is watching 50 numbers to be viewed on the TV 58 by pressing a single,
double or triple digit number, followed by the ok button 178
one of the program offerings on the TV 58 (or other AN
(FIG. Sa). For the remote control device 60 shown in in FIG.
devices). The list button 138 permits the user to view listings
4b, the user would depress the select button 155 after
for programs/services. In the presently preferred embodientering values using the keypad 176. In addition, the
ment, when TV is selected, the list button 138 permits the
user to view current and future TV listings up to one week 55 numeric keypad 176 may be used by the user to enter a
personal identification number ("PIN"), to be transmitted to
in advance. The categories button 140 permits the viewer to
the service provider 50 through the AN connect module 66
select programs/services by category. For example, and as
for payment of pay-per-view movies, products and the like.
will be described below, the user may desire to watch only
sports programs, only news programs, or the like. As wiII be
In addition to the control button groups, both embodidisclosed, the categories button 140, and the list button 138 60 ments comprise a transmitter (not shown) for transmitting
signals to the audio-visual system. Additionally, the model
operate in concert to provide listings of selected categories,
illustrated in FIGS. 5a-5b is further provided with a talk
such as sports or news programs, on the TV 58. The mark
button 181, a microphone 179, a speaker 182 and a receiver
button 142 permits the user to mark programs for reminders,
(not shown). The talk button 181 when depressed allows the
later recall, or switching between programs which have been
selected using the mark button 142. Ajump button 132 lets 65 user to speak into the microphone 179, thereby providing
the user switch to the program that was last marked, then to
input to the AN system through voice, and receive voice
the program that was marked before that, and so on, until all
feedback from the AN system through the speaker 182.

EXHIBIT 10
PAGE 58

5,583,560

13

14

Full Screen Viewing Function

("Showbits This Month" in this example) continues to be
viewed incthe main portion of the screen 180, while different
graphic overlay panels 190 are superimposed over the lower
third portion of the screen 180.
In the currently preferred embodiment, a user may selectively move between pages of information relating to the
currently viewed program by pressing the right arrow button
150 or the left arrow button 148 on the remote control device
60, or alternatively, the main module 62 will automatically
scroll through the various pages of information with a
predetermined amount of display time for each page of
information.
Referring now to FIG. 10, there is shown a currently
viewed baseball game entitled "Saint Louis at San Francisco" displayed on the screen 180. Assume now the user
also depresses the info button 136 on the remote control
device 60. As previously described, information relating to
the currently viewed program is displayed within the graphic
overlay panel 190. (n the present example, information
relating to the baseball player, "John Smith", is provided
within the overlay 190. In the event the down arrow 210 is
highlighted, ari additional information track is available to
the user. This additional information track may be displayed
through the use of down arrow button 146 on the remote
control device 60, and is referred to here as "Tid Bits". As
shown in FIG. 10, the ''Tid Bits" information track gives
additional information relating to the particular subject
currently being displayed on the screen 180. In the example
of FIG. 10, the first baseman John Smith is currently being
viewed on the screen 180, and information relating to John
Smith is provided in the graphic overlay panel 190. It will
also be noted that in FIG. 10, the down arrow 210 is
highlighted, indicating that yet another information track is
available.
Still referring to FIG. 10, it will be noted that a down
arrow 210 is shown. By depressing the down arrow button
146 on the remote control device 60 once again, an Info Mart
track information track as shown in FIG. 11 is displayed by
CPU 63 in the graphic overlay panel 190. As illustrated, the
Info Mart information track provides the viewer with the
opportunity to purchase products, services, programs and the
like related to the program which is currently being viewed
(in the present example, the baseball game "Saint Louis at
San Francisco"). The information track disappears when the
info button 136 is pressed again.
While the information has been described with the left
and arrow right buttons 148 and 150 causing different pages
of an information track to be displayed, and the up and down
arrow buttons 145 and 146 causing different information
tracks to be displayed, it is contemplated that the information displayed may be related to each other in other application dependent manners. It is further contemplated that
this information function can also be used with audio-visual
programs or services other than TV shows,' in particular
when the menu button 172 is used to display the output of
the VCR 56, the CD 70, a hard disk, a telephone or another
other A/V device 57 connected to the transceiver 54.
The data comprising the information of the various information tracks shown in FIGS. 7-11 are downloaded using
the method illustrated in FIG. 3b, i.e. the interleaved data
stream transmitted from the service provider 50, to the
transceiver 54. The data are stored in the system memory 65
and updated as new data are received over the digital
program listing channel 100. Alternatively, data displayed in
the information tracks such as the exemplary Tid Bits or
Info-Mart information tracks may be transmitted by the

Referring now to FIG. 6, there is shown a representative
full screen audio-visual program displayed on the screen 180
of the TV 58. By depressing the channel up/down buttons
130 on the remote in FIGS. 4a and 4b, (or by depressing the
up and down arrow buttons 145 and 146 on the remote
control device in FIGS. 5a-5b), users can switch channels as
is commonly done with regular TV remotes. As the user
selects a channel to view using the remote control device 60,
a graphic overlay panel 185 is displayed in the lower right
hand portion of screen 180. As illustrated, the graphic
overlay panel 185 includes the channel number currently
being viewed 186 along with the channel logo and identifier
188. Additionally, a marking identifier 189, such as a check
mark, is also displayed, as will be described, to indicate that
the program has been marked through the use of the mark
button 142 previously discussed with reference to the remote
control device of FIGS. 4a, 4b, and FIGS. Sa and 5b.

5

10

15

20

Information Function
Referring now to FIG. 7, there is shown the screen 180 in
which the information (info) function 136 has been
requested by the user. As the user is watching a particular
television program on the TV 58, he may depress the info
button 136 to obtain information in the form of a graphic
overlay panel 190 on the screen 180. In operation, the
transceiver 54 coupled with the main module 62 through the
A/V connect module 66 superimposes the graphic overlay
panel 190 against the currently displayed video image. This
process is also used in other functions, such as the full screen
viewing function, where the graphic overlay panel 185 is
superimposed on the screen 180.·The overlay graphic panel
190 contains an "i" logo 192 indicating that the information
button 136 has been depressed, and also displays the title of
the currently viewed program 194 ("Showbits This Month"
in this example), a category name 196 ("News" in this
example), and the length of the overall program ("60 min"
in this example). In addition, the channel number and
network logo ("8" and "CMM" in this example) are also
provided to the user. Also, as illustrated in FIG. 7, the overall
length of the currently viewed program is graphically illustrated, by a time bar 200 showing how much time remains
before the program ends. A left arrow 201, a right arrow 202,
and a down arrow 210 are displayed and highlighted to
indicate that the left arrow button 148, the right arrow button
150, and the down arrow button 146, of the remote control
device 60 can be depressed to cause additional information
to be depressed. An up arrow 205 is also displayed, but is not
highlighted, to indicate that no additional information will
be displayed if the up arrow button 145 is depressed.
Referring now to FIG. 8, which illustrates a new graphic
overlay panel 190 displayed in response to the user having
depressed the right arrow button 150 on the remote control
device 60 while using the information function. The transceiver 54 provides additional information on the currently
viewed program. As illustrated, an additional page of information on the program currently viewed is displayed. For
example, additional guests on the show "Showbits This
Month" include "Madeline, Harold Black and Harmer".
Referring now to FIG. 9, which illustrates another new
graphic overlay panel 190 displayed in response to the user
having depressed the right arrow button 150 again. A third
page of information is displayed on the screen 180 relating
to the currently viewed program "Showbits This Month". It
will be noted that the currently viewed television program

25

30

35

40

45

50

55

60

65

EXHIBIT 10
PAGE 59

5,583,560
15

16

service provider SO to the transceiver 54 using the vertical
blanking intervals, which are inherently part of the analog
TV channel 103 used to display the currently viewed program on the TV 58. In another embodiment, data comprising
the information tracks may be transmitted and downloaded
through the use of separate info-track channels comprising
additional channels in the electronic spectrum illustrated in
FIG.3a.

the higher channel numbers as illustrated in FIG. 14. Notably, movement of the program/service listing and the highlighting in the present invention are specifically calculated to
conform with an average person's expectations in using a
television remote control. While the list function has been
described with the down arrow button 146 causing highlighting to move towards the lower channel numbers, and
the up arrow button 145 causing highlighting to move
towards the higher channel numbers, it will be appreciated
that the correspondence between the arrow buttons 145 and
146 and the highlighting movement may be implemented in
other ways as best suits the anticipated user of the interface
system.
Skipping now to FIG. 16, a further feature of the list
function of the present invention is illustrated. In the event
the user depresses the right arrow button 150, the CPU 63
displays the next hourly time slot listing on the screen 180.
In the example of FIG. 16, the time slot for 9:00-10:00 p.m.
is illustrated. It will be appreciated that, had the user
depressed the left arrow button 148 three times, then the
program/service listing for 6:00-7:00 p.m. would have been
displayed. Additionally, as illustrated in FIG. 17, the continued depression of the right arrow button 150 for a
predetermined amount of time (in the present embodiment
for more than 2 seconds) results in the display of program!
service listings for subsequent days. In this case, the continued depression of the right arrow button 150 has resulted
in a scanning of the program/service listings through to
Saturday, Oct. 17. It will be noted that in the currently
preferred embodiment, a scanning of subsequent days
results in the display of the program/service listing from the
beginning of prime time, namely, 6:00 p.m. (or 7:00 p.m., at
the service provider's option). It will also be noted that the
info function can be used in conjunction with the list
function to display program/service information about the
currently highlighted program/service. It will further be
noted that the listing display will disappear if the user
depresses the list button 138 once again. It is also contemplated that this list function can be used with audio-visual
programs or services other than TV programs, in particular
when the menu button 172 is used to display the output of
other AN devices 57. For example, if the VCR 56 is
displayed on the TV 58, pressing the list button 138 on the
remote control device 60 would display a listing of all
programs recorded by the user on the VCR 56, highlighting
the program now displayed from the VCR 56. Similarly, the
list function would display CD ROM or telephone listings
when these AN devices are selected through the menu
function.

5

10

List Function
Referring now to FIG. 12, the list function of the present
invention will be described. While viewing a television
program displayed on the TV 58, the user may depress the
list button 138 on the remote control device 60 to obtain a
program/service listing for the current date and time during
which the user is watching television. As shown in FIG. 12,
upon depressing the list button 138, a program/service
listing 220 is displayed for the current date and time at which
the viewer has depressed the list button 138. As illustrated,
programs/services currently available for viewing and interaction are identified by their channel numbers, channel name
identifiers (for example, Channel 10, NTV), and by titles of
the programs/services. In the event that the program/service
listing includes two programs for a given time interval, then
the names of both programs are displayed. For example, in
FIG. 12, Channel 10 is NTV which beginning at 8:00 p.m.
is showing a program entitled "Rock Today", and beginning
at 8:10 p.m. is showing a program called "Party Time". In
addition, as shown in FIG. 12, a picture-in-picture window
250 continues to display the currently viewed program
which the user was last viewing. Accordingly, the user may
continue to view the currently selected program, and the
current program/service listing simultaneously. By pressing
the select button ISS in the current time slot, the user can
view the currently highlighted program in full screen, as
long as that program is playing live at the current time. For
example, in the case of FIG. 13, the pressing of the select
button ISS would display NTV's "Rock Today" program in
full screen. It will be noted that although the current implementation of the list function displays a list of programs/
services on single lines for one hour time slots, the invention's user interface also supports other styles of display, for
example, a grid showing programs on two lines, each line
representing a two hour time slot.
It wiII also be noted that the currently selected program
being displayed in the picture-in-picture window 250 is
initially highlighted in the program/service listing to assist
the user. In the example of FIG. 12, Channel 11, PBS, a show
entitled "Say Nay Billy" is highlighted. Moreover, channels
may be selected by pressing the up arrow button 145 or the
down arrow button 146 on the remote control device 60 to
move the highlighting over to alternate program/service
offerings. ·For example, if the user presses the down arrow
button 146, as shown in FIG. 13, the NTV listing is then
highlighted. Similarly, as shown in FIG. 14, by continuing to
depress the down arrow button 146, the highlighting continues to move upward towards the lower channel numbers
and once the highlighting reaches Channel 9, the entire
program/service listing scrolls to display consecutively
lower channel numbers previously not displayed. At the
same time, the higher channel numbers previously displayed
at the bottom part of the screen 180 will consecutively
disappear. As illustrated in FIG. 14, Channels 7 and 8 are
now displayed, and Channels 19 and 20 are no longer
displayed to the user. If the up arrow button 145 is now
pressed instead, the highlighting will move down towards

15

20

25

30

35

40

45

50

Record Function
Referring now to FIG. 18, which illustrates a program/
service listing displayed in response to the user having
55 continuously depressed the right arrow button 150 on the
remote control device 60. The exemplary program/service
listing displayed is for Saturday, Oct. 17 beginning at prime
time (6:00 p.m.). The up and down arrow buttons 145 and
146 on the remote control 60 are then used to highlight the
60 program "Specials Tonight". Assume now the user desires to
record the exemplary program "Specials Tonight". By pressing the record button 162 on the remote control device 60,
a record panel 259 is displayed by the CPU 63 on the screen
180, confirming the title and length of the program to be
65 recorded, along with a highlighted select ("bull's eye") icon
as shown in FIG. 18. It is contemplated that other functions
can be included in the record panel 259, such as a function

EXHIBIT 10
PAGE 60

5,583,560
17

18

for selecting different VCRs or other AIV recording devices,
a function for selecting different tapes or reading media,
and/or a function for selecting how often the recording is to
take place (once, weekly, daily). Once the select button 155
is pressed, the program listing is redisplayed without the
record panel 259. A record icon 260 is displayed on the
screen 180, as shown in FIG. 19. The "Specials Tonight"
program which will begin on Saturday, Oct. 17, will then be
recorded automatically. In the event the viewer changes his
mind and does not desire to record the program "Specials
Tonight", by pressing the record button 162 again and then
pressing the select button 155 once more, the program listing
will be redisplayed without the record icon 260 and no
recording will take place.

graphic overlay panel 300 which is superimposed over the
currently viewed program on screen 180. A plurality of
categories are identified by representative icons. The first
category which is selected by default is the "all" category
302, and is highlighted around the border. The "all" category
302 re~ers to all currently available programs/services being
transIIl1tted by the service provider 50. By depressing the
right arrow button 150 or left arrow button 148 on the
r~mote control device 60, other categories may be highlIghted. For example, by depressing the right arrow button
150, the next category "favorites", as indicated by a star icon
305, ~s highlighted (See FIG. 23). Similarly, by depressing
the nght arrow button 150 again, a movies icon 310 is
highlighted. By depressing the right arrow button 150
another time, a sports icon 315 is highlighted (see FIG. 24),
and so on. Additional categories may include a news icon
312 and a music video icon 313. However, it will be
appreciated that many other categories can be displayed. By
repeatedly pressing the right arrow button 150, the user can
make category icons appear, scrolling from right to left
against the category overlay panel 300.
In FIG. 24, the sports category has been highlighted by the
user as indicated by the highlighted sports icon 315. To
select a category that is currently highlighted, the user can
press the select button 155 and the category overlay panel
300 disappears while the currently viewed program displayed on the screen 180 is replaced by the nearest available
program in the category just selected. The selection of the
sports category results in the display of only sports related
programs on the TV 58. FIG. 25 illustrates an exemplary
baseball game which is displayed after the user presses the
select button 155. As illustrated, the sports icon 330 is also
displayed in the graphic overlay panel 185 along with the
channel identifier 188.
Assume now after the nearest sports program has been
displayed on the screen 180, the user desires to view
additional sports category programs on the TV 58. By
depressing the up channel button 130 or the up arrow button
145 on the remote control device 60, the transceiver 54
displays the next higher number channel which is currently
showing a sports program. In the example of FIG. 26, the
sports program relates to a windsurfing event. As shown, the
sports icon 330 continues to be displayed along with the
numerical channel indicator and the channel identifier 188
(in the example of FIG. 26 "Sportstime") in the graphic
overlay panel 185.
Referring now to FIG. 27, if the user has selected the
category of sports (icon 315), and depresses the list button
138, a program/service listing 360 of currently available
sports programs is displayed on screen 180. It will also be
noted that the currently viewed sports program will continue
to be displayed in a picture-in-picture window 365. As
previously discussed, with respect to the list function, programs which are shown during half hour intervals (See FIG.
27 "Windsurfing" and "Basketball") are identified and may
be individually selected using the remote control device 60.
Referring now to FIG. 28, which illustrates a program
listing with the sports icon 330 in the category overlay panel
highlighted, displayed in response to the user having pressed
the list button 138, and then presses the categories button
140. Assume now the user desires to break down the main
sports category into finer sub-categories such as baseball,
football, basketball, hockey, etc. By depressing the down
arrow button 146 on the remote control device 60, the user
causes the CPU 63 to select a default sport sub-category
(baseball in this example) and display a plurality of sports
sub-category icons against the categories overlay panel 300

5

10

15

Mark and Jump Functions
Skipping now to FIG. 15, the mark function of the present
invention and its operation will be described. Shown in FIG.
15 is a program listing reflecting Channel 10 (NTV) having
been selected using the up and down arrow buttons 145 and
146 on the remote control device 60. Assume now the
current time is 8:08:30 p.m. In accordance with the teachings of the present invention as described earlier, the program "Rock Today" would be highlighted on the screen 180.
The program "Party Time" may be highlighted by the viewer
by pressing the right arrow button 150. The mark function
may then be initiated by pressing the mark button 142 on the
remote control device 160. As illustrated in FIG. 15, a mark
identifier 252 is displayed adjacent to the selected program
(in the present example, "Party Time"). Skipping again to
FIG. 20, a reminder box 254 is displayed on the screen 180
at the time the program "Party Time" begins (in the present
example, 8:10 p.m.), regardless of what channel is currently
being viewed then by the user on TV 58. It should also be
noted that programs playing live in a current time slot can be
marked while displayed in full screen by depressing the
mark button 142. This causes a mark identifier 189 to be
displayed in the graphic overlay panel 185, as shown in
FIGS. 6 and 21.
Referring now to FIGS. 20 and 21, depressing the jump
button 132 while the reminder PIP window 254 is displayed
switches the program displayed in the main viewing area of
the screen 180 to the program which the user has previously
marked (see FIG. 21). If more than one program has been
marked, by repetitively depressing the jump button 132,
users can jump to the other previously marked programs
displaying them in full screen in the main display area of the
screen 180 one at a time. It will be appreciated that in an
alternate embodiment, the picture-in-picture window 254
illustrated in FIG. 20 may be preceded by a reminder icon
displayed automatically in the PIP window 254 a few
minutes prior to the actual start of the program. It will also
be noted that the mark and jump functions may be used with
audio-visual programs other than TV shows, allowing a user
to mark and subsequently jump between, for example, a
VCR program, a CD ROM program, an on-line service or an
answering machine message, in addition to the TV programs
previously marked while viewing television channels.

20

25

30

35

40

45

50

55

60

Categories Function
Referring now to FIGS. 22-32, the categories function of
the present invention will be described. To initiate the
categories function of the present invention, the .categories
button 140 is depressed on the remote control device 60. As
illustrated in FIG. 22, the CPU 63 displays a categories

65

EXHIBIT 10
PAGE 61

5,583,560

19

20

as illustrated in FIG. 29. The default sports sub-category
icon 315 is also highlighted, and the program listing 361
comprises only programs of that sports sub-category, i.e.
baseball in this example. By pressing the right arrow button
150 on the remote control device 60, the user can highlight
the football sub-category icon 317, then press the select
button 155 to cause only football programs to be listed.
Alternatively, the user can press the up arrow button 145 to
display the main categories again, with the sports icon 315
highlighted as illustrated in FIG. 28. The user can also press
the categories button 140 once more to make the categories
overlay panel 300 disappear.
Referring now to FIG. 30, which illustrates a program
listing 320 with the favorite icon 305 in the category overlay
panel highlighted, displayed in response to the use having
selected the "favorites" category using the left arrow button
148 on the remote control device 60. The program listing
320 comprises favorite programs for the current time slot
including frequently viewed programs, marked programs,
and programs to be recorded, as well as programs that match
specified user preferences. Assume now that the user desires
to resolve the "favorites" category into finer sub-categories.
By pressing the down arrow button 146 on the remote
control device 60, the user causes the CPU 63 to display a
plurality of favorites sub-category icons against the categories overlay panel 300, as illustrated in FIG. 31. The CPU 63
highlights the "frequent" programs sub-category icon 318
and displays a program listing 362 of programs most frequently watched by the user as determined by the CPU 63
from previous monitoring of the user's viewing habits. In the
normal course of operation, the CPU 63 monitors programs
which are most frequently viewed 'and compiles a prioritized
list of those frequent programs for each time slot, which list
is stored in the rewritable non-volatile part of the system
random access memory (such as SRAM) 65.
In the presently preferred embodiment, if a viewer views
a television program for more than 10 minutes, the current
program and channel identifiers are compared to frequent
programs already stored in system memory 65 for that
particular time slot, and an updated list of frequent programs
is created by CPU 63 and stored in the system memory 65.
In the event there are no frequent programs stored in the
system memory 65 for any particular time slot, then a
general list of most frequently watched programs is retrieved
from the system memory 65. It is anticipated that the
majority of frequent programs will be stored in time slots
corresponding to prime time, or during daytime television
hours.
By pressing the right arrow button 150, the user can move
the highlighting from the "frequent" programs sub-category
icon 318 to the "marked programs" sub-category icon 319,
as shown on FIG. 32. Accordingly, the CPU 63 highlights
the "marked programs" icon 319 and displays a program
listing 363 of only programs that have been previously
marked. Other favorite sub-categories include a record subcategory of all programs selected for recording, and personalized favorites sub-categories such as "Mom" 321 and
"Joe" 322. It is contemplated that new preferences can be
defined by individual users by selecting a special favorites
sub-category, represented by a "set-up" icon 323, then
marking preferred items from a list of programs and/or
categories, as well as specifying an icon and/or a name for
this new preference set. By first defining and subsequently
selecting, for example, the "Mom" sub-category icon 321, a
user can view only programs that match the interests defined
earlier by the mother of the household.
It is also contemplated that this category function can be
used in combination with AN programs or services other

than TV programs, in particular when the menu button 172
is used to select the VCR 56, or one of the other AN devices
57. For example, pressing the categories button 140 while
viewing a VCR listing (FIG. 36, below) would enable users
to find all previously recorded comedy programs. Similarly,
the categories function could enable users to find phone
listings in the "gardening" category, according to telephone
directory information accessed from the provider 50 via the
Trr cable 52.
While the present invention has been described with the
categories function having different effects on the displayed
image when used during full screen display and when used
in conjunction with list function, it will be appreciated that
the present invention also may be practiced with the categories function having consistent effect on the displayed
image in either situation, as desired.

5

10

15

Pix Function
20

25

30

35

40

45

50

55

60

65

Referring now to FIGS. 33 and 34, the present invention's
picture-in-picture (or pix function) will now be described.
The depression of the pix button 144 on the remote control
device 60 results in the display of a pix display 381 as
illustrated in FIG. 33. As shown in the present embodiment,
12 picture-in-picture video windows are arranged within the
screen 180, along the perimeter of the pix display 381, with
a central picture-in-picture window 375 also displayed.
Depressing the pix button 144 on the remote control device
60 results in the simultaneous display of multiple programs
currently transmitted over the Tff cable 52, the VCR 56, the
CD ROM 70 and/or the other AN devices 57. Moreover, one
of the picture-in-picture windows displaying the last program viewed in full screen, such as, for example, picturein-picture window 380, is highlighted. As shown in FIG. 33,
the highlighted smaller screen is also displayed in larger size
in the central picture-in-picture window 375, with the audio
track of the program also audible to the user. Accordingly, a
user may watch a motion picture or television program,
including the audio track, in window 375, while viewing
multiple other programs simultaneously. In the presently
preferred embodiment, the AN connect module 66 samples
(or "grabs") a video frame from each of the 12 programs
every pre-determined (N) number of frames. Accordingly, a
user viewing the pix display 381 perceives each of the 12
small picture-in-picture windows as sequential representative images of every N frames per window. However, the
picture-in-picture window 375 represents a full motion
display of the currently selected program, provided that a
second TV tuner 67 is included within the AN connect
module 66.
Additionally, in the preferred embodiment of the present
invention, it is contemplated that the user is able to view the
pix display 381 by category by pressing the categories
button 140 and selecting a particular category, such as
favorites or movies. For example, the pix display 381
illustrated in FIG. 33 is comprised of only favorite programs, as the user had previously selected the favorites
category. Had the user selected the "marked" sub-category,
only marked programs would be displayed on the pix
display 381, allowing users to easily add or delete programs
in the pix display 381 by using the mark button 142. If no
categories have been selected (which is equivalent to selecting the "all programs" category), then the pix display 381
displays the 12 programs nearest to the last selected program. Similarly, pressing the info button 136 would display
the graphic overlay panel 190 shown in FIG. 7 for program
information.

EXHIBIT 10
PAGE 62

5,583,560

22

21
Referring now to FIG. 34, which illustrates the selection
of a different pix program to be viewed in the central
picture-in-picture window 375. By depressing the-left arrow
button 148 on the remote control device 60, the highlighting
380 is moved one window to the left. In the example of FIG. 5
34, a "Discover It" program is now highlighted and shown
simultaneously in the central picture-in-picture window 375.
Similarly, through the use of the left arrow button 148 and
the right arrow button 150 along with the up arrow button
145 and the down arrow button 146 on the remote control 10
device 60, each of the pix programs in the pix display 381
may be viewed within the central picture-in-picture window
375, as desired. As in the list function, pressing the select
button 155 displays the currently highlighted program in full
screen. Alternatively, pressing the pix button 144 a second 15
time causes the pix display 381 to disappear and the last
highlighted program appears in full screen.
Referring to FIG. 35, selection of pay-per-view movie
currently shown in one of the picture-in-picture window 380
of the pix display 381 for viewing is illustrated. As described 20
earlier, by depressing the left arrow button 148 on the remote
control device 60, the PIP window 380 becomes highlighted.
In the example of FIG. 32, the pay-per-view channel shows.
a preview trailer for the motion picture "Jeff's World",
which is now displayed in the center PIP window 375 and 25
the perimeter PIP window 380 simultaneously. To select the
pay-per-view movie, the select button 155 is depressed on
the remote control device 60. It is contemplated that pressing
a single, double or triple digit number on the numeric
keypad 176, followed by the ok button 178 of the remote 30
control 60, will display the corresponding pay-per-view
channel, along with the 12 nearest channels. It is also
contemplated that this pix function can be used with audiovisual programs and services other than TV shows, in
particular when the menu button 172 is used to display the 35
output of the VCR 56, the CD 70, a hard disk or one of the
other NY devices 57 connected to the NY transceiver 54.

on the remote control device 60 to cause the CPU 63 to
instruct the VCR 56 through the NY connect module 66 or
the IR control 82 to playback a previously recorded program
on the VCR 56. Similarly, the user can press the stop arrow
button 166, the rewind button 164, andlor the fast-forward
button 170 on the remote control device 60 to further control
the program sourced from VCR 56. It is contemplated that
the CD ROM module 70 and some of the other NY devices
57 including other CDs, VCRs, laser disc players, hard
disks, telephones, answering machines, as well as transmitting devices connected to voice mail, electronic mail and
various on-line services can be similarly selected and controlled through the use of the menu function, as illustrated in
FIG. 36.
Other Functions

Other functions may be realized with the present invention, including numeric data entry. As shown in FIGS. 41
and 49, by pressing one or more buttons on the numeric
keypad 176 of the remote control device 60, a user can
confirm a financial transaction by entering his or her personal identification number (PIN) in a confirmation display
field 425. The numeric keypad 176 may also be used for
dialing TV channel numbers, telephone numbers and other
program source identifiers, as desired.
Yet another function which may be optionally realized by
the present invention is a talk function, whereby a user can
depress the talk button 181 on the remote control device 60,
and thereafter issue spoken commands into the microphone
179 located on the remote control device 60 in lieu of or in
addition to pressing buttons on the remote control device 60.
Through the use of voice recognition hardware and software
operating in conjunction with the CPU 63, the present
invention can be made to interpret the spoken commands
requested by the user, and invoke the corresponding functions. For example, a user may speak the word "sports"
while pressing talk button 181, which is equivalent to
Menu Function
40 invoking the category function and highlighting the sports
category, as if the user had pressed the corresponding
Referring now to FIG. 36, the menu function will now be
sequence of buttons, as described earlier. It is further condescribed. The menu function permits users to switch the
templated that the remote control device 60 in conjunction
television display to any NY source connected to the
with the TV 58 and a camera can be used as a video
transceiver 54, such as the VCR 56, the CD ROM module
70, or one of the other NY devices 57, and lets users control 45 telephone for video conferencing and the like, wherein
having engaged a telephone line (via the menu function) the
those NY sources directly. As shown in FIG. 36, upon
user speaks directly into the remote control device 60 as if
pressing the menu button 172 on the remote control device
it were a standard telephone.
60, the CPU 63 displays on the screen 180 a graphic overlay
panel 420 which is superimposed over the currently viewed
audio-visual program. As further shown in FIG. 36, a 50
Pay-Per-View Interface
plurality of audio-visual sources connected to the transceiver
The present invention includes one possible pay-per-view
54 are identified by representative icons. The currently
(PPV) interface which will now be described. Assume that
viewed NY source icon is automatically highlighted, typithe user has selected the TV source icon 421 using the menu
cally the TV icon 421. By depressing the right arrow button
150, or left arrow button 148 on the remote control 60, other 55 function displayed in the graphic overlay panel 420, and
further selects a pay-per-view program using either the list
NY source icons may be highlighted and previewed. For
or the pix function. This selection results in an initial title
example, by depressing the right arrow button 150, the next
screen to be displayed, as illustrated in FIG. 37, which may
NY source icon, VCR 422, is highlighted, as shown in FIG.
be static or animated. As shown in FIG. 38, upon the
36. Simultaneously, CPU 63 displays an audio-visual program derived from the highlighted source icon, in this case 60 selection of the PPV program and subsequent to the display
of the title screen of FIG. 37, a preview icon 382, a credit
a videotape playing in the VCR 56 onto the screen 180, as
icon 384, an info icon 386, and a ticket icon 388 are
illustrated in FIG. 36. To select and control any NY source
displayed, along with a preview trailer which is continuwhose icon is currently highlighted, in this case the VCR 56,
ously looping and is displayed in a half-screen picture-inthe user can press the select button 155, whereafter the
graphic overlay panel 420 disappears while the currently 65 picture window 390. It will be noted that in FIG. 38, the
preview icon 382 has been highlighted by default, thereby
viewed program sourced from the VCR 56 is displayed in
corresponding to the trailer being shown in the window 390.
full screen. Thereafter, the user can press the play button 168

EXHIBIT 10
PAGE 63

5,583,560

24

23

Referring now to FIG. 39, by depressing the right arrow
commercial was playing last is highlighted. Through the use
button 150 of the remote control device 60, the credit icon
of the up arrow button 145 or the down arrow button 146,
384 may be highlighted. The credits for the pay-per-view
the user may scroll through the advertisers, sequentially
movie are displayed on the screen 180 within the window
highlighting each of their advertising identifiers, along with
390. By depressing the right arrow button 150 once again, 5 a select icon 403 prompting a user to select that advertiser.
Moreover, as shown in FIG. 45, an "exit" icon 402, a
the info icon 386 is highlighted. As shown in FIG. 40,
information related to the pay-per-view movie is then pro"stores" icon 404, a "specials" icon 406, an info icon (or
vided to the user, such as the title, the production company,
"compare" icon) 408, an "order" icon 409 and other icons
a brief description of the movie, the rating and the total
may be presented to the viewer. As illustrated in FIG. 46 the
length. If the user desires to watch the pay-per-view movie, 10 selection of the "Attic at Casey's" advertiser with the select
button 155 (or the right arrow button 150) results in a display
he/she depresses the right arrow button 150 once again to
highlight the ticket icon 388, as illustrated in FIG. 41.
of the various specials available from that advertiser. In FIG.
46, there is shown cookware highlighted in conjunction with
As shown in FIG. 41, upon the selection ofthe highlighted
a still image of the cookware for sale. By depressing the
"ticket" icon 388, the CPU 63 generates and displays on the
screen 180 a request for the user to input a "movie pass" 15 down arrow button 146 on remote control device 60, the
highlighted section on the display can be moved down such
password using the numeric keypad 176 of the remote
that the mini-espresso (see FIG. 47) is selected. Simultacontrol device 60. After entering the personal identification
neously with the highlighting of the mini-espresso selection,
number (PIN) using the numeric keypad 176, causing a
a still image of the mini-espresso machine for sale is shown.
confirmation sound to be played and confirmation symbols
such as asterisks to fill in a blank confirmation display 425, 20
Referring to FIG. 48, to obtain further information regarding the mini-espresso machine illustrated in FIG. 47, the
the user depresses the ok button 178 or the select button 155
on the remote control device 60 to order and view the
user depresses the select button 155 (or the right arrow
button 150) on the remote control device 60 to highlight the
pay-per-view movie. As shown in FIG. 42, transceiver 54
then displays the ordered program if it is available, or
info icon 408. As illustrated in FIG. 48, information relating
displays a message informing the viewer that the pay-per- 25 to the particular product (i.e. espresso machine) is shown on
view movie will begin screening within X number of
the screen 180. To order the mini-espresso machine illustrated in FIGS. 47 and 48, the user depresses the select
minutes. In operation, depressing the ok button 178 or the
button 155 (or the right arrow button 150) to highlight the
select button 155 subsequent to the entering of the movie
pass personal identification number (PIN) results in the CPU
order icon 409. As illustrated in FIG. 49, a personal iden63 providing a signal to the cable service provider 50, over 30 tification order number window 420 is displayed in which
the TIT cable 52 using one of the back channels 102 (See
the user (using the numeric keypad 176) inputs a personal
FIG. 3a). It will further be appreciated that during the
identification number (PIN). After entering the personal
waiting period prior to the transmission of the pay-per-view
identification number (PIN), the ok button 178 or select
movie, additional trailers and other previews may be disbutton 155 is depressed to order the product. In the presently
played to the user on the screen 180. It will also be noted that 35 preferred embodiment, once the personal identification numif the user incorrectly enters hislher movie pass PIN number,
ber (PIN) of the user is entered into the system of the present
pressing the clear button 177 on the numeric keypad 176
invention and the ok button 178 or the select button 155 is
allows them to clear the confirmation display 425 and start
depressed, a signal is provided by the CPU 63 to the service
over again. It will also be appreciated that other pay-perprovider 50, using one of the back channels 102 (See FIG.
view interfaces may be designed and implemented using the 40 3)a. As illustrated in FIG. 50, a confirmation of the order,
teachings of the present invention on general purpose audioalong with a delivery time is displayed to the user. It will
visual user interface.
also be appreciated that other home shopping interface
variants may be designed and implemented using the teachHome Shopping Interface
ings of the present invention.
45

The present invention includes one possible home shopping interface, which will now be described. In FIG. 43, a
Summary
channel identified as "SHP" or "TV Shop" represents a
dedicated home shopping service within the electronic specThe present invention as described provides methods and
trum illustrated in FIG. 3a. It will be noted that although in 50 apparatus for a unique audio-visual interface for television
its present embodiment, TV Shop is presented as a television
and other AIV programs and services. The present invenchannel, it could also be presented as an on-line service
tion's functions and features provide a user interface which
through a transmitting device connected to the TIT cable 52.
helps users find and view programs of interest, access related
In the current embodiment, the selection of the TV Shop
information, control various audio-visual devices and order
service results in a continuous full-motion video display of 55 products or services from their own homes. While the
various paid-for commercials or advertising messages. As
present invention has been described with references to the
shown in the representative screen of FIG. 44, a TV Shop
various figures, it will be appreciated that the figures are for
icon is displayed along with a blinking select icon (or right
illustration only, and do not limit the spirit and scope of the
arrow icon) in a graphic overlay panel 400, at the start and
invention. For example, although the figures have, by necesend of each advertisement, prompting users to press the 60 sity, used example screens having certain attributes, icons,
select button 155 (or right arrow 150).
criteria or functions, it will be appreciated that the invention
Referring now to FIG. 45, by pressing the select button
is not limited by the specific examples provided. Rather, the
155 (or the right arrow button 150), the user can view
present invention has application in any television and/or
additional information relating to the advertisements shown
audio-visual display system. In addition, it will be noted that
in the full motion video section of the TV Shop. In the 65 the present invention may be realized using a variety of
example of FIG. 45, it will be noted that a listing of the
computer hardware and computer software, and is not limadvertisers is provided, in which the advertisers whose
ited to any particular hardware or software systems.

EXHIBIT 10
PAGE 64

5,583,560

25

26

Moreover, it will be appreciated that many of the functions and features of the present invention, such as graphic
overlay panels, icons, and selection methods, are not static
events and are perceived by the user as having motion, color,
sound and/or animation. Specifically, it will be appreciated
that the present audio-visual user interface also includes an
audio interface using a plurality of sound icons and backgrounds, which compliment the visual interface and are
played almost every time a function of the present invention
is used, and which cannot be illustrated graphically here, but
are part of the present invention. Due to the limitations of a
written specification, the reader is referred to a videotape
entitled "EZTV" submitted by the Applicants concurrent
with the filing of the application on which this patent is
based.
We claim:
1. An audio-visual system comprising:
a transceiver coupled to an audio-visual (AIV) display for
receiving audio-visual signals in an electronic signal
spectrum from a signal source, said transceiver including an interface generator for displaying a listing interface on said AIV display, said interface generator
including a list generator for selectively displaying
listing information related to audio-visual programs
which may be viewed on said AIV display;
a controller in communication with said transceiver for
permitting a user viewing said AIV display to selectively display said listing information on said AIV
display;
wherein said controller includes a control having an
acti vator for activating said list generator in said transceiver, the activation of said list generator resulting in
said listing interface being displayed on said AIV
display; and
wherein said listing interface includes a picture in a
picture (PIP) window in which an audio-visual program
viewed by said user prior to said activation of said list
generator is displayed in said PIP window.
2. The audio-visual system as defined by claim 1 wherein
upon said activation of said list generator a listing for the
audio-visual program displayed in said PIP window is
displayed and highlighted.
3. The audio-visual system of claim 1 wherein said
interface generator displays listings for time slots that are a
first interval in the future if said user presses a predefined
button on said controller, and wherein said interface generator displays listings for time slots that are a second
interval in the future if said user depresses the predefined
button on said controller longer than N seconds.
4. An audio-visual system comprising:
a transceiver coupled to an audio-visual (AIV) display for
receiving audio-visual signals in an electronic signal
spectrum from a signal source, said transceiver including an interface generator for displaying a listing interface on said AN display, said interface generator
including a list generator for selectively displaying
listing information related to audio-visual programs
which may be viewed on said AIV display;
a controller in communication with said transceiver for
permitting a user viewing said AIV display to selectively display said listing information on said AIV
display;
wherein said controller includes a control having an
activator for activating said list generator in said transceiver, the activation of said list generator resulting in
said listing interface being displayed on said AN display;

wherein said interface generator includes a reminder
mechanism for identifying audio-visual programs represented in said listing information which said user
desires to be reminded of; and
wherein said reminder mechanism is activated by said
user selecting a selected audio-visual program represented in said listing information using first or second
buttons and then activating a mark button on said
controller.
5. The audio-visual system as defined by claim 4 wherein
upon said user activating said mark button, said list generator displays a reminder mark adjacent to the listing information corresponding to said selected audio-visual program.
6. The audio-visual system as defined by claim 4 wherein
said list generator displays a reminder at a predefined
location on said audio-visual display, on the date and at the
time said selected audio-visual program is received by said
transceiver.
7. The audio-visual system as defined by claim 6 wherein
said reminder comprises a picture-in-a-picture window in
which said selected audio-visual program said user desires
to be reminded of is displayed.
8. A method for generating and displaying a listing
interface on an audio-visual (AIV) system, comprising the
steps of:
receiving audio-visual signals in an electronic signal
spectrum using a transceiver coupled to an AIV display,
said transceiver including an interface generator for
displaying said listing interface on said AIV display,
said interface generator including a list generator for
selectively displaying listing information related to
audio-visual programs which may be viewed on said
AIV display; and
a user selectively displaying said listing information on
said AIV display using a controller in communication
with said transceiver;
activating said list generator in said transceiver, the activation of said list generator resulting in said listing
interface being displayed on said AIV display; and
displaying in said listing interface a picture in a picture
(PIP) window in which an audio-visual program
viewed by said user prior to said activation of said list
generator is displayed.
9. The method as defined by claim 8 wherein upon said
activation of said list generator a listing in said listing
information for the audio-visual program displayed in said
PIP window is highlighted.
10. The method of claim 8 further comprising the step of:
wherein said interface generator displays listing information for future time slots that are a first interval in the
future if said user presses a predefined button on said
controller, and wherein said interface generator displays listing information for future time slots that are a
second interval in the future if said user depresses the
predefined button on said controller longer than N
seconds.
11. A method for generating and displaying a listing
interface on an auido-visual (AIV) system, comprising the
steps of:
receiving audio-visual signals in an electronic signal
spectrum using a transceiver coupled loan AIV display,
said transceiver including an interface generator for
displaying said listing interface on said AIV display,
said interface generator including a list generator for
selectively displaying listing information related to
audio-visual programs which may be viewed on said
AIV display;

5

10

15

20

25

30

35

40

45

50

55

60

65

EXHIBIT 10
PAGE 65

5,583,560

27

28

a user selectively displaying said listing information on
said AIV display using a controller in communication
with said transceiver;
wherein said interface generator includes a reminder
mechanism for identifying audio-visual programs rep- 5
resented in said listing information which said user
desires to be reminded of; and
activating said reminder mechanism by said user selecting
an audio-visual program represented in said listing
information using said first or second buttons and then 10
activating a mark button on said controller.
12. The method as defined by claim 11 wherein upon said
user activating said mark button, said list generator displays
a reminder mark adjacent to said listing information corre15
sponding to said audio-visual program.
13. The method as defined by claim 12 wherein said list
generator displays a reminder at a predefined location on

said AIV display, on the date and at the time said audiovisual program is received by said transceiver.
14. The method as defined by claim 13 wherein said
reminder displays a picture-in-a-picture window in which
said audio-visual program said user desires to be reminded
of is displayed.
15. The method as defined by claim 11 wherein said list
generator further includes a record mechanism for recording
selected audio-visual programs on a storage device coupled
to said transceiver.
16. The method as defined by claim 15 wherein said
record mechanism is activated by said user selecting an
audio-visual program represented in said listing information
using said first or second buttons and then acti vating a record
button on said controller.

EXHIBIT 10
PAGE 66

* * * * *