Emblaze Ltd. v. Apple Inc.

Filing 76

Declaration of Lisa A. Ferrari in Support of 75 Motion for Leave to Amend Complaint, filed by Emblaze Ltd.. (Attachments: # 1 Exhibit A, # 2 Exhibit B, # 3 Exhibit C)(Related document(s) 75 ) (Ferrari, Lisa) (Filed on 12/15/2011) Modified on 12/16/2011 (jlm, COURT STAFF).

EXHIBIT B

1
2
3
4

MARTIN L. FINEMAN (CA State Bar No. 104413)
DAVIS WRIGHT TREMAINE LLP
505 Montgomery Street, Suite 800
San Francisco, CA 94111
Telephone:
[phone]
Facsimile:
[phone]
Email:
[e-mail]

5
6
7
8
9
10

MARTIN B. PAVANE (admitted pro hac vice)
LISA A. FERRARI (admitted pro hac vice)
COZEN O'CONNOR
277 Park Avenue
New York, New York 10172
Telephone:
[phone]
[phone]
Facsimile:
Email:
[e-mail]
[e-mail]

11
UNITED STATES DISTRICT COURT
FOR THE NORTHERN DISTRICT OF CALIFORNIA
OAKLAND DIVISION

12
13
14
15
16
17
18

EMBLAZE LTD.
Plaintiff,
vs.

)

)
)
APPLE INC., A CALIFORNIA CORPORATION, )
Defendant.
)

Case No: 4:11-cv-01079-SBA
[PROPOSED] FIRST AMENDED
COMPLAINT FOR PATENT
INFRINGEMENT
(Trial by Jury Demanded)

19
20

[PROPOSED] FIRST AMENDED COMPLAINT FOR PATENT INFRINGEMENT

21
22
23
24
25
26
27
28

Plaintiff, Emblaze, Ltd. ("Emblaze"), formerly known as Geo Interactive Media Group, Ltd.,
for its Complaint against Defendant, Apple Inc. ("Apple"), alleges as follows:
I.

PARTIES
1.

Emblaze is an Israeli corporation doing business world wide in development and

marketing of innovative high-tech technologies and products. Emblaze's main offices are situated at
Emblaze House, 22 Zarhin Street, Ra'anana, Israel 43662.
2.

Emblaze is a publicly held company with shares registered and traded on the main

London stock exchange continuously since 1996.
1

1

3.

Upon information and belief, Apple is a California corporation, doing business

2

worldwide, with its principal offices situated in this district, at One Infinite Loop, Cupertino,

3

California 95014.

4

II. JURISDICTION AND VENUE

5
6

4.

action arises under the patent laws of the United States (35 U.S.C. § 1, et seq.).

7
8

Subject matter jurisdiction is based on 28 U.S.C. §§ 1331 and 1338(a), in that this

5.

Venue lies in this Court pursuant to 28 U.S.C. §§ 1391(b)-(c) and 1400(b).

III. COUNT I — PATENT INFRINGEMENT

9

6.

Emblaze is the owner of several United States Patents, including U.S. Patent No.

10

6,389,473 ("the '473 Patent"). A true and correct copy of the '473 Patent is attached hereto as Exhibit

11

A.

12

7.

13

over the Internet.

14

8.

15
16

The '473 patent claims methods for real-time broadcasting over a network, such as

Emblaze developed the technology described and claimed in the '473 patent and has

used this technology in its products.
9.

Emblaze first unveiled the technology described and claimed in the '473 patent in a

17

live video streaming broadcast from the White House during Easter, 1998. Emblaze's live streaming

18

technology allows transmission of live audio and video to multiple devices, saves on data traffic,

19

does not require devoted streaming servers, and allows reliable streaming even through firewalls.

20

10.

Upon information and belief, Apple has used and continues to use, sold and/or

21

offered to sell in New York and elsewhere and/or imported into New York and elsewhere products

22

incorporating "HTTP Live Streaming Standard technology" that have been and can be used for real-

23

time broadcasting and that infringe at least claims 1, 2, 8, 9, 10, 11, 12, 13, 14, 21, 23, 24, 25, 26, 27,

24

28, 29, 36, 37, 38, 40, and 41 of the '473 patent in violation of 35 U.S.C. § 271.

25

11.

Apple announced the introduction of its HTTP Live Streaming Standard technology

26

into its products on or about mid-2009, and such technology is embedded into Apple's best selling

27

products in this district and world wide (e.g., all Apple devices including Apple Stream Segmenter

28

software or Apple File Segmenter software (components of HTTP Live Streaming included with
2

MAC OS X version 10.6 and later, and available for download on the internet), including Macbook
Air, Macbook, Macbook Pro, Mac mini, iMac and Mac Pro; all Apple devices including Apple's
Safari browser version 5 and later for Mac OS, and available for download on the internet, included
in Macbook Air, Macbook, Macbook Pro, Mac mini, iMac and Mac Pro; all Apple devices including
Apple's Quicktime X and later, and available for download on the internet, included in Macbook
Air, Macbook, Macbook Pro, Mac mini, iMac and Mac Pro; all Apple devices including Apple iOS
3.0 and later, included in iPod touch 3 rd and 4th generations (1 st and 2hd generations of iPod touch
software upgradable to support HTTP Live Streaming), iPhone 3GS, 4 and 4S (iPhone and iPhone
3G software upgradable to support HTTP Live Streaming), iPad 1 and 2, and Apple TV 4.0, 4.3 and
4.4); all devices including iTunes 10.1 and later.
12.

Upon information and belief, the acts of infringement by Apple are willful,

intentional and in conscious disregard of Emblaze's rights under the '473 patent.
13.

Shortly after Apple announced Apple's adoption of the HTTP Live Streaming

Standard technology into its products, Emblaze informed Apple that Apple's HTTP Live Streaming
Standard technology infringes the '473 patent and offered Apple a license to practice under the '473
patent.
14.

To date, Apple has declined to take a license under the '473 patent.

15.

As a result of Apple's infringement of the claims of the '473 patent, Apple has made

and will continue to make unlawful gains and profits, and Emblaze has been and will continue to be
deprived of revenue that it would otherwise have generated but for such infringement.
16.

Emblaze has been and will continue to be irreparably harmed by Apple's infringement

of the '473 patent.
17.
IV.

The extent of Emblaze's damages cannot be determined except by an accounting.

JURY DEMAND
Pursuant to Fed. R. Civ. P. 38(b), Emblaze requests a trial by jury.

V.

PRAYER FOR RELIEF
Wherefore, Emblaze prays for relief as follows:

3

1
2
3

A.

A judgment that Apple has infringed the '473 Patent in violation of 35 U.S.C.

B.

An order enjoining and restraining Apple, its officers, directors, agents,

§§ 271(a)-(c);

4

servants, employees, affiliates, attorneys and all others in active concert or participation with Apple,

5

from infringing the '473 Patent, pursuant to 35 U.S.C. § 283;

6
7
8

C.

A judgment awarding Emblaze its damages, but not less than a reasonable

royalty, resulting from Apple's infringement, pursuant to 35 U.S.C. § 284;
D.

An accounting of Apple's revenue from the sale, licensing or other distribution

9

of Apple's infringing products;

10

E.

A judgment that Apple's acts of infringement have been in willful, knowing,

11

and deliberate disregard of Emblaze's patent rights, and awarding Emblaze enhanced damages

12

pursuant to 35 U.S.C. § 284;

13
14
15
16

F.

A judgment awarding Emblaze its costs, disbursements, and attorneys' fees

incurred in prosecuting this action pursuant to 35 U.S.C. §§ 284 and/or 285;
G.

A judgment awarding Emblaze pre- and post-judgment interest on any

monetary award; and

17
18
19
20

H.

Such other relief as the Court may deem just, equitable, and proper.

DATED: December , 2011

DAVIS WRIGHT TREMAINE LLP

By: /s Martin L. Fineman
Martin L. Fineman

21
22
23
24
25

MARTIN L. FINEMAN (CA State Bar No. 104413)
505 Montgomery Street, Suite 800
San Francisco, CA 94111
Telephone:
[phone]
Facsimile:
[phone]
Email:
[e-mail]

26
27
28
4

1

COZEN O'CONNOR
Martin B. Pavane (admitted pro hac vice)
Lisa A. Ferrari (admitted pro hac vice)
277 Park Avenue
New York, New York 10172
Telephone: [phone]
[phone]
Facsimile:
[e-mail]
Email:

2
3
4
5

[e-mail]

6

Attorneys for Plaintiff
Emblaze Ltd.

7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

NYC_MIDTOWN \ 1915837 \1

111111111111111111 1 1 1 1 1 1,1111!111,11111B111 1 111111111111 111111111 1
(12)

United States Patent

(10) Patent No.:
US 6,389,473 B1
May 14, 2002
(45) Date of Patent:

Cannel et aL

(54) NETWORK MEDIA STREAMING
(75) Inventors: Sharon Carmel; Tzur Daboosh, both
of Giv'atayim; Eli Reifman, Rishon le
Zion; Naftali Shani; Ziv Eliraz, both
of Tel Aviv; Dror Ginsberg, Karkur;
Edan Ayal, Kfar Saba, all of (IL)
(73) Assignee: Geo Interactive Media Group Ltd.,
Givatayim (IL)

(*

)

Subject to any disclaimer, the term of this
patent is extended or adjusted under 35
U.S.C. 154(6) by 0 days.

Notice:

(21) Appl. No.: 09/275,703
Mar. 24, 1999

(22) Filed:
(30)

Foreign Application Priority Data
123819

Mar. 24, 1998 (IL)

GO6F 13/00
(51) Int. Cl.7
709/231
(52) U.S. Cl.
707/500.1; 709/200,
(58) Field of Search
709/231, 236, 246, 247; 382/236, 239;
375/240.12
(56)

5,404,446 A
5,841,432 A

(57)

ABSTRACT

A method for real-time broadcasting from a transmitting
computer to one or more client computers over a network,
including providing at the transmitting computer a data
stream having a given data rate, and dividing the stream into
a sequence of slices, each slice having a predetermined data
size associated therewith. The slices are encoded in a
corresponding sequence of files, each file having a respective index, and the sequence is uploaded to a server at an
upload rate generally equal to the data rate of the stream,
such that the one or more client computers can download the
sequence over the network from the server at a download
rate generally equal to the data rate.
41 Claims, 11 Drawing Sheets
Microfiche Appendix Included
(2 Microfiche, 138 Pages)

U.S. PATENT DOCUMENTS
11/1993 Normille et al.

345/537
707/500.1

Primary Examiner—Robert B. Harrell
(74) Attorney, Agent, or Firm—Ladas & Parry

References Cited
5,267,334 A

4/1995 Bowater et al.
11/1998 Carmel et al.

382/236

STANDARD
NETWORK
SERVER

( 26

30
REALTIME
ENCODE

Wale d 'S'a

24-1

BROADCAST
SERVER

IIJO

•W IN

28

FIG. 1

30

20 VA'

30

III £L17`68£ `9

sa

PRIOR ART

U.S. Patent

May 14, 2002

Sheet 2 of 11

US 6,389,473 B1

U.S. Patent

May 14, 2002

Sheet 3 of 11

US 6,389,473 B1

1 IV.

1- 1 1 -1- 1 2-1— 1 3-1--- 141

SUCE
1

SUCE
2

SUCE
3

OH

SUCE
4

/
i

1

•
42

,

1
44

46

\
48

TIME
40

FIG. 3B
52

54
50

FIG. 30
56
J J+1

55

J+2 J+3

•• •

N

58

'a
rualud'S

FIG. 3D

41
57 59

/

A2

AUDIO
LEVEL #1
(1)
.
45

43

PTS I

1 1 PTS

A2

Al

PTS

SIZE PTSI

A2

SIZE PTS

TO
PTS

AUDIO
LEVEL #2
(1)
/
47

AUDIO
LEVEL #1
(2)
/
45

AUDIO
LEVEL #2
(2)
i
47

VIDEO
LEVEL #1
(1)

VIDEO
LEVEL #2
(1)

49

ID
(1 )

• ••

51

wits

HEADER

PTS j

57
Al r PTS
AUDIO
LEVEL #1
(3)
45

Al

PTS
AUDIO
LEVEL #2
(3)
47

Al

PTS
AUDIO
LEVEL #1
(4)
45

A2 1 PTS
AUDIO
LEVEL #2
(4)
47

Al

SIZE PTS
VIDEO
LEVEL #2
(2)
51

It Jo r

61
59
U
SIZE
PTS

U
SIZE
PTS

Al PTS

Al I PTS I

TEXT
(1)

URL
(1)

AUDIO
LEVEL #1
(5)

AUDIO
LEVEL #2
(5)

53

55

45

47

• ••

Ta £L17`68£`9

Al

57 61 59

FIG. 4
Ta£L17`68£`9Sfl

TI JO s p aTIS

NETwORK
SERVER

28

lu alu d 'S n

U.S. Patent

May 14, 2002

US 6,389,473 Bi

Sheet 6 of 11

FIG. 5

CONNECT
TO SERVER

INPUT
BROACAST
DATA

f

ENCODE

SLICE

82

4

FTP TO SERVER h84

UPDATE
INDEX
FILE

1
CHECK
LINK
FUNCTION

88

U.S. Patent

May 14, 2002

Sheet 7 of 11

US 6,389,473 B1

FIG. 6A

CONNECT
TO SERVER

FITTP FROM
SERVER

READ INDEX
FILE
CONNECT
NEW LINK

CONTINUE

SELECT SLICE

DECODE

OUTPUT
DATA

YES

LINK
FUNCTION
OK?

NO

U.S. Patent

May 14, 2002

Sheet 8 of 11

US 6,389,473 B1

FIG. 6B

CONNECT
TO SERVER

HTTP FROM
SERVER
READ HEADER
CONTINUE
CHOOSE
LEVEL

READ SLICE
DECODE
OUTPUT
DATA
DETERMINE
LINK RATE
RATE
TOO
LOW

YES

CHOOSE
LOWER
AUDIO/VIDEO
LEVEL

NO
RATE
HIGHER THAN
NEED

CHOOSE
HIGHER
AUDIO/VIDEO
LEVEL

•s•••••11111.^.

U.S. Patent

May 14, 2002

Sheet 9 of 11

US 6,389,473 Bl.

FIG. 7

INPUT DATA
i
i-90
CONTROL
FROM 88

SET COMPRESSION
RATIO
801

i
COMPRESS DATA
1

( 92
CONTROL
FROM 88

SET SLICE DURATION
821

i
PREPARE
SLICE I
i
TO FTP 84

+

U.S. Patent

May 14, 2002

Sheet 10 of 11

US 6,389,473 Bl.

FIG. 8
FROM
SLICE 82

84
N

1=1, J=1

9q

FP OPEN

94

LINK TIME-OUT FROM CHECK 88

FP SEND
SLICE I

LINK J

NEXT
LINK

1=1+1

J= +1
YES

TO UPDATE
INDEX 86

U.S. Patent

May 14, 2002

US 6,389,473 B1

Sheet 11 of 11

88

FIG. 9

FROM
UPDATE 86

CHECK SLICE
TIME TH

YES

RE—INITIALIZE
LINK
TO FTP
OPEN 94
INCREASE
COMPRESSION

I

_

TO SET
COMPRESSION 90

DECREASE
SLICE DURATION

TO SET
DURATION 92

YES

NEXT SLICE

INCREASE
LSLICE DURATION!

TO ENCODE
80

US 6,389,473 B1
1
NETWORK MEDIA STREAMING
A computer printout is attached hereto as an appendix in
microfiche form and is incorporated herein by reference. The
printout comprises executable program files in hexadecimal
format. This appendix includes 2 microfiches, containing a
total of 138 frames.
FIELD OF THE INVENTION
The present invention relates generally to network data
communications, and specifically to real-time multimedia
broadcasting over a network.
BACKGROUND OF THE INVENTION
In network broadcasting, data are transmitted over a
network in real time from a single transmitting computer to
a plurality of clients simultaneously. The network may be a
LAN, a WAN, an intranet or a public network such as the
Internet. Network broadcasting is most commonly used to
stream multimedia data, typically comprising images and
sound.
FIG. 1 is a schematic illustration showing a real-time
broadcasting system 20, as is known in the art. One or more
input devices 22 (for example, a video camera and/or
microphone) are used to generate a multimedia data stream
representing an entertainment or informational program to
be transmitted to a plurality of clients 30 via a network 28.
Because of bandwidth limitations of the network, the data
stream from host 22 must first be compressed by a real-time
encoder 24 and then routed to appropriate clients 30 by a
broadcast server 26 (since not all clients on the network are
necessarily intended to receive the broadcast).
Encoder 24 and server 26 typically comprise high-cost,
dedicated computer systems, such as a Sun Station
(produced by Sun Microsystems) or a Windows NT server,
running suitable RealSystem 5.0 software (produced by
RealNetworks Inc., Seattle, Wash.). These dedicated systems are required in order to ensure that the data stream is
distributed and received by clients 30 in real time. Similarly,
host 22 must typically be connected directly to encoder 24
by a high-speed data link or LAN, and not via the Internet
or other narrowband network. Therefore, real-time broadcasting is normally possible only for hosts having a suitable,
dedicated encoder and broadcast server and cannot be
offered by Internet service providers (ISPs) to their general
clientele.
SUMMARY OF THE INVENTION
It is an object of some aspects of the present invention to
provide substantially continuous, high-bandwidth data
streaming over a network using common, existing server
and network infrastructure.
It is a further object of some aspects of the present
invention to provide data broadcasting capability, particularly for multimedia data, without the need for a dedicated
broadcast computer system.
It is a further object of some aspects of the present
invention to provide apparatus and methods for data broadcasting at reduced cost by comparison with systems known
in the art.
It is still another object of some aspects of the present
invention to enable a personal computer to remotely broadcast a multimedia program through an Internet service
provider (ISP) using common, universally-supported Internet communication protocols.

2
In preferred embodiments of the present invention, a
transmitting computer generates a data stream and broadcasts the data stream via a network server to a plurality of
clients. The data stream is divided into a sequence of
5 segments or slices of the data, preferably time slices,
wherein the data are preferably compressed. Each slice is
preferably assigned a respective slice index. The transmitting computer uploads the sequence of slices to the server
substantially in real time, preferably using an Internet
to protocol, most preferably the File Transfer Protocol (FTP),
as is known in the art. The clients download the data stream
from the server, preferably using an Internet protocol, as
well, most preferably the Hypertext Transfer Protocol
(HTTP), or alternatively, using other protocols, such as UDP
15 or RTP, which are similarly known in the art. The clients use
the slice indices of the frames to maintain proper synchronization of the playback. The division of the data stream into
slices and the inclusion of the slice indices in the data stream
to be used by the clients in maintaining synchronization
zo allows the broadcast to go on substantially in real time
without the use of special-purpose hardware.
Preferably, each segment or slice is contained in a
separate, respective file. Alternatively, the segments or slices
may all be contained in a single indexed file, which is
25 streamed to the client in a series of packets, each covering
a range of one or more indices. Hi -IT version 1.1 supports
this sort of file streaming. Other protocols may also be used
for this purpose.
In some preferred embodiments of the present invention,
30
the data stream comprises multimedia data captured or
generated by the transmitting computer. The term "multimedia" as used in the context of the present patent application and in the claims refers to images or sound or to data
representative of images or of sound or a combination
35
thereof. Multimedia image data may include still images,
video, graphics, animation or any combination thereof,
including text displayed in conjunction therewith. It will be
appreciated, however, that the principles of the present
invention may similarly be applied to streaming of other
40
data types.
Preferably, the transmitting computer compresses the
frames in the data stream, most preferably using methods of
image and audio compression such as those described in
U.S. patent application Ser. No. 08/919,027, which is
45
assigned to the assignee of the present patent application and
incorporated herein by reference. Alternatively, any suitable
methods of compression known in the art may be used. The
compressed data are conveyed to the server and thence to the
clients, which decompress the data.
50 In some preferred embodiments of the present invention,
the transmitting computer and the clients monitor the
uploading and downloading of data to and from the server,
respectively, in order to determine the amount of time
55 required to convey each slice and to verify that the slices are
conveyed at a sufficient rate. When the data stream comprises multimedia data, the data rate should be generally
equal to or faster than the rate at which the data are generated
at the transmitting computer.
In some of these preferred embodiments, the transmitting
60
computer and/or the clients each open a plurality of FTP or
H YIP links, respectively, with the network server. The slices
are transferred over different ones of the links in alternation.
Although typically none of the plurality of links has suffi65 cient bandwidth on its own to convey the entire data stream
in real time, the combined bandwidths of the plurality of
links are generally sufficient for this purpose. Preferably,

US 6,389,473 B1
3

4

each of the links is monitored to determine its specific data
transfer rate. If the transfer rate of any of the links is below
a predetermined minimum, that link is preferably closed,
and a new link is opened in its place.
In other preferred embodiments, the slices are provided by
the server at multiple resolution or quality levels. Each such
level has a different degree of data compression, and thus
corresponds to a different data bandwidth requirement. The
client or the server monitors the data transfer rate of a data
link opened therebetween and selects the level that is
appropriate to the link bandwidth. If the monitored data
transfer rate changes during transmission, the quality level is
preferably reselected accordingly.
Preferably, the transmitting computer monitors the bandwidth of the data stream that it is uploading to the server, and
compares the data stream bandwidth to a known or estimated bandwidth of the link or links between the transmitting computer and the server. The transmitting computer
preferably compresses the data stream at a compression ratio
that is adjusted so as to match the data stream bandwidth to
the available link bandwidth, using methods described, for
example, in the above-mentioned U.S. patent application
Ser. No. 08/919,027.
There is therefore provided, in accordance with a preferred embodiment of the present invention, a method for
real-time broadcasting from a transmitting computer to one
or more client computers over a network, including:
providing at the transmitting computer a data stream
having a given data rate;
dividing the stream into a sequence of slices, each slice
having a predetermined data size associated therewith;
encoding the slices in a corresponding sequence of files,
each file having a respective index; and
uploading the sequence to a server at an upload rate
generally equal to the data rate of the stream, such that
the one or more client computers can download the
sequence over the network from the server at a download rate generally equal to the data rate.
Preferably, dividing the stream into the sequence of slices
includes dividing the stream into a sequence of time slices,
each having a predetermined duration associated therewith.
Preferably, uploading the sequence includes comparing
the upload rate to the data rate and adjusting the upload rate
responsive to the comparison. Further preferably, encoding
the stream includes compressing data in the stream at a
desired compression ratio, and adjusting the upload rate
includes changing the compression ratio. Alternatively or
additionally, adjusting the upload rate includes adjusting the
size of one or more of the slices.
Preferably, uploading the sequence includes opening a
plurality of file transfer links between the transmitting
computer and the server, each link characterized by a
respective link data rate, and uploading different files in the
sequence over different ones of the plurality of links. Further
preferably, opening the plurality of links includes opening
links such that the data rates of the links taken together are
sufficient to upload the sequence at the upload rate generally
equal to the data rate.
Preferably, uploading the sequence includes uploading a
sequence using an Internet Protocol, most preferably using
FTP.
Preferably, the method includes downloading the
sequence using an Internet protocol, most preferably H1'1 P,
or alternatively, UDP or RTP, over the network from the
server to the one or more client computers. Preferably, the
one or more client computers decode the sequence and play

back the data stream responsive to the indices of the files, at
a replay rate generally equal to the data rate.
Preferably, uploading the sequence includes uploading
and updating an index file containing the index of the file in
the sequence that was most recently uploaded, and the one
or more client computers read the index file to play back the
sequence. In a preferred embodiment, downloading the
sequence includes selecting a file in the sequence earlier
than the file whose index is contained in the index file and
downloading at least a portion of the sequence of files
beginning with the selected file.
Preferably, the one or more client computers include a
plurality of client computers, and downloading the sequence
includes downloading to the plurality of client computers
substantially simultaneously.
Preferably, downloading the sequence includes opening a
plurality of download links between one of the client computers and the server, each link characterized by a respective
link data rate, and downloading different files in the
sequence over different ones of the plurality of links. Most
preferably, opening the plurality of links includes opening
links such that the data rates of the links taken together are
sufficient to download the sequence at the download rate
generally equal to the data rate.
Preferably, opening the plurality of links includes monitoring the data rates of the links and opening a new link in
place of one of the links having a data rate lower than a
predetermined level.
In one preferred embodiment, opening the new link
includes retransmitting at least one of the files in the
sequence, wherein the at least one of the files was incompletely transmitted over the one of the links having the data
rate lower than the predetermined level.
In another preferred embodiment, opening the new link
includes dropping at least one file out of the sequence,
wherein the at least one of the files was incompletely
transmitted over the one or more of the links having the data
rate lower than the predetermined level.
In still another preferred embodiment, encoding the slices
includes encoding slices at a plurality of different quality
levels, such that the files corresponding to a given one of the
slices have a different, respective data size for each of the
quality levels. Preferably, downloading the sequence
includes determining a data bandwidth of the network
between the server and the client computer and selecting one
of the quality levels responsive to the determined bandwidth.
Preferably, the data stream includes multimedia data.
There is further provided, in accordance with a preferred
embodiment of the present invention, apparatus for real-time
broadcasting of a data stream having a given data rate over
a network, including:
a transmitting computer, which divides the stream into a
sequence of slices, each slice having a predetermined
data size associated therewith, and encodes the slices in
a corresponding sequence of files, each file having a
respective index, and
which uploads the sequence to a server at an upload rate
generally equal to the data rate, such that one or more
client computers can download the sequence over the
network from the server at a download rate generally
equal to the data rate.
Preferably, the transmitting computer compares the
upload rate to the data rate and adjusts the upload rate
responsive to the comparison. Most preferably, the transmitting computer compresses the data at a compression ratio
which is varied responsive to the comparison. Additionally

s

10

15

20

25

30

35

40

45

so

55

60

65

US 6,389,473 B1
6

5
or alternatively, the transmitting computer adjusts the size of
one or more of the slices responsive to the comparison.
Preferably, the transmitting computer opens a plurality of
links between the transmitting computer and the server, each
link characterized by a respective data rate, and transmits 5
different ones of the sequence of files over different ones of
the plurality of links. Most preferably, the transmitting
computer opens the plurality of links such that the data rates
of the links taken together are sufficient to upload the
sequence at the upload rate generally equal to the data rate. to
Further preferably, the transmitting computer monitors the
data rates of the links and opens a new link in place of one
of the links whose data rate is lower than a predetermined
level.
In a preferred embodiment, the slices are encoded at a is
plurality of different quality levels, such that the files corresponding to a given one of the slices have a different,
respective data size for each of the quality levels.
Preferably, the transmitting computer uploads the
sequence using an Internet upload protocol, most preferably 20
Preferably, the one or more client computers decode the
sequence and play back the data stream responsive to the
indices thereof, at a data replay rate generally equal to the
data rate. Preferably, the one or more client computers 25
download the encode sequence using an Internet download
protocol, most preferably HTTP or alternatively, UDP or
RTP.
Preferably, the one or more client computers include a
plurality of client computers, which download the sequence 30
substantially simultaneously.
Preferably, the network includes the Internet.
Further preferably, the data stream includes multimedia
data, and the predetermined data size of each of the slices
35
corresponds to a time duration of the slice.
The present invention will be more fully understood from
the following detailed description of the preferred embodiments thereof, taken together with the drawings in which:
BRIEF DESCRIPTION OF ME DRAWINGS
FIG. 1 is a schematic illustration of a computer broadcast
network, as is known in the art;
FIG. 2 is a schematic illustration of a computer broadcast
network, in accordance with a preferred embodiment of the
present invention;
FIG. 3A is a block diagram that schematically illustrates
a data structure of a broadcast sequence, in accordance with
a preferred embodiment of the present invention;
FIG. 3B is a block diagram that schematically illustrates
an index file associated with the data structure of FIG. 3B,
in accordance with a preferred embodiment of the present
invention;
FIG. 3C is a schematic illustration of a user interface
graphic, for use in conjunction with the data structure of
FIG. 3A, in accordance with a preferred embodiment of the
present invention;
FIG. 3D is a block diagram that schematically illustrates
a data structure of a broadcast sequence, in accordance with
another preferred embodiment of the present invention;
FIG. 4 is a block diagram that schematically illustrates a
network connection between a transmitting computer and a
network server, for use in broadcasting of a data sequence,
in accordance with a preferred embodiment of the present
invention;
FIG. 5 is a flow chart that schematically illustrates a
method of uploading broadcast data from a transmitting

40

45

50

SS

60

65

computer to a server, in accordance with a preferred embodiment of the present invention;
FIG. 6A is a flow chart that schematically illustrates a
method of downloading broadcast data from a server to a
client, in accordance with a preferred embodiment of the
present invention;
FIG. 6B is a flow chart that schematically illustrates a
method of downloading broadcast data from a server to a
client, in accordance with another preferred embodiment of
the present invention;
FIG. 7 is a flow chart that schematically illustrates a
method for preparing data ffies for transmission, in accordance with a preferred embodiment of the present invention;
FIG. 8 is a flow chart that schematically illustrates a
method of file transfer, in accordance with a preferred
embodiment of the present invention; and
FIG. 9 is a flow chart that schematically illustrates a
method for monitoring network links, in accordance with a
preferred embodiment of the present invention.
DETAILED DESCRIPTION OF PREFERRED
EMBODIMENTS
Reference is now made to FIG. 2, which is a schematic
illustration of a computer system 32 for remote broadcasting
of a multimedia sequence over a network 28, in accordance
with a preferred embodiment of the present invention.
System 32 comprises a transmitting computer 34, which
generates the sequence, a plurality of clients 30, and a
network server 36, all of which communicate over network
28, preferably using the well-known Internet Protocol (IP).
Computer 34 preferably receives audiovisual input from
input devices 22, although data inputs of other types may be
generated at or by computer 34 using any suitable means
known in the art.
Network 28 preferably comprises the Internet, although it
may equally comprise a LAN, WAN, intranet or other
computer network as is known in the art. Computer 34 and
clients 30 preferably comprise conventional personal computers or workstations. Server 36 may comprise any suitable
type of computer or computer system, for example, a Sun
Microsystems UltraSPARC station or a Windows NT server,
as are commonly used by Internet Service Providers (ISPs)
. In any case, it is noted that transmitting computer 34 can
be remotely located relative to server 36, and that the server
need not be equipped with any special-purpose hardware or
software for real-time data broadcasting, unlike broadcast
systems known in the art, such as the real-time encoder and
broadcast server shown in FIG. 1.
After preparing the multimedia sequence, computer 34
uploads the sequence over network 28, preferably using the
Internet File Transfer Protocol (FTP). Alternatively, other
Internet protocols may be used, such as the TCP/IP, UDP or
RT(x) protocols, which are known in the art. Preferably, the
data in the sequence are compressed, although compression
is not essential to implementation of the present invention.
The sequence is preferably generated and compressed in real
time, and could comprise, for example, an interview program or an entertainment or sports event, although a prerecorded sequence may similarly be broadcast in this manner.
Computer 34 is preferably equipped with suitable software
for preparing and compressing the multimedia sequence. For
example, for audio data, the computer may typically run
GSM 6.10 standard audio compression software, operating
at a sample rate of 8 kHz, with 16 bits/sample. Some useful
techniques for preparing, compressing and transmitting multimedia sequences are described in U.S. Pat. No. 5,841,432

US 6,389,473 B1
7

8

comprises a string followed by the index of the file. When
and in the above-mentioned U.S. patent application Ser. No.
one of computers 30 connects to server 36 and begins to
08/919,027, both of which are incorporated herein by refdownload the data stream, it first reads the index file in order
erence.
to identify at what point in stream 40 to begin and to start
Clients 30 connect to server 36 and receive the multimedia sequence, substantially in real time. Clients 30 prefer- 5 receiving the data stream substantially in real time, preferably with only a minimal lag, as it is transmitted from
ably download the sequence using the Hypertext Transfer
computer 34. Alternatively, a user of one of computers 30
Protocol (HTTP), although other Internet protocols may also
may choose to begin downloading data stream 40 from an
be used, such as UDP or RTP, as noted hereinabove with
earlier point in time than that indicated by ID 52. Further
reference to uploading by computer 34. Since FTP and
alternatively, stream 40 may be multicast to clients 30, as is
HI IP are supported by substantially all network servers, 10 known in the art, typically without the use of an index file.
server 36 need not include any special-purpose broadcasting
Index file 50 may further include a message 54, which is
hardware or software, as noted above. Similarly, because
read by computers 30 when they connect to server 36 to
HTTP is supported by substantially all modern Web
download data stream 40 or, alternatively or additionally, at
browsers, clients 30 will typically need only add a Java
any time the message is updated by computer 34. The
15
applet or plug-in to their existing Web browsers, as
message contains parameters relating generally to the data
described further hereinbelow, in order to receive and play
stream and/or instructions to computers 30, for example,
back the broadcast.
"transmission paused." FIG. 3C is a schematic representaFIG. 3A is a block diagram that schematically illustrates
tion of a user interface graphic "slider" 55, available to users
the structure of a stream of broadcast data 40 produced by
of computers 30, in accordance with a preferred embodiment
20
computer 34, typically corresponding to a multimedia data
of the present invention. Slider 55, which is preferably
sequence, in accordance with a preferred embodiment of the
displayed on the screens of computers 30, includes a bar 56
present invention. Data stream 40 comprises a series of data
and a movable indicator 58. The symbols 3, 3+1, 3+2, . . . N
slices 42, 44, 46, 48, etc. Each slice contains a segment of
in the figure are the indices of the slices of stream 40 that are
video and/or audio data, corresponding to a respective,
stored on server 36, wherein N is the index of the most
successive time interval labeled T„ T 2, T3, etc. The data are 25 recent slice, and J is the index of the earliest stored slice. J
preferably compressed, as described further hereinbelow.
may indicate the first slice in the sequence, if all of the files
Computer 34 stores each slice as a corresponding file,
are stored on server 36, or it may be the earliest file not yet
having a running slice index 1, 2, 3 . . . N. Preferably, each
erased. (The indices are marked in the figure on bar 56 for
file also includes one or more time stamps, indicating a real
clarity, and need not actually be shown on the computer
time at which the data in the file were recorded or an elapsed 30 screen.)
time relative to the beginning of stream 40. The files are
When one of computers 30 reads index file 50 and begins
uploaded to server 36, such that while any given slice (other
to download stream 40, indicator 58 preferably marks the
than first slice 42) is being created, one or more preceding
most recent slice, as shown in FIG. 3C. This is the point at
slices are in the process of being uploaded.
35 which the download will begin, unless the user of the
Computer 34 monitors the time codes as file 40 is
computer chooses otherwise. If the user wishes to begin the
transmitted, and clients 30 similarly monitor the time codes
download at an earlier point, he may move indicator 58 to
as the file is received, in order to ensure that the transmission
the left along bar 56 to that point, preferably using a mouse
or reception is "keeping up" with the input of the data to the
or other pointing device, as is known in the art. Indicator 58
computer. In the event that a lag is detected, steps are taken 40 may be moved back and forth along bar 56 to jump back and
to increase the data transmission or reception rate, as
forth along stream 40.
described further hereinbelow. For example, as shown in
FIG. 3D is a block diagram that schematically illustrates
FIG. 3A, time intervals T„ '1' 2, '1'3, etc., are not all equal, but
a file format of a multi-level data stream 41, in accordance
rather are adjusted by computer 34 in response to the
with another preferred embodiment of the present invention.
transmission rate. Alternatively or additionally, the compres- 45 The data stream is divided into audio slices 45, 47 and video
sion level of the data is varied, as is likewise described
slices 49, 51, and may also include other data formats, such
below, so as to adjust the data streaming rate to the available
as a text slice 53 and/or a URL slice 55. Each slice is
bandwidth over one or more channels between computer 34
preferably identified by a level identifier 57, a presentation
and server 36, and/or between server 36 and client 30.
time stamp (PTS) index 59 and, as appropriate, a size
Computer 34 continues to upload files 42, 44, 46, etc., so identifier 61. The function of these identifiers and indices is
until data stream 40 is finished or terminated by a user of
described further hereinbelow. A header 43 includes data
computer 34. All of the files in the data stream may be saved
such as the title, author, copyright and formats of the data in
on server 36 for any desired period of time, as long as the
the stream; the duration of the multimedia sequence represerver has sufficient free memory that is accessible to
sented by the stream; and a description of the available
computer 34. Typically, however, the memory available on 55 stream levels and associated data sizes.
server 36 is limited, and files 42, 44, 46, etc., will be stored
Each time slice in stream 41 includes multimedia data at
on the server and erased therefrom in a "first-in-first-out"
multiple quality levels. There are two such levels in the
sequence.
example shown in FIG. 3D, identified as level #1 and level
FIG. 3B is a block diagram that schematically illustrates
#2, but a larger number of levels may also be used. Typically,
an index file 50, which is created by computer 34, and is 60 the audio and video data in level #1, contained in slices 45
uploaded to server 36, in accordance with a preferred
and 49, are more highly compressed relative to the data in
embodiment of the present invention. The index file comslices 47 and 51 of level #2. In consequence, the level #1
prises a slice ID 52, indicating the index of the file in data
slices have smaller data volume than the level #2 slices and
stream 40 that was most recently uploaded by computer 34.
can therefore be transmitted over a lower-bandwidth data
Each time a new file 42, 44, 46, etc., is uploaded, ID 52 in 65 link, while maintaining the required slice timing indicated
file 50 on server 36 is updated. Preferably, ID 52 holds the
by time stamps 59. The lower data-rate transmission generfile name of the new file, wherein the name typically
ally comes at the expense of inferior sound and/or image

US 6,389,473 B1

9

10

files 42, 44, 46, 48, etc., as shown in FIG. 3A. Computer 34
quality. Size identifier 61 describes the size of those slices in
conveys file 40 to server 36 over links 60, 62, 64, 66 and 68,
stream 41 that have a fixed size associated therewith,
as described above, preferably using FTP, at step 84. Each
wherein typically the size (or the corresponding resolution)
time a new file is uploaded to the server, index file 50 (FIG.
of the level #1 video slices is smaller than that of the level
5 3B) is updated, at step 86.
#2 slices.
For each of the one or more links that is open, computer
Each of clients 30 chooses or is assigned the quality level
34 checks the link function at step 88, preferably by timing
appropriate to the bandwidth of its link on network 28 to
the transfer of the respective files being transmitted over the
server 36. A method for selecting and, as required, varying
link. In the event that one of the links, for example, link 60
the level is described hereinbelow with reference to FIG. 6B.
FIG. 4 is a block diagram that schematically illustrates 10 as shown in FIG. 4, is not responding or is responding
unacceptably slowly, computer 34 breaks the link. The file
communication between computer 34 and server 36 over
being transmitted over the broken link may be retransmitted
network 28, in accordance with a preferred embodiment of
over another link. Alternatively, the file may be dropped
the present invention. Computer 34 should preferably ensure
from the transmission, particularly in the case of a real-time
that there is sufficient communication bandwidth between
the computer and the server, particularly when network 28 15 multimedia broadcast, in which data arriving at one of
computers 30 will generally tend to disrupt reception of the
includes an Internet connection, which may be slow and
broadcast. When link 60 is broken, a new link, for example,
subject to bottlenecks. For this purpose, the computer preflink 70, is opened in its stead, as described further hereinerably opens a plurality of links, each link having its own
below with reference to FIG. 8. Alternatively or additionally,
URL (Uniform Resource Locator). Preferably, five links 60,
62, 64, 66 and 68 are opened and operate simultaneously 20 the encoding/quality level (step 80) or slicing (step 82) of the
data may be adjusted, as described hereinbelow with referover a single modem line. Alternatively, each link may be
ence to FIG. 7. This process continues until the broadcast
routed differently from the other links, over a different
program is completed.
telephone line and/or through different Internet nodes. Files
FIG. 6A is a flow chart illustrating the operation of clients
42, 44, 46, 48, etc., in stream 40 are transmitted respectively
over links 60, 62, 64, 66 and 68 in successive alternation, so 25 30 in downloading and playing back data stream 40 (FIG.
3A) transmitted by computer 34, in accordance with a
that at any given time (except at the very beginning of the
preferred embodiment of the present invention. The operasequence) up to five files are transmitted in parallel.
tion of client is controlled by a Java applet, which may be
Alternatively, more than five links may be opened, so that
downloaded from server 36, and includes facilities for
more than five files may accordingly be transmitted in
30
carrying out the steps shown in FIG. 6A, as well as for error
parallel.
detection and, optionally, correction in communications
Preferably, computer 34 monitors the rate of data being
received by the clients and for other functions known in the
transmitted over each of links 60, 62, 64, etc., and allocates
art. A sample applet of this sort is incorporated herein in the
files 42, 44, 46, 48, etc., according to the data rates. The sizes
of the files may be varied by adjusting slice durations T1 , T2, 35 software appendix, as further described hereinbelow.
T3, etc., and a relatively greater volume of data may be
Each client 30 connects to server 36, optionally using
transmitted through links exhibiting relatively greater data
multiple HITP links, in a manner similar to that shown and
rates. The bandwidth open for transmission between comdescribed above with reference to FIG. 4. Typically, client
puter 34 and server 36 is effectively roughly equal to a sum
30 opens one or two H flP links, over which files 42, 44, 46,
of the bandwidths of the plurality of open links. The number 40 etc., are downloaded in successive alternation, but as in the
of links that are actually opened between computer 34 and
case of transmitting computer 34, a greater number of links
server 36 may be less than or greater than the five links
may similarly be opened. The client first reads index file 50
shown in the example of FIG. 4, depending on the available
(FIG. 3B), and graphic 56 (FIG. 3C) is displayed by the
data rates of the open links, compared with the rate of data
client, so that a user can decide and indicate at which slice
in stream 40. Preferably at least two links are opened, so that 45 of data stream 40 to begin downloading. Responsive to a
preparation and transmission of files 42, 44, 46, 48, etc., may
user input, client 30 selects an appropriate starting slice and
be toggled back and forth between the links. A similar
begins to download and decode (decompress) files 42, 44,
technique is preferably employed by clients 30.
46, etc. In the case of a multimedia stream, client 30
reconstructs and outputs the multimedia data for the appreAlternatively or additionally, the link bandwidth may be
accommodated using the multi-level transmission concept 50 ciation of a user. Time stamps in the data stream are used to
synchronize the data, so that the multimedia sequence is
illustrated by FIG. 3D, over either a single link or over
played back just as it was input at computer 34, preferably
multiple simultaneous links with server 36.
with only a minimal necessary transmission and decoding
FIG. 5 is a flow chart that schematically shows an
delay.
overview of operations of computer 34 in preparing and
transmitting data stream 40 over network 28, in accordance 55 Client 30 preferably monitors the rate of data coming in
over each of its links with server 36. If any of the links is
with a preferred embodiment of the present invention.
non-operative or is operating unacceptably slowly, that link
Details of some of the steps in the operation are shown in
is closed, and a new link is opened in its place, as described
FIGS. 7-9 and described further with reference thereto.
above. Further preferably, the client compares the times
Exemplary programs for carrying out the functions illustrated in FIG. 5 are incorporated herein in a software 60 stamped in the data stream to a local real-time clock and, if
it determines that there is a significant lag in the time codes
appendix, which is described further hereinbelow.
relative to the real-time clock, opens additional links with
To begin the broadcast, computer 34 connects to server
server 36 in order to increase the overall data rate.
36, optionally opening the plurality of links shown in FIG.
FIG. 6B is a flow chart illustrating the operation of clients
4. Broadcast data are then input to the computer, for
example, from input devices 22, or from a video, audio or 65 30 in downloading and playing back multi-level data stream
animation sequence stored on disk or tape. The data are
41 (FIG. 3D) transmitted from server 36, in accordance with
another preferred embodiment of the present invention. As
compressed at step 80, and are then "sliced" at step 82 into

US 6,389,473 B1

11

12

upload file 42 is measured and compared to T„ at the same
in the method of FIG. 6A, each client 30 connects to the
time as file 44 (slice 2) is being encoded and prepared.
server, generally using a single HTTP link. After reading
Responsive to this measurement of upload time, the duration
header 43 and, preferably, making an initial assessment of
of subsequent slices, for example, times T, and 1' 4 for files
the link bandwidth, the client selects one of the available
quality levels in the stream. Responsive to the selection, 5 46 and 48, respectively, is adjusted. Thus, as illustrated in
FIG. 3A, T, and T4 are less than T, and T2. The shorter files
server 36 begins to transmit data slices at the chosen quality
46, 48, etc., that result from the change in slice duration are
level. The slices are received, decoded and output by the
more likely to reach server 36 in the proper sequence,
client.
without being held up by extended bottlenecks.
Periodically, client 30 makes an assessment of the rate of
data transfer over the link from the server and, if necessary, to Furthermore, when the slice durations are shorter, the effect
of "drop-out" of a slice due to failure of the corresponding
changes the quality level accordingly. For example, if the
link is less marked.
rate is low, such that time stamps 59 indicate that the slices
On the other hand, if it is determined that the upload time
need to be played as fast as or faster than they are being
for ffie 42 (or a subsequent file) is substantially shorter than
received, the client will preferably select a lower quality
level if one is available. On the other hand, if the rate is 15 duration T„ the duration of subsequent files may be
extended, and/or the compression ratio may be decreased, so
substantially higher than what is needed to receive the
as to take better advantage of the available bandwidth.
successive slices on time, the client may select a higher
quality level to take advantage of the available bandwidth.
FIG. 8 is a flow chart that schematically illustrates details
Preferably, upper and lower data rate thresholds, or
of 1,1? step 84 in the method shown in FIG. 5, in accordance
watermarks, are set dynamically in response to the data rate 20 with a preferred embodiment of the present invention. To
and are used in determining when a new quality level should
begin the process of FTP transfer to server 36, a FTP OPEN
be selected.
command is issued to the server by computer 34 at step 94,
thus opening link 60 (FIG. 4). The computer then issues a
FIG. 7 is a flow chart that schematically illustrates details
FTP SEND command to send file 42, corresponding to slice
of encoding step 80 and slicing step 82 in the method of FIG.
5, in accordance with a preferred embodiment of the present 25 1, over the link. These steps are repeated for each of a first
group of links 62, 64, 66 and 68 and for a corresponding first
invention. In encoding data stream 40, computer 34 prefergroup of files 44, 46, 48, etc., up to a maximum initial
ably compresses the data using any suitable compression
number of links .1. In the current example, J-5, since
method known in the art. For example, if data stream 40
the inventors have found that five 1 ,1? links provide a
comprises audio data, GSM 6.10 standard encoding may be
used, as is known in the art, to compress the data by about 30 reliable upload path with a sufficient overall bandwidth
when transferring data over the Internet. A larger or smaller
10:1. Alternatively or additionally, for video data, 11.263
number of links could be used, however, and Jm,, may also
standard compression, similarly known in the art, may be
be assigned the value 1, in which case the steps of the
used. These compression standards are advantageous in that
method of FIG. 5, and the details thereof shown in FIGS. 7,
common personal computers can perform such compression
in real time, in parallel with the other operations illustrated 35 8 and 9, are carried out over a single FIV link.
in FIG. 5. Other compression methods known in the art, such
As described above, links 60, 62, 64, 66 and 68 continue
as MPEG data compression, may similarly be used, as long
to operate until and unless it is determined that one of the
as computer 34 is sufficiently powerful.
links (for example, link 60 in FIG. 4) is operating at an
Computer 34 determines a compression ratio by which to 40 unacceptably slow data transfer rate. Assuming all of the
links to be functioning properly, after the first five slices of
compress the data, based on the collective bandwidth of its
stream 40 have been allocated in succession to the five links,
open links with server 36. Preferably, computer 34 receives
a file corresponding to a sixth slice of the stream is then
an indication of the bandwidths of the links, determined at
allocated again to link 60, or to whichever of the open links
step 88 in FIG. 5, and adjusts the compression ratio
accordingly, at a set compression step 90. For example, the 45 was the first to have completed transmission of its initial file.
This process continues in alternation for all of the slices of
compression ratio may be adjusted by changing compression
stream 40, until transmission is completed.
coefficients (e.g., MPEG coefficients) so as to match the data
stream bandwidth to the available link bandwidth. Methods
If link 60 has not completed transmission of file 42 by the
of adaptively varying the compression ratio of a multimedia
time the sixth file is ready for transmission, link 60 will have
data stream that can be used for this purpose are described, so timed out, and a time-out indication will be received from
for example, in the above-mentioned U.S. patent application
step 88 (FIG. 5). In this case, link 60 is terminated and is
Ser. No. 08/919,027.
replaced by link 70. Preferably, a "socket" opened for link
60 by a WINSOCK program running on computer 34 is
Similarly, at a set duration step 92, slice durations T„ T 2,
simply reinitialized to open link 70. Optionally, file 42 is
T,, etc., are optionally adjusted responsive to the link
bandwidths. Initially, duration T, of slice 1 for file 42 is set 55 retransmitted over link 70 or over one of the other links,
although in the case of a live broadcast transmission, it may
to a default value, typically between 1 and 5 sec. For
be preferable simply to drop the file rather than send it after
example, to transfer compressed audio data at 2 Kbytes/sec,
such a long delay.
file 42 may be assigned a file size of 10 Kbytes, with T 1 =5
sec. Assuming that computer 34 communicates over netFIG. 9 is a flow chart that schematically illustrates details
work 28 through a 28.8 Kbaud modem and maintains a 60 of check link step 88 in the method of FIG. 5, in accordance
typical FTP upload rate of 2 Kbytes/sec (allowing for
with a preferred embodiment of the present invention. As
moderate Internet bottlenecks), data stream 40 will be
noted above, for each file 42, 44, 46, etc., computer 34
uploaded to server 36 over link 60 (FIG. 4) substantially at
measures a slice transmission time T„, corresponding to the
the rate that the audio data are input to computer 34.
time required to transmit the entire file to server 36. If T„
Frequently, however, this will not be the case, and the FTP 65 is geater than a maximum permissible time TMAX, it is then
determined that the link over which the file was transmitted
upload rate over link 60 will fluctuate and may be slower
is not functioning adequately. In this case, a command is sent
than 2 Kbyte/sec. At step 88 (FIG. 5), the time required to

US 6,389,473 B1
14

13
to open a new FTP link at step 94, as described above. T ikm,
is preferably set to be a predetermined multiple of T„,
depending on the length of possible transmission delay that
can be tolerated. Typically, TmAx is set to an initial value of
about 20 sec, although when the slice durations are changed 5
(at step 92), TmAx is preferably adjusted accordingly.
For optimal, reliable functioning of the upload process
from computer 34 to server 36, T„ should desirably be close
to or less than a predetermined minimum time Tmm.
Typically, TM is set to be approximately equal to the slice 10
duration T„ T2 , etc., i.e., about 5 sec initially. If the
measured value of 1151. is greater than TA0N, although still
less than TmAg, then it will generally be desirable to either
increase the compression ratio, at step 90, or decrease the
slice duration, at step 92, or both. The reasons and methods 15
for changing the compression ratio and slice duration were
described hereinabove with reference to FIG. 7. Preferably,
computer 34 calculates optimal values of the compression
ratio and slice duration, depending inter alia on the relative
20
values of Tsy, and TA,IIN.
If Tsy is substantially less than Te„N, then the slice
duration may optionally be increased, as noted hereinabove.
The process shown in FIG. 5, including the interdependent steps of encoding 80, slicing 82, 1- ,T1? upload 84, 25
updating 86 and checking link function 88 thus continues
until the entire data stream 40 is uploaded (except for any of
files 42, 44, 46, 48, etc., that may be dropped due to
excessive transmission delay, as described above), or until
the transfer is terminated by a user of computer 34. Although 30
details of these steps have been described primarily with
reference to the uploading process of FIG. 5, it will be
understood that similar methods are applicable, mutatis
mutandis, to the method of downloading the files from
server 36 to clients 30, as shown in FIG. 6A.
35
Although the preferred embodiments described hereinabove refer primarily to broadcasting multimedia data over
the Internet, in other preferred embodiments of the present
invention, network 28 may comprise substantially any suitable sort of network, such as a LAN, WAN or intranet, and 40
the data may comprise substantially any type of continuously streaming data. It will be understood in this case that
the slices of the data stream corresponding to files 42, 44, 46,
etc., will not necessarily be time slices as described
hereinabove, but may rather have an appropriate, preferably 45
variable, data size associated therewith. Furthermore, it will
be appreciated that the principles of the present invention
may similarly be applied in other areas of real-time multimedia data streaming, such as video teleconferencing.
It will thus be understood that the preferred embodiments 50
described above are cited by way of example, and the full
scope of the invention is limited only by the claims.
Software Appendix
An appendix is attached hereto as an integral part of the
present patent application, including the following
computer-readable files, which exemplify aspects of the
operation of system 32 (FIG. 2) and of the file structures and
methods described hereinabove. This appendix contains
material covered by copyright, belonging to Geo Interactive
Media Group Ltd.:
1. LiveFeedClient.exe
2. LFRec.dll
3. Jsdec.class
4. Emb lazeAudio. class
S. CyclicDecoderStream.class

55

60

65

6. RandomAccessStream.class
7. Xbuttons.jpg
8, ALF.HTML
9. Addresses.dat
The above files should be entered into a personal
computer, preferably a desktop PC (Pentium 166 MHz, with
32 MB RAM), with a Windows 95 or Windows NT operating system installed. The files should be stored on disk in
a common folder or directory, together with a Microsoft
Basic DLL file named Msvbvm50.d11, available from
Microsoft Inc., Redmond, Wash. The computer should be
equipped with a Sound Blaster sound card, with a microphone connected to the line-in input jack thereon. The
computer should also be connected to the Internet via a
standard modem or LAN, and should have an Fl? account
with an Internet Service Provider (ISP).
What is claimed is:
1. A method for real-time broadcasting from a transmitting computer to one or more client computers over a
network, comprising:
providing at the transmitting computer a data stream
having a given data rate;
dividing the stream into a sequence of slices, each slice
having a predetermined data size associated therewith;
encoding the slices in a corresponding sequence of files,
each file having a respective index; and
uploading the sequence to a server at an upload rate
generally equal to the data rate of the stream, such that
the one or more client computers can download the
sequence over the network from the server at a download rate generally equal to the data rate.
2. A method according to claim 1, and comprising downloading the sequence using an Internet protocol over the
network from the server to the one or more client computers.
3. A method according to claim 2, wherein downloading
the sequence comprises opening a plurality of download
links between one of the client computers and the server,
each link characterized by a respective link data rate, and
downloading different files in the sequence over different
ones of the plurality of links.
4. A method according to claim 3, wherein opening the
plurality of links comprises monitoring the data rates of the
links and opening a new link in place of one of the links
having a data rate lower than a predetermined level.
5. A method according to claim 4, wherein opening the
new link comprises retransmitting at least one of the files in
the sequence, wherein the at least one of the files was
incompletely transmitted over the one of the links having the
data rate lower than the predetermined level.
6. A method according to claim 4, wherein opening the
new link comprises dropping at least one file out of the
sequence, wherein the at least one of the files was incompletely transmitted over the one or more of the links having
the data rate lower than the predetermined level.
7. A method according to claim 3, wherein opening the
plurality of links comprises opening links such that the data
rates of the links taken together are sufficient to download
the sequence at the download rate generally equal to the data
rate.
8. A method according to claim 2, wherein the one or
more client computers decode the sequence and play back
the data stream responsive to the indices of the files, at a
replay rate generally equal to the data rate.
9. A method according to claim 8, wherein uploading the
sequence comprises uploading and updating an index file
containing the index of the file in the sequence that was most

US 6,389,473 B1
15

16

data size associated therewith, and encodes the slices in
recently uploaded, and wherein the one or more client
a corresponding sequence of files, each file having a
computers read the index file to play back the sequence.
10.A method according to claim 9, wherein downloading
respective index, and
the sequence comprises selecting a ffie in the sequence
which uploads the sequence to a server at an upload rate
earlier than the file whose index is contained in the index file
generally equal to the data rate, such that one or more
and downloading at least a portion of the encoded sequence
client computers can download the sequence over the
of files beginning with the selected file.
network from the server at a download rate generally
11.A method according to claim 2, wherein encoding the
equal to the data rate.
slices comprises encoding slices at a plurality of different
26. Apparatus according to claim 25, wherein the one or
quality levels, such that the files corresponding to a given to
more client computers decode the sequence and play back
one of the slices have a different, respective data size for
the data stream responsive to the indices thereof, at a data
each of the quality levels.
replay rate generally equal to the data rate.
12. A method according to claim II, wherein download27. Apparatus according to claim 26, wherein the one or
ing the sequence comprises determining a data bandwidth of
the network between the server and the client computer and is more client computers download the sequence using an
Internet download protocol.
selecting one of the quality levels responsive to the deter28. Apparatus according to claim 27, wherein the Internet
mined bandwidth.
download protocol is selected from a group consisting of
13.A method according to claim 2, wherein downloading
the sequence comprises downloading the sequence using a
HTTP,UDP and RTP.
protocol selected from a group consisting of HTTP, UDP and 20 29. Apparatus according to claim 26, wherein the one or
RTP.
more client computers comprise a plurality of client
14. A method according to claim 2, wherein the one or
computers, which download the sequence substantially
more client computers comprise a plurality of client
simultaneously.
computers, and wherein downloading the sequence com30. Apparatus according to claim 25, wherein the transprises downloading to the plurality of client computers 25 mining computer compares the upload rate to the data rate
substantially simultaneously.
and adjusts the upload rate responsive to the comparison.
15.A method according to claim 1, wherein uploading the
31. Apparatus according to claim 30, wherein the transsequence comprises comparing the upload rate to the data
mitting computer compresses the data at a compression ratio
rate and adjusting the upload rate responsive to the comwhich is varied responsive to the comparison.
30
parison.
32. Apparatus according to claim 30, wherein the trans16.Amethod according to claim 15, wherein encoding the
mitting computer adjusts the size of one or more of the slices
stream comprises compressing data in the stream at a desired
responsive to the comparison.
compression ratio, and wherein adjusting the upload rate
33. Apparatus according to claim 25, wherein the transcomprises changing the compression ratio.
17.A method according to claim 15, wherein adjusting the 35 mitting computer opens a plurality of links between the
transmitting computer and the server, each link characterupload rate comprises adjusting the size of one or more of
ized by a respective data rate, and transmits different ones of
the slices.
the sequence of files over different ones of the plurality of
18.A method according to claim 1, wherein uploading the
links.
sequence comprises opening a plurality of file transfer links
between the transmitting computer and the server, each link 40 34. Apparatus according to claim 33, wherein the transcharacterized by a respective link data rate, and uploading
mitting computer opens the plurality of links such that the
different files in the sequence over different ones of the
data rates of the links taken together are sufficient to upload
plurality of links.
the sequence at the upload rate generally equal to the data
19.A method according to claim 18, wherein opening the
ratc.
plurality of links comprises opening links such that the data 45 35. Apparatus according to claim 33, wherein the transrates of the links taken together are sufficient to upload the
mitting computer monitors the data rates of the links and
sequence at the upload rate generally equal to the data rate.
opens a new link in place of one of the links whose data rate
20. A method according to claim 18, wherein opening the
is lower than a predetermined level.
plurality of links comprises monitoring the data rates of the
36. Apparatus according to claim 25, wherein the data
links and opening a new link in place of one of the links 50 stream comprises multimedia data.
having a data rate lower than a predetermined level.
37. Apparatus according to claim 36, wherein the prede21.A method according to claim 1, wherein uploading the
termined data size of each of the slices corresponds to a time
sequence comprises uploading a sequence using an Internet
duration of the slice.
Protocol.
38. Apparatus according to claim 25, wherein the trans22. A method according to claim 21, wherein uploading 55 mining computer uploads the encoded sequence using an
the sequence comprises uploading a sequence using FTP.
Internet upload protocol.
23. A method according to claim 1, wherein dividing the
39. Apparatus according to claim 38, wherein the Internet
stream into the sequence of slices comprises dividing the
upload protocol comprises HP.
stream into a sequence of time slices, each having a prede40. Apparatus according to claim 25, wherein the slices
termined duration associated therewith.
60 are encoded at a plurality of different quality levels, such
24. A method according to claim 1, wherein the data
that the files corresponding to a given one of the slices have
stream comprises multimedia data.
a different, respective data size for each of the quality levels.
25. Apparatus for real-time broadcasting of a data stream
41.Apparatus according to claim 25, wherein the network
having a given data rate over a network, comprising:
comprises the Internet.
a transmitting computer, which divides the stream into a 65
sequence of slices, each slice having a predetermined

UNITED STATES PATENT AND TRADEMARK 01-4-, ICE

CERTIFICATE OF CORRECTION
Page 1 of 1

PATENT NO. : 6,389,473 B1
: May 14, 2002
DATED
INVENTOR(S) : Sharon Carmel et al.

It is certified that error appears in the above-identified patent and that said Letters Patent is
hereby corrected as shown below:

Title page,
Item [75], "Ginsberg" should read -- Ginzberg

Signed and Sealed this
Ninth Day of July, 2002

Attest:

JAMES E. ROGAN
Attesting Officer

Director of the United States Patent and Trademark Office