Apple, Inc. v. Motorola, Inc. et al

Filing 12

AMENDED COMPLAINT for Patent Infringement against Motorola Mobility, Inc., Motorola, Inc., filed by Apple, Inc.. (Attachments: #1 Exhibit A - '949 patent, #2 Exhibit B - '002 patent, #3 Exhibit C - '315 patent, #4 Exhibit D - RE '486 patent, #5 Exhibit E - '354 patent, #6 Exhibit F - '263 patent, #7 Exhibit G - '983 patent, #8 Exhibit H - '705 patent, #9 Exhibit I - '647 patent, #10 Exhibit J - '852 patent, #11 Exhibit K - '131 patent, #12 Exhibit L - '337 patent, #13 Exhibit M - '867 patent, #14 Exhibit N - '721 patent, #15 Exhibit O - '599 patent) (Peterson, James) [Transferred from Wisconsin Western on 12/1/2011.]

EXHIBIT F

111111111111111111111111111111111111111111111111111111111111111111111111111
US006343263Bl

United States Patent

(10)

Nichols et al.

(12)

(45)

(54)

OTHER PUBLICATIONS

REAL-TIME SIGNAL PROCESSING SYSTEM
FOR SERIALLY TRANSMITTED DATA

(75)

Inventors: James B. Nichols, San Mateo; John
Lynch, San Jose, both of CA (US)

(73)

Notice:

(21)

Filed:

(51)
(52)
(58)

Frankel, "DSP Resource Manager Interface & Its Role in
DSP Multimedia", May 1994.*

Appl. No.: 08/284,061

(22)

Tanenbaum, Structured Computer Organization, 1984, pp.
10-12.*

Assignee: Apple Computer, Inc., Cupertino, CA
(US)

( *)

Patent No.:
US 6,343,263 BI
Date of Patent:
Jan. 29,2002

Int. CI?
G06F 13/00
U.S. CI.
702/189; 709/328
364/514,238.5,
Field of Search
364/924, 576, 725, 726; 395/2.09, 2.1,
2.91, 2.94, 406, 651, 680, 682, 821, 892,
561, 566, 733; 455/84, 3.1, 39; 370/210;
340/825; 342/195, 196; 360/39

Silberschatz et aI., Operating System Concepts, p. 489,
1994.*

Subject to any disclaimer, the term of this
patent is extended or adjusted under 35
U.S.c. 154(b) by 0 days.

Jon Udell, "Computer Telephony," Byte, vol. 19, No.7, Jul.
1994, pp. 80-96.
* cited by examiner

Primary Examiner-Patrick Assouad
(74) Attorney, Agent, or Firm-Burns, Doane, Swecker &
Mathis, L.L.P.

Aug. 2, 1994

(56)

(57)

A data transmission system having a real-time data engine
for processing isochronous streams of data includes an
interface device that provides a physical and logical connection of a computer to anyone or more of a variety of
different types of data networks. Data received at this device
is presented to a serial driver, which disassembles different
streams of data for presentation to appropriate data managers. A device handler associated with the interface device
sets up data flow paths, and also presents data and commands from the data managers to a real-time data processing
engine. Flexibility to handle any type of data, such as voice,
facsimile, video and the like, that is transmitted over any
type of communication network with any type of real-time
engine is made possible by abstracting the functions of each
of the elements of the system from one another. This
abstraction is provided through suitable interfaces that isolate the transmission medium, the data manager and the
real-time engine from one another.

References Cited
U.S. PATENT DOCUMENTS
5,327,558 A
5,363,315 A
5,381,346 A

* 7/1994 Burke et al.
395/650
* 11/1994 Weiss et al.
364/514
* 1/1995 Monaham-Mitchell et al. .. 364/

5,406,643
5,438,614
5,440,619
5,440,740
5,442,789
5,487,167

*
*
*
*
*
*

A
A
A
A
A
A

4/1995
8/1995
8/1995
8/1995
8/1995
1/1996

Burke et al.
Rozman et al.
Cann
Chen et al.
Baker et al.
Dinallo et al.

514
395/200
379/93
379/97
395/650
395/650
395/650

FOREIGN PATENT DOCUMENTS

EP

218859

ABSTRACT

41 Claims, 6 Drawing Sheets

4/1987

32

u.s.

Patent

Jan. 29,2002

FIGURE 1

US 6,343,263 BI

Sheet 1 of 6

10

14

12

,
Comm.
Application

,

Application
Program

Comm./
Application
~

Fax Sender
Printer Driver

Fax
Terminal

-~34

Fax
Extension _~

18

32

/

,

Serial' Driver
Switch

r

------

~

- - - -

-----~

~

-

-

- -- -

Modem Logic
I Communications
Modem Tool
16
Toolbox
20
I
I
I
I
Control/Command Line Interpreter
I
Modem
Modem Application
I
Control
Data Control
Telephone_
Fax
Panel
I
& Protocol
Protocol
Controller
I
2Q
28
JQ
I
I
I
L - - - - - - - - - - 1---- - - -

I

-

I

t

--

~I

t

- --- -

Computer Hardware Interface

22

--

u.s. Patent

Jan. 29,2002

US 6,343,263 BI

Sheet 2 of 6

RGURE2
.1Jl

Application Program

22

r
I
I
I
I
I
I
I
I
I
I
I
I
I
I
I
I
I
I

~---

2Q

Modem Logic

48

- -

--------t--- - • I API I

Realtime
Engine

-

I
I
I
I

Adapter
Handler

44

I FeBI

-

.

I
J

I API 1-

46

"I

l

49

1

Serial Driver

DMA

42

~
.I

~

Hardware Abstraction Layer

...0:"'"

L --------

-

-"""'>0

_.

- -- - -- - - --

_-1

~~ =k'~7
Telecom Adapter _~

36

(O~

38

A" b...

A'" ""'A

/

A'" ""'A

u.s.

Patent

US 6,343,263 BI

Sheet 3 of 6

Jan. 29,2002

FIGURE 3

..-

Realtime Engine

Video
Service
§g

46

-

~

48

I...-

,....-

Sound
Service
§g

-

,- - -

-- -

-

I

I
I
I
I
I
I
L.

Translation
Software 54

..

~

-

Telephone
Service

..
I

I
I

62

,.

,

..
-

-

,....-

-

I
I

,
Interface

-

-- - - -

-~-

64

-

- - --

M

Processor
§§.

-,

..--<-

Libraries
§Q

L-- 52

r-L---.48

I--

...--

49

~'

I

Processing Engine (DSP)
Oper. Sys.

L--L--- 48

"--

u.s.

Patent

Jan. 29,2002

Sheet 4 of 6

US 6,343,263 BI

u.s.

Patent

APPUCATION

Jan. 29,2002

MODEM

Sheet 5 of 6

HANDLER

SOfTWARE

US 6,343,263 BI

REAL-TIME
ENGINE

Call Remote

Parse

Command
Generate DTMF
For x
Generate PCM
andDTMF

Return

Next Digit

Generate DTMF
For y
Generate DTMF

Return

Next Digit

a
o
o

FIG. 48

u.s.

Patent

APPUCAWN

Jan. 29,2002

Sheet 6 of 6

MODEM

HANDI..EB

SCfIWABE

US 6,343,263 BI

REAL-TIME

ENGINE

0

0

Listen For
Answer
Answer

Determine

Tone
Call Tone
Detector

ToneDetected

Return

Set Up
Connection
Request
Connection
Generate PCM
Signals
Connected

Return

Return

Retrieve Data

Send Data

Send Data
PCM
Transforms

FIG.4C

US 6,343,263 Bl
1

2

REAL-TIME SIGNAL PROCESSING SYSTEM
FOR SERIALLY TRANSMITTED DATA

While the analog telephone network was the only practical medium for transmitting information between geographically distributed computers for quite some time, more
recently other, non-analog transmission mediums have
become available. Examples of these other mediums include
the integrated services digital network (ISDN), private
branch exchange (PBX) telephone systems, and TI digital
data links. Since information is transmitted over these mediums as digital data, conventional analog modem circuits are
not suited for use with them. Thus, for example, a standard
Group III facsimile machine cannot operate on a digital PBX
system.
Similarly, digital signal processing systems which are
designed to work with PCM-encoded analog data that is
received and transmitted via a modem are likewise not
suited for use with these other types of transmission mediums. While it is possible to incorporate another DSP system
into a computer that can handle data transmitted via any of
these digital networks, it would be more desirable to provide
a single system that can process data that is received over
any type of transmission medium, whether it be digital or
analog. Further in this regard, it is desirable to provide a
signal processing system that is not limited to one specific
DSP, but rather one that can operate with any of a variety of
different types of signal processors.
When personal computers communicate with one another
through non-modem transport facilities, they typically operate in a burst mode. While this mode of operation enables
data to be transferred at much higher rates than with
modems, it is still not suitable for applications such as video
or speech processing. These types of applications require
isochronous data handling, i.e. data that is transmitted at a
constant bit rate and that must be processed in real time.
Generally speaking, therefore, it is desirable to provide a
serial data transmitting and receiving system that is capable
of processing real-time isochronous data.
Further in this regard, it is desirable to provide such a
system that is capable of handling streams of data pertaining
to different functions. For example, in a video conferencing
application, speech data is transmitted at the same time as
video and other graphic information. However, each of these
types of data must be processed separately in real time. It is
desirable, therefore, to provide a data transfer system that
can handle each of two or more types of data at isochronous
rates.

FIELD OF THE INVENTION
The present invention is directed to the transmission of
data to and from a computer, and more particularly to a
system for performing real-time signal processing of data
that is serially transmitted to and from a computer.
BACKGROUND OF THE INVENTION
Various devices are known for transmitting data between
a computer and a remote site via wide-area telecommunications networks. One of the most widely used devices of
this type is the modem, which enables data to be transmitted
to and from a computer over a wide-area analog telephone
network. Generally speaking, the modem includes one or
more sets of registers, typically embodied in an UART or an
USART, for storing bits of digital data transmitted to or from
the computer, a processor for implementing modem
operations, such as dialing a telephone number or answering
a ringing signal, in response to commands sent from the
computer and stored in the UART, and a modulator/
demodulator for converting digital bits of data to be transmitted into analog signals, and vice versa. Originally, all of
these features were hardwired in a separate peripheral device
that could be externally connected to the computer via a
serial I/O port, or internally connected to the computer's
data bus. More recently, some of the functions associated
with these features, most notably the processing of commands to implement modem operations, have been removed
from the hardwired configuration and incorporated into the
computer itself. This approach has increased the versatility
of the modem. For example, while the hardwired modem
configuration had to be specifically designed for the telephone system requirements of a particular country, the later
approach could enable a single product to be used in a
variety of countries, each of which might have different
telephone signaling requirements. Similarly, since the computer itself was handling the data to be transmitted, additional services, such as the ability to send information as a
facsimile transmission, in which graphical data is processed,
became feasible. However, the presence of the UART, or
similar such device through which the data must flow, still
limits the effective rate at which the data can be exchanged
between the computer and the telephone lines.
To enhance the performance of modems, a digital signal
processor (DSP) has been incorporated into its structure. In
this arrangement, the modem software was designed to
cooperate with the DSP to provide data thereto for processing prior to transmission or after reception over the telephone line. While the addition of the DSP provided
increased capabilities in terms of the speed at which the data
could be transmitted over a telephone network and the ease
with which the modem could be configured, it was still
limited in the types of data that could be processed. More
particularly, because of the restrictions imposed by passing
the data through an UART or the like, even the most modem
modems are only capable of effectively transmitting data in
the range of 9.6-14.4 Kb/sec. While this rate of data transfer
may be useful for transmitting static information such as text
files or the like, it is not suitable for many real-time
applications in which the data is provided at much higher
rates, such as speech or video conferencing.
Further in this regard, the modem control software had to
be designed to work with the specific DSP incorporated into
the computer. If a different DSP was to be used, the modem
control software had to be reprogrammed to work with the
new DSP.

5

10

15

20

25

30

35

40

45

BRIEF STATEMENT OF THE INVENTION

50

55

60

65

The present invention provides a data transmission system
having a real-time data engine for processing isochronous
streams of data. An interface device provides a physical and
logical connection of the computer to anyone or more of a
variety of different types of data networks, including analog
telephone, ISDN, PBX and the like. Data received at this
device is presented to a serial driver, which disassembles
different streams of data for presentation to appropriate data
managers, such as the operating system of the host
computer, a service provider or an application program. A
device handler associated with the interface device sets up
data paths and issues service requests. The device handler
also presents data and commands from the data managers to
a real-time data processing engine, that can be used for a
variety of applications such as voice recognition, speech
compression, and fax/data modems. This real-time engine
can be shared by any application program running on the
host computer.
The invention enables any arbitrary type of data, such as
voice, facsimile, multimedia and the like, which is trans-

US 6,343,263 Bl

3

4

mitted over any type of communication network, to be
handled with any type of real-time engine, by abstracting the
functions of each of the elements of the system from one
another. This abstraction is provided through suitable interfaces that isolate the transmission medium, the data managers and the real-time engine from one another. The data is
provided to the real-time engine as multiple streams of
isochronous data, i.e. it is delivered as it arrives over the
network without data handling delays. This feature allows
more complex applications, such as speakerphones, videophones and high-speed modems to be readily implemented.
These features of the invention, as well as the advantages
offered thereby, are described in greater detail hereinafter
with reference to a specific embodiment illustrated in the
accompanying figures.

communications toolbox 16 contained within fax/data
modem logic modules 20. A hardware interface 22 transmits
the services provided by these modules over a transmission
medium to which the computer is connected, e.g. a telephone line.
Referring to the modem logic 20, a Modem Control and
Command Line Interpreter (CLI) 24 accepts command
inputs from the application programs to configure the
modem, dial, receive calls, initiate data or fax transmission,
hang up, and so on. Any operation of the modem must be
initiated by issuing commands to the CLI. The CLI 24 can
be an interface that is often referred to as the "AT command
set", which uses simple printable character sequences and
constitutes a de facto standard among modems.
A Telephone Controller module 26 dials, answers, and
hangs up the telephone line. The Telephone Controller can
be "worldwide" in nature. In such a case it configures the
modem to conform to the standards of telephone systems in
most major economic markets. This can be carried out by
storing a country code identifier that allows the Telephone
Controller to configure the modem properly.
A Data Control and Protocol module 28 provides all data
capabilities for the modem. It supports standard asynchronous text read and write, as well as the standard CCITT Y.42
and Y.42 bis and Microcom MNP class 2 through 5 reliable
link and data compression protocols, for example.
The Fax Protocol module 30 performs the functions of a
T.30 fax engine in a dedicated fax modem. It communicates
with a Fax Extension driver 32, a component of a Fax
Terminal 34, for example using an extension of the AT
command set known as +F ("plus F"), or TR.29.
The hardware interface 22 contains the appropriate
transport-dependent protocol components. The logical and
physical interface to the wide-area network is hidden in this
layer. This allows the fax/data modem modules to be used on
any wide-area connection, including PBX, Tl and ISDN, as
well as traditional POTS ("plain old telephone system")
channels. The components of this interface are illustrated in
the block diagram of FIG. 2.
Referring now to FIG. 2, the hardware interface includes
an external adapter 36 that provides the physical and logical
connection of the computer to the telephone line 38 or other
communications network, such as ISDN, PBX or Tl link.
This connection can be provided through a serial port 37 of
the computer. An example of such an adapter is described in
copending U.S. patent application Ser. No. 08/078,890 filed
May 10, 1993, and Ser. No. 08/180,925 filed Jan. 11, 1994,
the disclosures of which are incorporated herein by reference. Such an adapter preferably includes processing
capabilities, as disclosed in concurrently filed U.S. patent
application Ser. No. 08/288,144 now U.S. Pat. No 5,799,190
entitled Intelligent Communications Coprocessor, which
enables it to provide a constant stream of data to the
computer from one or more communications networks. This
data can be delivered in a time-division multiplexed manner,
or it can be delivered in a packetized form.
Data received at the adapter 36 from the telephone line 38
can be provided to a driver 40 which functions as a hardware
abstraction layer. This driver is a hardware-dependent
implementation of a serial controller, and is configured in
accordance with the particular characteristics of the communications port 37 to which the adapter 36 is connected. It
isolates the remainder of the software from the different
implementations for connecting the adapter to the computer,
e.g. serial port, parallel port or bus device.
A serial driver 42 operates to separate multiple incoming
data streams from one another, or to combine multiple

5

10

15

BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 is a block diagram of the software architecture for
a facsimile/modem communications engine;
FIG. 2 is a block diagram of the computer hardware
interface for the engine;
FIG. 3 is a more detailed block diagram of the architecture
of the real-time engine; and
FIGS. 4A-4C are flow diagrams of the steps that are
carried out in the operation of a virtual fax/data modem in
accordance with the invention.
DETAILED DESCRIPTION
The following description of an embodiment of the
present invention is presented in the context of its implementation in a moderate speed (e.g. Tl data rates), low cost,
digital interconnect to a wide-area network. An example of
such an interconnect is described in U.S. patent application
Ser. No. 08/180,926 filed Jan. 11, 1994, now U.S. Pat. No.
5,515,373 the disclosure of which is incorporated herein by
reference. The practical applications of the invention are not
limited to this particular embodiment, however. For
example, it can also be employed as a high-performance
interconnect to multimedia devices such as digital imaging
equipment and audio appliances.
To facilitate an understanding of the invention, it will be
described with reference to the specific example of a
telephone-based telecommunication subsystem that provides basic fax/data modem services, plus call management
and audio stream handling. This particular example is perhaps one of the more complex applications of the invention,
because of the number of different signaling requirements
associated with communications over the telephone lines.
Further complexity is added by the fact that these requirements are often country-specific, and therefore the handling
of a command, such as dialing a telephone number, can vary
greatly from one country to the next. Other implementations
of the invention, for example in the context of transmitting
sounds and video data, will become apparent from an
understanding of the principles of the invention explained
with respect to this particular example.
FIG. 1 is a block diagram of a communications engine
that is designed to provide modem and facsimile services.
The top level of the diagram represents communications
applications 10-14. These applications send commands for
communications services to a data service provider, or
application programming interface (API), such as the Telephone Manager provided by Apple Computer, Inc. or the
Telephone Application Programming Interface (TAPI) provided by Microsoft Corp. This interface is represented by the

20

25

30

35

40

45

50

55

60

65

US 6,343,263 Bl
5

6

outgoing data streams that are intended for respective transcommunications network, It IS provided to the real-time
mission to separate destinations, whether logical or physical.
engine as pulse-code modulated (PCM) data through the
For example, the adapter 36 might be connected to a desktop
DMA 50. If the computer does not have DMA capabilities,
telephone for voice communications and to the telephone
the data can be transferred between the adapter and the
line for wide-area communications. Data streams for these 5 handler as packetized data, as described in application Ser.
two connections can be combined in a multiplexed manner
No. 08/058,750 filed May 7, 1993 now abandoned.
by the serial driver, to be sent to the adapter 36 through the
As another example, the real-time engine may operate as
computer's serial port 37.
a virtual speech recognition device. To do so, the real-time
An adapter handler 44 is specific to the particular adapter
function block initializes the engine and installs the modules
36 and carries out features associated with that adapter. For 10
necessary to carry out this function. In this mode of
example, if the adapter is one that is designed to be conoperation, the engine may convert pulse-code modulated
nected to a telephone network, the handler implements
(PCM) data received from the adapter into vector-quantized
functions germane to that network such as ring detection,
speech components. The engine transforms this PCM data
pulse dialing, on/off hook control and the like, in response
into data appropriate for subsequent processing by a speech
to commands received from the application programs 10. 15
recognition application program, according to the comOne example of the manner in which data can be exchanged
mands from the handler 44, and the results of the transforms
between the adapter 36 and the handler 44 is described in
are provided through the shared FIFOs.
U.S. patent application Ser. No. 08/058,750 filed May 7,
With this configuration, the handler does not need to have
1993, now abandoned the disclosure of which is incorpoany knowledge of the real-time engine implementation.
rated herein by reference.
20 Communications with the real-time engine are carried out in
A real-time engine 46 can perform transforms on data
a robust fashion. In essence, the architecture of the system
streams provided to and received from the adapter 36. The
provides a message-passing capability between devices that
particular transforms to be performed are sent as commands
know nothing about the configuration or existence of one
to the real-time engine from the adapter handler 44 via
suitable application programming interfaces 48. For com- 25 another.
As illustrated in FIG. 2, there can be a number of
municating with the real-time engine, each interface
interfaces 48 situated between the handler 44 and the
includes shared command/control mailboxes in the computreal-time engine 46. Each interface represents services for a
er's RAM, as well as bi-directional first-in, first-out (FIFO)
particular class of functionality. For example, one interface
buffers for transferring data. As an example, if the system is
set up to operate as a fax/data modem, the real-time engine 30 may relate to the operation of the engine as a virtual
telephone, another interface can be associated with a virtual
functions as a virtual telephone. In such a case, the handler
sound device, e.g. stereo, and a third interface can pertain to
may instruct the engine 46 to send a facsimile in response to
a virtual video device. Each interface receives commands
a command from an application program. For this purpose,
from an application program, through the handler 44, and
the real-time engine is configured with a suite of modem and
call processing functions. This configuration is implemented 35 instructs the real-time engine to carry out the necessary
transforms which relate to the function of the virtual device
by a real-time function control block 49, which initializes
being implemented, e.g. text-to-speech conversion, video
and manages the operation of the real-time engine. Generimage processing, etc.
ally speaking, whenever a new task is to be carried out by
the real-time engine 46, the real-time function control block
The architecture of the real-time engine 46 is illustrated in
49 issues commands that are specific to the operating system 40 further detail in FIG. 3. Referring thereto, when configured
as a virtual device, the real-time engine is made up of two
of the real-time engine. These commands cause the engine
to start up, if it is not already running, and to configure itself
main components, a processing engine 52, such as a DSP,
with a library of routines that are necessary for it to
and translation software 54. The DSP comprises a processor
implement the task.
56, an operating system 58 for that processor, and a set of
When the handler 44 requests a facsimile transmission, 45 libraries 60 which enable the processor to perform designated signal processing functions. There are three possible
for example, the real-time function block issues commands
implementations of the DSP, respectively identified as hard,
to start the real-time engine and install the various modules
soft and native. In the hard implementation, all three comthat are needed for it to function as a virtual telephone.
ponents of the DSP are fixed within a piece of hardware, i.e.
Binary facsimile image data is transferred to the real-time
engine via the FIFO buffers, where it is encoded as PCM 50 an IC chip. In other words, the libraries and the operating
system are embedded as firmware, and cannot be reprodata which is further encoded according to the transport
grammed or updated without changing the chip. An example
medium over which it is to be transmitted. If the adapter is
of a hard implementation is the Rockwell 9623 data-pump.
connected to a telephone line, for example, these signals can
This type of DSP might be able to perform only one class of
be encoded as 16-bit pulse-code modulated (PCM) samples,
and forwarded directly to the adapter 36 via the serial driver 55 virtual device operation, i.e. Function as a modem. When a
hard DSP is employed for the real-time engine, the function
42. Alternatively, if the transport medium is an ISDN line,
control block 49 operates to initialize the processor at the
the modem signals are encoded as mulaw-companded 8-bit
outset of the operation of the real-time engine.
PCM signals. The different types of encoding are stored in
different tables, and the appropriate one to be used by the
The soft implementation differs from the hard implemenreal-time engine is installed by the real-time function block 60 tation in that the libraries, and possibly also the operating
during the initial configuration of the engine and/or desigsystem, are resident as software in the computer's memory.
nated by the API 48 at the time the command to transform
In this implementation, the libraries are programmable and
the data is issued.
can be updated as desired. The processor, however, is still
Transformed signals from the real-time engine that are to
resident as a separate piece of hardware. Because of this
be transmitted via the transmission medium are provided to 65 programmability, the DSP can carry out more functions than
a hard DSP, such as sound processing and Fourier transthe hardware abstraction layer 40 through direct memory
forms. An example of a soft DSP is the AT&T 3210. When
access (DMA) 50. When data is being received from the

US 6,343,263 Bl
7

8

a soft DSP is employed for the real-time engine, the function
The interface provides for the origination and answering
control block 49 operates to configure the appropriate librarof calls routed through traditional analog switches. To
ies for the transforms that are to be carried out by the DSP,
answer calls, the interface monitors incoming signals, as
in addition to initializing the processor.
reported by the real-time engine, for appropriate frequency
In the native implementation of the DSP, the processor 5 and cadence consistent with a particular country's requiredoes not reside in a separate piece of hardware. Rather, it's
ments. The interface also includes facilities for tracking call
functions are carried out by the CPU of the host computer.
progress, such as detection of dial tone, ring back and busy
As in the case of the soft implementation, the DSP operating
signals. It further includes the necessary information relating
system and the libraries are resident in the computer's
to the generation and detection of DTMF signals.
memory. When this implementation is employed, the func- 10
The interface generates calls that can be classified into
tion control block 49 operates to allocate system resources
two general categories, originating calls and callback calls.
to the DSP function, such as to enforce system time
Originating calls are those which are generated in response
management, to ensure that adequate processing time is
to commands from the application program. Callback calls
given to DSP operations.
are used to report progress information to the application
The translation software 54 is made up of two parts. The
main part of the software comprises a generic service 15 program. Most originating calls might take time to complete,
and are therefore asynchronous, so that the host processor
provider 62 which functions as a device driver. This part of
can suspend servicing of the calling application program
the software receives the commands from one of the APIs 48
until the task associated with the call is complete. This
and issues the instructions to the DSP to perform the
allows the interface to be called from within an interrupt
transforms that are required in the operation of the virtual
device being implemented. This part of the software is 20 handler as well as freeing the processor while waiting for
labeled as being generic because it is independent of the
some hardware to execute a task. Completion of the process
actual hardware that is used in the implementation of the
is indicated by executing a completion routine for an assoDSP 52. To enable the service provider to communicate with
ciated callback.
the DSP, an interface 64 is provided. This interface is
The originating calls are of two types, system task calls
specific to the particular DSP that is employed as the 25
and general purpose calls. The system task calls can include
processing engine for the real-time engine. In other words,
those such as "Open", which causes system resources to be
the generic service provider does not need to know whether
allocated to the real-time engine, and "Close", which dealthe processing engine is a hard, soft or native DSP. In
locates the resources. General purpose calls can include such
essence, therefore, the interface 64 functions as an additional
layer of abstraction which virtualizes the DSP, i.e. the 30 calls as "State", which returns the current state of the virtual
device, e.g. on-hook, ringing, off-hook or on-line, "Genergeneric service provider is aware of the existence of a DSP,
ateDtmf" which causes DTMF tones to be generated, and
but does not need to know how it is actually being imple"SetAutoAnswer", which instructs the engine to answer a
mented in order to operate.
call after a predetermined number of rings. Other examples
In a practical embodiment of the invention, separate
generic service providers can be employed for the different 35 of general purpose calls include "SetSilenceDuration",
which passes to the engine the length of a silence to be
virtual devices to be implemented. For example, one service
detected, "Hook", which is used to take the virtual phone
provider can be employed to provide the services of a virtual
off-hook and on-hook, and "DiaINumber", which dials a
telephone. Such a service provider might include a set of
number in a designated string. Similar types of general
calls which enable it to determine the capabilities of the
hardware being employed, e.g. whether it can support line 40 purpose calls can be included for functions associated with
facsimile types of operations.
current detection, remote wakeup, etc. Other sets of calls are
Examples of suitable callback calls include
used for control and status information, tone generation and
"DtmfDetected", which indicates that a particular DTMF
detection, data transfer, and power management.
digit has been detected, "RingIndicate", which identifies
Another service provider can be used for sound
when a valid ring has been detected on the line, and
applications, and a third service provider for video applica- 45
"DiaIToneDetected", which is called when a valid dial tone
tions. Depending upon the particular virtual device to be
is detected. Other appropriate callback calls will be readily
implemented, the function control block 49 calls up the
apparent from these examples.
appropriate service provider when configuring the real-time
The flow of events that occur when the fax/data modem
engine. Each service provider communicates with the handler 44 through a respective one of the APIs 48, and with the 50 is activated will now be described. These events are illustrated in the flow diagrams of FIGS. 4A-4C. From the
DSP 52 through the same interface 64.
perspective of an application program 10, it is "talking" to
An example of a suitable interface 48 for telephony
an external modem connected to the serial port. In fact,
applications will now be described in detail. The interface
however, it is actually obtaining communications services
basically operates to transmit high-level requests for service.
The functionality of such an interface can be divided into 55 from an internal virtual modem 20 and the hardware interface 22.
two main categories, namely functions that are used only on
At boot time, the computer's operating system determines
public service telephone network (PSTN) lines, such as ring
whether the computer is capable of supporting a communidetection, and those functions used on any telephone line,
cations system. This allows the operating system to notify
such as DlMF generation and detection. This arrangement
allows the interface to be used for ISDN and PBX lines as 60 the user on application program activation that a communications session is not possible on the computer. This deterwell as traditional analog lines for call progress and modem
mination is made by assessing system-dependent factors
functions. For PSTN lines, the interface generates comsuch as presence of a data stream processor, sufficient
mands for setting the appropriate electrical parameters, such
system resources, and so forth. In the following discussion
as voltage levels that comply with a particular country's
regulations. For this purpose, the interface can include data 65 it is assumed that the requisite resources are available.
At the outset, with reference to FIG. 4A, a communicatables containing information on all country-specific parameters.
tions application 10 is launched. A communications "con-

US 6,343,263 Bl

9

10

nection" is opened either implicitly on launch, or by comReferring now to FIG. 4C, upon completion of the virtual
mand. This directs the communications subsystem to
phone dial command, the fax/data module issues a command
initialize itself. The communications application's connecto the virtual phone, i.e. the hardware interface, directing it
to listen for an answering modem sequence. Again, this
tion establishment request is passed to the communications
toolbox 16. This in turn causes a driver command "Open" to 5 command is received by the handler 44 and is translated into
be issued by the fax/data modem logic 20 to the hardware
a virtual real-time engine command to detect tones (signal
interface 22.
energy) at certain specified frequencies and levels.
Optionally, the real-time engine may be commanded to
The hardware interface driver command "Open" is
concurrently listen for voice energy in case a human answers
received by the hardware interface adapter handler 44. As
noted, the handler has previously-typically at boot time- 10 the phone.
determined that the host computer has the resources to
Assuming that the answering station presents valid
establish a communications session, in this example being
answering modem tones, the handler will then be directed to
an analog modem over a telephone service line.
instigate a modem connection. This results in another command to the virtual real-time engine, this time requesting a
The handler calls function control block 49, to initialize
the real-time engine. The action taken by function control 15 modem connection compatible with the answer tone
sequence received. The actual modulation and demodulation
block depends on the real-time engine's implementation. If
of the hardware interface adapter's isochronous PCM data
a programmable DSP is used for the real-time engine, the
stream is accomplished entirely by the real-time engine.
function control block might issue a series of DSP operating
system specific commands to download and initialize the
Once the modem connection is established, the handler
DSP subsystem, followed by commands to download the 20 notifies the fax/data modem module that data transmission
DSP algorithms that perform the modem's analog modulamay begin. Digital data now flows between remote and local
tion. A native-mode DSP implementation may result in the
computers via handler Read and Write calls. Data is passed
function control block simply allocating system memory and
between the real-time engine and the handler via full-duplex
host processor resources needed for the modem algorithms.
FIFOs. This data is in turn passed between the handler and
In either case, it is significant to note that the hardware 25 the application program through the modem logic 20.
interface is designed to use a virtual real-time engine. The
The ability to communicate over different types of transentire real-time engine implementation is "hidden" from the
mission mediums in this single system is made possible by
handler 44 by the function control block. The handler does
the fact that each of the various components is isolated from
not communicate with the real-time engine directly, via
the particular features of the other through suitable levels of
DSP-specific commands. Rather, all communications take 30 abstraction implemented via the application programming
place over the virtual real-time engine interface 48 via the
interfaces. For example, to change the transmission medium
mailboxes and the full-duplex data FIFO registers.
from the telephone lines to an ISDN line, the telecom
The handler 44, as part of the Open process, prepares the
adapter 36 is disconnected from the serial port 37, and a new
attached telecom adapter 36 for operation. After 35 adapter appropriate for ISDN is plugged into the serial port.
initialization, the hardware interface telecom adapter delivThe associated adapter handler 44 is also loaded into the
ers a full duplex, isochronous data stream.
system. Thereafter, whenever the adapter handler issues a
command to the real-time engine to perform a transform, it
If the real-time engine is successfully configured, the
identifies the fact that the transformed data must be suitable
hardware interface is initialized properly, and all other
necessary resources are available, the handler Open opera- 40 for ISDN format. In response thereto, the API 48 which
receives these commands supplies the real-time engine with
tion will be successful, and an analog modem communicathe appropriate parameters for performing the transforms in
tions link over the hardware interface adapter can begin at
the required format, e.g. the proper number of bits per word,
any time. Otherwise, communications are impossible and an
etc.
error is reported to the application program.
Similarly, if the computer is transported from one country
Referring now to FIG. 4B, the application program now 45
to another, the only change that needs to be implemented to
initiates a modem connection with a remote station. For this
carry out telephone communications in the new country is to
purpose, the character sequence "ATDT5551212" might be
switch the adapter and its handler. Upon initialization, the
issued by the application program, signifying (in the "AT"
adapter identifies the fact that it is designed for a specific
modem command standard) that the communications subsystem should dial the remote station at number 5551212, 50 country. Whenever commands are to be sent to the real-time
engine, the handler instructs the API 48 of the country as
and instigate a modem connection with the answering
well as the command itself. For example, the command
modem, if present.
might be to generate a dial tone for country X. In response,
The dial command string is passed by the communicathe API 48 instructs the real-time engine to generate the dial
tions toolbox 16 to the CLI 24, where it is parsed and
converted into a virtual phone dial command. The virtual 55 tone, and provides it with the parameters pertinent to dial
tones in country X. The real-time engine then generates the
phone dial command is passed to the adapter handler 44 as
necessary PCM signals and supplies them to the adapter 36
a driver-level control call, where it is translated into a virtual
via the DMA. The adapter takes care of converting those
real-time engine command and placed on the virtual realsignals into the necessary electrical signals for transmission
time engine's command/response interface 48. This causes
the real-time engine to generate a PCM data stream that 60 on the telephone lines of that country.
directs the hardware interface telecom adapter to go offIn essence, the real-time engine allows any type of
hook, then to generate the DTMF tones corresponding to the
transform to be performed on any type of data delivered over
entered phone number. The handler 44 "sleeps", waiting for
any type of transmission medium. The application program
the real-time engine to signal that the real-time DTMF
which receives the transformed data does not have to have
command has been completed. In this operation, the handler 65 any knowledge of the fact that the transmissions are being
carried out over an ISDN line, rather than the telephone lines
has no involvement with the isochronous data stream created
that it might have been originally programmed for. Thus, for
by the real-time engine.

US 6,343,263 Bl
11

12

example, a modem connection can be established over an
and the realtime signal processing subsystem to allow
ISDN line without the application being aware of a change
the telecommunications subsystem to interoperate with
said realtime services.
in the transmission medium. The adapter 36, the hardware
4. The signal processing system as set forth in claim 3,
abstraction layer 40, the serial driver 42 and the adapter
handler 44 function to configure a real-time data stream 5 wherein the realtime signal processing subsystem comfrom the transmission medium to the real-time engine, and
prises:
vice versa. The speed at which this data can be delivered, as
at least one realtime communications module coupled to
well as the format of the data, is no longer limited by
receive a plurality of communications commands from
hardware devices that are employed in conventional hardsaid applications programs via said datastream handler
wired modems, particularly UARTs and the like. Rather, the 10
program and said realtime APIs, said realtime commudata is delivered at a real-time rate, where it is handled by
nications module in response to said communications
the computer's CPU.
commands issuing a plurality of requests for realtime
services to at least one realtime service provider;
The foregoing examples of the invention have been
presented to facilitate an understanding of its features and
a translation interface program coupled to receive said
operation. It will be appreciated, however, that the practical
requests for realtime services from said communicaapplications of the invention are not limited to these specific 15
tions modules; and
embodiments. Rather, the invention will find utility in any
a realtime processor including a realtime operating system
environment in which it is desirable to transmit and process
interoperating with said translation program for executdata at real-time rates. Thus, while the invention has been
ing a plurality of realtime operations comprising realdescribed in the context of communications over a wide-area
time functions in response to said requests.
network, it can be used in any type of data acquisition 20
5. The realtime data processing system as set forth in
claim 4, wherein the translation interface comprises a plusystem. The preceding description should therefore be
rality of realtime features to access a modem unit for
viewed as exemplary, rather than restrictive. The scope of
communicating over said WAN.
the invention is indicated by the following claims, rather
6. The signal processing system as set forth in claim 3
than the foregoing description, and all changes which come
within the meaning and range of equivalents thereof are 25 further comprising a direct memory access (DMA) unit
coupled between said realtime signal processing subsystem
intended to be embraced therein.
and a hardware abstraction portion of said telecommunicaWe claim:
tions subsystem, said DMA unit providing for transfer of
1. A signal processing system for providing a plurality of
datablocks from said telecommunications adapter module to
realtime services to and from a number of independent client
30 said realtime signal processing subsystem.
applications and devices, said system comprising:
7. A signal processing system for providing a plurality of
a subsystem comprising a host central processing unit
realtime services over a wide area network (WAN), said
(CPU) operating in accordance with at least one applisystem comprising:
cation program and a device handler program, said
a telecommunications subsystem comprising a host censubsystem further comprising an adapter subsystem
tral processing unit (CPU) operating in accordance with
35
interoperating with said host CPU and said device;
at least one applications program and a datastream
a realtime signal processing subsystem for performing a
handler program, said telecommunications subsystem
further comprising a telecommunications adapter subplurality of data transforms comprising a plurality of
system interoperating with said host CPU and said
realtime signal processing operations; and
WAN;
at least one realtime application program interface (API) 40
a virtual realtime device enabling a plurality of realtime
coupled between the subsystem and the realtime signal
processing subsystem to allow the subsystem to intersignal processing operations in accordance with at least
operate with said realtime services.
one realtime service request issued by said applications
program; and
2. The signal processing system as set forth in claim 1,
wherein said signal processing system receives and trans- 45
at least one realtime application program interface (API)
mits a plurality of datatypes over a plurality of different wide
interoperating with the telecommunications subsystem
area networks (WANs).
and the virtual realtime device to enable the telecom3. A signal processing system for providing a plurality of
munications subsystem to interoperate with said realrealtime services over a wide area network (WAN), said
time signal processing operations.
system comprising:
8. The signal processing system as set forth in claim 7,
50
wherein the virtual realtime device comprises a realtime
a telecommunications subsystem comprising a host centranslation interface program and virtual realtime engine,
tral processing unit (CPU) and a wide area network
said virtual realtime engine enabling said realtime services
interface, where said wide area network interface is
by performing a number of data translation operations in
comprised of a hardware interface to the network and
55 accordance with said realtime service request and said
driver software which executes on the host
realtime translation interface program.
a telecommunications subsystem comprising a host cen9. The signal processing system as set forth in claim 8,
tral processing unit (CPU) operating in accordance with
wherein the virtual realtime engine comprises:
at least one application program and a datastream
a realtime processor including a realtime operating
handler program, said telecommunications subsystem
system, and
further comprising a telecommunications adapter sub- 60
system interoperating with said host CPU and said
a plurality of realtime function libraries interoperatively
WAN;
coupled with said realtime processor for providing a
plurality of processing steps comprising said realtime
a realtime signal processing subsystem for performing a
signal processing operations,
plurality of transforms comprising a plurality of realtime signal processing operations; and
65
whereby said virtual realtime engine responds to commuat least one realtime application program interface (API)
nications commands initiated by said applications programs.
coupled between the telecommunications subsystem

US 6,343,263 Bl

13

14

10. The signal processing system as set forth in claim 7
further comprising a direct memory access (DMA) unit
coupled between said virtual realtime signal processing
subsystem and a hardware abstraction portion of said telecommunications subsystem, said DMA unit providing for
transfer of data from said telecommunications adapter module to said virtual realtime signal processing subsystem.
11. The signal processing system as set forth in claim 7,
wherein the virtual realtime translation interface comprises
a plurality of realtime features to access a modem unit for
communicating over said WAN.
12. The signal processing system as set forth in claim 11,
wherein the modem unit comprises at least a serial communications controller, a programmable timer, and a plurality
of input/output (I/O) lines.
13. The signal processing system as set forth in claim 7,
wherein realtime service requests are selected from the
group of realtime service request devices consisting of
telephone answering machines, automatic telephone dialing
machines, and remote control systems.
14. The signal processing system as claimed in claim 6,
wherein the realtime signal processing operations are
selected from the group of telecommunications transactions
consisting of fax (send/receive) and data transmission transactions.
15. The signal processing system as claimed in claim 14,
wherein the data transmission transaction comprises at least
one data framing format and at least one data protocol.
16. The virtual realtime data processing system as set
forth in claim 9, wherein the realtime processor comprises a
programmable processing unit which is controlled by said
realtime communications applications and said realtime
communications interface.
17. The virtual realtime data processing system as set
forth in claim 9, wherein the realtime processor comprises
said host CPU.
18. The signal processing system as set forth in claim 9,
wherein the realtime processor comprises a hard digital
signal processor in which the realtime operating system and
the realtime function libraries are fixedly embodied in a
hardware element.
19. The realtime data processing system of claim 4
comprising a plurality of realtime communications modules
which are respectively associated with different realtime
serVIces.
20. The realtime data processing system of claim 19
wherein at least some of said realtime communications
modules provide a service which implements a virtual
realtime device.
21. The realtime data processing system of claim 19
wherein one of said realtime services comprises a video
processing service.
22. The realtime data processing system of claim 19
wherein one of said realtime services comprises a sound
processing service.
23. The realtime data processing system of claim 19
wherein one of said realtime services comprises a telephone
serVIce.
24. The signal processing system of claim 1, wherein said
realtime signal processing subsystem comprises:
a realtime processor including an operating system for
executing a plurality of realtime functions;
a realtime communications module which is independent
of said realtime processor and is coupled to receive a
plurality of communications commands from said
application programs via said device handler program
and said realtime API, said realtime communications

module operating in response to said communications
commands to issue a plurality of requests for realtime
services to said realtime processor; and
a translation interface program which is specific to said
realtime processor and is coupled to receive said
requests for realtime services from said communications module and provide said requests to said realtime
processor.
25. The signal processing system of claim 24 comprising
a plurality of realtime communications modules which are
respectively associated with different realtime services.
26. The signal processing system of claim 25 wherein at
least some of said realtime communications modules provide a service which implements a virtual realtime device.
27. The signal processing system as set forth in claim 24,
wherein the realtime processor comprises a hard digital
signal processor in which said operating system and realtime
function libraries are fixedly embodied in a hardware element.
28. The signal processing system as set forth in claim 24,
wherein the realtime processor comprises said host CPU.
29. The signal processing system of claim 24, wherein
said realtime processor is embodied in a hardware device
and includes realtime function libraries that are embodied in
programmable software.
30. The signal processing system of claim 29 wherein said
operating system is also embodied in programmable software.
31. A signal processing system, comprising:
an input/output device for sending and/or receiving isochronous streams of data transmitted over a communications path;
a realtime engine for performing data transformations on
the isochronous streams of data, said realtime engine
being independent of said input/output device;
a device handler program associated with said input/
output device, for generating requests to the realtime
engine to perform data transformations on the isochronous streams of data; and
at least one application programming interface for receiving the requests generated by said device handler
program and issuing commands to said realtime engine
to perform the requested data transformations.
32. The signal processing system of claim 31 comprising
a plurality of said application programming interfaces which
are respectively associated with different types of services to
be provided by said realtime engine with respect to isochronous streams of data.
33. The signal processing system of claim 32 wherein one
of said application programming services relates to the
operation of the realtime engine as a virtual telephone
device.
34. The signal processing system of claim 32 wherein one
of said application programming services relates to the
operation of the realtime engine as a virtual sound device.
35. The signal processing system of claim 32 wherein one
of said application programming services relates to the
operation of the realtime engine as a virtual video device.
36. The signal processing system of claim 31 wherein said
application programming interface includes command/
control registers that are shared between said realtime
engine and said device handler program for transferring said
requests and responding thereto, and a buffer for transferring
isochronous streams of data between said device handler
program and said realtime engine.
37. The signal processing system of claim 31, wherein
said realtime engine comprises:

5

10

15

20

25

30

35

40

45

50

55

60

65

US 6,343,263 Bl
15

16

signal processor in which said operating system and realtime
function libraries are fixedly embodied in a hardware element.
a communications module which is independent of said
39. The signal processing system as set forth in claim 37,
realtime processor and is coupled to receive said com- 5 wherein said processing system is incorporated in a data
processing system having a host central processing unit
mands from said application programming interface,
(CPU), and wherein the realtime processor comprises said
said communications module operating in response to
host CPU operating in accordance with software instructions
said commands to issue a plurality of requests for
relating to said realtime functions.
realtime services to said realtime processor; and
40. The signal processing system of claim 37, wherein
a translation interface program which is specific to said 10 said realtime processor is embodied in a hardware device
and includes realtime function libraries that are embodied in
realtime processor and is coupled to receive said
programmable software.
requests for realtime services from said communica41. The signal processing system of claim 40 wherein said
tions module and provide said requests to said realtime
operating system is also embodied in programmable softprocessor.
ware.
38. The signal processing system as set forth in claim 37, 15
wherein the realtime processor comprises a hard digital
* * * * *
a realtime processor including an operating system for
executing a plurality of realtime functions;

UNITED STATES PATENT AND TRADEMARK OFFICE

CERTIFICATE OF CORRECTION
PATENT NO.
: 6,343,263 Bl
DATED
: January 29, 2002
INVENTOR(S) : James B. Nichols and John Lynch

Page 1 of 1

It is certified that error appears in the above-identified patent and that said Letters Patent is
hereby corrected as shown below:

Title page,
Item [75], Inventors, "James B. Nichols", please delete "San Mateo" as city and insert
-- Los Altos -- in it's place.
"John Lynch", please delete "San Jose" as city and insert -- Monte Sereno -- in it's
place.

Signed and Sealed this
Twenty-sixth Day of November, 2002

Attest:

Attesting Officer

JAMES E. ROGAN
Director ofthe United States Patent and Trademark Office