iLOR, LLC v. Google, Inc.

Filing 12

MOTION for Preliminary Injunction by iLOR, LLC (Attachments: # 1 Memorandum in Support Part 1# 2 Memorandum in Support Part 2# 3 Proposed Order # 4 Exhibit A# 5 Exhibit B# 6 Exhibit C# 7 Exhibit D# 8 Exhibit E# 9 Exhibit F# 10 Exhibit G# 11 Exhibit H# 12 Exhibit I# 13 Exhibit J# 14 Exhibit K# 15 Exhibit L# 16 Exhibit M# 17 Exhibit N# 18 Exhibit O# 19 Exhibit P# 20 Exhibit Q# 21 Exhibit R# 22 Exhibit S# 23 Exhibit T# 24 Exhibit U# 25 Exhibit V# 26 Exhibit W# 27 Exhibit X# 28 Exhibit Y# 29 Exhibit Z# 30 Exhibit AA# 31 Exhibit BB# 32 Exhibit CC# 33 Exhibit DD# 34 Exhibit EE# 35 Exhibit FF# 36 Exhibit GG# 37 Exhibit HH# 38 Exhibit II# 39 Exhibit JJ# 40 Exhibit KK# 41 Exhibit LL# 42 Exhibit MM# 43 Exhibit NN# 44 Exhibit OO# 45 Exhibit PP# 46 Exhibit QQ# 47 Exhibit RR# 48 Exhibit SS# 49 Exhibit TT# 50 Exhibit UU)(Faller, Susan)

iLOR, LLC v. Google, Inc.

Doc. 12 Att. 34

Case 5:07-cv-00109-JMH

Document 12

Filed 08/27/2007

Page 1 of 20

11111111111111111111111111111111111111111111111111111111111111I1111 11111111

USOO6567830B i

(12) United States Patent
Madduri

(10) Patent No.: (45) Date of Patent:

US 6,567,830 Bl
May 20, 2003

(54) METHOD, SYSTEM, AND PROGRA FOR DISI)!AYING ADDED TEXT TO AN ELECTRONIC MEDIA FILE
(75) Inventor: Hari Haranath Madduri. Austin, TX

OTHER PUBLICATONS Malcolm, Nicholas and Baines, Brian R., A Minimalist
Approach to the Development of a Word Processor Support-

(US)
(73) Asignee: International Business Machines

ing Group Writing Activities, Knowledge Science Institute, University of Calgary, Calgary, Alberta, Canada. Ackerman, Shirley S. and Threchek, Wiliam W., The Risks

an Rewards of Online Editing, IEEE Transactions on
processional Communication, vol. 31, No.3, Sep. 1988.

Corporation, Aronk NY (US)
( · ) Notice:

S. Ackerman, W. Threcheck, and R. Wright, Online Publication Editng System, IBM Technical Disclosure Bulletin,

Subject to any disclaimer, the term of this

patent is extended or adjusted under 35
U.S.c. 154(b) by 0 days.

vol. 36, No. 06A, Jun. 1993.
Microsoft Offce 6-in-l, New Edition, Que Corporation,

(21) Appl. No.: 09/249,203

(22) Filed: Feb. U, 1999

(51) Int. CI.7 ............................................... G06F 17/21 (52) U.S. CI. ........................................ 707/512; 707/531 (58) Field of Search ................................. 707/512. 531.
707/526; 395/145, 788

1994, pp. 425-428.' · cited by examiner Primary Examiner-Stephen S. Hong (74) Aiiorney, Agent, or Finn-David W. Victor; Konrad Raynes Victor & Mann LLP

(57) ABSTRACT
Provided are a method, system, and program for receiving

(56) References Cited
U.S. PATENT DOCUMENT

and displaying text inserted into a text document. The program displays lines of text from the document and
receives user inputted text. The program generates a graphical element to include the user inputted text and generates
the usr inputted text having a size capable of fitting between

5,146,552 A . 9/1992 Cassrla et al. ............ 395/145 5,530,794 A 6/1996 I.uebberl 5,572,656 A . 11/1996 lida et al. ................... 395n88 5,619,738 A . 4/1997 Petruchik et al. ........... 396/311 5,732,219 A 3/1998 Blumer et .1. 5,802,380 A . 9/1998 Bangs et al. ................ 395n77 6,230,170 B1' 5/201 Zellweger et al. .......... 7rJ/512

two lines of the text from the document. The program then displays the graphical element in the document. The user
inputted text within the graphical element is displayed between two lies of texl from the document.

35 Claims, 7 Drawing Sheets

12)
The preferred embodiments may be implemented as a method, apparatus or article of
manufacture using standard programming and/or engineering techniques to produce software,

firmware, hardware, or any combination thereof. The term "article of manufacture" (or
110--. :iñïiiisdõëunÏeniãñd ih-rÕugÏiaut-lhIS diseuSsiöni

alternatively, "computer program product") as u~ëa:~~~r_e~~~~~~~_~d_e_d_~c:~~.i.o-i~~~~.?_n:i_~r more

computer programs and data files accessible from one or more computer-readable devices,

carriers, or media, such as a magnetic storage media, "floppy disk" CD ROM, a file server
112--,___J ~~ble li~~ ~¡ ~;;~te-i~Ir~r~d~~~~I~ - - - - - - - - - n_ - - -:

providing access to the programs via a;ncl'Norl¡ tFaRsrnissioA lino, holograpAie unit, etc.: Of
---- - ------- - -- - -------- -- -- _ _ _._ _ _ _ _ _ __J

course, those skilled in the art will recognize many markings may be made to this configuration
without departing from the scope of the invention.

EXHIBIT

l £E
Dockets.Justia.com

Case 5:07-cv-00109-JMH

Document 12

Filed 08/27/2007

Page 2 of 20

u.s. Patent

May 20, 2003

Sheet 1 of 7

US 6,567,830 Bl

tJir. !
Computer 2

Memory 4

Viewer Program
8

Operating System 6

Document 10a

Document 10b

Document 10c

Network Card 18
I

Video Controller 14
I
I

I

I

Input

Means
16

Display Monitor 12

Case 5:07-cv-00109-JMH

Document 12

Filed 08/27/2007

Page 3 of 20

u.s. Patent

May 20, 2003

Sheet 2 of 7

US 6,567,830 Bl

Vlie. 2
lOa

20

22

24

Case 5:07-cv-00109-JMH

Document 12

Filed 08/27/2007

Page 4 of 20

U.S. Patent

May 20, 2003

Sheet 3 of 7

US 6,567,830 Bl

30

óJÚf. 3
32 34
Yes Display pop-up
to enter text.

38

window for user

42
No
Store text user entered
in marking infonnation as note and determined address information of position of note in text in marking file.

40
Determine address

location of cursor in

document when note
marking selected.

36
No

46
Display non-textual marking options.

52
Display textual

marking options.

Store marking type and determined address information in marking file.

Determine address

location of cursor in document when

marking invoked.

58
Determine address

information for start and end of highlighted

text in document.

Store marking type in marking information along with detennined address Infonnation in

marking fie.

Case 5:07-cv-00109-JMH

Document 12

Filed 08/27/2007

Page 5 of 20

U.S. Patent
70

May 20, 2003

Sheet 4 of 7

US 6,567,830 Bl

86
No
Generate display information for
markings usin~

72
Display marking files associated
with displayed

color indicated in

marking fie.

document.

84

Yes

88
Display markings
according to generated
display information.

Cause video controller to generate display information for

markings using
user selected color.

Cause video controller to generate display

information for overlapping
markings using user selected color.

No

Video controller displays overlapping

markings according to generated
display information.

94
Go to 82 to cause video controller to generate display information for non-

qk¡. if

overlapping markings.

12)
VJkf. 5

The preferred embodiments may be implemented as a method, apparatus or article of

~ ~ = ~

~ S" ~ =

Case 5:07-cv-00109-JMH

manufacture using standard programming and/or engineering techniques to produce software,

'= .

3:

firmware, hardware, or any combination thereof. The term "article of manufacture" (or

.?

110 - -, : i¡Úîiis deeurñëni ãñd thrõughaiJI-iiis diseiJssiõni

alternatively, "computer program product" ) as usedl~~~e~~~~~~~~.?_e_~~~~~~~~~~:~~!l_e_pr more

, .. i

N N o

6

Document 12

computer programs and data files accessible from one or more computer-readable devices,

= ~~. .

00

carriers, or media, such as a magnetic storage media, "floppy disk" CD ROM, a file server
112--,_ i cable line or remote infrared signals i

Ul Q

r - - - - - - - - --- -- ----- -- - -- - --- - - - - - - - -- ---1

-.

..

providing access to the programs via-ã:nctworl~ tranomi33ion line, holographic unit, eto.: Of
- --- -- --- --------------- -- ---___ __ ___ __J

course, those skilled in the art will recognize many markings may be made to this configuration

without departing from the scope of the invention.

~ ~
Q'

Filed 08/27/2007

'&

Q'

õo

..

w Q

Page 6 of 20

t: ~

12)
qú¡. 6

~ = -

~ ~ ~ ~

The preferred embodiments may be implemented as a method, apparatus or article of

Case 5:07-cv-00109-JMH

manufacture using standard programming and/or engineering techniques to produce software,

~ = ~

firmware, hardware, or any combination thereof. Th~_ t-eJD1 ~~lj:Lcl~ _oLrr~!Jl,laQt-LiS~ !Q( ---i
114 --____~in_ __ document and_ __ __ __ _ __ discussion J -- this ___ __ __ __ throughout this __ _____ i

r o

N N

S

alternatively, "computer program product" ) as used herein~is intended to encompass one or more
(l ( .l.

computer programs and data files accessible from one or more computer-readable devices,

r; =o. .
="

Document 12

carriers, or media, such as a magnetic storage media, "floppy disk" CD ROM, a file server
116 --'---.-C~P!ejii~-r iiniii~Inj~!~d~~inliIsJ

-i

providing access to the programs via a network transmission line, holoQraphic unit, etc. Of

course, those skiled in the art wil recognize many markings may be made to this configuration

Filed 08/27/2007

without departing from the scope of the invention.

~ r; ~ '& ~ ..
Qc

c.

==

Page 7 of 20

tI:

Case 5:07-cv-00109-JMH

Document 12

Filed 08/27/2007

Page 8 of 20

u.s. Patent

May 20, 2003

Sheet 7 of 7

US 6,567,830 Bl

120
Wait for user to select option to insert text.

VJú¡. 7

122
Generate transparent graphical text box in which user may enter text between the lines of text.

No

132
Save information on graphical text box in marking fie, including addressable location of film, size dimensions, and inserted text.

126

130

134
Display user entered text in

graphical text box; expand
transparent fim horizontally to receive further input.

~

End logic.

)

Case 5:07-cv-00109-JMH

Document 12

Filed 08/27/2007

Page 9 of 20

US 6,567,830 Bl
1

2
then diffculties may arse tracking the different versions reflecting the mark-ups and edits from different users in the network system. There is thus a need in the art for an improved system, melhod, and program for adding text to an eleclronic media
file.

METHOD, SYSTEM, AND PROGRA FOR DISPlAYING ADDED TEXT TO AN ELECTRONIC MEDIA FILE

CROSS-REFERENCE TO RELAD
APPLICATIONS
"Method, System,

And Program For Displaying Markings SUMMARY OF THE PREFERRED to an Electronic Media File," by Rari Madduri, having U.S, EMBODIMENTS application Ser. No. 09/249,201; and "Method, System, And 10 To overcome the limitations in the prior art described Program For Sioring And Retrieving Markings for Display above, preferrcd embodiments disclose a method, system, to an Electronic Media File," by Rari Madduri, having U.S. and program for receiving text to add to a displayed text application Ser. No. 09/249,202.

document. The program displays lines of text from the
document and receives usr inputted text. The program
text and generates the user inputted text having a size

BACKGROUND OF THE INVENTON
1. Field of the Invention

15 generates a graphical element to include the user inputted

The present invention relates to a melhod, system, and
program for receiving user inputted text to add to a text documenl and displaying the added text in ihe document.

capable of fittig between two lines of the text from the
doument. The program then displays the graphical element

20 element is displayed between two lines of text from the 2. Descripiion of the Related Ar docment. In current document editing systems, users typically enter In further embodiments, a markig file includes informamarkings and modifications to a documenl and store such
markigs and modifcaiions with the document. A marking

in the document. The user inputted text within the graphical

can be any highlighiing, sirieoul, line, font change, handwritten indication added to text, underlining, double

tion indicating a graphical element including insert text and an addressable location indicating a position of the graphical 25 element in the document. The program processes the mark-

underline, redline, bold, italics, etc. A highlightig is a thin color film through which the underlying text is visible. For

ing file to generate the graphical element to superimpose with respect to the addressable location in the document.
The program then displays the insert text in the graphical

group editing projects, a group member may edit and

element such that the displayed insert text is displayed m.rkup ihe document and transmit the revised and edited 30 between two lines of text in the document. document 10 another group member for review aod further editing. In such group editing environments, groups of users In preferred emhodiments, information on the graphical can edit a document on-line by entering textual notes into the element is stored in a file separate from the text document. document being editcd. Such notes are typically entered into This separate file includes an addressable location of where pop-up widows which appear in the document. The user 35 to position the graphical element, user inputted text, and the may enter lextual memos into the pop-up window.
generated size of the graphical element and user inputted
text. U.S. Pat. No. 5,146,552, cniitlcd "Meihod for Asociating Annotation with Electronically Published Material," which With preferred embodiments, users may use an enhanced is assigned to I ntemational Business Machines Corporation viewer to insert text into a document. This preferred ("IBM"), the assignee of the subject patent application and 40 enhanced viewer would generate a graphical element or which is incorporated herein by reference in its entirety, graphical text box to store the user inputted text. The viewer describes a program which allows a user to review and program would then superimpose the graphical text box or
display annotations or notes created by the user or shared

element in the text such that the user inputted text would

and exchanged between users with access to a copy of the appear between two lines and the underlying text from the same published electronic document. The annotations may 45 document would appear through another portion of the be associated with a particular context within the document. graphical text hox. This .llows a user to enter insert text Users can rcview and merge notes from other users, and which is then automatically processed and generated to display such notes with the electronic document. display as inserted between lines within a document. Many of the current systems for editing and markig-up In further embodiments, information on the generated documents are intcnded for documents which are read/wnte 50 graphical element, including the user inputted text, is stored capable, thereby alowig the usr to make changes directly in a separate file. In this way, the displayed overlay of Ihe into the document. However, in many cases, a user may be graphical elemenl including the usr inserted text onto the viewig a document in a read-only format. For instance, a underlying document text does not aller the formatting,
usr may be reviewing a Hypertext Mark-Up Language
spacing or content of the electronic media file because

(HTML) document with an HTML browser, e.g., 55 information on the inserted text is stored separately from the
NETSCAPE Communicator, the MICROSOFT Internet Explorer, Spry Mosaic, NCSA Mosaic, Lynx, Opera, GNUs-

cape Navigator et. With such HTML browsers and olher
viewers for viewig read-only documents, such as the
ADOBE

text document. Preferred embodiments are particularly applicable to network environments where multiple users may edit a docu-

Acrobat Reader, users typically cannot directly edit 60 viewer program of the preferred embodiments. Later, the file the HTML document while in viewing mode.' storing the usr inserted text may be selected and the viewer Text editors, such as WORDPERFECT and program would then superimpose the inserted text onto the MICROSOFT WORD, allow uscrs to mark-up and edit a displayed document. This allows users in a network to share document.' However, with such text editors, the user must marking files including insert text to a document without save the modifed and marked-up file over the original 65 altering the document. A usr need only receive the marking document or save the original and edited document in file transmitted over a network if the user otherwise has a separate files. If many users are group editing a document, copy of the asciated document to which the markigs

ment. Each user may insert text into a docuent using ihe

Case 5:07-cv-00109-JMH

Document 12

Filed 08/27/2007

Page 10 of 20

US 6,567,830 Bl
3
apply. Further, users may selectively display insert text stored in graphical elements from other people without altering the underlying text document. This alIows multiple users to insert text and edit the document without generating multiple versions of the electronic media file throughout the network.

4
However, the documents lOa, b, and c including text may also include other electronic media content, such as graphi-

cal images, video clips, audio files, etc. In preferred
embodiments, the documents lOa, b, and c are in a read only
format. The operating system 6 may be any suitable oper-

ating system such as AlXil. OS/390TM, UNIX, OS/21i,
MYSTM, WINDOWS, WINDOWS NT, etc.

In stil further embodiments, users may retrieve markig
files from different human editors and asociate colors with

A user views and controls computer 2 operations via a the diferent markig files to identify the editor of the display monitor 12, which may be any suitable display markigs. In this way, when the user displays graphical 10 device known in the art. The computer 2 includes a video elements including insert text and other markigs from controlIer 14 to generate display control information and to
diferent marking fies, the user can readily identify the
control the display of data and graphics on the display

source of the markig or insert text based on the color in
which the marking or insert text is displayed.

monitor 12. The video controlIer 14 may be any suitable graphics card or video controller device known in the art. 15 The computer 2 further includes an input means 16, which BRIEF DESCRIPTION OF THE DRAWINGS may be a keyboard, mouse, pen-stylus, voice activated input, touch-sensitive display screen, etc. Alo included is a netReferring now to the drawings in which like reference
work card 18 to alIow the computer 2 10 communicate with

numbers represent corresponding parts throughout:
FIG. 1 is a block diagram ilustrating a software and 20

other computers over a network, such as TCPIIP, LAN,

WAN, ETHERNET, infrared communication means, etc. The network card 18 may be a modem or any other network the present invention are implemented; card interface known in the art to alIow the computer 2 to FIG. 2 ilustrates the arrangement of a marking file in communicate with the network. In preferred embodiments, accordance with preferred embodiments of the presnt the computer 2 may communicate with other computers over 25 the network to transfer documents lOa, b, and c. The other invention; computers with which the computer 2 communicates may FIG. 3 ilustrates program logic to create and store markhardware environment in which preferred embodiments of
ings in accordance with preferred embodiments of the
present invention;

FIG. 4 ilustrates program logic to select and display
the present invention.
FIGS. 5 and 6 ilustrate how a graphical element may

include the viewer program 8 to view, or view and add, markigs in accordance with the preferred embodiments. In preferred embodiments, a user inputs markigs to the
the display 12. In preferred embodiments, the operating

markig files in accordance with preferred embodiments of 30 document lOa, b, and c while the document is displayed on
system 6 provides for a windows graphical display environment in which different operating system tasks are executed display insert text in accordance with preferred embodiin different windows. Within the viewer program 8, the user ments of the present invention; and 35 would select, with the input means 16, a document lOa, b,
FIG. 7 ilustrates logic to receive and process user insert

text in accordance with preferred embodiments of the

present invention.

or c to open in the viewer program 8. The selected document lOa, b, or c would be loaded into a volatile memory area of

the computer, e.g., a RA, DRA, SRA, etc. The video
controller 14 would then generate display control informa40 tion for document lOa, b. or c to display the electronic media
within document lOa, b, or c on the display monitor 12.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

In the following description, reference is made to the accompanying drawigs which form a part hereof, and which ilustrate several embodiments of the present invention. It is understood that other embodiments may be utilied and structural and operational changes may be made without
departing from the scope of the present invention.
Hardware and Software Environment

When the user selects a marking file to display markings superiposed on the electronic media of the electronic
45 media fie, there are at least two possible methods to

generate the display control information. With one method, the video controlIer 14 may generate markings directly into a copy of the document IDa, b, or c maintained in a volatile
memory device as a temporary file. Such markings would

FIG. 1 ilustrates a hardware and software environment in which preferred embodiments of the present invention are

50 not be stored in the document tOa, b, or c in non-volatile
play control information for the modified document

memory. The video controller 14 would then generate dis-

(temporary file) maintained in the volatile memory area to display the markings superimposed on the electronic media computer 2 may be a personal computer, workstation, mainframe, personal digital assistant (PDA), etc., operating 55 (text, images, etc.). With a second method, the video controlIer 14 could generate display control information for the under microprocessor control. The memory 4 may be any markings and separately display the electronic media and suitable volatile memory device known in the art, e.g., markigs on the screen such that the displayed markigs are RA, DRA, SRAM, etc., or a combination of volatile and displayed superimposed onto the displayed electronic non-volatile memory devices, e.g., hard disk drives, to provide storage space for programs, whether they are being 60 media. Those skiled in the art wil appreciate that there are alternatives ways for a video controller 14 to generate and executed or not. If the memory 4 is compris of volatile and display data from two separate files. non-volatile memory devices, then data and programs may be swapped between the volatile and non-volatile devices in The Viewer Program a manner known in the art.
The memory 4 includes an operatig system 6 and a 65
viewer program 8. Certain preferred embodiments are
described with respect to electronic media compried of

implemented. A computer 2 includes a memory 4. The

The viewer program 8 includes viewer capabilities to
alIow the user to view documents lOa, b, or c comprised of

text.

multiple types of electronic media. The viewer program 8

Case 5:07-cv-00109-JMH

Document 12

Filed 08/27/2007

Page 11 of 20

US 6,567,830 Bl
5
includes the viewing capability of browser programs such as HlML browsers, e.g., Netscape Communicator, Microsoft Internet Explorer, etc, and may also include tbe capability of
various add-on programs, such as video players and audio

6
The document lOa, b, c may be organized according to any documenl format or content addressing system known in

players, to enable thc display and procssing of diferent
electronic media types. The documents lOa, b, and c (electronic media files) may include text in any text format known in the art, including HlML, ASCII, ANSI, Rich Text Formal (RTF), a format specifc to a word processor docu-

the art. For instance, if the marked electronic media is text, then the address information 22 may indicate the start and end for the corresponding markings in the text as an offset
number from the beginning of the document. Alternatively,

text in the documents lOa, b, c may be addressed according to paragraph, wherein the paragraph is the addressable entity
and text is addressed accrding to an offet within an

ment (e.g., WordPerfect, Microsoft Word) or any other 10 addressble paragraph. In such case, paragraph delimiter information is used to identif the location of text in each format known in the art for displaying text. The documents paragraph grouping and the application of fonts, margins, lOa, b, and c may also include other media types, such as
end of paragrapb for eacb block of text in a document. If the the format of the document lOa, b, and c is capable of maintaining address information for multiple electronic 15 text in the document lOa was grouped according to

images, pictures, video clips, sound clips, etc. In such case,

and justification are dependent on accurately denoting the

paragraphs, then the address information 22 would include such paragi~ph delimiter information to identify where the modifications apply in the document lOa. Further, docusuperimposed onto the text or images of a displayed documents may be addressed according to levels of objects or ment lOa, b, and c, including highlighting of the text or other sections. Addressing according to levels of objects images in diferent colors, addition of markings, strikeout of 20 permits addressing of non-textual electronic media, such as text or any modification or editing known in the art. These images. In embodiments where the documents IDa, b, c viewing and marking features are known in the art and include non-textual data, such as images or video-clips, the included in many types of text editors. documents lOa, b, c would include a format to address such In preferrd embodiments, the viewer program 8 would images andlor video clips and portions therein within the display the document lOa, b, or c content in a single 25 docment lOa, b, c. graphical window. Using the input means 16, the user would The markigs file 20 would als include document idenindicate markings and display such markings directly onto tification information to identify the document lOa, b, c to which the markig file 20 applies. Th would prevent a usr the content of the document lOa, b, c displayed in the from applying a markigs file 20 to a document that was not window. The user may make markigs by selecting or highlighting a particular section of the electronic media 30 the document for which the markings werc generated. As discussd, the viewer program 8 permits any docu(text, image) and then selecting from a displayed menu or ment markings known in the art, including strikeout, tool bar a particular type of marking to apply to the highmedia types. Thc viewer program 8 also includes the capability to allow a user to enter and display markings as

lighted electronic media, e.g., redlining, strikeout, underlining, italics, bold, etc. Alternatively, thc uscr may
make markings by "free-hand" drawing markigs using the 35

highlightig, underlining, double underlie, redline, bold, italics, etc. Further, the marking information 24 may describe an annotation note added to the text. Stil further,
the user may add text to the electronic media. An addition to

mouse pointer or pen-stylus. In preferred embodiments, the markigs the user enters with the viewer program 8 arc stored in a fie separate from the document lOa, b, or c. Thus, the entered markigs that
not alter the format, spacing or content of the underlying

text is new text superimposed on the underlying text or
image within the document lOa, b or c as desribed below.

The viewer program 8 would further include menu
dow in which the viewer program 8 is executing. Such menu options would include a featue to alow the user to retrieve one or more previously stored marking files 20. Upon

are displayed superimposed on the document lOa, b, or c do 40 options the user could selectively display in the GUI wi-

document lOa, b, or c because the markings are stored

separately. Further, in preferred embodiments, the viewer selecting a markig file 20 for a dislayed document, the may be restricted to not allow the user to add markigs that would alter the underlying structure of the document lOa, b 45 video controller 14 would generate display control information to superimpos the markigs on the text in the document or c, such as altering the positioning of text within the lOa, b, c displayed on the display monitor 12. In preferred document. Moreover, because users may input markings to thc document lOa, b or c and store the markings separately, embodiments, the usr may concurrently superimpose on a new version of the document is not created each time a user electronic media markings from multiple markig files. Preferred embodiments are particularly applicable to reading documents on-line and, in particular, documcnts, FIG. 2 ilustrates a preferred embodiment of a markings such as on-line books, provided in a read-only format. With filc 20 which stores markings a user makes to a document,

edits or revises the document. 50
e,g., document lOa with the viewer program 8. The mark-

the viewer program 8 of the preferred embodimenl~, the user ings file 20, which is a separate file from the corresponding may display a read only document and create and save document lOa, includes address 22 and marking 24 infor- 55 markings to the document within the window in which the

mation. The markig information 24 describes thc type of markig made through the viewer program 8. The address
information 22 describes the address or position of the

document is displayed. Users could als use the viewer

described markig in the electronic media within the document lOa. The address information 22 maps the markings 24 60

program to comment upon the appearance or content of an image or video clip. Storing the markings would not modif the actual contents of the document lOa, b, c because the

markigs are stored separately in the markig fie 20.
Further, viewers in accordance with preferred embodiments
do not allow the usr to alter the format or positioning of the

to the document. The address information 22 would include information on the addressing context of the document lOa to properly locate the markings described in the modification information 24 to the electronic media, e.g., text, images, in the document lOa. A markigs fie 20 may include multiple 65 instances or records of addrcss 22 and markings 24 information for each marking and modifcation made.

text in the underlying document. Preferred embodiments
only allow the superimposition of markings over the underlying text.

With preferred embodiments, a usr may enter a marking,

annotation or addition to text and choose to have the

Case 5:07-cv-00109-JMH

Document 12

Filed 08/27/2007

Page 12 of 20

US 6,567,830 Bl
7
marking to be displayed in a selected color. A user may then transmit the marking fie 20 to another usr over a network. In this way, a user at a computer 2 may maintain several

8
each users markings on the display monitor 12. The natural combined color would form on the display monitor 12 as a result of the display effect of superiposing the display of diferent colors. Those skiled in the art will appreciate that there are numerous ways to display markings from different
usrs to superimpose overlapping markings and colors

markig files from diferent users, wherein the markigs of each user are maintained in a diferent, distinctive color to
identify the u~er that created the markings. A user, using the

menu selections displayed in the viewer program 8, may then select the markig fies for multiple usrs. The computer 2 would generate display information to superimpose the markings from multiple marking files from diferent
usrs over the electronic media within the document lOa, b
or c.

resulting from such overlapping markings onto electronic media. When selecting a markig file to display on the displayed 10 electronic media, the user may select the color used to

display the selected markings. In this way, a user can

identify colors with particular reviewers that created the markings. The user may als select a color to display ng markings from two embodimenl~, the viewer program 8 would display a dialog 15 overlappicolor may differ from or more other users. This selected the color formed by the box on the monitor 12 to allow the user to selectively
When a user selects a marking fie to import and super-

impose on the document lOa, b or c, in preferred

editor and the color asciated with overlapping markigs diferent markig files. Moreover, when the marking files are from multiple editors. generated by diferent reviewers of the document lOa, b or U ers may indicate their own markings c, the usr can select a color to asociate with a paricular 2S thanscolor, such as a displayed pattern of by means other thc markigs or reviewer when displaying that reviewers markings superimdiferent fonts or formats, e.g., bold, italics, font size, font posed on the electronic media of the document lOa, b or c. type, etc. Moreover, the usr may select to block out or not If the user does not select a particular color to asociate with display markigs from one or more markig files for the markings from a selected marking file, then the viewer selec ed pages program 8 may select a default color. The default color may 30 menttlOa, b or or portions of the underlying text in docuc. be a color provided by the user who created the markig file Preferred embodiments are particularly useful for allowor may be a color the viewer program 8 selects in a manner ing a user to view the markings, notes or suggested revisions to avoid conficting with colors already used to display from different users. Moreover, a user may select markigs markings. In further embodiments, the viewer program 8 may only 3S indicated by the author or creator of the document, a famous critic or other esteemed individual to view how especially allow the user to select a color to display markings that not insightful people marked-up ihe document lOa, b or c. If the only does not confict with the current colors used to display markigs include annotation notes associated with particular markings but also does not conflict with the color in which
locations in the document lOa, b or c, then the user may the user is entering markigs onto the electronic media. For intance, if the user is markig displayed text in red and the 40 review the notes others made into the text or image. In group editing environments, the editor may review the suggested user selects to display the markings from another markigs modifications of diferent people and then determine which file, then the viewer program 8 would not permit the usr to final revisions to apply based on the received markings. select red to display the markings from the selected marking file. In further embodiments, the viewer program 8 may Because, in preferred embodiments, the marking file 20 is include an algorithm to determine similar colors that may 45 maintained separate from the document, users need only conflict, such as red or magenta. In such case, the viewer exchange markig files over a network and not the entire program 8 would not allow the user to select a color that is document. However, a user that wants to display the markdifferent yet conflicts with the colors used to display the ings from a marking file 20 with electronic media from the current markings in the document. document lOa, b or C must maintain a copy of the document If markings from different users overlap in the displayed 50 lOa, b or c to which the marking file applies 20. Otherwse, the user would not be able to display the markings superdocument, then the ovcrlapping markings may cause the imposed on the underlying electronic media content as the display of a color that is a combination of the colors of the content is stored separately in the document lOa. overlapping markings. For instance, yellow may be used to display editor A's markings and red may be used to display FIGS. 3 and 4 ilustrate logic implemented in the viewer editor B's markings (wherein editor one and editor two's 55 program 8 to proces markings to a document lOa, b or c. markings arc maintained in separate markig files). In such Control begins al block 30 which represents the viewer case, superimposing markings from editor A and editor B on program 8 waiting for the user to select a markig option. As the same section of electrnic media would display the color discussd, the user may select markig options from a menu formed from the combination of the overlapping marking bar displaycd in tbe window in which the viewer program 8 colors, which in the example would be orange. 60 is executing. Alternatively, the user may display marking
The computer 2 may store the combination color formed

markings. For instance, if editor A's markigs are in yellow and editor B's markings are in red, then the user may select th markings from editor the dialog box would present the user colors to select that are 20 A e color green to display overlapping 8 may furter display and editor B. The viewer program not currently being used to display markings. In this way, the a legend showig the colors asociated with a particular same color will not be usd to display markings from

associate a color with the markings from the selected

combination of the colors assciated with the overlapping

markings file 20. In preferred embodiments, if there are markigs from other markig files already displayed, then

options upon selecting a mouse button or "hot" key from the

by superiposing markigs from different users into a keyboard input means 16. Afer the user selects a marking temporary copy of the document in volatile memory. The option, control transfers to block 32 where the viewer video controller 14 may then generate display information program 8 determines whether the user has selected the note from this temporary file including the markings from dif- 65 annotation option to enter annotation text in the document.
ferent users and the conten t of the electronic media.
If so, control transfers to block 34; otherwise, control

Alternatively, the video controller 14 may separately display

transfers to block 36. At block 34, the viewer program 8

Case 5:07-cv-00109-JMH

Document 12

Filed 08/27/2007

Page 13 of 20

US 6,567,830 Bl
9
displays a pop-up window in the window in which the
viewer program 8 is executing in which the user may enter annotation text. Control then transfers to block 38 where the viewer program 8 wailS for the user to save the note or close the note pop-up window without saving note annotations. If
the user closes the pop-up window or note annotation

10
without generating additional version numbers of the document and without having 10 transmit the actual document. The underlying marked-up document need not be transferred as long as the user applying the marking fie 20 has a copy
of the underlying document lOa, b or c to which the

without saving text, then the program would return to block 30 to wait for further user action. Otherwse, control transfers to block 40 where the viewer program 8 determines the
address location of a pointer or cursor the user positioned

markings apply. This feature reduces network traffc because those groups editing the document do not need to transfer the
actual document, which could be quite large, every time they

send a copy of their markigs to others involved in editing
10 the document.
FIG. 4 ilustrates logic implemented in the viewer pro-

gram 8 to display the markings on the displayed document lOa, b or c. Control begins at block 70 which represents the 8 stores the annotative text the user entered in the pop-up viewer program 8 waiting for the user to select the markigs window in the marking information 24 field and the deter- 15 menu. Upon selection, control transfers to block 72 which mined address information of where the marking was made represents the viewer program 8 displaying one or more in the electronic media in the address information field 22 of marking files 20 assoiated with the document lOa, b or c the separate marking file 20. being displayed in the active widow on the display 12. If the user did not select the note annotation marking Control transfers to block 74 where the viewer program 8 feature, then control transfers to block 36 where the viewer 20 waits for the user to select with the input means 16 one of program 8 determines whether a block of text in the disthe displayed marking files 20. In preferred embodiments. played document is selected, e.g., highlighted with the after the user selects a markig fie 20, a dialog box may be cursor or pointer. If so. control transfers to block 44; displayed in which the user would select a color in which to otherwe, control transfers to block 46 where the viewer display the markings form the selected marking fie. The program 8 displays marking options. The marking options 25 markings would then be displayed in the selected color. The may allow the usr to add markup to the location of the user may also be presented with a dialog box in which to cursor or pointer in the document. Altcrnatively, the user select a color to display overlapping markings from separate may tag the location of the cursor or pointer to associate a marking files that mark the same location of electronic bookmark or tag to the location of the pointer. The markig media, i.e., the same portion of text or same part of an image. or tag information would be stored in a separate marking fie 30 Control then transfers to block 78 where the viewer 20, such that the markig or tagging would not alter the program 8 processes the selected marking fie to determine format, spacing or content of the document lOa in which the whether the selected markigs overlap any markigs curmarkig is indicated. From block 46, control transfers to rently displayed in the document lOa, b, c displayed on the block 48 where the viewer program 8 waits for the user to monitor 12. If so, control transfers to block 80; otherw select a marking or taggg option to add to the document 35 control transfers to block 82. At block 82. the viewer lOa, b or c. Afer selection, control transfers to block SO program 8 determines whether the user selected a color for where the viewer program 8 determines the address location the selected markings. If so, control transfers to block 84 to of the cursr or pointer in the displayed document when the cause the video controller 14 to generate the display control markig or tagging option was invoked. Control then transinformation for the selected markigs in the user selected fers to block 52 where the viewer program 8 stores the 40 color. Otherwse, control transfers to block 86 to cause the marking tagging type in the markig information 22 field video controller 14 to generate the display control informaand the determined addrcss location, where the mark or tag tion for the selected markings in the color indicated in the was made, in the addres information field 24. marking information 24 of the marking file 20 or in a default If a block of text, such as a plurality of characters, is color maintained by the viewer program 8. From blocks 84 selected, then at block 44 the viewer program 8 displays 45 or 86, control transfers to block 88 where the video conmarking options that may apply to a block of text, referred troller 14 displays the markings according to the generated to herein as textual marking options, such as highlighting. display control information. As discussed, the video conredlining, strikeout, or other markings to be made to the troller 14 may generate the display control information and selected text. Further, a tagging option may be provided to store the display control information in a video memory for allow the user to tag or bookmark the selected text. The user 50 display on the display monitor 12. The video controller 14 may select a color for the markig, e.g., a red highlight. may generate the display control information to display over Control then transfers to block 54 where the viewer program any electronic media from the document lOa, b, c or 8 waits for the user to select a displayed marking option. markings presently displayed. Alternatively, the viewer proAfter the user selects a marking option or tag for the selected gram 8 may store the markings directly in a temporary file text, control transfers to block 56 wherc the viewer program 55 including the document, and the video controller 14 would 8 determines address information for the start and end in the generate the display control information from such tempoIf the markings from the selected markig file overlap any storing the markig type (including color information if markigs already displayed on the display monitor 12, then selected) in a marking information field 24 along with the 50 control transfers from block 78 to block 80 which represents determined address information in an address information the viewer program 8 determining whether the usr selected
22 field.
a color for the specifc overlapping markings. If so, control

with the input means 16 in the displayed document lOa, b or c content when the annotation marking feature was invoked. Control then transfers 10 block 42 where the viewer program

document lOa, b or c of the highlighted text. Control then transfers to block 58 which represents the viewer program 8

rary file.

In this way, a user can enter various markings or tags to transfers to block 90; otherwise, control transfers to block 82 a document without altering the actual document. Moreover, to display the markings in the displayed document lOa, b, c. because, in preferred embodiments, the markings are stored 65 The video controller 14 may generate display information to in a separate marking file 20, the user's markings file can be display the markings over the curent displayed document, transmitted to other users for review with the document including any currently displayed markings. In such case,

Case 5:07-cv-00109-JMH

Document 12

Filed 08/27/2007

Page 14 of 20

US 6,567,830 Bl
11
any markings from the selected marking file will be displayed as directly superimposed over the displayed electronic media, including markings from other markig fies. To the extent the superimposed markings are in one color and the displayed electronic media (including other
markings) that overlap the location of the selected markings are in another color, a color that is a natural combination of the overlapping marking colors will be displayed. If the user
selected a specific ovcrlap color for the overlapping

12
between the lines of text. With FIG. 6, the lower left comer of the graphical text element 114 and 116 is positioned between the lines of text. In this way, the graphical text element 114 and 116 is positioned at a vertical offet from the addressable location in the underlying text where the usr selected the option to insert or add text. This vertical

offset locates the graphical text element 114 and 116
between lines of text in the underlying text.

The information to generate the transparent graphical text markings, then control transfers to block 90 to cause the box the underlying text is stored in the video controller 14 to generate display information for the 10 markwith respect toaddress information 22 would indicate ing file 20. The overlapping markings in the user selected overlap color. the addressable location in the underlyiog text in document Control transfers to block 92 where the video controller 14 lOa, b or c where the user indicated to add the text. In the displays the overlapping markigs according to the display case of the embodiments of FIG. 5, this addressable location control information. Control then transfers to block 94 which represents the viewer program 8 proceeding to block 15 is where the lower left comer of the transparent film graphical text element is positioned. In the cas of the embodi72 to cause the video controller 14 to generate display ments of FIG. 6, vertical offet ioformation is also maincontrol information to display the non-overlapping mark-

graphical text element 114 and 116 at a vertical offset from created by multiple users to display over tbe document 20 the addressable location selected by the user, between two lines oftext. The marking information 24 would indicate the displayed on tbe display monitor 12. In this way, the user can dimensions of the graphical text box and the text the user view markings, comments or highighting of tbe electronic entered to be displayed in the upper half of the graphical text media from diferent users with access to the underlyiog box. When the marking file 20 is selected, the viewer documcnt lOa, b or c. Further as discussd, with preferred embodiments, the user can select desired color schemes for 2S program 8 would generate any graphical text boxes 110, 112, 114, and 116 in the marking file 20 to superimpose in the any markings in a file and for specifc overlapping marking document lOa, b or c. combinations. Because the usr can manipulate markings The graphical text boxes 110, LU, 114, and 116 shown in without altering the underlying document, only one version FIGS. 5 and 6 are bordered by dashed lines. In preferred of the underlying document need be maintained by each user 30 embodiments, the transparent graphical text box would be reviewing the markings.

ings.

tained in the markig file 20 for us in positioning the

With preferred embodiments, a user can select markings

Adding Text to the Displayed Electronic Media
File

displayed without any dashed lines so all that would be
viewed would be the inserted text superiposed between the

lines of underlying text. In the embodiments of FIG. 6, the In certain instances, users may want to add text above a underlying tcxt is displayed through the lower half of the line of text displayed in the document lOa, b or c. Preferred 3S transparent graphical text boxes 110 and LU and the inserted embodiments include a feature to allow the user to insert text text the user entered would be displayed in the upper balf of above a line of text in the displayed document lOa, b or c, the graphical text box 10 and 112, displayed between the in between two lines of text displayed in the underlying lines. In alternative embodiments, the usr may select to document lOa, b or c. This is accomplihed by the viewer display lies around the graphical text box 110 and 112 or program 8 generating a graphical text box in which the user 40 shade the graphical text box 110 and 112 box. Fuher, the may enter text. This graphical box, which appears as a user may select to have a carrol or other markig superimtransparent film superimposed on the text in the document posed on the underlying text, such as the case with graphical lOa, b or c, is displayed at an addressable location within the text elements 110 and 114, wherein the carrot marking document. identifes where to insert the added text in the underlying FIGS. 5 and 6 illustrate embodiments showing how the 4S document. Alternatively, the graphical text element may be graphical text box may be displayed as a transparent film or displayed without any location marking, such as the case label inserted at places within the document to add text. FIG. witb graphical text elements 112 and 116. 5 ilustrates text displayed from an electronic file lOa, b or FIG. 7 ilustrates logic implemented in the viewer proc on display monitor U. The user has inserted text which is gram 8 to alow the usr to insert text. Control begins al maintained in graphical text boxes 110 and 112 for display 50 block UO where the viewer program 8 waits for the user to in the document lOa, b or c. To insert text, the user selects select, using the input means 16, the option to insert text. At an addressable location within the text in the document lOa, this point, the user's cursor is preferably at an addressable b or c. The viewer program 8 then displays a graphical text location within the text. This may occur at blocks 48 or 54 box in which the user enters text. The viewer program 8 in FIG. 3. Control then transfers to block 122 (FIG. 6) where superimposes the graphical text box 110, 1U over the 5S the viewer program 8 generates the graphical text box in the underlying text such that the lower left corner of the graphidocument. Initially, before any text is entered, the graphical cal text is at the addressable location in the underlying text text box may have no width and consist of a blining cursor. where the user selected the option to insert or add text. The Control then transfers to iJlock U4 to await user action. user may then insert the addcd tcxt using the keyboard or Upon user action, control transfers to block 126 where tbe pen stylus to be displayed in the upper half of the film above 60 viewer program 8 determines whether the user enteted text. the underlying text displayed through the Inwer half of the If so, control transfers to block 128 to display the user transparent film. In preferred embodiments, the graphical entered text in the graphical text box. When displaying the text box is dislayed as a transparent film such that underuser entered text, the viewer program 8 will size the entered lying text in the document displays through the lower text, and if necessry determine a smaller font size, such that portion of the graphical text box superimposed thereon. 6S the entered text is positioned above the line of underlying FIG. 6 ilustrates an alternative embodiment in which the text where the cursor is positioned, but below the line of text graphical element, shown as 114 and 116, are positioned directly above. The viewer program 8 may then expand the

Case 5:07-cv-00109-JMH

Document 12

Filed 08/27/2007

Page 15 of 20

US 6,567,830 Bl
13
graphical text box 110, 112, 114 or 116 horizontally to provide more space in the graphical text box for the user to enter further text. In alternative embodiments, the viewer

14
110 and 112 would not appear in the color associated with the marking file including the inserted text, and would appear unaffected by the graphical text box 110 and 112.

program 8 may display the graphical text box without a blinkng cursor to allow the user to enter text with a pen
stylus to add to the upper half of the graphical text box,

Further, to the extent other markings were made to the
underlying text and displayed in a color superimposed on the text, then those colored markings to the underlying text

would appear through the lower half of the graphical text box 110 and 112 in a manner that would not alter the color In the embodiments of FIG. 5, the graphical text boxes of the underlying text and any other markings superimposed 110 and 112 display the user entered text in the upper half 10 thereon. For instance, if markings from one markig file of the box, between the lines of underlying text, and the superiposed blue strieouts on the underlying text and underlying text is displayed through the lower half of the another marking file superimposed a transparent graphical transparent graphical text boxes 110 and 112. text box adding text in yellow, then the strikeout would From block 128, control transfers back to block 124 to appear as blue over the underlying text and the added text await further user input. If, at block 126, the user did not 15 would appear yellow above the line of stried-out text. enter text, then control transfers to block 130 to determine In certin embodiments, the graphical text boxes 110, 112, whether the user selected to end text insertion. If so, control 114, and 116 and the graphical interface of the viewer transfers to block 132; otherwse, control transfers back to
graphical text box.

above the underlying text displayed in the lower half of the

block 124 to wait for the user to select a recognizable action. Classes, including the Java Swig Components. Using Java At block 132, the viewer program 8 saves the information on 20 Swig Components, the graphical text boxes may be implethe graphical text box including inserted text in the marking mented as Swing text components. In further embodiments, file 20. In preferred embodiments, the viewer program 8 the graphical box may be a graphical element having a shape would save the addressable location of the underlying text other than a box that is capable of being transparent and where the user pointing device, e.g., mouse pointer or cursor, displaying user inputted text in the document. Moreover, if was located when the user invoked the text insert option. 25 the graphical element has shapes diferent from a box, then With the graphical text boxes 110 and 112 in the embodithe user inputted text and underlying text may be displayed ment of FIG. 5, the saved address information would be used with respect to different portions of the graphical element,

program 8 may be implemented using the Java* Foundation

to determine where to display the lower left comer of the other than described with respect to text boxes 110, 112, 114, transparent graphical text box 110 or 112. With the embodi. and 116. ment of FIG. 6, the saved address information would further 30 In further embodiments, after transparent graphical text include a vertical offset, such that the graphical text box 114
or 116 would be displayed at a position that is a vertical
offet from the saved addressable location in the underlying

boxes are superimposed on the document lOa, bore, the

usr may, using the input means 16, move the graphical text

box around the display U to other, preferably blank, areas text. In the markig information 24, the viewer program 8 would save information on the dimensions of the graphical 35 of the underlying text document. When moving the graphical text box, the user would expse the area previously text box 110, 1U, the font details (size, bold, type, etc.) of covered by the moved graphical text box. In further the entered text, and the entered text. From this saved
information, the viewer program 8 would be able to generate
for display the graphical text box 110, 1U, 114 or 116. After
saving the information on the graphical text box 110, 112, 40

embodiments, if the user were to deselect the moved graphical text box, the graphical text box may then return, lie an elastic band, back to the original addressble location of the
transparent graphical text box, before the box was moved.

114 or 116, control transfers to block 134 to end the text
insertion logic.

In further emhodiments, the user may invoke the insert text option when text is selected, e.g., higWighted. In such
case, in addition to entering text, the user may select 45

This feature allows the user to move the inserted text box to a "white" or blank area of the underlying document to view without intedering text or markings.

marking options to perform with respect to the selected text. For instance, the usr may strkeout or highlight the selected text. Such markings may be stored separately in the markig

Alternative Embodiments and Conclusions
This concludes the description of the preferred embodi-

file 20 as discussed with respect to FIGS. 4 and S. In the
displayed in the lower half of the graphical text box to

ments of the invention. The followig describes some alternative embodiments for accomplishing the present inven-

embodiments of FIG. 5, markings to the text may be 50 tion.

The pteferred embodiments may be implemented as a
method, apparatus or article of manufacture using standard

superimpose over the selected underlying text. For example,

the strikeout may be displayed within the lower half of
graphical text box 112, such that when the graphical text box

programming and/or engineering techniques to produce

software, firmware, hardware, or any combination thereof. 1U is displayed as superimposed over the underlying text, 55 The term "article of manufacture" (or alternatively, "comthe strieout within the graphical text box 112 is displayed puter program product") as used herein is intended to over the selected underlying text. encompass one or more computer programs and data files accessible from one or more computer-readable devices, In preferred embodiments, the graphical text boxes 110, carrers, or media, such as a magnetic storage media, "floppy LU, 114 and 116 would be displayed in a color corresponding to a color the user selectively associated with the 60

disk," CD-ROM, a file server providing access to the
programs via a network transmission line. holographic unit, etc. Of course, those skiled in the art wil recognize many changes may be made to this configuration without departing from the scope of the present invention.
The computer 2, operating system 6, viewer program 8,

marking file 20 including the transparent graphical text
boxes. With respect to the embodiments of FI G. 5, the

viewer program 8 may only display the upper half of the
graphical text boxes 110 and 112 in the color asociated with the marking fie 20 including such inserted text. In this way, 65

only the inserted text wmùd appear in the color and the underlying text in the lower half of the graphical text box

and video controller 14 were described as separate components in preferred embodiments. However, in alternative

Case 5:07-cv-00109-JMH

Document 12

Filed 08/27/2007

Page 16 of 20

US 6,567,830 Bl
15
embodiments, anyone of the functions described as being performed by anyone or combination of components 2, 6, 8, and 14 may b~ performed by another component 2, 6, 8 or 14, or combination thereof. In preferred embodiments, the logic of FIGS. 3 and 4 is implemented as software logic in the viewer program 8. Certain of the logic, such as the logic for gencrating display control information, may be implcmcnted in the viewer

16
In summary, preferred embodiments in accordance with the present invention disclose a method, system, and program for receiving text to add to a displayed text document. The program displays lines of text from the document and receives user inputted text. The program generates a graphical element to include the user inputted text and genera tes

program B and/or the video controller 14, or in other
hardwired or softwired logic within the computer 2. 10

the user inputted text having a size capable of fitting between two lines of the text from the document. The program then

displays the graphical element in the document. The user
inputted (ext within the graphical element is displayed

Moreover, certain of the logic described as being implemented in the video controller 14 may be implemented in the

viewer program 8 or elsewhere in the computer 2. The
steps changed.

between two lines of text from the document. The foregoing description of the preferred embodiments

of the invention has been presented for the purses of preferred logic of FIGS. 3 and 4 is for ilustrative purses, such that steps may be added, removed, and the sequence of 15 ilustration aod description. It is not intended to be exhausThe format of the document may be capable of addressing above teaching. It is intended that the scope of the invention numerous types of electronic media within a document lOa, be limited not by this detailed description, but rather by the b, c, including text, images video clips, audio, etc. In such case, the marking fie would include address information to 20 claims appended hereto. The above specifcation, examples and data provide a complete description of the manufacture address the location of particular markigs to an image, and use of the composition of the invention. Since many video clip, or audio clip. For instance, an input device, such embodiments of the invention can be made without departas a pen-stylus, mouse, touch sensitive screen, joystick, etc., ing from the spirit and scope of the invention, the invention may be used to make markings to a displayed image or video resides in the claims hereinafter appended.
clip frame. In such case, the address information could 25

tive or to limit the invention to the precise form dislosed. Many markings and varations are possible in light of the

address a particular location within the image or a location within an image within a frame of a video clip. For instance, various graphic programs, such as Microsoft Paint, allow a
user to enter and store markings in different colors onto an

"NETSCAPE is a registered trademark of the Netscape
Communications Corpration; MICROSOFT is a registered trademark of Microsoft Corpration; ADOBE is a registered
trademark of Adobe Systems Incorporated; WORDPERFECT is a registered trademark of the Corel Corporation;

imported image. Preferred embodients would save the 30
markigs along with addres information mapping the mark-

AlX, OS/390, MVS, and OS/2 are trademarks of IBM;
WINDOWS and WINDOWS NT are registered trademarks of Microsoft Corporation; UNIX is a registered trademark licensed by the X/Open Company LID; JAVA is a registered
trademark of Sun Microsystern Corporation. What is claimed is:

ings to the image in a file separate from the image.
Preferred embodiments were described as storing the

markigs in a file separate from the document to which the

1. A method for receiving text to add to a displayed text embodiments, the markings may be stored in the same fie document, compriing the steps of: as the underlying document in a manner that does not aller displaying lines of text from the document; the format, spacing or content of the underlying document. receiving a user selected addressable location to insert For instance, the marking file 20 may be stored as an object 40 text in the document and user selection of a text editing within the file in which the underlying document is stored in

markigs apply to avoid alterig the format, spacing or 35 content of the underlying document. In alternative

such a manner that the marking object wil not alter the
underlying document. In this way, the marking object is a module of the document that may be invoked and superimposd onto the content of the underlying document without 45 ever altering the content of the