Apple Inc. v. Samsung Electronics Co. Ltd. et al

Filing 559

Declaration of Deok Keun Matthew Ahn in Support of 558 Response ( Non Motion ), filed byApple Inc.. (Attachments: # 1 Exhibit 1.01, # 2 Exhibit 1.02, # 3 Exhibit 1.03, # 4 Exhibit 1.04, # 5 Exhibit 1.05, # 6 Exhibit 1.06, # 7 Exhibit 1.07, # 8 Exhibit 1.08, # 9 Exhibit 1.09, # 10 Exhibit 1.10, # 11 Exhibit 1.11, # 12 Exhibit 1.12, # 13 Exhibit 1.13)(Related document(s) 558 ) (Jacobs, Michael) (Filed on 12/29/2011)

Download PDF
EXHIBIT 1.05 9 5,202,961 lected flow velocity and the selected flow direction being selected in a one of a tentative set mode and a fixed set mode, in the tentative set mode the se- 10 9. A controller as recited in claim 1, wherein said control icon image includes a first image when said control icon is operating in said tentative set mode and lected flow velocity and the selected flow direction wherein said control icon image includes a second being continuously variable according to the se- 5 image when said control icon is operating in said fixed lected position of the control icon image while the set mode, button is in a one of two button states, in the fixed 10. A controller as recited in claim 9, wherein said set mode the selected flow velocity and the seslide indicator includes at least one interactive control lected flow direction being independent of control dial for indicating a range of gradational flow velocities icon image position, said tentative set mode an said 10 for said sequential information in a predetermined flow fixed set mode being alternatively selectable direction and includes at least one interactive control through operation of said control device; and button for indicating a predetermined flow velocity for means for detecting the presence of said control icon said sequential information in said predetermined flow image at said selected position and for generating direction, wherein said control icon image is operative said control signals corresponding to said selected 15 to be moved to said selected position in response to said flow direction and said selected flow velocity for control dial being selected by said control device, and use in regulating the flow of said sequential inforwherein said control icon image is operative to be mation. moved to said selected position in response to said con2. A controller as recited in claim 1, wherein said trol button being selected by said control device. slide indicator includes at least one interactive control 20 11. A controller as recited in claim 10, wherein said dial for indicating a range of gradational flow velocities slide indicator image has a first region for indicating a for said seguential information in a predetermined flow zero flow velocity for said sequential information, and a direction and includes at least one interactive control second region contiguous to said first region for indicatbutton for indicating a predetermined flow velocity for ing a forward flow velocity of said sequential informasaid sequential information in said predetermined flow 25 tion. direction, wherein said control icon image is operative 12. A controller as recited in claim 11, and further to be moved to said selected position in response to said including a third region contiguous to said first region control dial being selected by said control device, and for indicating a backward flow velocity of said sequenwherein said control icon image is operative to be tial information. moved to said selected position in response to said con- 30 13. A controller as recited in claim 12, wherein said trol button being selected by said control device. 3. A controller as recited in claim 2, wherein slide indicator includes a plurality of said control tons, and wherein said control buttons are radio tons 4. A controller as recited in claim 1, wherein first region includes one of said interactive control but- said butbut- tons, wherein said second region includes one of said control dials, and wherein said third region includes one of said control dials. 35 14. A controller for producing control signals for use said in regulating the flow of sequential information, com- slide indicator image has a first region for indicating a zero flow velocity for said sequential information, and a second region contiguous to said first region for indicating a forward flow velocity of said sequential informa- 40 tion 5. A controller as recited in claim 4, and further including a third region contiguous to said first region for indicating a backward flow velocity of said sequential information. 45 6. A controller as recited in claim 5, wherein said slide indicator includes at least one interactive control dial for indicating a range of gradational flow velocities for said sequential information in a predetermined flow direction and includes at least one interactive control 50 button for indicating a predetermined flow velocity for said sequential information in said predetermined flow direction, wherein said control icon image is operative to be moved to said selected position in response to said prising: a first means for indicating a range of selectable flow directions and flow velocities for said sequential information; a second means responsive to a user for choosing a selection position from said range of selectable flow directions and flow velocities, said selection position corresponding to a selected flow direction and a selected means for operating in a first mode for tentatively maintaining said selection position, means for operating in a second mode for fixedly maintaining said selection position, and means for switching between said first and second modes while maintaining said selection position; and a third means responsive to said second means for detecting said selection position and producing said control signals for regulating the flow of said sequential information as selected by said second control dial being selected by said control device, and 55 means in said selected flow direction and at said wherein said control icon image is operative to be moved to said selected position in response to said control button being selected by said control device. 7. A controller as recited in claim 6, wherein said first region includes one of said interactive control buttons, 60 wherein said second region includes one of said control dials, and wherein said third region includes one of said control dials. 8. A controller as recited in claim 7, wherein said control buttons are radio buttons, wherein said second 65 re on further includes a plurality of said radio buttons, an wherein said third region further includes a plurality of said radio buttons. selected flow velocity. 15. A controller as recited in claim 14, wherein said first means includes a slide mdicator having at least one interactive control dial for indicating a range of gradational flow velocities for said sequential information m a predetermined flow direction and having at least one interactive control button for indicating a predetermined flow velocity for said sequential information m said predetermined flow direction. 16. A controller as recited in claim 15, wherein said slide indicator includes a plurality of said control buttons, and wherein said control buttons are radio buttons. APLNDC00028552 11 5,202,961 12 17. A controller as recited in claim 14, wherein said 27. A controller as recited in claim 26, wherein said first means includes a slide indicator having a first refirst means includes at least one interactive control dial gion for indicating a zero flow velocity for said sequenfor indicating a range of gradational flow velocities for tial information, and a second region contiguous to said said sequential information in a predetermined flow first region for indicating a forward flow velocity for 5 direction and includes one or more mteractive control said sequential information. buttons for indicating a predetermined flow velocity for 18. A controller as recited in claim 17, and further said sequential information in said predetermined flow including a third region contiguous to said first region direction. for indicating a backward flow velocity of said sequen- 10 first region includesas recited in interactivewherein but28. A controller one of said claim 27, control said tial information. 19. A controller as recited in claim 18, wherein said slide indicator includes at least one interactive control tons, wherein said second region includes one of said control dials, and wherein said third region includes one dial for indicating a range of gradational flow velocities of said control dials. for said sequential information in a predetermined flow 29. A controller for use in combmation with a display direction and includes at least one interactive control 15 system and a user operable control device for producing button for indicating said predetermined flow velocity control signals for regulating the flow direction and the for said sequential information in a predetermined flow flow velocity of sequential information, said control device including means for selecting and manipulating dr20 controller as recited in claim 19, wherein said images on a display of said display system, comprising: first region includes one of said interactive control buta first means for generating an interactive controller tons, wherein said second region includes one of said image for display on said display, said controller control dials and a plurality of said control buttons, and image including a plurality of selectable markers wherein said third region includes one of said control and a slide indicator, each said marker corresponddials and a plurality of said control buttons. 25 ing to a predetermined flow velocity for said se21. A controller as recited in claim 17, wherein said quential information in a predetermined flow direccontrol buttons are radio buttons. 22. A controller as recited in claim 14, wherein said controller is used in combination with a display system and a user operable control device including means for selecting and manipulating images on a display of said display system, wherein said first means includes means for generating an interactive slide indicator image portraying said range on said display, and wherein said tion, said slide indicator corresponding to a range of gradational flow velocities for said sequential information in one or more predetermined flow directions; a second means responsive to a user for selecting a selection position along said controller image corresponding to a selected flow direction and a selected flow velocity for said sequential informa- second means further includes means for generating an 35 interactive control icon image on said display visually roximate to said slide indicator image, said control con image being operative to be moved by said control don; means responsive to said second means for a third and detecting said selection position and generatmg said control signals for regulating the flow of said device to said selected position along said slide indica- sequential information as selected by said second tor image, said third means being operative to detect the y sition of said control icon image along said slide indi- means in said selected flow direction and at said selected flow velocity, said second means having a tor image as said selection position and to generate first operating mode, a second operating mode, and said control signals corresponding to said selected flow direction and said selected flow velocity for use in regulating the flow of said sequential information45 a third operating mode, said third means bemg operative to tentatively generate said control signals in response to said first operating mode, said 23. A controller as recited in claim 22, wherein said control icon image has a first image when said second means is operating in said first mode and wherein said control icon image has a second image when said sec- third means being operative to fixedly generate said control signals in response to said second oper- on A co ro err as ecm din 2 herein said y first means includes at least one interactive control dial for indicating a range of gradational flow velocities for d ential information in a predetermined flow ec n and includes at least one interactive control 55 button for indicating a predetermined flow velocity for said sequential information in said predetermined flow ating mode, said third operating mode being operative to switch between said first o ratm m e w n u t en temsni contreol signals. 30. A controller as recited in claim 29, wherein said slide indicator includes at least one interactive control dial representing said range of gradational flow velocities, and wherein each of said markers includes at least one interactive control button representing a predetermined flow velocity and a predetermined flow direc- da25œ. ntroller as recited in claim 22, wherein said . A controller as recited in claim 30, wherein said slide indicator image has a first region for indicating a 60 slide indicator includes a first region corresponding to a zero flow velocity for said sequential information, and a zero flow velocity for said sequential information, and a second region contiguous to said first region for indicatsecond region contiguous to said first region corrg ing a forward flow velocity for said sequential informasponding to a forward range of gradational flow velocaties for said sequential information. tio . A controller as recited in claim 25, and further 65 32. A controller as recited in claim 31, and further clud a third region contiguous to said first region including a third region contiguous to said first region r ind ting a backward flow velocity of said sequencorresponding to a backward range of gradational flow velocities for said sequential information. tial information. APLNDC00028553 5,202,961 13 14 33. A controller as recited in claim 32, wherein said mined flow velocity for said sequential information in first region includes one of said interactive control butone of said predetermined flow directions. tons, wherein said second region includes one of said 41. A controller as recited in claim 40, wherein said control dials and a plurality of said control buttons, and first region includes one of said interactive control butwherein said third region includes one of said control -5 tons, wherein said second region includes one of said dials and a plurality of said control buttons. control dials, and wherein said third region includes one 34. A controller as recited in claim 33, wherein said of said control dials. control buttons are radio buttons. 42. A method for producing control signals for con35. A controller as recited in claim 29, wherein said trolling the flow of sequential information, comprising second means further includes means for generating an 10 the steps of: interactive control icon for display on said display visuselecting a flow operation mode for the flow of the ally proximate to said slide indicator, said control icon sequential information, said flow operation mode be rative to be moved by said control device to including a tentative flow operation mode for consai se tion position along said slide indicator, said trolling the flow of said sequential information third means being operative to detect the position of 15 according to a position of a control icon image said control icon along said slide indicator as said selection position and to generate said control signals corresponding to said selected flow direction and said se- while a selecting means of a control device is in a one of two states, on a temporary basis and a fixed flow operation mode for maintaining a fixed flow lected flow velocity for use in regulating the flow of ial inf rmation 20 said sequ troller as recited in claim 35, wherein said control icon has a first image when said second means is of said sequential information; selecting a selected flow direction and a selected flow velocity for the sequential information; and producing control signals for regulating the flow of said first mode and wherein said control c nm as amsecond image when said second means is the sequential information in accordance with e selected flow direction and the selected flow veloc- o e t 1 esr ec e ein claim 35, wherein said 25 it e method of claim 42, and further including the slide indicator includes at least one interactive control steps of n e 30 di 38. A controller as recited in claim 35, wherein to a slide indicator includes a first region corresponding said 35 44.velocity. The method of claim 43, wherein said control icon zero flow velocity for said sequential information, and a has a first image when said tentative flow operation """!.. °"ro°°"":"°"'s.'°r'ida ne ion odse edmí eew e te ti forAsai uential inf rma aim 38, and further 40 of selecting said first image to select said tentative flow including a third region contiguous to said first region operation mode and selecting said second image to corresponding to a backward range of gradational flow seleect d fmix flow ope ion mdheerein said step of ve i esc r 11 rœa r d c , wherein said selecting said flow direction and said flow velocity slid indicator includes at least one interactive control 45 includes the step of moving said control icon to a seer se e ti 1ri f rm to noe of d pred termined flow directions and includes one or more in- fl w-dir t onna dæs d flo vbe oc y ano d flow of the sequential information accordingly. teractive control buttons representing said predeter50 55 60 65 APLNDC00028554 llllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllIl USOO5428730A United States Patent (19] Baker et al• [54) MULTIMEDIA SYSTEM HAVING SOFTWARE -· -ANISM PROVIDING STANDARDIZED INTERFACES AND CONTROLS FOR THE OPERATION OF MULTIMEDIA DEVICES [75] Inventors: Randolph S. Baker, Atlanta, Ga.; Richard E. DuLaney, Lighthouse [11] [45] Warinner, Woodstock, both of Ga.; Scott J. Broussard, Boca Raton, Fla. [73] Assignee: International Business Machines Corporation, Armonk, N.Y. [21] Appl. No.: 991,034 [22] Filed: Dec. 15, 1992 [51] Int. CI.6 .............................................. GO6F 15/00 [52] U.S. CI. ....... 395/154; 395/161 [58] Field of Search ............... 395/154, 155, 157, 159, 395/161 [56] References Cited U.S. PA a sa x DOCUMENTS 4,845,644 7/ 989 Anthias et al. ...................... 395/157 4,866,638 9/ 989 Cosentino et al. .................. _95/155 4,901,221 2/ 990 Kodosky et al. ................... .95/159 4,914,568 4/ 990 Kodosky et al. ......... ....... ¯95/275 5,040,131 8/ 99 5,041,992 8/ 99 Torres ----..---........ 195/156 Cunningham et al. ............. 295/155 5,060,140 10/ 99 Brown et al. ....................... 364/200 5,065,345 11/199 Knowles et al. .................... 395/154 5,109,515 4/ 992 Laggis et al- ..................-.... 395/725 5,291,587 3/ 994 Kodosky et al. ................... 395/500 5,293,476 3/ 994 Wolber et al. ------................ 395/ 59 5,309,556 5/ 994 Sismilich ............................. 395/ 61 5,313,574 5/ 994 Beethe ..--.....----..-................. 395/ 59 5,333,299 7/ 994 Koval et al. .--..................... 395/550 OTHER PUBLICATIONS Cohen, "LabVIEW Controls DOS, Tape Systems", MacWEEK, v. 6, No. 42, Nov. 23, 1992, p. 12 (1). 5,428,730 Jun. 27, 1995 Rizzo, "LabVIEW 2", MacUSER, v. 7, No. 5, May 1991, p. 58 (2). Santori, "An Instrument That Isn't Really", IEEE Spectrum, v. 27, No. 8, Aug. 1990, pp. 36-39. Schmalzel, et al., "Instrumentation Curriculum:.From ATE to VXI," IEEE IMTC '92, May 1992, pp. 265-267. Point; Patricia A. Esack, Boca Raton, both of Fla.; Stephen R. Henson, Atlanta, Ga.; Scott R. Swix, Half Moon Bay, Calif.; George E. Wagener, Marietta; Frank D. Patent Number: Date of Patent: (List continued on next page.) Primary Examiner--Heather R. Herndon Assistant Examiner-Joseph R. Burwell Attorney, Agent, or Firm-Bernard D. Bogdon; Stephen A. Becker [57] --•«CT A multimedia system includes a multimedia device con- trol program having a plurality of application program selectable controls for controlling operation of a like plurality of different classes of multimedia devices. Each class has a different set of controllable device functions or attributes. In response to selection of a class by an application program, a unique control panel is created in a user interface for controlling operation of a device. The selected control performs all of the processing actions necessary to control the device in response to user selections on the interface, without involvement of the application program in such actions. The multi- media control program includes a plurality of panel templates from which control panels are created in a control screen. Each panel has a plurality of controllers that can manipulated by a user to control a corresponding function of a multimedia device. The templates provide a consistent interfaces having a common look and feel. The multimedia control program also includes a plurality of class control programs one or more of which are selected by an application program to be attached to a like number of multimedia devices for controlling operation of the devices in response to user selections on the user interface, thereby relieving the application program of performing the necessary control functions. 20 Claims, 7 Drawing Sheets APLNDC00028555 5,428,730 Page 2 O-·---- PUBLICATIONS Petzold, "An Introduction to the Media Control Interface", PC Magazine, v. 11, No. 1, Jan. 14, 1992, p. 407 (5). Petzold, "MCI Command Strings: The Easy Multimedia Interface", PC Magazine, v. I I, No. 2, Jan. 28, 1992, p. 329 (5). Varney, "Multimedia Development Kit Now Runs Under Motif", Digital News & Review, v. 9, No. 17, Sep. 14, 1992, p. 15 (1). Arnett, "Workstations-a Software Advantage?", Multimedia Computing & Presentations, v. 4, No. 7, Apr. 30, 1992, p. 1 (3). Volkman, "The Digital Video Interface for Windows Multimedia", Windows-DOS Developer's Journal, v. 3, No. 9, Sep. 1992, p. 5 (9). "Microsoft Ships Toolkit for Multimedia Windows", PC Magazine, v. 10, No. 19, Nov, 12, 1991, p. 61 (1). Seltzer, "05/2 PDK a Boom for Developers", PC Week, v. 9, No. 39, Sep. 28, 1992, p. 1 (2). Thomsen, "A Software Architecture for VXI Bus Systems", Autotestcon '89 Conference Record, 1989, pp. 327-334. Southerton, Programmer's Guide to Presentation Man- ager, 1989, Addison-Wesley, pp. 32-36, 75-78, 181, 205-211, 312-313. Hamilton et al., "The ATTC Laboratory Automation System", National Association of Broadcasters, Apr. 1992. Pyxis Corp. (Fairfax, Va.) News Release, "New VI Libraries Expand LabView TM Capabilities for Macintosh", Nov. 6, 1992. Pyxis Corp. (Fairfax, Va.), "Multimedia VIs Application Note", 1992. Letter from Scott Hamilton, President, Pyxis Corp. (Fairfax, Va.), Sep. 12, 1994. IBM TDB vol. 34, No. 11, Apr. 1992, p. 29. APLNDC00028556 U.S. Patent June 27, 1995 12 MICRO PROCESSOR 10 A 16 Sheet 1 of 7 MATH COPROCESSOR 5,428,730 , 20 BC 22 14D 45 37 BUFFER 102 98 OP A 6 SCSI HD DATA MAIN MEMORY APPUCATION -PROGRAMS _ MMAP 100 NVRAM 38 J 40 ROM POST 32 ROM 2 FLOPPY 73 PRINTER BIOS 74 KEYBOARD 76 46 42 G lOC - MMPM - MMDCP SPEAKER SERIAL 44 48 VSP 60 7Û cr AUX. VIDEO 110 AIC 112 MONITOR CABLE 8g 49 90 n?P \ 108 VIDEO DISC PLAYER 68 i 82 79 MOUSE MONITOR 02 TEXT AND GRAPHICS VRAM 78 80 114 DAC MUX 88 a 24 I 39 3 ADAPTER 41 AUDIO \ INST. RAM 84Á ---- DATA RAM 86 - 50 . / 92 94 ADAPTER EC HEADPHONE MICROPHONE LINE IN LINE OUT SPEAKER MIDI CARD 96 MMD APLNDC00028557 U.S. Patent June 27, 1995 Sheet 2 of 7 5,428,730 FIG. 2 200 204 ) 213 21L - Audio 202 ) 212 Sample VIDPLAY 22o) 206 21s il 2 231 Mute Freeze 22 N 226 --232 Volume 227 222- -233 Balance 223- 3° File Coritrols S_ource Image Help 'j' . Video - - Contrast 21e - 2 6 234 Player Control Treble 24 Color 44 LJ Frarrie 242 243 224- i i i i Bass 236 Tint 249" 245 i 246 \247 \248 Sh ness APLNDC00028558 U.S. Patent June 27, 1995 126 Sheet 3 of 7 USER INTERFACE (Uly 102 106 MULTIMEDIA APPLICATION PROGRAM (MMAP) AUDIO CONTROL 103 OS/2 MULTIMEDIA PRESENTATION MANAGER (MMPM) VfDEO CONTROL 105 107 10 OS/2 PRESENTATION MANAGER (PM) FIG.3 / MULTIMEDIA DEVICE CONTROL PROGRAM (MMDCP) 101 104 5,428,730 PLAYER CONTROL €ONTROL iNSTANCE(S) - PANEL TEMPLATES 111% 113¾ INTERFACE LOGIC - DEVICE LOGIC OS/2 KERNEL 120 OS/2 DEVICE DRIVERS 122 DEVICE COMMUNICATIONS SERIAL, PARALLEL, SYSTEM BUS 42 BIOS 124 m DEVICES 108 VIDEODISC VIDEOTAPE DIGITAL AUD1O CD MMD 96 DIGITAL VIDEO ETC. APLNDC00028559 U.S. Patent June 27, 1995 Sheet 4 of 7 5,428,730 FIG. 4 ( PRIOR ART) 126 128 USER INTERFACE (UI) * 130 Ý 132 A OTHER k UI SELECTIONS k CONTROL PANEL DISPLAY APPLICATION (BUTTONS, SL1DERS, & UPDATE, CHANGE IN INTERACTIONS DIALS, ETC.) STATUS, ENABLE/DISABLE, ETC. MMAP 13 CREATE/DESTROY CONTROL DISPLAY WITH SELECTION COMPONENTS 136 UI DETECTION Ul EVALUATION- WHAT WAS SELECTED? SELECTION EVALUATION- VALID/INVALID DEVICE COMMANDS NECESSARY COMMAND RESPONSE EVALUATION UI UPDATE TO REFLECT NEW CONDITIONS ERROR PROCESSING TIMED EVENT NOTIFICATION PROCESSING 138 140 CONTINUOUS STATUS NOTIFICATIONS COMMAND RESULTS TIMED EVENT NOTIFICATIONS STATUS REQUESTS 104 Y DEVICE OPENICLOSE DEVICE COMMANDS COMMAND TERMINATES DEVICE ERROR NOTIFICATIONS w MMPM 120 DEVlCE DRIVER ' DEVICE COMMUNICATIONS 'S DEVICE APLNDC00028560 U.S. Patent June 27, 1995 Sheet 5 of 7 5,428,730 FIG. 5 126 128 102 USER INTERFACE (Ul) A 130 Ý 132 OTHER k Ul SELECTIONS i CONTROL PANEL DISPLAY APPLICATION (BUTTONS, SLIDERS, & UPDATE, CHANGE IN INTERACTIONS DIALS, ETC.) STATUS, ENABLE/DISABLE, ETC. 106 A MMAP ,l 150 CREATE 102/ CREATE WINDOW OPEN DEVICE CREATE CONTROL 107 156' BUILD CONTROL DISPLAY y O NIBol DESTROY 158 UI DETECTION om amur <152 - U1 EVALUATION- WHAT WAS SE wm 2 DEVICE ID > SELECTION EVALUATION- VALID/lNVALID DEVICE COMMANDS NECESSARY MISC. /DENTIFY DEVICE MSGS· > (O ) 160s 154 1623 w DEVICE OPEN/CLOSE 104 MMDCP COMMAND RESPONSE EVALUATION UI UPDATE TO REFLECT NEW CONDITIONS ERROR PROCESSING TIMED EVENT PROCESSING 164 CONTINUOUS STATUS NOTIFICATIONS i COMMAND RESULTS TIMED EVENT NOTIFICATIONS DEVICE ERROR NOTIFICATIONS DEVICE COMMANDS COMMAND TERMINATES STATUS REQUESTS o MMPM 120 DEVICE DRIVER 122 DEVICE COMMUNICATIONS 124 DEVICE APLNDC00028561 U.S. Patent June 27, 1995 Sheet 6 of 7 5,428,730 FIG. 6 126' USER INTERFACE (Ul) 130-1 132-1 - 130-2 107P 102 107A I DU- I PLAYER CONTROL INSTANCE 7 U-2' <- AUDIO CONTROL INSTANCE MMAP * 152-1 164-1 - 132-2 152-2 162-1 162-2 164-2 104 MMPM 120-1 120-2 LAYER DEVICE DRIVER AUDIO DEVlCE DRIVER 122-1 122-2 DEVICE COMMUNICATIONS DEVlCE COMMUNICATIONS 124-1 SERIAL PORT PLAYER DEVICE 124-2 SYSTEM BUS AUDIO DEVICE APLNDC00028562 U.S. Patent 170 June 27, 1995 172 APPLICATION INPUT CREATE FIG. 7 USER INTERFACE INPUT DESTROY DEVICE ID MISC. Sheet 7 of 7 SELECTIONS 5,428,730 c 17 OS INPUT MOVE SHOW HIDE PAINT ETC. 176 DEVICE INPUT (VIA MMPM) ERRORS TIMED EVENTS 106 MMDCP CONTROL M ^GE PROCESSING 178 CREATE MESSAGES FROM APPLICATION ACQUIRE RESOURCES (MEMORY, ETC.) MAKE CONTROL PANEL PLACE IN WINDOW 180 DESTROY MESSAGES FROM APPLICATION RELEASE RESOURCES (MEMORY, ETC.) DISCARD WINDOWS, CONTROL PANEL 182 DEVICE ID MESSAGES FROM APPLICATION CHECK ID VALIDITY ATTACH CONTROL INSTANCE TO DEVICE DETERMINE DEVICE CAPABILITIES QUERY DEVICE STATUS UPDATE CONTROL PANEL (ENABLE/DISABLE COMPONENTS, CHANGE CURRENT SETTINGS) 184 MISCELLANEOUS APPLICATION MESSAGES TOGGLE APPLICATION NOTIFY ONIOFF TOGGLE DEVICE COMMUNICATION ONIOFF USE CUSTOM CONTROL PANEL LAYOUT 186 OPERATING SYSTEM MESSAGES MOVE, PAINT, SHOW, HIDE, ETC. THE WINDOW UPDATE THE CONTROL PANEL AS NECESSARY 788 DEVICE NOTIFICATION MESSAGES ERROR? IF SO, A l I vlPT RECOVERY OR DISABLE COMPONENTS UPDATE CONTROL PANEL AS NECESSARY 190 UI SELECTION MESSAGES / CHECK CURRENT CONTROL PANEL/DEVICE COly i s i DETERMINE ACTIONS NECFSSARY, INCLUDING: CANCEL CURRENT OPERATION(S) SEND DEVICE COMMAND(S) EXAMINE DEVICE COMMAND RESPONSES IF ERROR, ATTEMPT RECOVERY UPDATE CONTROL PANEL AS NECESSARY APLNDC00028563 1 5,428,730 MULTIMEDIA SYSTEM HAVING SOFTWARE ME· ••-=SMPROVIDING STANDARDIZED INTERFACES AND CONTROLS FOR THE OPERATION OF MUL • = - = ·IA DEVICES 2 gram having a plurality of application program select- able controls for a like plurality of different classes of multimedia devices. Each class has a different set of controllable device functions or attributes. In response 5 to selection of a control by an application program, a unique control panel is created in a user interface for BACKGROUND OF THE Iix veis1ION controlling operation of a device. The selected control performs all of the processing actions necessary to conThis invention relates to the field of data processing, trol the device in response to user selections on the and, more particularly, to an improved multimedia system having a software mechanism that provides stan- 10 interface, without involvement of the application pro- dardized interfaces and controls for the operation of multimedia devices. gram in such actions. In accordance with another aspect of the invention, A multimedia system is designed to present various multimedia materials in various combinations of text, the multimedia application program includes a plurality of predetermined templates defining different control graphics, video, image, animation, sound, etc. Such a 15 panels which contain user manipulable controllers. The system is a combination of hardware and software. The application program sends a message to the control hardware may include a personal computer to which program indicating its selection and the control provarious multimedia devices can be attached. The hardgram creates the desired control screen from the seware runs under the control of an operating system and lected template. multimedia application programs. Recent commercially 20 DRAWINGS available multimedia systems provide multimedia application developers the capability to control the multimeOther objects and advantages of the invention will be dia devices via software. In turn, application developers apparent from the following description taken in concreate programs that allow end users to control a device nection with the accompanying drawings wherein: by manipulating simulated controls, such as buttons, 25 FIG. 1 is a schematic diagram of a multimedia data dials, and other elements, on a computer display screen. processing system embodying the invention; The application programmers have been required to FIG. 2 is a schematic diagram of an exemplary concreate their own custom designed device control trol screen on a display; screens as well as handle all user interactions, device FIG. 3 is a schematic diagram of the major compocommunications, and screen changes. Such custom 30 nents of the software mechanism included in the system development has resulted in inconsistent user interfaces, requires knowledge of the device and device control methods, and duplicates effort for multiple applications. shown in FIG. 1; FIG. 4 is a schematic diagram illustrating communi- Another object of the invention is to provide a multimedia control program that interacts with a multimedia application program to control a multimedia device without requiring the application program developer to have an in-depth knowledge of the multimedia device to be controlled. A further object of the invention is to provide a multimedia control program which provides common functions available for use by multimedia application programs, so as to relieve application programmers of duplicate development effort and to thereby increase multimedia programmer productivity. Still another object of the invention is to provide a multimedia control program which creates user interfaces comprising display screens having a common look and feel, the screens containing graphic controllers that can be manipulated by users to control multimedia devices. A still further object of the invention is to provide a multimedia device control program which creates a control panel with minimum involvement of an application program and which thereafter controls operation of the device without intervention of the application program. Briefly, in accordance with the invention, a multimedia system includes a multimedia device control pro- FIG. 7 is a block diagram illustrating different inputs into the MMDCP, and the actions responsive to the 45 inputs. DET A e av DESCRIPTION Referring now to the drawings, and first to FIG. 1, cations between the major components of a prior art These problems reduce useability and increase development expense. 35 system which the present invention improves; FIG. 5 is a diagram similar to FIG. 4 but showing the SUMMARY OF THE I- - · ION improved processing in accordance with the invention for multimedia device control and user interface interOne of the objects of the invention is to solve the action; problems noted above by providing a multimedia sysFIG. 6 is a simplified diagram similar to FIG. 5 but tem with a set of standardized multimedia controls by 40 showing processing for controlling a plurality of differwhich a consistent user interface is presented from difent multimedia devices; and ferent application programs. there is shown an exemplary multimedia data process50 ing system comprising a personal computer 10 operable under a multitasking operating system such as OS/2 Version 2.0, to execute application programs. Computer 10 comprises a microprocessor 12 connected to a local bus 14 which, in turn, is connected to a bus interface 55 controller (BIC) 16, a math coprocessor 18, and a small computer system interface (SCSI) adapter 20. Microprocessor 12 is preferably one of the family of 80xxx microp. _ -, such as an 80386 microprocessor, and local bus 14 includes conventional data, address, and 60 control lines conforming to the architecture of such processor. Adapter 20 is also connected to a SCSI bus 22 which is connected to a SCSI hard drive (HD) 24 designated as the C:drive, the bus also being connectable to other SCSI devices (not shown). Adapter 20 is 65 also connected to a NVRAM 30 and to a read only memory (ROM) 32. BIC 16 performs two primary functions, one being that of a memory controller for accessing a main mem- APLNDC00028564 3 5,428,730 ory 36 and a ROM 38. Main memory 16 is a dynamic random access memory (RAM) that comprises stores data and programs for execution by microprocessor 12 and math coprocessor 18. ROM 38 stores a POST program 40 and a BIOS 42. POST program 40 performs a 5 standard power-on, self-test of the system when computer 10 is started by turning the power on or by a keyboard reset. An address and control bus 37 connects BIC 16 with memory 36 and ROM 38. A data bus 39 connects memory 36 and ROM 38 with a data buffer 41 10 that is further connected to data bus 14D of bus 14. Control lines 45 interconnect BIC 16 and data buffer 41. The other primary function of BIC 16 is to interface between bus 14 and an I/O bus 44 designed in conformance with Micro Channel (MC) architecture. Bus 44 is 15 further connected to an input/output controller (IOC) 46, a video signal processor (VSP) 48, a digital signal processor (DSP) 49, and a plurality of expansion connectors (EC) or slots 50. VSP 48 is further connected to a video RAM (VRAM) 60 and a multiplexor (MUX) 62. 20 VRAM 60 stores text and graphic information for controlling what appears on the screen of a monitor 68. MUX 62 is further connected to a digital to analog converter (DAC) 66 and to a connector or terminal 70 that is connectable to a video feature bus (VFB). DAC 25 66 is connected to monitor 68 that provides a conventional output screen or display for viewing by a user. IOC46 controls operation of plurality of I/O devices including a floppy disc drive 72 designated as the A:drive, a printer 74, and a keyboard 76. Drive 72 com- 30 prises a controller (not shown) and a removable floppy disc or diskette 73. IOC 46 also is connected to a mouse connector 78, a serial port connector 80, and a speaker connector 82, which allow various optional devices to be connected into the system. A mouse 79 is connected 35 to connector 78. DSP 49 is further connected to an instruction RAM 84, a data RAM 96, an analog interface controller (AIC) 88, and an audio controller (90). RAMS 84 and 86 respectively hold instructions and data used by DSP 49 40 for processing signals. Audio controller 90 controls various audio inputs and outputs and is connected to a plurality of connectors 92 by which various devices can be connected to the system. Such devices include a headphone, a microphone, a speaker, a musical instru- 45 ment digitizing interface (MIDI), and devices requiring an audio line-in and line-out functions. Various other multimedia devices (MMD) 96 can be also attached to 4 invention is embodied primarily in MMDCP 106 and its interaction with operating system 100 and the hardware. The combination and interaction of hardware and software is referred to herein as a "software mechanism". FIG. 2 illustrates an exemplary control screen 200 in which a plurality of windows or panels are displayed on a monitor in response to execution of a sample program named "VIDPLAY". This program creates a display of a control screen 200 that includes a video window 202, an audio control panel 204, a video control panel 206, and a player control panel 208. Also presented in the screen is a standard cursor 210 that is manipulated through use of mouse 79. Window 202 contains a title bar 212 bearing the title "Sample VIDPLAY", a window control button 213, a sizing button 215, a menu bar 214, and a video viewing area 216. The window itself is conventional and can be manipulated by means of cursor 210 to change the position and size of the window. Audio control panel 204 contains a title bar 218 identifying the panel as an "Audio" control panel, and a control area containing various simulated controllers or controls 220-224. Control 220 is a mute button, änd controls 221-224 are dials for respectively controlling volume, balance, treble, and bass. With reference to control 221, each dial includes reference marks 228 around the periphery of a circular knob, a movable pointer 229, a decrease button 226, and an increase button 227. Buttons 226 and 227 respectively display graphic symbols "-" and "+" which generally denote the button function. Also included in control 221 is a legend "Volume" signifying the function of the device being controlled. When the cursor 210 is placed on one of buttons and the mouse is clicked, the pointer appears to move until the mouse button is released. Such movement would be --panied by signals being sent to the physical device whose controls are being simulated to cause the corresponding action to occur. For example, if the cursor is clicked on button 227, pointer 229 rotates clockwise and thé actual volume of the audio device would increase. Video control panel 206 has a title bar 230 displaying the legend "Video", a freeze button 231, and dial controls 232-236 for controlling the brightness, contrast, color, tint, and sharpness, of the video device, e.g., monitor 112 (FIG. 1). Player control panel 208 displays the legend "Player Control" in title bar 240, push buttons 241-248, and a slider control 249. Buttons 241-248 the system through an EC 50 and adapter card 94. include different symbols symbolic of different player For purposes of illustrating the invention, system 10 50 functions controlled thereby. Such buttons provide, for further includes a video disc player 108 having a control video disc player 108, controls of eject, pause, play, line connected to connector 80 of the serial port for play backward, record, repeat, rewind, scan forward, receiving start/stop signals through the serial port. scan backward, step backward (frame backward), step Player 108 is also connected to and transmits video forward (frame advance), and stop. Slider control 249 signals to an auxiliary video card 110 that has a connec- 55 displays the current device position in time or frame tion to a monitor 112 and another connection 108 to based units. VFB connector 70. Preferably, each button presents a visual differentiaMemory 36 stores various programs for execution in tion between actuated and non-actuated conditions. the system, which programs include application proThis can be accomplished by shadowing the buttons to grams 100, including multimedia application programs 60 give appearances of being pushed in or sticking out (MMAP) 102, and an operating system 98 which infrom the plane of the display. Some of the buttons may cludes OS/2 multimedia presentation manager also be animated. For example, scan forward and scan (MMPM) 104 and a multimedia device control program backward buttons may have a series of arrows that are (MMDCP) 106. It is to be noted that while FIG. 1 successively highlighted to simulate the direction of illustrates an exemplary multimedia system, the operat- 65 movement. ing system is general purpose and is designed to run or It is to be noted that the above discussion of FIG. 2 control data processing systems having configurations provides nomenclature facilitating an understanding of that are different from the one shown in FIG. 1. The the invention as described below. The contents of con- APLNDC00028565 5 5,428, 730 trol screen 200 as shown in FIG. 2 is intended to primarily show what samples of different control panels look like. Quite obviously, in any given system, the type and number of panels is dependent upon what multimedia devices are connected to the system, and what controls are selected to be shown. Further variations in such control panels and how they are created are described hereinafter. FIG. 3 illustrates the layered structure of the software mechanism and the primary programs that are executed concurrently in a multitasking mode, in accordance with the invention. At the top, a user interface 126 communicates with MMAP 102 and MMDCP 106, both of which are above MMPM 104. The latter program interacts with the IBM OS/2 Presentation Manager (PM) program 99 which is above the operating system kernel 98, which sits on top of device drivers 120. PM 99 also communicates with MMAP 102 and MMDCP 106. A device communications program 122 is between drivers 120 and BIOS 42 which is the lowest software layer and provides device specific processing for operating multimedia devices 124. These devices include video disc player 118, and any MMD 96 such as video tape, digital audio, compact disc (CD), and digital video devices. 6 the commands can be sent to the proper device. After attachment, the control instance performs its primary function of controlling the particular device, as described in more detail hereinafter. 5 FIG. 4 illustrates various functions 134, 136, and 138 as performed in the prior art by a typical application program 102' and the interfaces between such program and user interface (UI) 126' and MMPM 104. The illustration is for a system having a single multimedia device 10 124. The graphics on the screen display together with the means for manipulating the graphics and making selections, form the user interface 126' through which the user interacts with the system to control operation of the multimedia devices. Input messages 130 from the 15 user into the application program 102' include UI selections made by the user actuating the various buttons, sliders, dials, etc. Output communications or messages 132 from the application to the user interface include controlling and updating the panel display, indicating 20 changes in status of the multimedia devices, enabling and disabling buttons or controls which are or are not appropriate for the current device state. For example,if a device is not playing, then the pause button is disabled because its use is not applicable. Other application inter25 actions 128, such as those not pertinent to the control of MMAP 102 is a user oriented application program a MMD, may also occur with the user interface. providing access to services of multimedia devices apVarious input/output messages 140 and 142 also compropriate to the application, e.g., a video disc browsing municate between the application and MMPM 104. program. MMAP 102 interacts with MMDCP 106 for Application 102' may receive as input messages 142 the purpose of providing a user interface for the appro- 30 continuous notifications on the status of the multimedia priate device, and with MMPM 104 for opening and devices, the results of commands, timed event notificaclosing the device and accessing specific features or tions and device error notifications. The application performing custom processing not available via the output messages 140 include device open/close and more generic MMDCP 106. MMPM 104 implements a other commands, command terminates, and status rehigh level, device independent interface to multimedia 35 quests. devices for application use and includes support for In response to the application program being run or device context switching, data streaming, and manageexecuted, the program is initially operative to create, by ment of device contention. PM 99 is used for its screen function or step 134, a control screen on a monitor management functions and capability of creating winwhich screen includes one or more controllers. The dows on a display screen. OS/2 kernel 98 represents the 40 size, appearance, location, contents, etc. of the control services provided by the IBM OS/2 operating system. screen are the full responsibility of the application. These services generally include functions for process Once the controllers are displayed, the user can thereafand .. ... management. Kernel 98 and PM 99 are ter manipulate the controls and make selections to conomitted from later drawing figures for simplicity of trol operation of a MMD. When the application is being illustration. OS/2 device drivers 120 are device specific 45 ended, a destroy message 134 destroys the control control programs providing a software interface for screen initially created. Functions 136 are performed in controlling individual devices A device driver at this response to the application program receivmg messages level generally provides services for a device of a spe130 and include detecting UI actions, evaluating the cific type and manufacturer. Device communications selection as to what action was selected, determining if 122 indicates a communication mechanism by which a 50 the selection is valid or invalid, and generating various device driver __ - an attached device. Some deoutput messages 140. In response to the application vices are card type adapters installed in the system for program receivmg messages 142, application 102' evalu- access by the system bus. External devices are generally ates responses to commands, updates the UI to reflect attached to the computer by a cable plugged into the new conditions, and processes the notifications of timed parallel port or the serial port. 55 events and device errors. The application also sends MMDCP 106 comprises an audio control 101, a video messages 132 as appropriate in response to such processcontrol 103, a player control 105, one or more control ing. In summary, the messages and interactions exinstances 107, panel templates 109, interface logic 111, plained with reference to FIG. 4 indicate what an appliand device logic 113. Controls 101, 103, and 105 are cation program would have to do in the absence of the separate generic routines or programs for controlling 60 invention. It is to be appreciated that application promultimedia devices having audio, video, and player grams normally perform functions in addition to those attributes. In general, an application program first opens related to MMD control but inasmuch as the invention a device, and then sends a message to MMDCP to creis concerned with MMD control, the additional funcate a control screen which is done by making an intions have been omitted for simplicity. stance or copy of the appropriate control and then at- 65 With reference to FIG. 5, in accordance with the taching the control instance to the device. That is, once invention, MMAP 102 is greatly simplified and reduced the instance has been made, it is necessary to inform it of relative to the equivalent prior art application program the identity of the device it will be controlling so that 102' because MMDCP 106 performs most of the func- APLNDC00028566 i 7 5,428,730 8 tions for generating and receiving communications with display player control panel208 and audio control panel the user interface and with MMPM 104. When MMAP 204. Once the control instances are attached, each con102 is run, a series of instructions 102I are executed. The trol instance manages its attached device and configures first instruction is to create a window and this is done by itself according to the device capabilities. During operPM 99. The second instruction is to open the device to 5 ation, player control instance 107P receives communibe controlled and this is done by sending by sending a cations 130-1 for the UI and sends messages 132-1 to the DEVICE OPEN command 162. The third instruction is UI, while audio control instance 107A receives mes- to create a control and this is done by sending a CREsages 130-2 from the UI and sends messages 132-2 to the ATE message 150. The fourth instruction IDENTIFY UI. In the prior art, the application program has to DEVICE sends a DEVICE ID message 152 identifying 10 implement and manage the additional functionality to the particular device to be controlled to MMDCP. In address and handle the functional differences between response to messages 150 and 152, MMDCP 106 creates player and audio devices. a control instance 107 and attaches it to open device FIG. 7 shows further details of control message pro124. After that, control processing including generating cessing that is done in response to ......Ing messages or and receiving communications from the UI and the 15 inputs 170, 172, 174, and 176 from four sources, namely, MMPM, is done by control instance 107 thereby reliev¯ MMAP 102, UI 126, the operating system, and multimeing the application from doing similar functions and dia device 124 inputs through MMPM 104. In response being concerned with knowledge of individual devices. to receiving a DRE A TE control screen command, Control instance 107 receives input messages 130 from MMDCP is operative to perform the functions 178 and UI 126, transmits messages 132 to the UI, and communi- 20 acquire the resources (e.g., memory) for the display, cates with MMPM 104 via two way communications make the desired control panels, and place them on the 164. Control instance 107, in response to receiving variscreen. In response to receiving a DESTROY message ous inputs, performs the functions described below with from the application, actions 180 release the resources, reference to FIG. 7. The control screen presents a conand discard the window and any control panels on the trol interface to the end user, and control instance 107 25 translates actions and selections on the interface into screen. In response to receipt of a DEVICE ID message appropriate media control and system commands. from the application, actions 182 comprise checking the MMDCP 106 can create three different types of convalidity of the device ID, making a control instance by trol panels: a video control panel for a device, such as a copying the control and then attaching the control graphics/video overlay adapter, having video attri- 30 instance to the open device, determining device capabilbutes; an audio control panel for a device, such as an ities, querying the device status, and updating the conamp/mixer device, having audio attributes; and a player trol panels, e.g., by enabling or disabling components control panel for a device, such as a videodisc player, and changing current settings. The attaching is simply having player attributes. A video control panel typinotifying the control instance of the ID of device to be cally includes a freeze push button, and circular sliders 35 controlled. Actions 184 are functions or actions peror dials to control brightness, contrast, saturation, hue, formed in response to miscellaneous messages from the and sharpness. An audio control panel typically inapplication and include enabling or disabling the sendcludes a mute push button and a series of circular sliders ing of notification messages to the application, toggling to control volume, balance, treble, and bass. A player application notify on/off, toggling device communicacontrol panel typically includes push buttons for combi- 40 tion on/off, and using custom control panel layout. nations of the following player actions: eject, pause, Action 186 updates control panels as necessary in play backward,play forward, play reverse, record, re. response to receiving messages from the operatmg syspeat, rewind, scan backward, scan forward, scan retem. Such messages include hide, show, move, paint, verse, step backward, step forward, and stop. A player etc. In response to receiving device notification mescontrol panel might also include a linear slider for posi- 45 sages, actions 188 include updating the control panel as tion display. It should be obvious that the individual necessary. If an error has occurred, a recovery attempt controls depend on the capabilities of the attached mulis made or the component is disabled. Timed events are timedia device(s) and the configuration of the player control ... events which occur at a device asynchronously with respect to a command issued to the device. For examFIG. 6 illustrates the manner in which MMDCP 106 50 ple, if the device is instructed to play for a certain amount of time, the completion of the play command is controls two multimedia operations for a MMO that a timed event which results in a notification being sent runs two multimedia devices. The devices might be , to MMDCP 106. At this point, MMDCP 106 stops e.g., a player device 124-1 such as videodisc player 108 animating the play button and changes the appearance connected to the serial port, and an audio device 124-2 connected to one of the expansion connectors 50 and 55 of the play button to be consistent with a "stop" state. Other timed events might include position notifications accessible over the system bus. For such a configuraand cuepoints. When an input 172 is received, actions tion, MMAP 102 needs to send a first DEVICE OPEN 190 are performed to check the -.-..t control pamessage 162-1, a first DEVICE ID message 152-1, and nel/device context, and determine what actions are a first ^TE message 150-1 to create a player control instance 107P, and to then send a second DEVICE 60 necessary. If for example, a device is already playing and the play button is selected, the context indicates no OPEN message 162-2, a second DEVICE ID message 152-2, and a second CREATE message 150-2 to create action is necessary and the selection of the play button is ignored. The actions to be done include cancelling an audio control instance 107A. Also, respective player current operations, sending device commands, examinand audio device drivers 120-1 and 120-2 and respective device communications 122-1 and 122-2 have to have 65 ing device command responses, attempting error recovbeen loaded into the system, for establishing communiery, and updating the control panel as necessary. The cations with the respective multimedia devices. With specific details of the actions described relative to FIG. two devices, the control screen 200 in FIG. 2 could 7 should be obvious to those skilled in the art and have APLNDC00028567 9 5,428,730 been omitted to shorten description particularly since the details are not claimed. A typical sequence of actions that occur can be illus- trated by the example of audio control panel 204 and volume controller 221 (FIG. 2). Suppose the user de- 5 10 The following application code is written for OS/2 operating system with OS/2 multimedia extensions, and provides an example of the code simplicity of an application program useable with the invention: sires a volume increase. To accomplish this using the ******** BEGIN EXAMPLE CODE ******** mouse, the mouse is manipulated to position cursor 210 //Open a videodisc device with an alias name and obtain a on button 227 and then clicked and held until the vol//device ID ume increases to the desired level. In response to the OpenDevice ((LPSTR)szVdiscDevName, (LPSTR)szVdiscAlias, mouse being clicked, the following sequence of actions 10 &wVdiscDevlD); //Create a window to hold the Player Control Panel on the occur. First, presentation manager 99 (FIG. 3) deter//screen mines which application, the cursor is currently being hwndVPCDIg = WinLoadDIg(HWND_DESKTOP, operated under, which window the cursor is being ophwndClient, erated in, and which object in the window has been pVPCDigProc, 0, selected. In this example, the object is the audio control- 15 ID_DLG_VPC, ler volume increase button. PM 99 then sends a user NULL); interface message 172 (FIG. 7) to the audio control //Create the Player Control Panel using MINIMAL instance which message identifies the object and the //configuration. "hwndVPC" is the handle uses for any further control action desired. In response to receiving the //communication with the player control functions 190 and determines what actions are necessary. The first is to decide whether to cancel any cur- WC-PLAYERCTL, rent operation. If the mute button is on, it would be WS VISIBLE ] MCS-MINIMAL //Select panel style audio device to increase the volume by an increment. HWND-TOP ID_VPC_WND, message, the audio control instance then performs the 20 hwndVPC = WincreateWindow (hwndVPCDig, //Create Player Control //Misc OS/2 item X_MMCTL, Y MMCTL cancelled to allow audio to be produced by the device. 200,100 Next, a volume increase device command is sent to the 25 hwndVPCDig, At this point the device responds and if the response indicates such action has been done without error, the next action is to update the pointer of the volume dial and this is done by sending a message through PM 99 and user interface which thereupon redraws the volume dial pointer to a new position representing an increment of volume increase. The sequence of actions repeat until the mouse button is released, or until the maximum volume is reached. An optional action (not shown) could be performed before communicating with the device, to give the application program the opportunity to handle the user selection. This is useful in situations where operational customization is done by the application, instead of operating by the control instance. With this option, the application would send a return code indicating it will or will not handle the message. As indicated above, MMDCP 106 includes a variety of panel templates 109 that are used in making a control panel in 178. MMDCP provides the application programmer with the options of accepting a default panel, a minimal panel, a complete panel, or a custom panel. A complete panel includes all the supported dials, buttons, sliders, etc. for controlling a particular device. A minimal panel includes only those controllers generally considered absolutely necessary. Naturally, the minimal panel occupies less screen area than a complete panel. A custom panel could be defined by the application and use any subset of the components of a complete panel, change relative positions, include other functions, etc. The templates also include vertically oriented panels, such as the audio and video panels shown in FIG. 2, and horizontally oriented panels, for both complete and minimal panels. Additional options include animated buttons (such as movement of arrows on a Play button when a device is playing), and the ability for an application to request that the control only present a user inter- //Use the window above 30 //Starting screen location //Size //Misc OS/2 item //Misc OS/2 item //Misc OS/2 item NULL, //Misc OS/2 item NULL); //Misc OS/2 item //Inform the Player Control to use the videodisc player that //was opened earlier WinSendMsg (hwndVPC, M - -- - EVICEALIAS, szVdiscAlias, 0) ********END SAMPLE CODE ******** 35 In response to successful execution of the above code, a device ID is obtained, a window is created, a player control instance is created, and the device is set. The player control instance will display the control screen 40 including the player control panel, handle all user interface actions, communicate with the videodisc player, etc. No other code in the application program is needed to perform or process selections from the control panel. As indicated previously, an application program must 45 first create an instance of a control and second, attach 50 55 60 face but not communicate with the device. This option may be useful if the application desires to control other devices while taking advantage of the user interface 65 standards and processing. The designs of the panels provide a uniform appearance and common look and feel between different applications. the instance to an open device before the control instance can assume responsibility for managing the user interface with the attached device. The first action involves creating a window of the proper window class. There is a window class for each type of supported device to be controlled: audio attributes, video attributes, and player attributes. There are also separate control programs (which may share a function) for each window class. In presentation manager (PM), an application program creates a window using the PM call WinCreateWindow. WinCreateWindow has several parameters including the window class and window style. The window style of the MMDCP specifies, among other things, the orientation of the control panel (vertical or horizontal) and the degree of desired device control (minimal, complete). The control program uses the style to select one of four layouts for the initial appearance. For example, audio control 204 in FIG. 2 is a vertical, complete control. The result thus promotes a consistent user interface. The panel templates 109 each specify the type of components and their positions using a dialog template APLNDC00028568 11 5,428,730 of PM. By way of example, the dialog template corresponding to the layout of audio control panel 204 is: 12 multimedia device (hereinafter referred to as open MMD) and for thereafter transmitting signals to DGLTEMPLATE ID-DLG_AACCOMPLETE_V LOADONCALL MOVEABLE DI- --· --LE BEGIN DIALOG "", ID_DLG_AACCOMPLETE_V, 0, 0, 42, 174, NOT FS DLGBORDER | FS_NOBYTEALIGN PRESPARAMS PP_FONTNAMESIZE, "8.Helv" BEGIN PUSHBUTTON " ", ID-MUTE, I, 153, 40, 20, WS-TABSTOP "Mute", ID_TEXT-MUTE 1,145,40,8 PRESPARAMS PP-FOREGROUNDCOLORINDEX, SYSCLR_WINDOWTEXT --· -- = - ·· ·N "Volume" ID_VOLUME 1,IO9,40,36,WS-TABSTOP PUSHBUTTON "Balance", ID_BALANCE 1,73,40,36, WS_TABSTOP PUSHBUTTON "Treble", ID_TREBLE, 1,37,40,36, WS-TABSTOP PUSHBUTTON "Bass", ID_BASS 1,1,40,36, WS TABSTOP END END For the purpose of defining the layout of a control panel, the window class specified in the dialog template 20 is ignored for each of the predefined components such as the mute button or the volume dial. To create the control panel display, the MMDCP uses the information in the template to create each of the desired components. The subset of components present influences the 25 subsequent operation of the MMDCP. For example, a minimal audio control includes only a volume dial and a mute button. A request to update the control panel would query the device only for the applicable information, i.e., the volume and mute status. A complete audio 30 control would cause a query of the device for all of the components, i.e., the mute, volume, bass, and treble status, The template mechanism also allows the MMAP to specify an alternative layout for a control panel. This is 35 done through an optional miscellaneous application message 184 to USE CUSTOM CONTROL PANEL LAYOUT. In this case, the MMAP provides the desired template to the MMDCP after the panel is created. The custom template supersedes any layout selected at 40 MMDCP creation time. If neither minimal nor complete is specified in the style, then the MMDCP assumes that the MMAP will subsequently provide a layout and no components are created until this is done. While the invention has been described in connection 45 with a specific operating system, it should be obvious that other operating systems could also be used. Further, it should also be obvious that additional classes of control programs can be added or substituted for the audio, video and player control programs. It should be 50 further apparent to those skilled in the art that many changes can be made in the details and arrangements of steps and parts without departing from the scope of the invention as defined in the appended claims. What is claimed is: 55 1. In a multimedia data processing system (MMDPS) comprismg a processor, a storage system including a memory for storing at least one multimedia application program (MMAP) and a multitasking operating system for execution by said processor, a monitor for display- 60 ing screens of information, and connector means for said open MMD in response to said first means receiving device commands, said signals being operative to control said controllable functions of said open MMD; second means comprising a user interface including user manipulable objects on a display screen of said monitor; and multimedia device control means (MMDCM) layered between said first means and said second means, said MMDCM comprising a plurality of application-program-selectable class-specific controls, each control being for a different one of said classes of multimedia devices; said MunCM being responsive to a CREATE message from the application program to select one of said controls (hereinafter referred to as selected control) having the same class as said open MMD; said selected control comprising third means for creating a control screen on said monitor in said user interface, said control screen having a control panel comprising a plurality of user actuated controllers for controlling different functions of said open MMD; said user interface being operative to send a user selection message in response to a user manipulating one of said controllers; and, said MMDCM further comprising processing means operative, in response to receiving said user selection message, to send a device command to said first means for changing a function of said open MMD and to send a message to said user interface to update the one of said controllers manipulated by the user to reflect the changed function of said open MMD. 2. A MMDPS in accordance with claim 1 wherein: said selected control is a copy of one of said controls (which copy is referred to hereinafter as a control instance); and said MMDCM is operative to attach said control instance to said open MMD in response to receiv- ing an identification of said MMD. 3. A MMDPS in accordance with claim 2 wherein said control instance is made in response to said CRE- connecting one or more multimedia devices into said ATE message and said identification is done by a DEMMDPS, each device having a plurality of controllable VICE ID message from the application program. functions, each device being in one class of a plurality of 4. A MMDPS in accordance with claim 3 wherein: different classes of multimedia devices, the improve- 65 said MMDCM further comprises a plurality of predefined control panels for each class of said controls, ment comprising: first means responsive to a DEVICE OPEN command from said application program for opening a said MMDCM being further operative in response to said on = ^ TE message to select one of said APLNDC00028569 13 5,428,730 predefined control panels for display in said control screen; and said processing means is further operative in response to receiving said DEVICE ID message to determine device capabilities and device status, and 5 send messages to said user interface for updating said control panel in accordance with said device capabilities and said device status. 5. A MMDPS in accordance with claim 4 wherein said processing means is operative to selectively enable 10 and disable said controllers in said control screen according to whether or not said MMD has a controllable function corresponding to the control function of each controller. 6. A MMDPS in accordance with claim 4 wherein: 15 said MMDCM is further operative in response to receiving a second CREATE message to create a second control instance and a second control panel for a second MMD. 7. A MMDPS in accordance with claim 1 wherein 20 said MMDCM further comprises a plurality of predefined control panels for each class of said controls, said MMDCM being further operative in response to said CREATE message to select one of said predefined control panels for display in said control screen. 25 8. A MMDPS in accordance with claim 7 wherein said predefined control panels for each class include a minimal control panel and a complete control panel, said minimal control panel having a minimum number of controllers for controlling a minimum number of 30 functions, and said complete control panel having a plurality of controllers for controlling a maximum number of functions of each class. 9. A MMDPS in accordance with claim 8 wherein: said predefined control panels further comprise con- 35 trol panels of horizontal and vertical orientations. 10. A MMDPS in accordance with claim 1 wherein: said first means is operative to send to said control instance notification messages notifying said control instance of errors occurring in said MMD and 40 timed events occurring in said MMD; and said processing means is further operative to process said notification messages. 11. In a multimedia data processing system (MMDPS) comprising a processor, a storage system 45 including a memory for storing at least one multimedia application program (MMAP) and a multitasking oper- 14 trols, said templates for each class include a minimal control panel template and a complete control panel template, said minimal control panel template defining a minimum number of controllers for con- trolling a minimum number of functions, and said complete control panel template defining a plurality of controllers for controlling a maximum number of functions of each class; said MMDCM being responsive to a CREATE message from the application program to select one of said templates and send a message to said user mterface to create a control panel in said control screen in accordance with the definition in the template so selected; said control panel in said control screen comprising a plurality of user actuated controllers for controlling different functions of said open MMD; said user interface being operative to send a user selection message to said MMDCM in response to a user manipulating one of said controllers; and, said MMDCM further comprising processing means operative, in response to receiving said user selection message, to send a device command to said first means for changing a function of said open MMD and to send a message to said user interface to update the one of said controllers manipulated by the user to reflect the changed function of said open MMD. 12. A MMDPS in accordance with claim 11 wherein said minimal control panel templates and said complete control panel templates further define control panels of horizontal and vertical orientations. 13. A MMDPS in accordance with claim 12 wherein said controllers include buttons, dials, and sliders. 14. The method of operating a multimedia data processing system (MMDPS) comprising a processor, a storage system including a memory for storing at least one multimedia application program (MMAP) and a multitasking operating system for execution by said processor, a monitor for displaying screens of information, and connector means for connectmg one or more multimedia devices into said MMDPS, each device having a plurality of controllable functions, each device being in one class of a plurality of different classes of multimedia devices, said method comprising: (a) opening a multimedia device (hereinafter referred ating system for execution by said processor, a monitor for displaying screens of information, and connector means for connecting one or more multimedia devices 50 to as open MMD); (b) storing in said storage system a plurality of application-program-selectable class-specific controls, each control being for a different one of said into said MMDPS, each device having a plurality of controllable functions, each device being in one class of classes of multimedia devices; (c) creating, in response to a CREATE message from a plurality of different classes of multimedia devices, the improvement comprising: the application program, a control instance from the one of said controls (hereinafter referred to as first means responsive to a DEVICE OPEN com- 55 mand from said application program for opening a multimedia device (hereinafter referred to as open MMD) and for thereafter transmitting signals to said open MMD in response to said first means receiving device commands, said signals being 60 selected control) having the same class as said open MMD and attaching said control instance to said open MMD; (d) creating a control screen on said monitor in said user interface, said control screen including a control panel comprising a plurality of user actuated operative to control said controllable functions of said open MMD; second means comprising a user interface including a display screen on said monitor; and multimedia device control means (MMDCM) 65 layered between said first means and said second controllers for controlling different functions of said open MMD; (e) sending a user selection message from said user interface to said control instance in response to a user manipulating one of said controllers; (f) generating, in response to said control instance means, said MMDCM comprising a plurality of receiving said user selection message, at least one control panel templates for each class of said con- device command for changing a function of said APLNDC00028570 15 5,428,730 16 . open MMD, based upon said user selection mes- accordance with said device capabilities and said sage and current operational state of said open device status. 18. The method according to claim 15 comprising: sending to said control instance notification messages notifying said control instance of errors occurring in said MMD and timed events occurring in said MMD; (g) transmitting signals to said open MMD in response to said device commands, said signals being 5 operative to change one of said controllable func- tions of said open MMD; MMD; and (h) sending a message to said user interface to and processing said notification messages. update the one of said controllers manipulated by 19. The method according to claim 14 further comthe user, to reflect the changed function of said 10 pnsmg: open MMD. storing in said storage system a plurality of control 15. The method according to claim 14 further companel templates for each class of said controls, said pnsmg: templates for each class include a minimal control storing in said storage system a plurality of control panel template and a complete control panel tempanel templates for each class of said controls, said 15 plate, said minimal control panel template defining templates for each class include a minimal control a minimum number of controllers for controlling a panel template and a complete control panel temminimum number of functions, and said complete plate, said minimal control panel template defining control panel template defining a plurality of cona minimum number of controllers for controlling a trollers for controlling a maximum number of funcminimum number of functions, and said complete 20 tions of each class; control panel template defining a plurality of conincluding in said application program a template detrollers for controlling a ...J..um number of funcfining a custom control panel; tions of each class; specifying in said no ^TE message which template and selecting one of said templates; is to be used in step (d) said step (d) being performed by creating said control 25 said step (d) creates a custom control panel in said panel in said control screen in accordance with the control screen when said custom control panel definition in the one template so selected. template is specified in said ""=^TE message. 16. The method according to claim 15 comprising: 20. The method according to claim 14 comprising: selectively enabling and disabling said controllers in said control screen according to whether or not 30 sending a message to said application program in response to receiving said user selection message in said MMD has a controllable function correspondstep (e), which message requires a response from ing to the control function of each controller. 17. The method according to claim 15 comprising: said application program indicating whether or not determining device capabilities and device status, and said application program will process said user sending a message from said control instance to 35 selection message. said user interface to update said control panel in ***** 40 45 50 55 60 65 APLNDC00028571 04860.P1365THE UNITED STATES PATENT AND TRADEMARK tion of St FeFntCE ) W. chutensen ) Examiner: ) Art Unit: o. 08/316,237 Dela Torre, C. 2415 Filing Date: September 30, 1994 ) For: METHOD AND APPARATUS FOR DISPLAYING AND ACCESSING CONTROL AND STATUS INFORMATION IN A CO- -R ) ) ) ) evenu ) Commissioner of Patents and Trademarks Washington, D.C. 20231 PETITION FOR EXTENSION OF TIME (37 C.F.R. § 1.136(a) Sir: Pursuant to Rule 1.136(a) Applicant hereby petitions for an extension of time to respond to the outstanding Office Action dated March 20, 1996. Our check in the amount of $380.00, the extension fee required for a response filed within the second month (37 C.F.R. § 1.17(b)), is submitted concurrently herewith. Please charge any debits and credit any overages to our Deposit Account Number 02-2666. Respectfully submitted, BL LA KB I Y, SOKOLOFF, TAYLOR & ZAFMAN Dated: ½ , 1996 Michaei'J. Mallie Reg. No. 36,591 12400 Wilshire Blvd. Seventh Floor Los Angeles, CA 90025-1026 (408) 720-8598 (37 C.F.R. § 1.8( ) I hereb certi y that this correspondence is being deposited with the United States Postal Service as first class mail w th sufacient postage in an envelope addressed to the Commissioner of Patents and Trademarks, Washington, D.C. 20231 Date of Deposit Name o Signature erson Mailing Correspondence Date APLNDC00028572 Att 's In in ge A ati Applic PATENT . Steven W. Christensen 8/316.237 Filed: For: 004R60 P13As 9/30/94 .METHOD AND APPARATUS FOR DISPLAYlNG AND ACCESSING CONTROL AND STATUS INFORMATION IN A COMPUTER SYSTEM (title) THE COMMISSIONER OF PATENTS AND TRADEMARKS Washington, D.C. 20231 SIR: Transmitted herewith is an Amendment for the above application. / Small entity status of this application under 37 C.F.R. §§ 1.9 and 1.27 has been established by averified statement previously submitted. / A verified statement to establish small entity status under 37 C.F.R. §§ 1.9 and 1.27 is endlosed. I No additional fee is required. I The fee has been calculated as shown below: (Col. 1) (Col. 2) (Col. 3) Claims remaining after amendment Highest no. previously paid for Present extra Indep. Claims: Rate Additional fee minus 20 4* x $11.00= x $22.00= minus 3 0 x $39.00= x $78.00= + $125.00 + $250.00 Fir t presentation c Multip e Dependent Claim(s) *If the difference in Col. ? Is less than zero, ente "0" in Col. 3 A check in the amount of I fee Other than a Small Entity 3 Claims: XXX Additional 24 Total | XXX Small Entity Rate m a nno Total Add. Fee: $88.00 $0.00 Total Add. Fee: $88.00 is attached for presentation of additional claim(s). Applicant(s) hereby Petition(s) for an Extension of Time of twn 191 month(s) pursuant to 37 C.F.R. § 1.136(a). A check for * n ^^ is attached for processing fees under 37 C.F.R. § 1.17. Please charge my Deposit Account No. 09-2666 the amount of . A duplicate copy of this sheet is enclosed. The Commissioner is hereby authorized to charge payment of the following fees assodated with this communication or credit any overpayment to Deposit Account No. no oman (a duplicate copy of this sheet is enclosed): Any additional filing fees required under 37 C.F.R. § 1.16 for presentation of extra claims. VW Any extension or petition fees under 37 C.F.R. § 1.17. BLAKELY, S-..-.., TAYLOR & ZAFMAN Date: Michael allie Reg. No. 36,591 12400 Wilshire Blvd., 7th Floor Los Angeles, California 90025 (408) 720-8596 I hereby certify that this correspondence is being deposited with the United States Postal Service as first class mail with sufficient postage in an envelope addressed to: Commissioner of Patents & Trademarks, Was on, D.C. 20231 on: Angdfa M. Quinn vare LJV/ccm (1/31/96) APLNDC00028573 PATENT COOPERATION TREATY From the INTERNATIONAL PRELIMINARY EXAMINING AUTHORITY To: JAMES C. SCHELLER BLAKELY, SOKOLOFF, TAYLOR & ZAFMAN 12400 WILSHIRE BOULEVARD, 7TH FLOOR GELES C - -1026 WRTITEN OPINION O (PCT Rule 66) OC I 1 1 199ô BgY, 80KOLOFF, TAYLOR & 2AFMAN LOS ANGELES Date of Mailing (day/month/year) Applicant's or agent's file reference 04860.P1365 REPLY DUE / International application No. International filing date (day/monthlyear) PCTIUS95/11025 | 29 AUGUST 1995 aancT W98 " ¯ ~ within TWO months from the above date of mailing Priority date (day/monthlyear) 30 --- --BER 1994 International Patent Classification (IPC) or.both national classification and IPC IPC(6): GO6F 9/44; and US Cl.: 395/161 Applicant APPLE COMPUTER, INC. 1. This written opinion is the Et (first, etc.) drawn by this International Preliminary Examining Authority. 2. This opinion contains indications relating to the following items: I Basis of the opinion II Priority III Non-establishment of opinion with regard to novelty, inventive step or industrial applicability IV Lack of unity of invention V Reasoned statement under Rule 66.2(a)(ii) with regard to novelty, inventive step or industrial applicability; citations and explanations supporting such statement VI Certain documents cited VII Certain defects in the international application VIII Certain observations on the international application 3. The applicant is hereby invited to reply to this opinion. When? See the time limit indicated above. How? By submitting a written reply, accompanied, where appropriate, by amendments, according to Rule 66.3. For the form and the language of the amendments, see Rules 66.8 and 66.9. Also For an additional opportunity to submit amendments, see Rule 66.4. For the examiner's obligation to consider amendments and/or arguments, see Rule 66.4 bis. For an informal communication with the examiner, see Rule 66.6. If no reply is filed, the international preliminary examination report will be established on the basis of this opinion. 4. The final date by which the international preliminary examination report must be established according to Rule 69.2 is: 30 JANUARY 1997 Name and mailing address of the IPEA/US Authorize officer Commissioner of Patents and Trademarks Box PCI' Washington, D.C. 20231 Facsimile No. (703) 305-3230 Form PCT/IPEA/408 (cover sheet) (January 1994)* WiARK R. POWELL Telephone No. (703) 305-9703 APLNDC00028574 -.. ...n. OPINION I. International application No. PCT/US95/11025 Basis of the opinion 1. This opinion has been drawn on the basis of (Substitute sheets which have beenfurnished to the receiving Ojice in response to an invitation under Article 14 are referred to in this opinion as "originaHyfded".): the international application as originally filed. the description, pages 1-54 pages NONE , filed with the demand. pages NONE , filed with the letter of the claims, Nos. 1-18 , as originally ñled. , as originally filed. Nos. NONE , filed with the demand. Nos. NONE the drawings, , as amended under Article 19. Nos. NONE , filed with the letter of sheets/Ag 1-13 , as originally filed. shœtsing NONE , filed with the demand. sheets/Ag NONE , filed with the letter of 2. The amendments have resulted in the cancellation of: the description, pages NONE the claims, Nos. NON the drawings, shots/Rg 3. NONE This opinion has been established as if (some of) the amendments had not been made, since they have been considered to go beyond the disclosure as filed, as indicated in the,SupplemmtabBox Additional observations below (Rule 70.2(c)). 4. Additional observations, if necessary: NONE Form PCT/IPEA/408 (Box I) (January 1994)* APLNDC00028575 International application No. WRITTEN OPINION PCT/US95/11025 V. Reasoned statement under Rule 66.2(a)(ii) with regard to novelty, inventive step or industrial applicability; citations and explanations supporting such statement 1. STATEMENT Novelty (N) 2. YES 1-18 NO Claims NONE YES Claims Industrial Applicability (IA) NONE Claims Inventive Step (IS) Claims 1-18 NO Claims Claims 1-18 NONE YES NO CITATIONS AND EXPLANATIONS (See Supplemental Sheet.) Form PCT/IPEA/408 (Box V) (January 1994)* APLNDC00028576 International application No. WRITTEN OPINION VI. PCT/US95/11025 Certain documents cited 1. Certain published documents (Rule 70.10) Application No. Patent No. Filing Date (dayhnonth/year) US, A, 5,416,895 16 MAY 1995 8 APRU 1992 NONE US, A, 5,428,730 2. Publication Date (day/month/year) Priority date (valid claim) (day/month/year) 27 JUNE 1995 15 DECEMBER 1992 NONE Non-written disclosures (Rule 70.9) Kind of non-written disclosuæ Date of non-written disclosure (day/monthlyear) Date of written disclosure referring to non-written disclosure (day/month/year) Form PCT/IPEA/408 (Box VI) (January 1994)* APLNDC00028577 Intemational application No. - "N OPINION PCT/US95/11025 Supplemental Box (To be used when the space in any of the preceding boxes is not sufficient) Continuation of: Boxes I - VIII Sheet 10 TIME LIMIT: The time limit set for response to a Written Opinion may not be extended. 37 CFR 1.484(d). Any response received after the expiration of the time limit set in the Written Opinion will not be considered in preparing the International Preliminary Examination Report. V. 2. REASONED STATEMENTS - CITATIONS AND EXPLANATIONS (Continued): Claims 1-18 lack novelty under PCT Article 33(2) as being anticipated by the patent to Mills. As to claim 1, see Fig. 1, which shows a processor 14, display screen 18, and cursor control device 19. Mills also teaches 'window generation logic' to 'generate and display a first window region' with view window 20, at Fig. 2, and at col. 3, line 68; and 'indicia generation logic' to 'generate data for display in at least one display area in the first window' such as windows 22,24 at Fig. 2, such that the 'indicia generation logic use message-based communications to exchange information to coordinate activities' at col. 3, lines 22-26. Mills also teaches a 'control strip' [claim 2] with controller 36, at Fig. 2, and at col. 4, lines 32-40; wherein at least one display area is variably sized [claim 3] using size boxes, at Fig. 2, and at col. 4, lines 8-9; and that the size of the first window region is variable [claim 4] also at col. 4, lines 8-9. In addition, Mills teaches sizing the first window region so that none of the display areas are visible [claim 5] with close box 28, at Fig. 2, and at col. 4, lines 7-8, or all [claim 6] or a portion [claim 7] of the display areas are visible, both at col. 4, lines 8-9. Mills also teaches that one of the display areas displays only information [claim 8] with video window 22, at Fig. 2, or provides -s to control information [claim 9] with video window 24, at Fig. 2, or displays an additional display element [claim 10] with video window 22, at Fig. 2. As to claim 11, it corresponds to claim 1, and also includes the following: using the 'indicia generation logic' to 'generate user sensitive graphics for display' at col. 4, lines 32-40; wherein the 'window generation logic' determines when a data display area has been selected, signals the indicia generation logic, which then initiates a response, at col. 2, lines 9-13. Claims 12-14 correspond respectively to claims 6,2,3. Regarding claim 15, Mills teaches the following: 'generating a first window' with view window 22, at Fig. 2, to accommodate one display area, as with control window 24, for 'indicia' with control buttons 42-54, at Fig. 2, and col. 4, lines 37-40, by 'executing a first programming module' at col. 3, line 66; 'displaying an indicia' at col. 4, lines 32-40; 'selecting one of the indicia' at col. 4, lines 48-55, and sending a message for generating the display at col. 3, lines 22-26; with the programming module performing a function, at col. 4, lines 38-40. Mills also teaches 'status information' [claim 16] such as tentative-set state at col. 5, lines 1014, or fixed-set state at col. 5, lines 23-35, and describes the operating stat_e or mode of the controller at col. 6, lines 21-32. Form PCT/IPEA/408 (Supplemental Box) (January 1994)* APLNDC00028578 w m OPINION International application No. PCT/US95/11025 Supplemental Box (To be used when the space in any of the preceding boxes is not sufficient) Continuation of: Boxes I - VIII Sheet 11 In addition, Mills teaches 'control information' [claim 17] in control window 24, at Fig. 2, and at col. 4, lines 32-40. As to claim 18, Mills teaches that the first programming module requests a set of features at col. 4, lines 20-25, sends a message to the programming module indicative of features, and the programming module returns a message; such that the programming modules interact with each other in response to user interaction with the first programming module, all at col. 4, lines 25-40. Claims 1-18 lack an inventive step under PCT Article 33(3) for the reasons given forth above. Claims 1-18 have industrial applicability as defined by PCT Article 33(4) because the subject matter claimed can be made or used in industry. -- NEW CITATIONS US, A, 4,896,291 (Gest et al) 23 JANUARY 1990, see Fig. 1, which shows graphical user interface tools, such as slide bars 12, dials 11, and menus 14. US, A, 5,202,961 (Mills et al) 13 APRIL 1993, see Figs. 2,3, which show a slider bar 36, with velocity and direction indicators, and a control icon 40. Forn PCT/IPEA/408 (Supplemental Box) (January 1994)* APLNDC00028579 65 JDS I Paten ' E UNITED STATES PATENT AND TRADEMARK OFFICE In re Application of ) Steven W. Christensen ) Examiner: Dela Torre, C. Serial No. 08/316,237 ) Art Unit: 2415 Filing Date: September 30, 1994 ) For: METHOD Amn APP^R^TUS FOR DISPLAYING AND ACCESSING COIN 1xOL AND STATUS INFORMATION IN A COMPUTER ) ) ) ) SYSTEM ) AMENDMRNT Commissioner of Patents and Trademarks Washington, D.C. 20231 Sir: In response to the Office Action mailed March 20, 1996, Applicant respectfully requests the Examiner to enter the following amendments and consider the following remarks: IN · - SPECTFICATION At page 2, line 1 At page 4, line 10, ease replace "it" with --It--. ase replace "individiual" with --individual--. At page 21, line 23, please insert --not-- after "has". Serial No. 08/316,237 1 04860.P1365 APLNDC00028580 At page 28, lines 11-12, please delete "If the cursor location is determined to be within the control strip." AT page 28, line 15, please insert -within the control strip-- after "occurs". IN THF. CT.ATMS . 2 (Amended) An interactive computer-controlÏ&d d s¯pÏay system comprisin : 3 a pro ssor; 4 a data d play screen coupled to the processor; 5 a cursor c 6 7 trol device coupled to said processor for positioning a cursor on said data isplay - a w; a window gen ation and contr.ol logic coupled to the processor and 8 data display screen to c ate an operating environment for a plurality of 9 individual programming odules that provide status and control funci;ions, 10 wherein the window genera n and control lo ic venerates and dis lavs a 11 first window region having a pl ality of display areas on said data display 12 screen, wherein each of the plural 13 the plurality of individual programuking modules; of display areas is associated with one of 14 an_ indicia generation logic couple to the data display screen to executa 15 at least one of the plurality of programmin modules to generate inform=+1an 16 [data] for display in [at least] one of the plurah 17 window reyton wherein [a] at least one of the p rality of display areas and its 18 associated_prog---_---imodule is sensitive to us r input, and further 19 wherein the window generation anri can+ral logic an the indicia generation 20 logic use message-based communication to exchange in rmation to Serial No. 08/316,237 2 of display areas in the first 04860.P1365 APLNDC00028581 21 coordinate activities 22 display activity. 1 5. the indicia generation logic to enable interactive (Amended) The display system defined in Claim 4 wherein the 2 first window region[s] is sized such that none of the phiralg of display areas 3 à [are] visible. 1 6. (Amended) The display system defined in Claim 4 wherein the 2 first window region[s] is sized such that all of the plurality of display areas are 3 visible. 1 7. (Amended) The display system defined in Claim 4 wherein the 2 first window region[s] is sized such that a portion of the plurality of display 3 areas n [are] visible. 1 2 1 8. (Amended) The display system defined in Claim 1 wherein at least one of the pluralg of the display [data] areas only displays information. 9. (Amended) The display system defined in Claim 1 wherein at 2 least one of the display [data] areas acts to provide access to control 3 information when selected. 1 10. (Amended) The display system defined in Claim 9 wherein said 2 at least one of the plurality of display [data] areas displays an additional 3 display element. Serial No. 08/316,237 3 04860.P1365 L APLNDC00028582 2 1 . (Amended) An interactive computer-controlled display system comprisin : 3 a proc ssor; 4 a data d play screen coupled to the processor; 5 a cursor c 6 trol device coupled to said processor for positioning a cursor on said data isplay s--; 7 window gener tion and control logic coupled to the processor and data 8 display reen to create an operating environment for a plurality_of 9 individual programming modules that provide status and control functions 10 wherein the window gener tion and control logic generates and displays a 11 first window region having 12 scre 13 the plurality of individual prog mming modules [, wherein the first 14 window region comprises at least wherein each of the lurality of display areas on said data display ality of display areas is associated with one of e data display area]; 15 at least one indicia graphics g neration logic coupled to the processor 16 and the window generation logic, wh ein said at least one indicia graphics 17 generation logic generates user sensitive 18 one data display area by executing at leas one of the plurality of programming 19 modules; 20 aphics for display in said at least wherein the window generation and c ntrol logic determines when 21 said at least one data display area has been sele 22 at least one indicia graphics generation logic in r ponse to user selection, and 23 further wherein said at least one indicia graphics g neration logic initiates a 24 response from said at least one of the plurality of pr gramming modules. 1 2 d by the user and signals said (Amended) The display system defined in Claim wherein the first window regn is always visible to the user. Serial No. 08/316,237 4 04860.P1365 APLNDC00028583 1 . ' 2 3 4 ' 5 (Amended) A method for generating control information comprisin the steps of: create n operating environment for a plurality of individual programming odules that provide status and control fimetions· generating first window sized to accommodate a pluraligof [at least 6 one] display areas 7 the first window com rises] executing [a first] at least one of the plurality of 8 individual programm° 9 areas is associated with oe of the plurality of individuaLyrog 10 r indicia resulting_Jrom [, wherein the step of generating modules, wherein each of the plurality of displaX a maanle=· 11 displaying an indicia ° each of said at least one display area by 12 executing one of a plurality of rogramming modules corresponding to each 13 indicia; 14 selecting one of the indicia, herein the step of selecting comprises the 15 first progr ------- --g module determin 16 area is selected and sending a message 17 plurality of programming modules respon ible for generating the display of 18 the selected indicia; 19 20 which of said at least one display the programming module of said said progr-------ing module performing function in response to the selection. Please add the following new claims: I( 1 2 (New) The display system defined in Claim 1 wherein each of the plurality of display areas is individually and variably sized. Serial No. 08/316,237 5 04860.P1365 APLNDC00028584 1 2 (New) The display system defined in Claim 1 wherein the first window region always appears in front of application windows. / 3 1 (New) The display system defined in Claim 1 wherein the first 2 window region is implemented in a private window layer that appears in 3 front of windows for all applications layers. 1 2 (New) The display system defined in Claim the plurality of display areas is individually and variably sized. 1 2 wherein each of (New) The display system defined in Claim wherein the first window region always appears in front of application windows. 1 (New) The display system defined in Claim wherein the first 2 window region is implemented in a private window layer that appears in 3 front of windows for all applications layers. Applicant respectfully requests reconsideration of this application as amended. Claims 1-18 remain in the application. Claim 1, 5-12, and have been amended. Claims 19-24 have been added. No claims have been canceled. The Examiner has listed a number of informalities and errors in the application. The Applicant has corrected these informalities and errors, as well as others, to put the application in correct form for allowance. The Examiner has rejected Claims 1-18 under 35 U.S.C. § 102(b) as being unpatentable over \ Serial No. 08/316,237 111= * =1 Mills teaches the use of a slider control bar for 6 04860.P1365 L APLNDC00028585 controlling the rate of display of sequential information. Specifically, Mills teaches the use of such a control window to control the playback rate of video. This control window is defined as having certain components such as standard playback direction/velocity indicators, reverse, stop and fast forward. Mills. uses control icon to select one of these rates of display. Claim 1 of the present invention as amended states, in part: a window generation and control logic coupled to the processor and data display screen to create an operating environment for a plurality of individual progr-------ing modules that provide status and control functions, wherein the window generation and control logic generates and displays a first window region having a plurality of display areas on said data display screen, wherein each of the plurality of display areas is associated with one of the plurality of individual progr-,sing modules; Thus, the present invention provides logic that creates an operating environment like a shell for other programming modules to provide status and control functions. Mills does not provide such an environment. In Mille the control window is used for controlling video generated by an application. Claim 1 of the present invention also includes that "window generation and control logic and the indicia generation logic use messagebased communication to exchange information to coordinate activities of the indicia generation logic to enable interactive display activity" Contrary to the Ex22,.°1L€r'S RSsertion, Mills does not teach the use of message based communication for information exchange, particularly between control logic that creates an envi-------ent for the plurality of programming modules Serial No. 08/316,237 7 04860.P1365 APLNDC00028586 which control the indicia generation logic. In one described embodiment, the message based communication comprises the control strip of the present invention passing messages to a module to either tell it what to do or to obtain information about the module and its capabilities (e.g., See Specification, pg. 32, lines 5-7). There is no indication in Mills that such message based communication is used. In view of the above discussion, Applicant respectfully submits that Mills does not anticipate the present invention as claimed in Claim 1, nor any of its dependent claims. Independent Claims 11 and 15 also include a similar limitation regarding the creation of the operating environment. Therefore, based on the same rationale given above, Applicant respectfully submits Claims 11 and 15 and their dependent claims are not anticipated by Mills. Applicant has added Claims 19-24. Claims 19-21 are dependent on Claim 1, and Claims 22-24 are dependent on Claim 11. Claims 19 and 22 provide that each of the plurality of display areas is individually and variably sized. Mills does not disclose such sizing. Claims 20 and 23 provide that the first window region always appears in front of application windows. Similarly, Claims 21 and 24 provide that the first window region is implemented in a private window layer that appears in front of windows for all applications layers. Mills clearly does not show this. In fact, as soon as another application is used in Mills, the control window will be overlapped and at least partially non-visible. Therefore, based on this, Applicant respectfully submits that Claims 19-24 are allowable over the art of record. Accordingly, Applicant respectfully submits that the rejection under 35 U.S.C. § 102(b) has been overcome by the amendments and the remarks and withdrawal of these rejections is respectfully requested. Applicant submits Serial No. 08/316,237 8 04860.P1365 L APLNDC00028587 that Claims 1-18 as amended and Claims 19-24 as added are now in condition for allowance and such action is earnestly solicited. Please charge any shortages and credit any overcharges to our Deposit Account No. 02-2666. Respectfully submitted, Bl .« K NLY, SOKOLOFF, TAYLOR & ZAFMAN Dated: , 1996 Micha J Mallie Attorney for Applicant Registration No. 36,591 12400 Wilshire Boulevard Seventh Floor Los Angeles, CA wo-1026 (408) 720-8598 I hereby certify that this correspondence is being deposited with the United States Postal Service as first class mail with sufficient postage in an envelo e addressed to the ommissioner of Patents and rademarks, Washington, D.C. 0231 on Augmt on Angel& Quinn Serial No. 08/316,237 9 ' 6 10 A August 20, 1996 Date 04860.P1365 APLNDC00028588 Attorney's Docket No. 04860.Pl365 PA IN THE • " •• ~••STATES PATENT AND = a = mviAKE OFFICE In Re Patent Application of: ) Steven W. Christensen Application No.: Filed: For: 08/316,237 September 30, 1994 Examiner: ) Art Unit: C. Dela Torre 2306 ) METHOD AND APP A RATUS ) FOR DISPLAYING AND STATUS) INFORMATION IN A COMUTER ) SYSTRM ) Assistant Co......:-sioner for Patents Washington, D.C. 20231 REGENED INFORMATION DISCLOSURE STATEMENT Sir: 6800, () Enclosed is a copy of Information Disclosure Citation Form PTO-1449 together with copies of the documents cited on that form. It is respectfully requested that the cited --nts be considered and that the enclosed copy of Information Disclosure Citation Form PTO-1449 be initialed by the E--··-----er to indicate such consideration and a copy thereof returned to applicant(s). Pursuant to 37 C.F.R. § 1.97, the submission of this Information Disclosure Statement is not to be construed as a representation that a search has been made and is not to be construed as an admission that the information cited in this statement is material to patentability. Pursuant to 37 C.F.R. § 1.97, this Information Disclosure Statement is being submitted under one of the following (as indicated by an "X" to the left of the appropriate paragraph): 37 C.F.R. §1.97(b). X 37 C.F.R. §1.97(c). If so, then enclosed with this InformationÛ Disclosure Statement is one of the following: C X A certification pursuant to 37 C.F.R. §1.97(e) y A check for $ LJV/cak (10/01/96) for the fee under 37 C.F.R. § 1.17(p). - 1 - APLNDC00028589 37 C.F.R. §1.97(d). If so, then enclosed with this Information Disclosure Statement are the following: (1) A certification pursuant to 37 C.F.R. §1.97(e); (2) A petition requesting consideration of the Information Disclosure Statement; and (3) A check for $ for the fee under 37 C.F.R. § l.17(i) for submission of the Information Disclosure Statement. If there are any additional charges, please charge Deposit Account No. 02-2666. Respectfully submitted, BLAKELY, SOKOLOFF, TAYLOR & Dated: , \ -- LLP , 1996 s C. Scheller, Jr. g. No. 31,195 12400 Wilshire Blvd. Seventh Floor Los Angeles, CA 90025-1026 (408) 720-8598 I hereby certify that this correspondence is being deposited with the United States Postal Service as first class mail with sufficient postage in an envelope adoressed to the Assistant Commissioner for Patents, Washington, D. C. 20231on L \¾. ( 9 p (Date of Deposit) Dehemh McDovem (Typed or printed na e of person mailing correspondence) (Signature of person mailing correspondence) LJV/cak (10/01/96) to o, -2- APLNDC00028590 AVAC.B To: PAPER & Sheet U.S. DEPARTMENT OF COMMERCE ATTY. DOCKET NO. PATENT AND WADEMARK OFFICE rm PTO-1449 EV. 8-83) 1 of 1 SERIAL NO. 04860.P1365 08 APPLICANT 4 TRA06f INFORMATION DISCLOSURE CITATION Steven W. Christensen FILING DATE (Use several sheets if necessary) GROUP 9/30/94 289& 246 U.S. PATENT DOCUMENTS *E I ER -umm. NUMBER DATE NAME 5 4 1 6 8 9 5 5/16/95 Anderson, et al. CLASS 395- - †4W IF FILING DATAETE 4/8/92 FOREIGN PATENT DOCUMENTS DOC"MENT GUMBER DATE COUNTRY CLASS SUBCLASS TRANSLATION YES NO I I THER DOCUMENTS (including Author, 71tle, Date, Pertinent Pages, Etc.) fj3 c "PCT Written Opinion", October 4, 1996 Ch EXAMINER DATE CONSIDERED Q .ORLA -TORí2g i\/(2/96 *EXAMINER: Initial if citation considered, whether or not citation is in conformance with MPEP 609; Draw line through citation if not in conformance and not considered. Include copy of this form with next communication to applicant. APLNDC00028591 lllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll! United States Patent [191 Anderson et al. [11] [45] [54] SYSI'EM AND METHODS FOR IMPROVED SPREADSHEET INTERFACE WITH USER-Famr sax OBJECTS Lotus Development Corporation, Lotus User's Guide: Lotus 1-2-3 for Windows Release 1.0, 1991, pp. 37-46, 307-317. Proceedingr Advanced Computer Technology, Reliable Systems and Applications, 5th Annual European Computer Conf., CompEuro '91, May 13, 1991, Italy, pp. 282-286. Andreas Hutflesz et al., The R-File·An Effcient Access Structurefor Proximity Queries, Proceedings Sixth International Conference on Data Engineering, Feb. 5-9, 1990, Los Angeles Airport Hilton and Towers, Los Angeles, Calif., USA, pp. 372-379. D. Cakmakov et al., "A Multimedia Cognitive-Based System Model" Advanced Computer Technology, Reliable Systems and Applications, 5th Annual European Computer Conference, Bologna, May 13-16, 1991, pp. [73] Assignee: Borland International, Inc., Scotts Valley, Calif. [21] Appl. No.: 866,658 [22] Filed: Apr. 8, 1992 [52] U.S. CI. . GO6F 17/30; GO6F 17/40 .............. 395/148; 395/159; 395/161; 395/158; 395/155 [58] Field of Search ............... 395/155, 148, 158, 149, 395/159, 160, 161; 340/747, 748; 364/401; 345/24, 25 [56] References Cited U.S. PAxen1 DOCUMENTS 5,021,973 6/199 5,033,009 7/199 282-286. Birkholz, "Die Datenbank in der Tabellenkalkulation," CIIIP 2eitschnf fur Mikrocomputer-Technik, No. 3, Mar. 1991, pp. 450-454. "Hyperpaper," IBM Technical Disclosure Bulletin, vol. 33, No. 9, Feb. 1991, pp. 283-384. Hernandez et al. ................ 364/518 Dubnoff .............................. 364/523 Pdmary Examiner-Mark K. Zimmerman Assistant Examiner-J. Feild Attorney, Agent, or Firm-John A. Smart; David N. Slone; Vernon A. Norviel 5,055,998 10/ 99 Wright et al. ....................... 364/200 5,072,412 12/ 99 Henderson, Jr. et al. .......... 395/159 5,093,907 3/ 992 Hwong et al. ...... 395/152 5,095,429 5,121,499 5,140,678 5,182,793 5,231,577 5,237,651 5,283,864 3/ 992 8/1992 1/1993 7/1993 8/1993 2/1994 Harris et aL ........................ 364/408 Torres ........-........................ 395/159 Alexander et al. .---............... 395/13 Koss .................-......--........ 364/419 Randall ........ 395/148 Knowlton ........................... 395/158 6/1992 McCaskill et al- .................. 395/700 [57] ABSTRACT An electronic spreadsheet system of the present invention includes a notebook interface having a plurality of notebook pages, each of which may contain a spread of information cells, or other desired page type (e.g., Graphs page). Methods are provided for rapidly accessing and processing information on the different pages, including displaying a plurality of page identifiers for selecting individual pages, and further including a preferred syntax for referencing information. Additional methods are provided for editing cells and blocks of cells. FOREIGN PA mm DOCUMENTS 0428261 0431638A2 0443184A2 WO92/04678 WO92/08199 5/1991 6/1991 8/1991 3/1§92 5/1992 5,416,895 May 16, 1995 Lotus User's Guide: Lotus1-2-3for Windows Release l.0; 1991; pp. 159-171. Lucia 3-D Tutorial Guide, Chapter 16, pp. 1-18, 1987. [75] Inventors: Charles R. Anderson, Santa Cruz; Robert W. Warfield, Aptos; Istvan Cseri, Scotts Valley; Murray K. Low, Santa Cruz; Weikuo Liaw, Scotts Valley; Alan M. Bush, Palo Alto, all of Calif. [51] Int. CL6 ........ - ' USOO5416895A Patent Number: Date of Patent: European Pat. Off. . European Pat. Off. . European Pat. OfT . WIPO . WIPO . OTHER - CATIONS Microsoft Excel User's Guide, 1990, pp. 76-79· . 17 Claims, 35 Drawing Sheets 250 APLNDC00028592 U.S. Patent May 16, 1995 Sheet 1 of 35 5,416,895 104 KEYBOARD 105 PONTING DEVICE ¯ ios 102 D-- DISPLAY 107 103 MASS STORAGE 108 PRINTING D••·, - MAIN MEMORY Ilo CONTROLLER - 101 CENTRAL PROCESSOR 110 FIG. 1A 150 152 151 153 APPLICATION T SOFTWARE OPERATING F SYSTEM A USER FIG. iB APLNDC00028593 172 160 -- 174 - 175 • . Sample Appilcation EIIe Edit Iext grawing Ereferences +Y The quick brown 182 {TEXTOBÆCT) f0x jumped over ... r- m 165 181 (GRAPHIC OBÆCT (D FIG. 1C 178 APLNDC00028594 r U.S. Patent C‡ Sheet 3 of 35 5,416,895 C3 w· May 16, 1995 CO N N S APLNDC00028595 220 221 224 \ l cut I copy pastel C I i 22L 22L . 226 227 229 Ililli Normal 225 • a...2 7 22 FIG. 28 A 261a 262a 263a FIG. 2D 260b b FIG. 2E 'y APLNDC00028596 ' A 250 ,253 ,,256(CELLC4") 255 254 NOTEBK1.WB1 liilt 118 257··· 252- .* cilD (0 | E F I'd 259 e+ GlH 4 5 a 7 1 25 10 11 12 13 14 DJ DY © 15 18 271 260 272 273 FIG. 2C 278 CA 00 APLNDC00028597 U.S. Patent May 16, 1995 Sheet 6 of 35 5,416,895 260a I 2eaa 271a 261a 272a FIG. 3A 273 2eoo I 262c 271c Graphs 261c E 272c FIG. 38 F G .. H . +I 273 259 VERTICAL NAVIGATION HORIZONTÆ NAVIGATION FIG. 3C APLNDC00028598 U.S. Patent May 16, 1995 Sheet 7 of 35 5,416,895 COLB 410 I 411 13-ir I 2 3 - I * fŒff/MI F////ff//l - Y/////'7/A i I 6 f////////d 6 V/ff(//fd FIG. 4A 421 420ROW3 -i 4 - A - e _ r - V/////Æ/////// .I 6 410m L-i . 420 ¡ \ myyNI 1 I N 1 2 , YR/«///1 V/ /// I * - Yl//f///d I 6 - FIG.48 c («/€///1 ! 5 - VN¥ \ .! FIG. 4C 431 - 4\ /////////////////////////A FIG. 4D 441 ' 4 , , , , , , //////// /////M 27////E, ..... 442 FIG. 4E APLNDC00028599 U.S. Patent May 16, 1995 Sheet 8 of 35 5,416,895 to 8 CO APLNDC00028600 I A - Elle B Edit Block \ C | D E I F H 1 c Ouattro Pro for Windows - PROFITS.WB1 Ioola §raph Eroperty ndow Belp D.ata IBIBIRI IHeading2 le] Salads:04 G Igla. 1 1 PROFITS: SALADS 2 3 4 7 8 9 10 La Food Cost (D þ|g J am liÏ Total Öõi 1.00 6Ï smummuma . ofit 2 397.22% Las ... fahd 73 4 17 «D 51.00 $2.75 24 $0.50 $2.75 $1.75 22 52.25 00% FIG. 4G 0.75 $2.75 07 36Ò ËϾ 00 APLNDC00028601

Disclaimer: Justia Dockets & Filings provides public litigation records from the federal appellate and district courts. These filings and docket sheets should not be considered findings of fact or liability, nor do they necessarily reflect the view of Justia.


Why Is My Information Online?