Mirror Worlds, LLC v. Apple, Inc.

Filing 389

MOTION to Strike the Untimely Supplemental Expert Rebuttal Report of John Levy, Ph.D. Regarding Validity and To Preclude Dr. Levy from Testifying at Trial About His New Opinions by Apple, Inc.. (Attachments: #1 Affidavit Declaration of J. Randall, #2 Exhibit 1, #3 Exhibit 2 (Part 1 of 2), #4 Exhibit 2 (Part 2 of 2), #5 Exhibit 3, #6 Exhibit 4, #7 Exhibit 5, #8 Exhibit 6, #9 Exhibit 7, #10 Exhibit 8, #11 Exhibit 9, #12 Exhibit 10, #13 Exhibit 11, #14 Text of Proposed Order)(Randall, Jeffrey)

Mirror Worlds, LLC v. Apple, Inc.

Doc. 389 Att. 3

Exhibit 2 Part 1 of 2

Dockets.Justia.com

IN THE UNITED STATES DISTRICT COURT FOR THE EASTERN DISTRICT OF TEXAS TYLER DIVISION MIRROR WORLDS, LLC, Plaintiff, v. APPLE INC., Defendant. JURY TRIAL DEMANDED Civil Action No. 6:08­CV­88 LED

APPLE INC., Counterclaim Plaintiff, v. MIRROR WORLDS LLC, MIRROR WORLDS TECHNOLOGIES, INC., Counterclaim Defendants EXPERT REPORT OF STEVEN K. FEINER, Ph.D. RE: INVALIDITY OF U.S. PATENT NO. 6,006,227, U.S. PATENT NO. 6,638,313, U.S. PATENT NO. 6,725,427 AND U.S. PATENT NO. 6,768,999

HIGHLY CONFIDENTIAL ATTORNEYS' EYES ONLY

TABLE OF CONTENTS Page

I. II. III. IV. V.

INTRODUCTION ............................................................................................................. 1 QUALIFICATIONS .......................................................................................................... 1 SUMMARY OF OPINIONS ............................................................................................. 2 MATERIALS CONSIDERED .......................................................................................... 3 TECHNOLOGY BACKGROUND ................................................................................... 4 A. Pre­Computer Organization .................................................................................. 8 1. 2. 3. B. C. Stacks of information................................................................................. 8 Partial overlapping of documents for greater accessibility...................... 11 Document ordering .................................................................................. 22

Speculative Methods for Organizing and Accessing Documents........................ 28 Computer­Based Methods for Organizing and Accessing Documents.............. 31 1. 2. 3. 4. 5. 6. 7. 8. 9. File cabinets and folders .......................................................................... 31 Windows for displaying multiple documents or programs simultaneously ......................................................................................... 51 Partially overlapping windows ................................................................ 53 Representation of 3D objects in 2D systems ........................................... 56 3D representations in 2D computer graphics systems ............................. 60 GUIs for displaying 3D representations in 2D systems........................... 61 Documents displayed in a receding foreshortened stack ......................... 62 Timelines.................................................................................................. 64 Document indexing and searching........................................................... 66

D. E. VI. A. B. C. D. VII. A.

Metadata............................................................................................................... 67 Apple's Advanced Technology Group ................................................................ 67 U.S. Patent No. 6,006,227.................................................................................... 72 U.S. Patent No. 6,638,313.................................................................................... 75 U.S. Patent No. 6,725,427.................................................................................... 78 U.S. Patent No. 6,768,999.................................................................................... 79 Legal Standards.................................................................................................... 82 -i-

MIRROR WORLDS PATENTS ..................................................................................... 69

INVALIDITY: ANTICIPATION AND OBVIOUSNESS.............................................. 82

TABLE OF CONTENTS (continued) Page

1. 2. B. 1. 2. C. D.

Anticipation.............................................................................................. 82 Obviousness ............................................................................................. 83 Construed Terms...................................................................................... 89 Indefinite Terms....................................................................................... 93

Claim Construction .............................................................................................. 89

Level of Ordinary Skill in the Art........................................................................ 93 Anticipation/Obviousness References ................................................................. 95 1. 2. 3. 4. 5. 6. 7. 8. 9. Mander 724/Piles Project......................................................................... 96 Lucas '330/Workscape............................................................................. 97 Smart Folders ........................................................................................... 98 MEMOIRS............................................................................................... 99 AAAI Fall '95 Symposium Paper.......................................................... 100 TR­1070/Lifestreams ............................................................................ 101 '227 Patent ............................................................................................. 102 Additional Prior Art References ............................................................ 102 Claim Charts .......................................................................................... 111

VIII.

SUMMARY OF PRIOR ART REFERENCES............................................................. 113 A. B. C. Mander '724....................................................................................................... 113 Retrospect .......................................................................................................... 117 1. 1. 2. D. E. 1. 1. F. G. Motivation to Combine Mander '724 with Retrospect .......................... 119 Workscape.............................................................................................. 124 Motivation to Combine Lucas '330/Workscape with Mander '724 ...... 132 Motivation to Combine Lotus Magellan with Lucas '330..................... 135 A Person of Ordinary Skill in the Art would be Motivated to Combine Thompson­Rohrlich and Inside Macintosh ........................... 141 Lucas '330.......................................................................................................... 121

Lotus Magellan .................................................................................................. 133 Thomson­Rohrlich '852 and Inside Macintosh................................................. 137

TR­1070 ............................................................................................................ 143 AAAI Fall '95 Symposium Paper...................................................................... 145 -ii-

TABLE OF CONTENTS (continued) Page

H. I. J.

Spatial Data­Management ................................................................................. 147 MEMOIRS......................................................................................................... 155 Additional Prior Art References ........................................................................ 158 1. SIGIR '93 "Content Awareness in a File System Interface: Implementing the 'Pile' Metaphor for Organizing information" by Rose, Mander, Oren, Ponceleon, Saloon and Wong (APMW0000812­APMW0000821) ...................................................... 159 CHI '92: "A 'Pile' Metaphor for Supporting Casual Organization of Information." by Mander, Salomon and Wong (APMW0000846­APMW0000862) ...................................................... 160 Japanese Publication No. 06­180661 .................................................... 161 U.S. Patent No. 5,621,906 (O'Neil)....................................................... 162 U.S. Patent No. 5,758,324 (Hartman et al.) ........................................... 163 U.S. Patent No. 6,396,513 (Helfman et al.) ........................................... 165 U.S. Patent No. 5,724,567 (Rose et al.) ................................................. 167 U.S. Patent No. 6,202,058 (Rose et al.) ................................................. 168 U.S. Patent No. 5,649,188 (Nomura et al.) ............................................ 169 The HyperCard Basics (Apple Computer, 1990)................................... 170 U.S. Patent No. 6,006,227 (Freeman et al.) ........................................... 171 "Semantic File Systems," by Gifford, Jouvelot, Sheldon and O'Toole (ACM '91) (APMW0018268­APMW0018277) .................... 171 On Location 2.0.1, by On Technology, Inc. (1989­90)......................... 172 Vannevar Bush Article........................................................................... 176 Kullberg Thesis--Dynamic Timelines .................................................. 176 The Eyes Have It: A Task By Data Type Taxonomy for Information Visualizations..................................................................... 178 Robert Spence Office of the Future ....................................................... 179 Document Management Systems........................................................... 183 "Representation in Virtual Space: Visual Convention in the Graphical User Interface" by L. Staples (1993) (APMW0018360­ APMW0018366) .................................................................................... 185 -iii-

2.

3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. K. 1. 2. 3. 4. 5. 6.

Additional Obviousness References Showing the State of the Art.................... 176

TABLE OF CONTENTS (continued) Page

7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. IX. A.

United States Patent No. 5,060,135 (Levine et al.)................................ 185 United States Patent No. 6,262,732 (Coleman et al.) ............................ 186 "The Role of Time in Information Processing: A Survey," by Bolour et al., ACM SIGART Bulletin (Apr. 1982) ............................... 187 United States Patent No. 5,764,972 (Crouse) ........................................ 187 United States Patent No. 5,479,602 (Baecker & Small) ........................ 187 "Recovery Concepts for Data Sharing Systems," by Ehrard Rahm (1991)..................................................................................................... 188 Email clients and systems (E.g. Elm, Pine, Eudora, Outlook, Lotus Notes, cc:Mail)....................................................................................... 188 Software­Distribution and Change Management Software (E.g. Novadigm's EDM)................................................................................. 188 The World Wide Web............................................................................ 189 "Names should mean What, not Where" by O'Toole & Gifford (1992)..................................................................................................... 189 United States Patent No. 5,649,182 (Reitz) ........................................... 190 "Using Collaborative Filtering To Weave An Information Tapestry" by D. Goldberg et al. (1992) ................................................. 190 United States Patent No. 5,729,730 (Wlaschin '730 patent) ................. 190 Washington Post Article, "The Cyber­Road Not Taken" by David Gelernter (1994)..................................................................................... 191 "LifeLines CHI '96" article ................................................................... 192 Trellis Architecture ................................................................................ 192

CLAIM ELEMENTS ARE WELL KNOWN AND OBVIOUS................................... 194 Organizing Data Units/Documents Into A Main Stream................................... 195 1. 2. 3. B. C. 1. Organizing Locally Generated Data ...................................................... 198 Organizing Data Units/Documents Received From Other Computers .............................................................................................. 199 Documents From Diverse Applications/Document Object Models ...... 200 Generating Persistent/Live Substreams ................................................. 205

Searching/Filtering to Generate Substreams...................................................... 203 Timestamps Identifying Each Data Unit............................................................ 207 -iv-

TABLE OF CONTENTS (continued) Page

1. 2. D.

Chronological Indicator having the Respective Timestamp.................. 209 Include Each Data Unit in the Mainstream According to Timestamp in Chronological Indicator .................................................. 211 Archiving Data Units with Timestamps Older Than A Specified Time ....................................................................................................... 213 Archiving Data Units while Retaining Chronological Indicators.......... 214

Archiving ........................................................................................................... 212 1. 2.

E. F. G.

Using Subsystems From Another Operating System......................................... 215 Enterprise Information Management System .................................................... 217 Graphical User Interface Elements .................................................................... 218 1. 2. 3. 4. 5. Receding, Foreshortened Stack.............................................................. 218 Document Representations and Glance Views ...................................... 221 Sliding Without Clicking To Display The Glance View ....................... 225 Document Representations With Markings Common To A Class Of Documents ........................................................................................ 226 Glance View with Command Buttons ................................................... 227

H. I. X. A. XI.

Motivation to Combine References ................................................................... 228 Lack of Secondary Considerations of Non-Obviousness .................................. 229 Lack of Antecedent Basis for Claims 9, 10, 13, 15, 17, 18, 19, 22, 24, 29, 31, 37 and 39 of the '427 patent ........................................................................ 236 1. Legal Standards: Materiality and Cumulative Art ................................. 237

INVALIDITY UNDER 35 USC §101 AND §112 ........................................................ 235

MATERIALITY ............................................................................................................ 237 B. C. D. Materiality of the CHI '92 "Piles" Article to the '227 Patent............................ 238 Materiality of TR­1070 to the '227 Patent ........................................................ 240 Materiality of MEMOIRS to the '227 patent..................................................... 243

-v-

I.

INTRODUCTION I have been retained by the law firm of Paul Hastings Janofsky & Walker, LLP,

on behalf of defendants Apple Inc., as an expert witness in this litigation between Mirror Worlds, LLC and Mirror Worlds Technology, Inc. (collectively, "Mirror Worlds"), and Apple Inc. ("Apple"). In this report, I provide my opinions regarding the invalidity of the claims of U.S. Patent No. 6,006,227 ("'227 patent"), U.S. Patent No. 6,638,313 ("'313 patent"), U.S. Patent No. 6,725,427 ("'427 patent") and U.S. Patent No. 6,768,999 ("'999 Patent") that have been asserted against Apple by Mirror Worlds (collectively referred to the "Mirror Worlds patents"). Attached to this report as Attachment 1 is a copy of my curriculum vitae, which includes my contact information and describes my prior employment and work history in detail. A list of the consulting and litigation matters in which I have been involved over the last four years is attached to this report as Attachment 2. This list also identifies all of the cases in which I have testified as an expert witness in deposition and/or at trial. A list of the documents, tangible things, reports, models and/or data compilations that I have been provided, reviewed, considered, and/or relied upon in the preparation of this report is attached as Attachment 3. I am being compensated in this matter at a rate of $600/hour. My compensation is in no way affected by the outcome of this litigation. II. QUALIFICATIONS I am a Professor of Computer Science at Columbia University in New York, New York. I earned a Ph.D. in Computer Science from Brown University in 1987. As mentioned above, a copy of my curriculum vitae describing my qualifications, responsibilities, employment

EXPERT REPORT OF DR. STEVEN K. FEINER RE: INVALIDITY OF U.S. PATENT NOS. 6,006,227, 6,638,313, 6,725,427 & 6,678,999

1

Case No. 6:08­CV­88 LED

history, honors, awards, and appointments, and a listing of the papers, articles, books, videos, presentations, tutorials, and other material I have authored, is attached to this report as Attachment 1. I direct the Computer Graphics and User Interfaces Laboratory at Columbia University and, as set forth in my curriculum vitae, I have been involved in research in a variety of technical areas related to the Mirror Worlds patents for over 30 years. I am also the co­author of Computer Graphics: Principles and Practice (Addison­Wesley, 1990) and Introduction to Computer Graphics (Addison­Wesley, 1993), which are used as textbooks in computer science curricula in a number of universities. As part of my testimony, I plan to discuss technology covered in my textbooks, both of which are hereby incorporated into this report by reference in their entirety. III. SUMMARY OF OPINIONS As set forth in more detail in the other sections of this report, it is my opinion that: Claims 13­17, 20 and 22 of the '227 patent are anticipated and/or rendered obvious in view of the prior art, as discussed below; Claims 1­4, and 9­11 of the '313 patent are anticipated and/or rendered obvious in view of the prior art, as discussed below; Claims 1, 2, 5, 7­10, 13, 15­19, 22, 24­26, 29, 31­34, 37 and 39 of the '427 patent are anticipated and/or rendered obvious in view of the prior art, as discussed below; and Claim 1 of the '999 patent is anticipated and/or rendered obvious in view of the prior art, as discussed below. It is also my opinion that claims 9, 10, 13, 15, 17, 18, 19, 22, 24, 29, 31, 37, and 39 of the '427 patent are indefinite for lack of proper antecedent basis.

EXPERT REPORT OF DR. STEVEN K. FEINER RE: INVALIDITY OF U.S. PATENT NOS. 6,006,227, 6,638,313, 6,725,427 & 6,678,999

2

Case No. 6:08­CV­88 LED

It is also my opinion that certain prior art and information discussed in this report would have been material to the prosecution history of the '227 patent and are non­cumulative of the art and information actually disclosed to the United States Patent and Trademark Office ("USPTO"). In addition to explaining the opinions presented in this report at trial, I may also provide a tutorial that discusses the technology relating to the Mirror Worlds patents and the prior art. My tutorial may also discuss the history of the technologies implicated by the Mirror Worlds patents, to provide an understanding of the background technologies and state of the art at the time the Mirror Worlds patents were filed. I reserve the right to supplement this report for any reason, including any modifications or revisions to the Court's Claim Construction Order, arguments made by Mirror Worlds or its experts regarding the validity or materiality of the Mirror Worlds patents, and/or additional information or facts that I become aware of or are brought to my attention. IV. MATERIALS CONSIDERED As previously mentioned, Attachment 3 sets forth the materials I have considered for the opinions expressed in this report. I reserve the right to reference these materials in addition to the materials expressly identified in my report to supplement and/or explain any of the opinions and analyses contained in this report. In addition, I hereby expressly incorporate by reference into this report Apple's Second Amended Invalidity Contentions dated June 18, 2010, Apple's (First) Amended Invalidity Contentions dated May 11, 2009 and Apple's Invalidity Contentions dated November 3, 2008, including the cited references and accompanying attachments and claim charts. In addition, Attachment 5 contains a list of the materials and

EXPERT REPORT OF DR. STEVEN K. FEINER RE: INVALIDITY OF U.S. PATENT NOS. 6,006,227, 6,638,313, 6,725,427 & 6,678,999

3

Case No. 6:08­CV­88 LED

references cited in this report and in the claim charts accompanying or incorporated by reference into this chart. V. TECHNOLOGY BACKGROUND When space is limited, most people are naturally inclined to search for or devise ways to more efficiently store items that they must store in that space. In this pursuit, one quickly recognizes that there can be tradeoffs between storing items efficiently and making them easily accessible. Managers of supermarkets continually deal with this as they look to offer more and more products in a way that will be noticed and accessible by customers. Employees in physicians' offices do this as the number of patient records grows. In fact, everyone, at one time or another, looks for ways to make more efficient use of their space, while still enabling items occupying that spacing to be readily accessible. Almost any type of item could be the subject of organization and retrieval, including documents such as, for example, reports, memoranda, bills, records, pictures, and letters. To illustrate the conundrum often faced when addressing document storage, consider an office desktop. People often keep items on their desktop for easy access. When the number of documents being accessed becomes large, the desktop can become cluttered. In this situation, accessing the documents becomes difficult. This is just one example of how limited physical space affects storage and access. Other examples involve space on walls, floors, or other places in which information can be stored.

EXPERT REPORT OF DR. STEVEN K. FEINER RE: INVALIDITY OF U.S. PATENT NOS. 6,006,227, 6,638,313, 6,725,427 & 6,678,999

4

Case No. 6:08­CV­88 LED

Desk image from web.tradekorea.com. This problem not only applies to physical spaces, such as office desktops, but also applies to computer system monitors. Just as the sizes of physical spaces are limited, a computer monitor's display area is limited as well.

Computer monitor image from www.amazon.com. In fact, the space available on a typical computer monitor at the time at which the Mirror Worlds patents were filed was only a small fraction of that available on a physical desktop, as the following figure from 1986 schematically shows.

EXPERT REPORT OF DR. STEVEN K. FEINER RE: INVALIDITY OF U.S. PATENT NOS. 6,006,227, 6,638,313, 6,725,427 & 6,678,999

5

Case No. 6:08­CV­88 LED

[Henderson, at p. 213. (APMW0074226)] Therefore, just as there can be difficulties in efficiently organizing information in physical space, there can be even greater difficulties in efficiently organizing information on a computer display monitor. In the discussion that follows, I will show that many of the techniques used to efficiently organize documents in physical space have been used to efficiently display information on a computer monitor. As mentioned in U.S. Patent 6,249,724 by Richard Mander et al. ("Mander '724"), computer systems for organizing information in the mid­1990s typically fell into two types of file systems: flat file systems and hierarchical file systems. These file systems were often implemented as part of the operating system of the computer system and were intended to allow the user to organize information in a manner desired by the user. In a flat file system, the computer stores all files (e.g., documents) at the same level, such that the user when examining these files sees all files at this level. Thus, a flat file system is similar to a desk where all documents on the desk are spread out, with none in any

EXPERT REPORT OF DR. STEVEN K. FEINER RE: INVALIDITY OF U.S. PATENT NOS. 6,006,227, 6,638,313, 6,725,427 & 6,678,999

6

Case No. 6:08­CV­88 LED

folders or other containers. Such a file system becomes increasingly cumbersome as the number of documents grows. Hierarchical filing systems on computers attempt to improve on flat file systems by providing a hierarchy of folders or subdirectories within which the user may store documents. In a hierarchical file system, the user can file a document into a folder that may itself be within another folder, or, more generally, within a hierarchically nested set of folders. One way of looking for any documents within a first folder involves opening the first folder after opening the second folder that contains the first folder (and, similarly, opening in sequence any folders within which the second folder was nested). In this manner, the user is not presented with a bewildering number of documents at the same level, as in the flat file system, and the user's files are maintained in an orderly and systematic way within the hierarchy specified by the user. According to Mander '724, versions of Finder on Apple's Macintosh used hierarchical file systems with graphical user interfaces. As Mander '724 points out (see Mander '724 at 1:18­2:58), this hierarchical structure forces a computer user to be as organized as possible in filing information. If the user has difficulty in filing documents because of the difficulty in deciding the proper categories for the documents, then the user may store the documents in the highest directory of the hierarchy. As more and more documents are stored in the highest directory, a bewildering clutter of documents may appear in that highest directory, which then makes the system more like a flat file system. Mander '724 notes several problems caused by the use of hierarchical file systems and their user interfaces, and highlights how these problems can be particularly troublesome for email.

EXPERT REPORT OF DR. STEVEN K. FEINER RE: INVALIDITY OF U.S. PATENT NOS. 6,006,227, 6,638,313, 6,725,427 & 6,678,999

7

Case No. 6:08­CV­88 LED

With these challenges presented to computer developers in the 1990s in mind, the following technical overview is provided. See Attachment 4 for a list of the references cited in the technical background of this report. A. Pre­Computer Organization The industrialization of printing systems and methods in the 18th and 19th centuries resulted in an information deluge. With all of this new information, there became an ever pressing need to devise ways to efficiently store and organize this information so that it was accessible to those interested in it. A common method used to store documents efficiently involves stacking them directly on top of each other or directly side by side. This technique has been used in many environments, including in libraries where books are arranged in "stacks;" and in offices where documents are "stacked" on desks or in filing cabinets. While this method is efficient in terms of making use of storage space, it leaves something to be desired when it comes to enabling a user to easily access specific documents in the stack. As I will show, a number of methods and systems have been devised to improve the accessibility of documents in a stack. 1. Stacks of information a. File cabinets

File cabinets (see http://en.wikipedia.org/wiki/Filing_cabinet), also known as filing cabinets, date to the late 19th century and contain a set of drawers in which documents are stored side­by­side or back­to­back. Before file cabinets existed, businesses organized papers in envelopes, which they stored in arrays of pigeonholes often lining a wall or the interior of a desk. Finding and opening envelopes, and unfolding papers, proved to be inefficient, prompting the development of file cabinets.

EXPERT REPORT OF DR. STEVEN K. FEINER RE: INVALIDITY OF U.S. PATENT NOS. 6,006,227, 6,638,313, 6,725,427 & 6,678,999

8

Case No. 6:08­CV­88 LED

File cabinets can be used to efficiently organize and store a wide range of documents. A typical file cabinet could be used to hold 8.5"×11" paper documents. Also popular were file cabinets or bureaus that hold 3"×5" index cards. While these index card bureaus have been used in many different locations, including homes and offices, they were most often found in libraries. Libraries utilize such bureaus to hold cards, each of which lists information such as the call number, title, author, and publication date, for an item in the library's collection. The cards are ordered and thus could be used by library users to locate and retrieve items from the library's collection. Examples of a typical file cabinet and index card bureau drawer of the 1920s are shown below.

"System: The Magazine of Business" at p. 378 (September 1921). (APMW0073702) b. File Folders

"System: The Magazine of Business" at p. 112 (July 1921). (APMW0073496)

A file folder (see http://en.wikipedia.org/wiki/File_folder) is a folder that holds loose paper documents together for organization. File folders usually consist of a sheet of heavy
EXPERT REPORT OF DR. STEVEN K. FEINER RE: INVALIDITY OF U.S. PATENT NOS. 6,006,227, 6,638,313, 6,725,427 & 6,678,999

9

Case No. 6:08­CV­88 LED

paper stock, plastic, or similar material, which is used to keep related documents together. File folders are often used in conjunction with a file cabinet to further organize paper documents. The following figures, taken from magazine advertisements dating to 1921, lament a problem with common file cabinets and demonstrate that by using file folders to further organize documents in a file cabinet, specific documents are more easily accessible to the user.

"System: The Magazine of Business" at p. 450 (October 1921). (APMW0073758) c.

"System: The Magazine of Business" at p. 70 (July 1921). (APMW0073464)

Limitations of file cabinets and file folders

While file cabinets and file folders are extremely useful for organizing documents in a space efficient manner, it was widely recognized at least eighty years ago that there was
EXPERT REPORT OF DR. STEVEN K. FEINER RE: INVALIDITY OF U.S. PATENT NOS. 6,006,227, 6,638,313, 6,725,427 & 6,678,999

10

Case No. 6:08­CV­88 LED

room for improvement in terms of allowing users easier access to individual documents. The following figures, taken from a 1921 periodical, advertise products that were aimed at enabling users to glance at specific individual documents stacked in the file cabinets or file folders. By enabling this glance view, a user's accessibility to specific documents was improved.

"System: The Magazine of Business," p. 190, Aug. 1921. (APMW0073556) 2.

"System: The Magazine of Business," p. 178, Aug. 1921. (APMW0073550)

"System: The Magazine of Business," p. 459, Oct. 1921. (APMW0073763)

Partial overlapping of documents for greater accessibility

It was readily apparent, even as far back as the 1920s, that by partially overlapping documents in a stack instead of directly or fully overlapping documents in a stack, accessibility to individual documents in the stack could be improved. This idea of partially overlapping documents in a stack was commonly known, as is shown in the following figures.

EXPERT REPORT OF DR. STEVEN K. FEINER RE: INVALIDITY OF U.S. PATENT NOS. 6,006,227, 6,638,313, 6,725,427 & 6,678,999

11

Case No. 6:08­CV­88 LED

"System: The Magazine of Business" at p. 739 (December 1921). (APMW0073981)

"System: The Magazine of Business" at p. 55 (July 1921). (APMW0073455)

The "Universal Displayor" product described in the following figures could be used to organize a wide variety of information in a partially overlapping stack. The advertisement lists maps, charts, posters, pictures, bulletins, drawings, exhibitions, routing charts, advertising copy, directors' meetings, traveling exhibits, health records, merchandise, blue prints, specimens, statistics, samples, and surveys as information that could be organized and displayed using this product. As the advertisement further shows, users could just use their hands to flip between different items of information.

EXPERT REPORT OF DR. STEVEN K. FEINER RE: INVALIDITY OF U.S. PATENT NOS. 6,006,227, 6,638,313, 6,725,427 & 6,678,999

12

Case No. 6:08­CV­88 LED

"System: The Magazine of Business" at p. 472 (October 1921). (APMW0073772)

"System: The Magazine of Business" at p. 576 (November 1921). (APMW0073858)

This concept of organizing documents in partially overlapping stacks has been well­known and widely used for many years. I will now show several examples where this partially overlapping document concept has been used for organizing magazines and periodicals, timecards, maps and sales information, game cards, posters, music records, compact discs, and jukebox song selections. Some of these examples date to the 19th century, although it is likely these concepts may have been considered obvious even earlier. a. Magazines or Periodicals

Displays for magazines and periodicals are commonplace in newspaper stands, convenience stores, and physicians' offices. These displays typically orient the magazines and periodicals such that they partially overlap. Doing this makes it possible for the magazines and
EXPERT REPORT OF DR. STEVEN K. FEINER RE: INVALIDITY OF U.S. PATENT NOS. 6,006,227, 6,638,313, 6,725,427 & 6,678,999

13

Case No. 6:08­CV­88 LED

periodicals to be efficiently stored in a small space, while at the same time providing users with easy access to individual items. An example of this sort of rack is shown in the following picture from an article dating to the 1920s.

"System: The Magazine of Business" at p. 30 (July 1921). (APMW0073432) b. Timecard holders

Another example of how people have determined that it makes sense to display documents in a partially overlapping stack is a common timecard holder that holds the timesheets of many employees. There are several options for storing and arranging these timecards. One option would be to stack all of these timecards in a fully overlapped manner. While this may be efficient in terms of space storage, it would be problematic for accessibility in cases in which each of tens or even hundreds of employees in a line needs to find their timecard so they could clock in to work. Another option would be to lay out all the timecards in a non­overlapped manner. While one would be able to see the entire timecard of every individual, this method would take up a lot
EXPERT REPORT OF DR. STEVEN K. FEINER RE: INVALIDITY OF U.S. PATENT NOS. 6,006,227, 6,638,313, 6,725,427 & 6,678,999

14

Case No. 6:08­CV­88 LED

of unnecessary storage space, since the employee would need to see only a portion of the full timecard to identify it quickly. A third option is to arrange the timecards in a partially overlapping stack. This method has been commonly adopted in practice, likely because it is effective in its use of storage space and because it offers workers easy accessibility to their individual timecard. Holders like this for storing timecards in this partially overlapped manner have been common for some time as the following picture from an article dating to the 1920s demonstrates.

"System: The Magazine of Business" at p. 680 (December 1921). (APMW0073930) c. Sales information holders

Another example where people have chosen to display documents in a partially overlapped manner is in the field of sales. In this field, the ability to display items in a format that allows for easily accessibility for display, while at the same time not consuming too much space, is highly valued. The Multiplex automatic Window Display (shown below) is an example of a product that has these desired attributes. This product was hailed as being a "way of advertising goods by window display." It displayed a significant amount of information in a smaller area than was typically
EXPERT REPORT OF DR. STEVEN K. FEINER RE: INVALIDITY OF U.S. PATENT NOS. 6,006,227, 6,638,313, 6,725,427 & 6,678,999

15

Case No. 6:08­CV­88 LED

required: The advertisement mentions "that on the five wings of the fixture there can be displayed as many samples or as many advertising announcements as could be gotten on a table three feet wide and thirty feet long, yet the fixture occupies only about four square feet of display space." ["Commercial America," p. 35, July 1911. (APMW0072258)]

["Commercial America" at p. 35 (July 1911). (APMW0072258)] Another Multiplex product is shown below in advertisements from a periodical from 1921. These advertisements clearly show the multiplex product as displaying partially overlapped documents that appear to the user as a receding foreshortened stack.

["System: The Magazine of Business," p. 217, August 1921. (APMW0073575)]
EXPERT REPORT OF DR. STEVEN K. FEINER RE: INVALIDITY OF U.S. PATENT NOS. 6,006,227, 6,638,313, 6,725,427 & 6,678,999

["System: The Magazine of Business" at p. 92 (July 1921). (APMW0073482)]

16

Case No. 6:08­CV­88 LED

Yet another example of a system for displaying multiple documents in a stack is shown in the following advertisement from the same magazine.

["System: The Magazine of Business" at p. 118 (July 1921). (APMW0073502)] d. Holding multiple documents by hand

When an individual holds in his or her hand multiple documents and would like to display or view them, it is natural for that person to partially overlap the documents so that a portion of each document is visible. This method of holding and displaying multiple documents has been practiced for over one hundred years. Consider the following common example in which individuals have thought it natural to display multiple documents in this partially overlapped manner. e. Card players holding hands of cards

Players of card games have long held their cards in a convenient manner in which card are overlapped and easily accessible. This common configuration corresponds to a partially overlapped stack. Below are an engraving (ca. 1816, after a medieval illustration), a photograph (ca. 1882), and a painting (ca. 1903) of card players holding their cards in this partially overlapped stack configuration.

EXPERT REPORT OF DR. STEVEN K. FEINER RE: INVALIDITY OF U.S. PATENT NOS. 6,006,227, 6,638,313, 6,725,427 & 6,678,999

17

Case No. 6:08­CV­88 LED

[Poker1882 (APMW0072222)]. [Singer at p. 68 (1816) (APMW0071873)].

[Dogs Playing Poker (His Station and Four Aces) by C. M. Coolidge, 1903] f. Music record album organizers

The picture shown below, which was taken in 1969 at the Music Millennium store in Portland, Oregon, shows a typical way in which music record albums were organized. The albums are arranged in a partially overlapped stack. To access a specific album, a customer could
EXPERT REPORT OF DR. STEVEN K. FEINER RE: INVALIDITY OF U.S. PATENT NOS. 6,006,227, 6,638,313, 6,725,427 & 6,678,999

18

Case No. 6:08­CV­88 LED

easily scan the rack for the desired album and once located, just pull out the album by hand to take a closer view. As shown below, similar approaches have been used for compact disks.

[Calamar at p. 88 (c. 1969) (APMW0074633).] g. Compact disc organizers

["Billboard Magazine" at p. 52 (July 4, 1992] (APMW0075696)] h. Jukeboxes

The jukebox (see http://en.wikipedia.org/wiki/Jukebox), which is a music­playing device that can play specially selected songs from self­contained media, was popularized over
EXPERT REPORT OF DR. STEVEN K. FEINER RE: INVALIDITY OF U.S. PATENT NOS. 6,006,227, 6,638,313, 6,725,427 & 6,678,999

19

Case No. 6:08­CV­88 LED

70 years ago. Early jukeboxes were rather large decorative devices, often constructed of fine wood, with colored lighting on the front of the machine and on its vertical sides. The classic jukebox has buttons with letters and numbers on them that, when combined, are used to indicate a specific song whose name or other identifying characteristic is displayed on the machine. An example of such a jukebox is shown below, in which the record being played is displayed at the center, surrounded by the other records in the stack.

[Adams at p. 86 (1955). (APMW0074350)] Early jukeboxes displayed song information, such as the song name, for every song they stored. When small, wall­mounted, jukeboxes had the capability of playing twenty songs or less, displaying each song name on a single display menu was feasible. However, when the number of songs stored on the jukebox became substantial, displaying each song name or identifying characteristic on a single menu was problematic. As was quickly realized, a way to overcome this problem was to replace the single song selection menu with several smaller menus. Each new menu listed a subset of the songs that could be played by the jukebox. These menus were situated in a partially overlapped
EXPERT REPORT OF DR. STEVEN K. FEINER RE: INVALIDITY OF U.S. PATENT NOS. 6,006,227, 6,638,313, 6,725,427 & 6,678,999

20

Case No. 6:08­CV­88 LED

fashion. A user could flip from menu to menu by using controls provided on the jukebox. Two examples of jukeboxes that utilized the flip­through menu structure to provide the user with the capability to select from hundreds of songs, are shown below.

Wurlitzer 5250 jukebox (ca. 1961) [Adams at page 105 (APMW0074353)] and Wall­Ette WRC jukebox (ca. 1968) [Wallette (APMW0076432)] With the development of compact discs in the 1980s, the media used in modern jukeboxes moved from vinyl records to compact discs. These CD jukeboxes, for the most part, continued to use the flip through overlapping menu structure that earlier record jukeboxes had used. In addition, the available songs identified on these CD jukeboxes were not only identifiable by the song name. Graphical representations corresponding to the song's album cover were also added to the flip through menu to identify and distinguish songs. An example of a CD jukebox, the Rowe Laser Star 100B CD Jukebox (1990), which utilized the flip through overlapping menu structure with individual song album cover images, is shown below.

EXPERT REPORT OF DR. STEVEN K. FEINER RE: INVALIDITY OF U.S. PATENT NOS. 6,006,227, 6,638,313, 6,725,427 & 6,678,999

21

Case No. 6:08­CV­88 LED

Image of Rowe Laser Star 100B CD Jukebox (circa 1990) from www.belamny.com/images/rowecd100b.jpg 3. Document ordering

There are many ways in which physical documents can and have been ordered for access and retrieval. A number of obvious ways to organize documents have been known for well over half a century, including ordering them numerically, alphabetically, and chronologically, as shown below.

EXPERT REPORT OF DR. STEVEN K. FEINER RE: INVALIDITY OF U.S. PATENT NOS. 6,006,227, 6,638,313, 6,725,427 & 6,678,999

22

Case No. 6:08­CV­88 LED

[Robek at p. 160 (1987). (APMW0074261)]

[Robek at p. 163 (1987). (APMW0074264)]

[Robek at p. 179 (1987). (APMW0074265)] These ways of ordering documents have been around for over eighty five years as the following advertisement from 1921 demonstrates. This product being advertised is said to be capable of organizing documents "alphabetically, numerically, or by states, months, or days."

["System: The Magazine of Business" at p. 478 (October 1921). (APMW0073776)]

EXPERT REPORT OF DR. STEVEN K. FEINER RE: INVALIDITY OF U.S. PATENT NOS. 6,006,227, 6,638,313, 6,725,427 & 6,678,999

23

Case No. 6:08­CV­88 LED

The following are additional examples in which documents have been ordered by number, by alphabet, by color, and by time. a. Ordered by number

There are many instances where documents are organized numerically. For example, checks, invoices, vouchers, licenses, purchase orders, warranties, and insurance policies are just a few of many examples where people have chosen to order a set of documents by number. Consider the following two systems by which many libraries order their collections by number. b. Library collections ordered according to the Dewey Decimal Classification System

Most libraries utilize a system, referred to as the Dewey Decimal Classification (DDC) system (see http://en.wikipedia.org/wiki/Dewey_Decimal_Classification), to organize their collections. This system was first designed by Melvil Dewey in 1876. The DDC is comprised of ten main classes. Each class is divided into ten divisions and each division is divided into ten sections. The DDC represents these categories using decimals. Items in a library's collection are organized in increasing numerical order of the decimal number. An item can be any information entity (e.g., book, computer file, graphic, or map), that is considered library material. Libraries order their collection on shelves in increasing numerical order of the decimal number, (e.g., 050, 210, 350, 350.973, 351) corresponding to the DDC. When two items in the collection have the same classification number, the conflicting items are placed according to the second line of the call number (usually the first letter or letters of the author's last name, or the title if there is no identifiable author) in alphabetical order.

EXPERT REPORT OF DR. STEVEN K. FEINER RE: INVALIDITY OF U.S. PATENT NOS. 6,006,227, 6,638,313, 6,725,427 & 6,678,999

24

Case No. 6:08­CV­88 LED

c.

Universal Decimal Classification

Another system that is used by many libraries to organize and order their collections by number is the Universal Decimal Classification (UDC) system (see http://en.wikipedia.org/wiki/Universal_Decimal_Classification). The UDC was developed by the Belgian bibliographers Paul Otlet and Henri La Fontaine at the end of the 19th century. It is based on the Dewey Decimal Classification, but allows for the assignment of multiple classifications to an item, enabling multiple orderings. d. Ordered by alphabet

The use of alphabetical ordering to order items or documents is well­known and has been utilized for many years. Examples of items that are typically ordered alphabetically are dictionaries, encyclopedias, book indexes, telephone books, and address books. Many document file systems are geared to be ordered alphabetically. This has been done for a long time, as substantiated in the following figures from a 1920s periodical.

["System: The Magazine of Business" at p. 82 (July 1921). (APMW0073472)]

["System: The Magazine of Business" at p. 112 (July 1921). (APMW0073496)]

["System: The Magazine of Business" at p. 471 (October 1921). (APMW0073771)]

EXPERT REPORT OF DR. STEVEN K. FEINER RE: INVALIDITY OF U.S. PATENT NOS. 6,006,227, 6,638,313, 6,725,427 & 6,678,999

25

Case No. 6:08­CV­88 LED

e.

Ordered by number and alphabet

There are many examples where documents are organized by combination of numerical ordering and alphabetical ordering. One such example is now described. 1. Library collections ordered according to the Library of Congress Classification

Some libraries, including the United States Library of Congress utilize an alphanumeric classification and ordering system. The Library of Congress Classification (LCC) system (see http://en.wikipedia.org/wiki/Library_of_Congress_Classification and http://www.loc.gov/catdir/cpso/lcc.html) was first developed in the late nineteenth and early twentieth centuries to organize and arrange the collections at the Library of Congress. The system is comprised of twenty­one main classes, each identified by a single letter of the alphabet. Most of these alphabetical classes are further divided into more specific subclasses, identified by two­letter, or occasionally three­letter, combinations. Examples of these representations include class N (Art) with subclasses NA (Architecture); NB (Sculpture), ND (Painting); as well as several additional subclasses. Each subclass includes topics relevant to the subclass. Each topic is assigned a single number or a span of numbers. Whole numbers used in LCC may range from one to four digits in length, and may be further extended by the use of decimal numbers. Some subtopics appear in alphabetical, rather than hierarchical, lists and are represented by decimal numbers that combine a letter of the alphabet with a numeral, e.g. .B72 or .K535. Relationships among topics in LCC are shown not by the numbers that are assigned to them, but by indenting subtopics under the larger topics that they are a part of, much like an outline [http://www.loc.gov/catdir/cpso/lcc.html]. The items in a library organized by the LCC are ordered by taking into account both alphabetical and numerical elements of the call number, as the following figure shows.
EXPERT REPORT OF DR. STEVEN K. FEINER RE: INVALIDITY OF U.S. PATENT NOS. 6,006,227, 6,638,313, 6,725,427 & 6,678,999

26

Case No. 6:08­CV­88 LED

[Library 2 (APMW0074601)] f. Ordered by color

There are even examples where documents are ordered by color, as is shown in the following advertisement circa 1921.

["System: The Magazine of Business" at p. 576 (November 1921). (APMW0073858)] g. Ordered by time

Calendars are the simplest example in which a number of items, including, for example, appointments, and meetings, are ordered by time. Two examples of such calendars are shown below.

EXPERT REPORT OF DR. STEVEN K. FEINER RE: INVALIDITY OF U.S. PATENT NOS. 6,006,227, 6,638,313, 6,725,427 & 6,678,999

27

Case No. 6:08­CV­88 LED

["System: The Magazine of Business" at p. 114 (July 1921). (APMW0073498)] ["System: The Magazine of Business" at p. 482 (October 1921). (APMW0073780)] There are many other examples where documents are ordered by time. These examples include a person's medical record folder, a museum's collection, a credit card statement, a bank statement, and the list goes on and on. B. Speculative Methods for Organizing and Accessing Documents In July 1945, Vannevar Bush published an article entitled "As We May Think" in The Atlantic Monthly.1 In this article, Bush communicates his concern that the immense volume of information becoming available to scientists would overwhelm traditional methods of acquisition, storage, organization, and analysis. He suggests that scientists need to create new methods for dealing with this information. See, for example, the Vannevar Bush article at APMW0055075.
1

A version of this article also appeared in the September 10, 1945 issue of Life Magazine

(APMW0080776­ APMW0080785) which is hereby incorporated by reference.
EXPERT REPORT OF DR. STEVEN K. FEINER RE: INVALIDITY OF U.S. PATENT NOS. 6,006,227, 6,638,313, 6,725,427 & 6,678,999

28

Case No. 6:08­CV­88 LED

Bush argues that "[a] record if it is to be useful to science, must be continually extended, it must be stored, and above all it must be consulted." See, for example, the Vannevar Bush article at APMW0055076. Bush then goes on to describe a system that was suited to the task of extending, storing, and enabling a user to access the record (i.e., information). In this system, records are stored as a miniaturized representation so that these records occupy little space. Bush looked for advances in microfilm to allow pictures to be stored in a small amount of space. At the time the Vannevar Bush article was published, microfilm already existed with a factor of 20 linear reduction. Bush noted that, with 100 times linear reduction, the entire "Encyclopaedia Britannica could be reduced to the volume of a matchbox." With such an efficient storage medium, improved distribution of information could be realized. See, for example, the Vannevar Bush article at APMW0055077. Bush recognized the problems with traditional methods of providing access to information: "Our ineptitude in getting at the record is largely caused by the artificiality of systems of indexing." To enable the information to be utilized by an end­user, Bush describes a proposed system that incorporates "slanting translucent screens, on which material can be projected for convenient reading." See, for example, the Vannevar Bush article at APMW0055082. This system could display the equivalent of quick glance views of desired records or pages. See, for example, the Vannevar Bush article at APMW0055083. Bush further describes how the system enables the user to easily access a specific record: "If the user wishes to consult a certain book, he taps its code on the keyboard, and the title page of the book promptly appears before him, projected onto one of his viewing positions." See, for example, the Vannevar Bush article at APMW0055082.

EXPERT REPORT OF DR. STEVEN K. FEINER RE: INVALIDITY OF U.S. PATENT NOS. 6,006,227, 6,638,313, 6,725,427 & 6,678,999

29

Case No. 6:08­CV­88 LED

Bush's proposed system enabled the organization and management of many different types of information: "A memex is a device in which an individual stores all his books, records, and communications, and which is mechanized so that it may be consulted with exceeding speed and flexibility. It is an enlarged intimate supplement to his memory." See, for example, the Vannevar Bush article at APMW0055082 and the speculative image below from the Life Magazine version (APMW0080785) showing the use of the memex:

EXPERT REPORT OF DR. STEVEN K. FEINER RE: INVALIDITY OF U.S. PATENT NOS. 6,006,227, 6,638,313, 6,725,427 & 6,678,999

30

Case No. 6:08­CV­88 LED

Bush also described how records in a system could be organized according to time: "Time is automatically recorded to tie the two records together." See, for example, the Vannevar Bush article at APMW0055078. Bush described a system that could store and provide access to an immense amount of information from one's life experience: "The lawyer has at his touch the associated opinions and decisions of his whole experience, and of the experience of friends and authorities. The patent attorney has on call the millions of issued patents, with familiar trails to every point of his client's interest. The physician, puzzled by a patient's reactions, strikes the trail established in studying an earlier similar case, and runs rapidly through analogous case histories, with side references to the classics for the pertinent anatomy and histology. The chemist, struggling with the synthesis of an organic compound, has all the chemical literature before him in his laboratory, with trails following the analogies of compounds, and side trails to their physical and chemical behavior. The historian, with a vast chronological account of a people, parallels it with a skip trail which stops only on the salient items, and can follow at any time contemporary trails which lead him all over civilization at a particular epoch. There is a new profession of trail blazers, those who find delight in the task of establishing useful trails through the enormous mass of the common record. The inheritance from the master becomes, not only his additions to the world's record, but for his disciples the entire scaffolding by which they were erected." Vannevar Bush article at APMW0055084. C. Computer­Based Methods for Organizing and Accessing Documents 1. File cabinets and folders

Just as filing cabinets and folders are used in peoples' homes and offices to organize and store paper documents, electronic folders are used on computer systems to organize and store electronic documents. An electronic folder operates just like a physical folder, in that it can be used to hold multiple documents. In the case of an electronic folder, the documents are

EXPERT REPORT OF DR. STEVEN K. FEINER RE: INVALIDITY OF U.S. PATENT NOS. 6,006,227, 6,638,313, 6,725,427 & 6,678,999

31

Case No. 6:08­CV­88 LED

also electronic. Electronic documents, like paper documents, can correspond to any number of file types, such as digital pictures, spreadsheets, letters, and video clips. The metaphor of a filing cabinet or folder works well here. If your file cabinet or folder is stuffed with paper documents and they are not in any given order, you will have a hard time finding a specific desired document. If all the documents stored on your computer (i.e., in your computer filing cabinet) or in individual computer file folders become poorly organized and disorderly, it will likewise be difficult to easily locate specific documents. a. Folder and document representations on text­based user interfaces

Early personal computer operating systems, such as MS­DOS, have textual user interfaces. To interact with a computer having a textual user interface, a user types commands into the command­line interface. For MS­DOS, users type commands at the so­called MS­DOS prompt "C:\Temp>", where "C:" identifies the default disk drive and "\Temp" identifies the default path. The top level directory, identified by a single "\", as in "C:\", is called the root directory. b. Document represented by a characteristic such as its name

All documents or files in an MS­DOS operated computer system are specified by a four-part naming convention. The four required parts are (1) drive, (2) path or folder, (3) filename, and (4) filename extension. As an example, a file specified by "C:\cars\.ford.txt" is specified as being located on the "C:" drive in the "cars" folder, has the name "ford," and is a text document. A user can see the names of all the files and folders in a given directory by typing the command "dir" at the command prompt. The dir command typed by itself, displays the disk's volume label and serial number; one directory or filename per line, including the filename
EXPERT REPORT OF DR. STEVEN K. FEINER RE: INVALIDITY OF U.S. PATENT NOS. 6,006,227, 6,638,313, 6,725,427 & 6,678,999

32

Case No. 6:08­CV­88 LED

extension, the file size in bytes, and the date and time the file was last modified; and the total number of files listed, their cumulative size, and the free space (in bytes) remaining on the disk. To display the file or folder attributes of the files or folders contained in a specific folder, the user can type the command "dir [drive:][path][filename] [parameters]". For instance, to list all the files and folders in the "cars" folder on the C: drive, as mentioned above, the user would type "dir C:\cars\". By adding the proper parameter to the command line statement, a user can have the files and folders sorted or ordered according to alphabet (i.e., name), size of the file or folder, type of file (i.e., extension), date and time, and last access date. c. Folders and document representations on direct manipulation user interfaces

For over fifty years, scientists and engineers have researched systems and methods aimed at what was later termed "direct manipulation user interfaces" [Myers at p. 46]. This research and development is embodied in graphical user interfaces (GUIs), comprised of windows, icons, and widgets (e.g., buttons, menus, and scroll bars), which can be directly manipulated by techniques that mimic in part human interaction with physical objects. These computer systems require a way for users to specify the operations that they would like the computer to perform. Direct manipulation interfaces allow a user to directly manipulate visible objects on a display, typically with a pointing device. In the 1950s, 1960s, and 1970s, many different pointing devices were developed to specify desired operations. These pointing devices included the light pen, the trackball, the joystick, cursor keys, the digitizing tablet, the touch screen, and the computer mouse. The concept of a direct manipulation interface was first demonstrated in 1963 by MIT graduate student Ivan Sutherland in his Ph.D. dissertation, entitled "Sketchpad: A Man­ Machine Graphical Communication System." Sketchpad interprets information specified
EXPERT REPORT OF DR. STEVEN K. FEINER RE: INVALIDITY OF U.S. PATENT NOS. 6,006,227, 6,638,313, 6,725,427 & 6,678,999

33

Case No. 6:08­CV­88 LED

directly on a computer display with a light pen. A Sketchpad user could use the light pen to draw new objects, and select, move, and modify existing ones. Sutherland described his system as one that "opens up a new area of man­machine communication", since systems prior to Sketchpad required most communication to be typed and Sketchpad "eliminat[ed] typed statement (except for legends) in favor of line drawings" [Cambridge at p. 17].

[Cambridge at p. 20. (APMW0074438)]

[Cambridge at p. 54. (APMW0074472]

In 1967, William Newman demonstrated the "Reaction Handler," which supported direct manipulation of graphics, and featured "Light Handles," a kind of graphical potentiometer [Myers at p.47]. In the 1960s, Doug Engelbart and his colleagues at the Stanford Research Institute developed a computer system called the oNLine System (NLS), whose user interface is shown below. This system utilized a new pointing device, called a mouse, to manipulate graphics on a computer monitor. The mouse was intended to be a lower cost replacement for the light pen that was used in Sketchpad and earlier systems. In 1968, Engelbart and his colleagues gave an historic demonstration of NLS at the Fall Joint Computer Conference.
EXPERT REPORT OF DR. STEVEN K. FEINER RE: INVALIDITY OF U.S. PATENT NOS. 6,006,227, 6,638,313, 6,725,427 & 6,678,999

34

Case No. 6:08­CV­88 LED

[Reimer at p. 3. (APMW0074575)]

[Reimer at p. 5. (APMW0074577)]

Also in 1968, the AMBIT/G system for the manipulation of directed graphs was demonstrated at MIT Lincoln Labs. This system included interface elements such as iconic representations, gesture recognition, dynamic menus with items selected using a pointing device, and selection of icons by pointing [Meyers at p. 47. (APMW0074367)]. Researchers at the Xerox Palo Alto Research Center (PARC) extensively explored GUIs in the 1970s. They developed what was in effect the first personal computer, the Alto, offering a GUI in 1973. This system was not a commercial product, but several thousand units were built and were heavily used at Xerox and at several universities for a number of years. The Alto (shown below) used a mouse to manipulate graphical elements such as windows, menus, radio buttons, check boxes, and icons on a bitmapped screen.

EXPERT REPORT OF DR. STEVEN K. FEINER RE: INVALIDITY OF U.S. PATENT NOS. 6,006,227, 6,638,313, 6,725,427 & 6,678,999

35

Case No. 6:08­CV­88 LED

[Reimer at p. 6. (APMW0074578)]

[Reimer at p. 7. (APMW0074579)]

In addition, Xerox PARC researcher Alan Kay published an article in 1977 on the "Dynabook," a speculative hand­held networked computer, in which the idea of direct manipulation interfaces for everyone was outlined. Many of the user interfaces developed at Xerox PARC exemplify what has been referred to as "WYSIWYG" (what you see is what you get) in that they allow users to view and edit information on the computer in a form that is intended to look as much as possible like the end result. Early examples of WYSIWYG interfaces were the Xerox PARC "Bravo" text editor and "Draw" drawing program [Meyers at p. 47. (APMW0074367)]. A modern day example of a WYSIWYG interface is Microsoft Word, which allows a user to view and edit a document in a form that is very similar to the end product. A 1976 Xerox PARC internal memo showing a schematic of the plan for a GUI using a "desktop metaphor" [Bardini at pp.158­159 (APMW0074395­6)] is displayed below. Notice the iconic representations of computer items and the receding stack of documents.

EXPERT REPORT OF DR. STEVEN K. FEINER RE: INVALIDITY OF U.S. PATENT NOS. 6,006,227, 6,638,313, 6,725,427 & 6,678,999

36

Case No. 6:08­CV­88 LED

[Bardini at p. 164. (APMW0074401)] In the 1970s, Alan Kay worked with a team to develop a programming language and integrated development environment called Smalltalk, which would turn out to be the first commercial product to employ overlapping windows. A number of iterations of this Smalltalk environment were developed: Smalltalk­72, Smalltalk­74, Smalltalk­76 and Smalltalk­80. In the early 1980s, several commercial systems that employed GUIs were produced. These include the Xerox Star (1981), the Apple Lisa (1983), and the Apple Macintosh (1984), and the Atari ST and Commodore Amiga in 1985. The GUIs on these commercial systems provided visual iconic representations of folders, documents, and other computer items. d. Iconic representations of documents and folders

In 1975, David Canfield Smith, in his Ph.D. thesis at Stanford, was the first to use the term "icon" to refer to a GUI element. Icons typically refer to "any nontextual symbol on the display" [Johnson at p. 16 (APMW0076284)]. Smith worked at Xerox PARC, and contributed to the development of icon­based file management system GUIs there. It was widely recognized,
EXPERT REPORT OF DR. STEVEN K. FEINER RE: INVALIDITY OF U.S. PATENT NOS. 6,006,227, 6,638,313, 6,725,427 & 6,678,999

37

Case No. 6:08­CV­88 LED

during Smith's time at Xerox PARC, that the textual user interfaces used in operating systems were limited in terms of organizing and maintaining files or documents. In the article "The Xerox Star: A Retrospective," published in IEEE Computer in September 1989, the authors lamented the difficulty with these textual user interfaces: "[A colleague] complained that she couldn't keep track of the files on her disk. An inspection of her system revealed file