Polaris IP, LLC v. Google Inc. et al

Filing 507

RESPONSE in Opposition re 460 MOTION in Limine Number 1 (Uncharted Prior Art References): Motion to Preclude Defendants' Reliance on or Reference to Uncharted Prior Art References (EZ Reader Manual and CBR Express Manuals) and Hearsay MOTION in Limine Number 1 (Uncharted Prior Art References): Motion to Preclude Defendants' Reliance on or Reference to Uncharted Prior Art References (EZ Reader Manual and CBR Express Manuals) and Hearsay MOTION in Limine Number 1 (Uncharted Prior Art References): Motion to Preclude Defendants' Reliance on or Reference to Uncharted Prior Art References (EZ Reader Manual and CBR Express Manuals) and Hearsay filed by AOL, LLC., Google Inc., Yahoo!, Inc.. (Attachments: # 1 Sistos Decl. ISO Defendants' Response, # 2 Branting Decl ISO Defendants' Response, # 3 Exhibit, # 4 Exhibit, # 5 Exhibit, # 6 Exhibit, # 7 Exhibit, # 8 Exhibit, # 9 Exhibit, # 10 Exhibit, # 11 Exhibit, # 12 Exhibit, # 13 Exhibit, # 14 Exhibit, # 15 Exhibit, # 16 Exhibit, # 17 Exhibit, # 18 Exhibit, # 19 Exhibit, # 20 Exhibit, # 21 Exhibit, # 22 Exhibit, # 23 Exhibit, # 24 Exhibit, # 25 Exhibit, # 26 Exhibit, # 27 Exhibit, # 28 Exhibit)(Sistos, Antonio)

Polaris IP, LLC v. Google Inc. et al

Doc. 507 Att. 23

Dockets.Justia.com

IN THE UNITED STATES DISTRICT COURT FOR THE EASTERN DISTRICT OF TEXAS MARSHALL DIVISION

BRIGHT RESPONSE, LLC F/K/A POLARIS IP, LLC NO.2:07-CV-371-TJW-CE V. GOOGLE INC., et al. JURY TRIAL DEMANDED

DECLARATION OF PHIL KLAHR

I, Phil Klahr, declare: 1. The facts set forth in this declaration are true and correct and of my personal

knowledge. If called as a witness, I could and would testify competently thereto.
2. 1 served on the program committee of the Conference on Innovative Applications

of Artificial Intelligence ("IAAI Conference") between the years of 1991 and 2000. As a member of the program committee, I reviewed case study papers submitted for publication as part of the conference. 3. The IAAI program committee would not consider a case study paper for

publication in connection with the 1996 Eighth IAAI Conference (and in prior IAAI conferences) unless it described an application of artificial intelligence that had actually been deployed in the field.

01980 .51452/3423329.1

1

4.

Attached as Exhibit A is the Call for Papers for the Eighth IAAI Conference,

which took place August 4-8, 1996 in Portland, Oregon. The Call for Papers correctly states that IAAI case study papers "must describe deployed applications with measurable benefits." The document also correctly provides that case study papers must "[d]escribe the task the application performs or the problem it solves," and must identify "[w]hat measurable benefits have resulted from its use." 5. Attached as Exhibit B to this declaration is "EZ Reader: Embedded Al for

Automatic Electronic Mail Interpretation and Routing" (the "EZ Reader Case Study"), accepted for publication as part of the 1996 IAAI Conference. 6. In accordance with the 1996 IAAI Conference's requirements as set forth in the

Call for Papers, the Case Study states that the described EZ Reader application had been deployed in the field. The Abstract states that "Phase I of EZ Reader was deployed in the first quarter of 1996 and handles up to 80% of incoming mail automatically, depending on message content.". (Ex. A.) The body of the Case Study states that during the early months of initial testing, the percentage of messages that were processed by the EZ Reader system at Chase Manhattan Bank without manual intervention was as high as 80%. (Ex. A at 1516.) 7. In my experience, the program committee of the IAAI Conference would not have

published the EZ Reader Case Study, or any other case study published in 1996 or before, if it had knowledge that the described application had not been deployed in the field. The very purpose of the 1996 IAAI conference (and prior conferences) was to highlight deployed applications of artificial intelligence.

01980.51452/3423329.1

2

12/12/2009 00:24

14153808049

PHILIP KLAHR

PAGE

02

8.

In nay experience, the program committee of the IAAI Conference would not have

knowingly published any paper containing inaccurate information. I declare under penalty of perjury of the laws of the United States that the foregoing is true and correct. y Executed this /-_ day of A- ri 2010 in VkI ,

Phil Klahr

ot9w5145213423329.1

3

Ex hibit A

the DOA ARRUdl 1000MO Appliotiops of Adiddl lotdi?fou (00krou
(al for Papers, Panels, and Ioted Talks
PORTn, O NO , AUQU5T 5 -1,1996

Sromom or iRr GntBw boanoo roR Win lRTruima
tant issues Involved in delivering effective and successful AT systems. Each panel should address a practical problem or theme of broad interest to developers of Al applications. Panelists should focus on specific issues, such as problems and solutions, technical trade-offs, managerial Issues, organizational concerns, and challenges for the future. Case studies can be used to illustrate the issues, but should not be the principal focus of the panel presentations. Panel proposals should include a description of the sessions, the name and full address and phone of the principal contact of the organizer, the name of the proposed panel moderator, and a list of proposed panelists. Panel proposals will be judged based on the importance and relevance of the proposed topic, and on the range and experience of the proposed panelists.

ntelligent systems are becoming a reality for many enterprises in the 1990s. Rather than being independent technologies, these systems are generally used as one component in a complex system. The Eighth Annual Conference on Innovative Applications of Artificial Intelligence will explore how such systems are being used as enabling technologies. Through case studies, meetthe-author sessions, Invited talks, and interactive panel discussions, the conference will showcase and promote Af technology at work in today's global enterprises. It will highlight the most Innovative deployed Al applications worldwide and explore critical issues involved in AI development and deployment. Successful authors will receive the Annual Innovative Applications Award and will be highlighted in Al Magazine. - Howard E. Shrobe, Program Chair, &'[ed Senator, Cochair

191 USE Study Capers
Case-study presentation papers, highlighting any area of AJ technology, must describe deployed applications with measurable benefits. Papers should be 5-10 pages in length, plus figures. The proceedings on the conference will be published by AAAJ Press; authors will be required to transfer copyright, We request that your paper include the following sections: Title Page: Each copy of the paper must have a title page, including the title of the paper; the names, affiliations, postal and email addresses, and telephone and fax numbers of all authors; the name of the tool or language used; a designation of the application domain; and a short abstract of less than 200 words. Task or Problem Description: Describe the task the application performs or the problem it solves. State the objectives of the application and explain why an Al solution was important. If other solutions were tried and failed, outline these solutions and the reasons for their failure. Application Description: Describe the application, providing technical details about design and implementation, What key aspects of Al technology enabled the application to succeed? If applicable, describe how Al technology is integrated with other technology. If a commercial tool is used, explain the decision criteria used to select it. Describe any insights gained about the, application of Al technology. Application Use and Payoff. How long has this application been deployed? Explain how widely, how often, and by whom the application is being used. Also describe the application's payoff. What measurable benefits have resulted from its use? What additional benefits do you expect over time? Application Development and Deployment: Describe the development and deployment processes. I-low long did they take? How many developers were involved? What were the costs? Maintenance: Describe your experience with and plans for maintenance of the application. Who maintains the knowledge base? How often is update needed? Is knowledge in this domain expected to changeover time? How sloes the design of the application facilitate update? Have you automated knowledge-base update in anyway?

Panels in previous years have included topics such as: · Al and business process reengineering
· What's next in knowledge-based technology? · How to get intelligence/Al into the system

· Gaining support for Al technologies within an organization

IMI luuited Talks
Each year IAAI invites distinguished speakers from government, business, and research to share their experiences, as well as their thoughts for the future direction of Al development and applications worldwide. Suggestions for speakers should include the full name, affiliation, postal and email addresses, phone and fax numbers of the speaker; a preliminary title for the talk; and a short abstract of the proposed talk of no more 200 words. One of the Invited speakers will be honored with the Dennis O'Connor Memorial Lecture Award.

i mefiable
Submit five (5) clearly legible complete copies of papers, panel proposals, and invited talk proposals (hard copies only--we cannot accept computer files) to the AAAI office by January 17, 1996. Papers or proposals received after this date will be returned unopened. Notification of receipt will be mailed to the first author (or designated author) soon after receipt. All inquiries regarding lost papers must be made by January 24, 1996, Papers, panel proposals, and Invited talk proposals will be reviewed by the Program Committee in March and notification of acceptance or rejection will be mailed to the first author (or designated author) by April S. Camera-ready copy of accepted papers will be due about one month later. Mftn Please send papers, panel proposals, and conference registration inquiries to IAAI-96, American Association for Artificial Intelligence 445 Burgess Drive Menlo Park, CA 94025-3496 Phone: [phone] Pax: 415-321-4457 Email: [e-mail] No paper submissions will be accepted via fax or email . However, registration or clarification inquiries may be sent by fax or email.

16Ai raoen
The IAAI panels will focus on critical issues involved in Al development and deployment. This series of panels will explore impor-

Exhibit B

Jl
Proceedings of the Thirteenth National Conference on Ar cial 1nteDigence and the Eighth lnnoyatirye Applications of Ai tificial Intelligence Conference

Volume Two

Spa mored &y the American Assoda onfor Articial Intelligence

AAAT Press I The MIT Press
MZAI>PZA · Cambridge - Imaun

BR 001250 .

Copyoghl v m6 American Association for Artificial Intelligence 465 Burgess Ihivc. Menlo Park. California 94025 USA All Rigbts Reserved. No pan of this book may be reproduced in any fomt byany electronic or mechanical means ( including. photocopying. recording, or infnrmarion storage and r 0c"t)without permiWan in writing from ihe publisher. The trademarked terms that appear throughout this book are used in an editorial nature only, to the benefit of the trademark owner. No infringement of any trademarked term is intended. C.apublished and distributed by The MF7'f1ress, Massachusetts Institute afTc6nology Gmhridge. Massachusetts and London, Pstgland ISAN D-262-5M91-X

Manufactured in the Unircd States ofAmerica on acid-free paper.

i

OR 001251

P

EZ :leader: E:Tlbat:derd Al for Automatic Mectrowe Mail Interpretation and Routing
Amy Rica and Julie Nset 184ghMaw, Inc. Anthony Angolli and Rosanna Piccolo Chase Manhattan Bank, N.A. AbA i art EZ Rcado is an inid ligcnt electronic mail (email) made: Amt employs a unique combination of Julebased parsing and case- baseJ 'easoiing to automatically and with a high level of accuracy classify and respond to large volumes of incoming email. CZ Rcadct tcduces the time and human icsotsrecs requhcd to handle incoming email by selecting icsponscs and 7idd ng anaehmenss and advice io e=b incoming message based on bow prevlous similol messages were handlcil. The application , developdd tot Chase Marthanaa 13an1: using BlIgitl + varc , Int.'s ARTwEnlerp. isetW toot, answers rmails aaomatically and decreases processing time fur ihasc icquhing manual review. Phase i of EZ Rcado was dcploved in itte tnsi quarter of 1996, and Imndlcs up to g0°! of incoming mail au ' omalleatly, depending an mm mgc conical. Ulm phases will enable amnmmic p'ncessing td a wlder variety of messages. By dramatically icducing the cffon associated with inanuot processing. @Z Reader will pay its own development costs ecith'tn six months and will resuh in svbstantial.iccurring dothu savings each yea'. This pope' dcacribcs F2 Rcadw in detail . including its Al-based dcsitln, testing. hnplcmcniation and dcvclopmcni history. Dank N. A.. Regional Shark, began to provtdc electronic hankittg services using phone and personal computer technology in 1995. Mmkciing campaigns advertised that email cnuld be used to icqucsi information and servicm opening a new electronic channel of communication with customer sand piospcots, The success of ins ar&WITg campaigns ciealed o cha)lengc fm ChaseDircet from the beginning 10 quickly ,and con · etfcciivcly ^xoccss email from multiple sources. including the Imcmel. Microsoft Money email. and anodho tnlernal DOS-based money manago program with vniail capability. In addition to ChasuDirccl 's comminarnl in pinvide excellent, ttutcly sdrvicc to its' customers. Occuonic commove laws icquircd the bank w respond to certain types of ciccuonic correspondence within speei0c bate frames. Although more than 80% or incoming messages were simple icqucsts for product ini'MMatlln, the stairoflan got backlogged and uoikcd alto tows and on wcckcads to keep up with the rcqubcd analysis aid icsponses. Faced with the huge p ' ojeetcd i;ooreasc in Internet email volume due jr, the planned inrrodumion ofa new Maid Wide Web scrvci. as illustrated in Fieuro I Wow, ChaseDiiva aggcssivcry sought con tcft'cctivc. bigh-qualiw %%-Jys to process em ails. This urgcni business peobkin attracted aacntion from Chase's Regional Bank Knowledge Base (KB) technology scum. Tbc Ieam'S geneial chwgc was to apply anif'ciai imclligcncc (AI) technology in key areas of the Rcgfonat Bank whclc appropriate to opUmize opeiiW onal decisions. To address ChascNrtci ' s business problem, the Knowkdgc Base team crewed EZ Reader, an, embedded A] application operating as an invisible layer between the The Lotus NotmO email system and ClmwD b cd. applitalion cominuoasly rciriovcs Incoming Interne email from Chase piospras and Customers thiough an iutcrj[ocC to Lotus Notes. aadalso adsas a ftitaring aadromisg

Problem Description
lake oiba businesses that sought in cxp,md access to ihcii products and services Iloough the Internet and othci online channels'. Chaseiitcct, a unit of Chase Manhattan

Contact irilbrolation Collows. Amy Rice: 301 Tresscr Blvd 131h 1100;, ' Stamford . CT 06901. iicc{ Julie Min. brighhmu .com. 2010 Corpgralc Ridge, Stiflc 700. McLean, VA 22102 hsuQb+FgnRvaracKyn, A MbprryAngvt1* One Chase.Maahattan Plaza.' 191h floor, Nrw Yank . WY I00E1 . anlhauy , angottiQchasacom. Rosumw Pkcolo: 15 E- 26th St. New Yolk NY 10010, ' osanua.plccolo@ch asacom.

t l3anka facer sig'iricant ducats to the rciail banking franchise horn advmocs1n online banking (Tort u, Whig & WUMLr 1996)

CrscSiadics

1507

13R 0.07252

Volume

Jum·95

oet-95

C).$-5

Flxum I. Prel"led Ceowmh ar Irimminx t:nv2h by Source

mcchanirm, cidtcr replying to the email automatically or attaching a suggested response arrd tcl'crsing the message for manual review The Ka team's overall business goal for EZ. Reader was to reduce the lumber or emails that needed manual processing by more than Ma, with an accuracy rate or 9s%n- Other goals were to provide rapid turnaround pme for return messages to customers and prospects and coniisiency in responses. RZ Rcadrr'a Putomwcd reasosning capabilities enabled Choscnircci ttl reach these goals and significantly reduce the manual crfon nwdeJ for email processing,

F-7, Reader Design
The Chv-%tMrccl email processing problem offered several clear opponunitics for the im*mcnimion or automated ecosoning. A] ire Z'G ]header The email review process involved complex reasoning 77se process was distinctly knowlcdgc·inlensivc; ......_.-sprcialisls applied domain-spcciffe knowledge using hcurisifcs based ou knowledge of (:hasc products and services. The reliance on knowledge as a process enmponanl was evident as Junior team members routinely sexed upon the cxporienec of their supervisor and other senior team mcmbels to explain Irmv to classify ambiguom messages and formulate =pottses for new 'classes or messages. Finally, cofnmon reasoning tasks were rclltcted is Ac process mtput; claskijxwioa of *c email and c4ogairarion ofa response from a limimdset of pscpartdicxtaotxlules. F,Z Rieadc's A] masoning compmcnt is a awa - driven forward-ebaiding tole psrscr operating in concert v id; ease - based reasoning written in Brightworc, Inc.'s

A V Esrterprisr, a commercially sucecssrul knowledge. based application dccclopmcnl tool. As described below, On: YC,iowlcdgc Base team considered and rcjce:cd several 311CJT12tiec so(MMC appfiealions tools and rot implementing iltc reasoning component or n Rcader. In-house procedural appNeadon lan&aiages such as 'COBOL could not handle the reasoning component of the email process bemuse ahc input to the process was so complex and unpredictable. An approach or coding branches for each potential Input us'ing' procedural methods alorro would have been impraclically camplcx. The Rete algorithm used by AltT'Fiircrprbvs palter matcher addressed this input complexity issue for FZ Reader. ART"Fntoprisr's integrated rule stripling language, case based wsd objtct modeling capabilities cnebtcd automWCd reasoning to tcvcrat or change conclusions drawn throughout sequential parsing and to dlsambiguatc text while ignoring irrelevant portions, such as signature limes. wilhow coding rxp)icii subrotAnscs. Although input was in the rotor of narmt language. the output was -a simple classifrcalioo baud on the application or heuristic rules and cxpericncc. For this reason, syntactic analysis and interpretation , discourse analysis and pragmatic inrercncing in commercial natural language processing (NLP) products Guth as SPA or Logicon, or augmented transition networks in custom developed applications such as the Intelligent Bantling System (Sahink Sawyer 1989) were viewed as providing only an , prc-processor function to the classification =ftsoning task, and nab providing a value-added output per x. This is illustmled by the fact that the ChaSeDircp laanam email ravic-wcM acre able to classify o forge majority of messages by using a rcfailvely limited scr or key linguistic clues Ihat could be expressed as simple roles. Furlhcnnorq, when human reNicw.Cls ilad trouble idcnlifyiny cononon linguistic clues in a pari(cular message, they relied upnrt team atpericnoc to help them classify the mcmgc and conftgurc or synthesize a re ),Kin;c. Compared with the lire to create rules and case structures in A971$nrrrpruc, the time frame to My analyse the semantics of the text interpretation domain and develop a comprehensive NLP module was deemed too long to meet business needs, and would have 'provided , at bast, sccom.Wy runaliatality at greatly iacrcascd rAM and technical complexiy. Fundamcnial Iechniral rcqufremots rot in automated solution included compatibility wilt existing hardware and standards for run -time aad netvrork ptiformarl= stability; salability, vendor SUppolt ' and comtnerPial viability. Widcly tlvailablc shaacwme. limpcretary intelligent progtxms speoilicaAy written for eltestonic Forts processm&(Compton & Wolrc 1993)aad pecT:agtsd email roatcslresysnndcr soilwam surb W Prolog-based

F.

JSOa

IMI-9s

-i
BR 001253

14;;i1boF4 Miixiron Laboraro % es. ine.) were Cicrcd Col the.e masons, in contrast. dw 4%RT·Fnicrorrse ;not hurl ahead.- barn 6epio; cd in anod= es application ar C,yaic Where broad functiornlily ono cusromrzabRhy , proJuct rliabilhy and vendor Support ncuessary eommerria ) appUcariomwas previously validated. Sincc Ilhe content, nornber wW ordering of concepts in incoming emails was impredietabie· oulvmarie processing rouid our cosily be accomplished by conventional procedural progiunming mchniqucs. The full·reaturcd Al capabilitic$ of ncc in - house knowledge- based development tool affvrded o quickly implemcinhic· one-tool technique for rionsfaming the tinguislie clues in emails to outpul cras*Mcit6ons. rcoccas Flow Figure 2 iilusiiara dic now of an emull though die 62 Rader sysTern W described below: 1, The eusromci sends an email to Chase Manhattan Bank's lmcraeraddress, 2. Chase s corporate email tourer passes ncc message froth rbe domain server To ChaseDhect's t.otus Notes server. 3, EZ Reads petiodiealiy cheeks 'be inbox (a Lotus Notes mail database) fa new mnil. When o nnv email arrives in the inbox. CZ Reader renievcs the message and"inlcrprcts' n by pcrformingnrle·bnsed parsing and asobascd unlwal. Tire outcome of its interpretation K one oriwo possibilities: a) EZ Reatbr can respond lu the cmril autonralically. Anaurotnatic resp=c,whicfi is roared directly ro the Cb2SCDirCet ourbox· consists of ncc witintl email and one or move abavhrients. or prepared ieplfcs· they me retrieved fram a Lams Note% repository ur standard responses. b) EZ Header eannor respond ro the rmaii automatically. It refers the. email to Chusi:01=7-Tar human ieView and response_ Before placing Me email is the manual review inbox· EZ Reader assigns a category and priority ra the mcssagc and suggests one or more standard ropllcs basal on message cannot. (Categories and priorirlrs are described In more detail later.) 4. ChaseDirm spstiaiists review no write rCrpanses to all messages rsferred'by EZ' Reader and place these eloctrarde rcpiics in the outbox. Chase's corpraarc cmatT router routes the message S. farm C]tasc'VirccC5 Lotm Notes Tait mmtagamcnt system and places it in Clrosc's domain scrvtr rot reply back through the Internet..'

6.

Thecustoms race :vesChtutAirrct'semai^rcpk·,

t

Customer

it

GrarrDirr" r f_oANora Setter "^ c rs I
C7dxrNote, datai-s Oulbox Inbox Mal Rc

In bo x

74csp^nse

F
NS-. EZ Reader
Aua»naut x ChrueUircct ftmuof He.icw

11

1

Figoro S. Email Path lbl oueb Cbaaepir act

EZ Resder F1yhrld;Cnowlcdge Base ApproucL
EZ Reach 's hybrid tcawning approach ieOccts the actual iorcrprermirm process used by human =ail revicwets in ChaseDirect. The application combines proproccssing rules for parsing and eascbasod tcuicvat with a drranainspecitic kntrsvledge ·base . Otha text inrapreudion, applimlyc. save successfully MScd * Siywd npprv9pb v (Sahbt & Sawyer 1989) (GoodmM 1991). A hybrid Al desivi PmWcd both a functional and maoagcable progrun-Ak tcpmrlatation of the bwin= kaowiirdgc and itrhs for unail interpretation.

Caaestudrrs

ISOP

BR 001754

The combined - roles and ease-based aFpruacb was first :voluwvd aftermL team knuwledgc cnghtar observed and analyzed the ChaseDirccr uanil Inrewetatfon process. Human email reviewers read each nlessat:e from begirullag to. crrd white continuous l y evolving a final lmcrtnetation . ' Ilu email reviewer reaotsivcly applied business knowledge to message content throughout lbe review. RCVitwers modified their conclusions throughout the review. since an email contained any number of eonctpts in unpredictable order. The appllcallon emulates the recursive nature of evolving fatetprnation by first daccting combinations of ptominew words and patrcros of text in any 'order thruughot an incoming message , then setting object atlribme values ( hot both trigger and influence the casebascd reasooing process. Tltc application's ease-based reasoning process then provides data to the rule-base to infer a classification by comparing the mnsaagc content against the mposh ory of messages In the easc·base.

entails within the Lonss Notes Tsic.r. so >=Z Read: does not iroerfere with Lotus Naves as the standard Chase platform for rmail-relared word processing, mrlrivine, rtferenee, and reprinting functions.

i-

L

CEL

r..,..
Break
continuous

1-

START , DA INI' ZEE ' ..,.._.J

Continuous p ro c essing

call API
API; rOrrlCYC Np

I

promos sire anytime

new message?

internal Processing Flow
Within a Readcr, program flow is controlled lhrougb the bring of declarative rules which nigger, monitor and control processing in the application programming imerfaee (AP)), rule-brsse and casabasa Figure 3 depicts the knowledge-base processing flow, described in detail btlnw. Lotus Notes is Chase Monhanan Bank's corporate email standard; Chase's corporate cmdil router routes emails to and from the Internet domain and 1:01us Nines databases. Accordingly, F.Z Reader was boiR to opcrote coatinuousty and outnmatlealN in eonjunmion with Lotus Notes mail functions. "The standard replies available for 5elacdon by EZ Reader arc sturcd in a separate Lotus Notes database. 1;'L Reader input and outpur is performed automatically through its coraleetion to Lotus Notes, which was programmed via a Windows"' 3.1 API using the Vendor independent Messaging (VIM) proiwol and the API provided by Lotto Notes. The F,Z Reader API performs three important tasks: It retrieves an email from the Lotus Nines iobox and rctums it to EZ Reader. 2. it writes EZ Xcadcr's output to either the manual review inboz (referral) of the ouibox (auwrcply). 3. N marks the carXent email in the LdM Notes inbox as read. 71ba API enablts IM Revco to send its output (the original message and EZ Xcadcr's chosen response) to incised Lotus Notes databases ttal can be viewed and edkcd by business usersihmi;) anntomrindLotusNotesinterface that Itsts 0wpuls by category. Cliascl)ired business users have access to the Lotus Notes databases and all incoming 1.

i
Fmgrrc3. F,Z Rcrdcr Interned PIVm3ing Ftow The AP) also contains functions to cosurc transaction intc&Tay in east of eormestivity problems, Ordionry Locos Notes prOO=iag tags: a message xx "read" whtw to unread maiT.mt ssagc is opened for die Test tiant and than closed Typically, an API program that retrieves wail will mimic this action by marking a message as read irawrdiatety

*10

LVJ -%

BR 001256

after rctricv¢!. Mow-vcr- in EZ Kcadc,mcssagcatarltiag is dcfar.ed until after the Icsponvc has been sent ip cnltrR Hitt no Cato will be las(. For cvamplc. 6 dtb EZ Rcader client machine loses ! is connection io ilic Lone .-totes server in ncc midd)c or processing inn email, upon applicotion resttui· EZ Reader will again reirk--t that cmui) and aGcmpt to compictc ploccssing.

Al Emables Email Classification
£Z Reader uses chtssiftcalfon rules and- case-based reasoning to assign a business category and priority to each incoming email . CZ Rtader then tests the inftrtrd classification to seta and anach a standard response from the lotus Notes database orsiandard rcrtwnsrs. Calegorles . Using Al techniques described in the follotving saotiow. EZ Reader classifias each incoming email based on totai message contain into any of the roila'N ing three catagorics: Atrfomolic Response. EZ Rcadcr assigns a category or Auromarie A=porly[ to items that can be associated with a response from the Lotus Notes icposito7 of standard resportser and direall· rnalicd hack to the wade: wnbnW manual review or revision. Referrvl. EZ Rcadcr nssigns a category of Reyr, .nl in messn,_,cs that cannot be processed solely as automatic EZ Render then assigns a Airtlier subresponses . classification to the mtsxagc to assist ChasoDircet staff with inicrpretatlo » later. The sub- classi fications reflect ChascMrcci's organization and opem) lons, and arc expeett:d to tdmngc over time. Currestdy, there are two' sub-twegorfts for referred email: Sales and Service. and 4 levels ofprfority. Detecred EZ Reader assigns a c lcgory of Derecred to cmal)s that contain Phrases or punams that imply a panleular monual hand)ing pwccd= or tnicrprewtivc aid for a referred rnexsage In these cases. EZRaadm selects the appropriate remark to anacft io incoming mail for manna ) rcview. F$wnp)cs of Detected remarks Include: "cicleetc(I a phone number '-dctcm d a foreign addresi'.

Ru!c·bascw F7.Rcadcrusccnricstorcprcs ntChasclyircebusiness knoivlcJgc about Sow email content should be interpr -. fed turd handled. RZ Reader ruics oho control appiitvion pnocessfng Slow. FZ Reader's ruks obscrvc slandiud ART`SmtTprxrc syntax. Thew rules arc represented In an 1P THEN format with a 10 hand s)dc containing it set of conditions and a right hand side containing conclusions . The rules fire whenever the conditions sin forth in the left hand side of the rule arc met roprdkss or the sequencing of conditions . 'flit scncral syntax ora rule follows. _ RULE Rl2ll ^·HAMtr-· condition, eundhlunr ^^ . actions performed if all conditions ate Irut:
acrionr arionr

An simple cxampic of a rule lot detecting foreign phone numbers is shown below. In this example, if the ypteoi format ora foreign phone number is found in the message body (bv calling a function tow moskcd·mcmber) oeif other specified keywords arc.ptcsem. item all actions in the right-stand side of the rule will occur, Including the printing of the Imi "Foreign Phone Number dctcave
RULE foreign-0)inrr (or '

(n=)Wd·membtrs' + 99 Ymcssagc·body) (rnaskcd · mcmber5"+99 ·"? mcssagc bod;') (moskcd · mcmbcis "(-tg9)'l'trtcssagabody) CM.

cad' whir rondirfmtr... v
(printout "Foreign phone number detected") mry wheracnons... 'me left hand sides or the business knowledge rules in F.Z Reader rcptcscm key linguistic clues that directly imply intarpredvc conclusions, including lhctzts, wild card p pants, variables and segments. or choices of pattern sacs. Fos taamplc, me wild C" f»= rule infers the presence arm foreign phone number by looking for paldcros of ice that resemble a phone number with a pri caling plus sign. The inAxv*Kc or a foreign phooc nurnber is then usod .by thc-astbased search process ID . trigger an outputtilaraititation,

Y

Fraud Ilap cards

Promotional content (erg- Miaaafi lktwork)
Scr'd blpn v p Kit -

F

2

SensilivK in o ( tsE·.
account rwanlxr inahk&d)

3 4

MbocDoncousscrytce Lbmmeals ,. FYI

MultipieetoesSixu
err lap^Fty m cs

fttonef

7bbte I. Rrrerrals - Cateyorin a,na PiroriR"

Case Stvd^c

ISYI

BR 001256

Case-Dasc. E7 Reade) comnitc a case·bcsc enmuoncci 1ha. anabies the application to emulate the reasuning of C awDircc; slat` :Chen they use cxperlcncc to determine flow to handle :re- ambigunus entail. When rule-bloc ptoccssing fans io elcarly idenlipj a classit)cation fur an ambiguous incoming email, EZ Reader auvoilt s 10 Irmo cases that clnsc)y icsemblc it. )fa similar previous email Is found SZ Readct inlets Thar the response used pteviausly can tic used [a adapted) for the incoming anal). Technica)ly, the E7. Rcadcr casc.basc is a scatchablc database or cmails associnicd with spcc)fic actinns and object instances stored within the R Reader application. It consists of an ART'Direrprise object model of an dual, , called a Case, Ile Case obicet chss contains attributes, or slots, for the Important fcamres ormaits as defined by the knowledge used by CI*=Direet to WrMlci and respond to the message. In'nic) ah-rawics of the Ctuc Wuct include tefcrcnccs to addresses, spucirle types of ctonputcts, investment options, cic. The Cmc object also ineludas control attributes such aS a We fat the cote insgnce. Annthcr anribuic of the case objet. lists the results associated with the case, scrving to link the casebase with the role-bas. and also to drrcci Lotus Notes to ittrievc sPcc)fte standard responses. Cach actual email samptc in the east-base is defined as an object instance. Within the AR.T't)rrtrprise deMolimcnt cnvlrowncre an nptimized case base is prepared for scarchitig bg dcvetoptr functions that create a =e·hase index. a highly optitnizcd internal data structure that enables stored cast add fcatute values to be matched very qufck)y whit a an input case, called a jxerunntd c=. EZ Ruled" $catches the case-base assigning relative scores to each stored case based on the number of kwuros, the mismatch of feanue Calttez and the abstncc of features at compared wil h the prescmcd case urlnl; cuslomizablC caw-based 1caa0o(og eomponcros suppficd in lbu ART·£nrer,wrrc tool. 'Y7(aractcr match419 with At'rams was uhoscn In drive eau-base scoring in E-% Rcadcr. A trigram is a ,;. ChalralaCi scqucnce. For example, tlic word "CnASEgcncratcs 7 consecutive rrigmits: _ Ci _ CH: CAA: HAS; ASE: SF_: B _. Man charade, matching is used, the vatuc of the character reauxt: is Ixokcn tip into 000sconive h-igroms, and the trigrams of a srort d case arc inatebod against thZ t6grams of the pimento) case. lbc dtew gmc of partial maicbb4 is based on the proportion or O bigrams in the Presmtetl case that match trigranis yethe stotta e'aw- 21st trip= wnlchksg luffiniquC minimizes the importance of low nrdn of the individual wads in rite incontIng orts=er. Standard ease- btno scoring for the mac test of an email ['es for all tvxl type (eatures). is driven by

k

AIT'Enrerprire 's default iftram charactermatching algorithm:

)'cmttnc-wcight [n)= !{tx·tm)tts) -fmisinatc1%+(tntttxy Matchi
Wltctc · tm is the number of aigrams in common between the picsemcd case and the stored case
· . tx is the total number of trigrams I. the (,Cswcd case feature

jtrli5matchi is the mismatch wcighr of rcatuic/for the;-th case J match, is the match weight of reaturc j rer the i-th case. Thu standard algorithm works as follows, if the value in a feature of the stored tsnntil dwlchcs the value in the corresponding featurt of the Incoming email, the feature's match weight is add to ' the shred crrwil' s score- if the rcarwc's value mismatchm the feature ' s mismatch weight , sypically a negatlvc valug is added to dtc scot c·
In a Reader. each ali buic, or, feature , used by the mc·basc was acalsnal a dcrault malch - wcigbl and a customized sitstnatch-wciyln of rFro. In EZ Reader, the minnaicb.weight of zero leads to better diffetcnlialion of stoics,, because of the incidence of misspellings in incoming emallg combined with the well-bounded knowlc%u domain. The actual weight that any fcararc conuibolcs is raeaningfnt only within the cunlrxt of a particular case and relative to the weights of other rwcures. Since stored cases calf contain difl'arcm numbers of rcarures. a picsenicd last's raw score is normalized by d)vfding the raw score b, rho maximum PussWc match score for the case.

Mismatches are nor entirely ignored by EZ Reader. Another faetor. atswang oases in 1:Z Rcadcr is 11tsl a glubai absence Wd& is assigned to ar emcd sired cum throughout the casc·basc. lltc total contribution of the absence wcigM 10 a stored cases match score is calculated by muhiplying the value by the number of features in the presented ea se which are not in the states, ease. The total absence weight is lhw added into the env match score for the cast print to awmalkinion. 11o default absrnco weight is -f; lx Reader utilises an absence weight orQ la ),edcrc the littpat" oftnawng feat xes. Thu aasebase process independent upon xv)a to derive irs·pk*crthed vase ftatturc values , to M Reader, tales fire baforc the tawbased iirasoribtg process to W==1 feati)scs or sdtaraetcristies or the email that help distinguisb the cwtlcul.ofihemttssage . 00pcrtding on the Content of the

t

TM2 t"1-%

13R 001257

meaaa¢c, any of tilt casohxc snatch fcarores may be wt n the prathotessihs idle phase. Any fealmcs scl w'01 nhca after , the scoring cz culaliuns perrofrded by r12T'Snlerprrsr 't ease - bated reasoning engine. Dmwing out sarwin anrae:aisties ur the message content using rite; combined with inesaet msabased : anieval allows toe more powtnful wed precise email inierinctation then simple keyword parsing an ease.boscd tcnieval b'Wed on mossagc baly only. For e.+ttunpk , if ez Reader iTtfers front incoming email [exr that the Vander does not want ill lx telephoned by ChaseVirca , the rufc rot diLner _eltt-cnsrom 4 tires and SaS Than atrribnte in the case to ,'Yes". features sit to " Yes" then cottributc to the exsc - lr4Cd sealCII by addinj weight fnr similar siotcd vacs during ease - bascieirieval. A sample or EZ Rradt7 hybrid processing now, including the inreraetion between rule iidngs and rasa base matching, is son forth billow. The importance of set attributes ro the ease-base srocii is clearly illustiated in Orrsc two cxemplcs. T%vo sampic arstances or ncc obj-ct sIAL £qSr are Show if below: CA SE001: Title - "Sign-up Xh rnlvar,Reren." Subject --close dlncci` maSagelcxt" "Plivruscud inea CiascMra sig,t·vn lit. My odd=s Isaddr e567 - "Yet"
aeriwn = tekruten-pp·)ch, dclcctAmddr . avtn:YrErt-np·art

CASEGDZ: 00c m "Sign-Up Kh requcstrno wdnas; Altio Respond,"
suven- "Chase dila : message Ian- "Prdu sand mca ChateDirrct Sion-up kit."

arliou - chase-dircowtd Suppose ChaseNrcet ecccivcs atr email with The body of the message as follows: Dcar ChastrMrccl, Please scud the ChascMccl Sigal-Up Kit to my houie Address, Thanks, Juba Doc 121 E1tirSt. NY, NY 7 DOD T

A rule for dellic4ng an address trill fire . resulting i:s selling the case atuibuie addirsi ' to "Yes ." Next, FZ Reader w411 perlorm a search againnsr the easrbosr. raakinc CASE-001 with o store higher than CAS&002 because of the match of Vdreys2 . The email will be rcfcaed because the sig n-Tip kit must be scnrout via postal mail, and the Venda will receive an electronic acknowledgment Thar their request has been received and Thar h is being processed, The doccied anion empl-v aids the ChtescUlneet sta1T In quickly determining imporraar eomads in the entail. Nesir suppose anmha pcisoa regttvstr the kit bur does nor include his postal address in the avail, i n which case the nequca cannot be fulfilled. Tier ease-base scotch will sesult in CAS£OOx scaring highera » d being seleaed over CAS £ 001. The Benda will their receive as autornmic standard ChaxDitod response with instructions me how to icccivc The sign.up kb. Om or 11te man, bentrits of case based retrieval is that the cases raiieved from ncc ease-base do not have to match rite airaia exactly ( as in the t, tvazS..S atrriboicL but the desired precision or a match can be easily specified. This "li (y is important ro the success of email inrerpteiatinn, Because CZ Reader priicnws rice-rormm text- h canna s-unply rely on an exact match bcnween the incoming message body and the ... A: attribute , As a eonsaqucnee, some sulicifluous lhc,als, such as the actual address of the message scads, were removed from message tcxr attribute values durhtg The case creation process. Since the ease-based icuicval atgo then pet forms rrigtam marching on the message text fmlmc. Therals such as addresses cleft unhucriviDazelly aMM a casrbasc match score. In add'uion to the awibutrrsming roles ti¢scxibed above, E7; Reada's ruic-base cdnsists or scierral "'aaionsctting" riles . The rules can dam inrtmralion that a human ieadeer may overlook. Some aspects or tire castom i s email reveal valuable infi rmmion for ChaseMce: bui do nor neecssarily contribute to the reply. i PDT example, ChaseDirea keeps traeSc of prmpeets' slid eusromeW phone numbers . This informanon is Important to Most: inxt, but ChieseDiiccr does not neecssarily wart IQ respond ro the eusto es in a difraenl marina. EZ Reader can tagihis eritaiT with a message aboth dctcming a phone number . which will consequently be easily seen by stafrorembers when manually mviewhil; the errmil.. The mr-basc clemently contains over 700 rases; the introductioot or more temple cmaiis over time will enable io·iat^e-prcn a wide, varfaty ormesstigcs, and EZ Rc-+ rtxthcr · case feature ihacase piccisirnT dtrough nefinancnli ....

i

P

Cascsrpdia

*Y1

$R 007258

F':.eption Messancc Hanaline. ?here arc some message that EZ Rradcr i s not able To irderplc:, o.t., exeepuun mcrmge= The eoment of a execpfion Incsctae is ae.bigucus eycn tar a knowledgeable persnn to interpret. for C;Cmnric, a message that ron%ulm qtly the wosti Tbc L:e Rcadct applicalion comains case "Test". examples orpmviousiy received eaeeptfon messages, so it is able to forward them and all oft: uninterprelable massages so the Lotus Notes database I'w manual evaluation Eliot Processing. P7 Rcadel chocks to; errors both in its knowkdge · bare ptocrssing and in the API connection 16 Lotus Notes . Furors generated by the Lotus Notes .API do not cause FZ Readci 10 terminate. instead. FZ Rcades will continue iryintg ro access Lotus Notes until a connection is Ftx instance, it the Lows Notes saves is made. temporality down, the API wilt send the jeltvart) savor message back to 9Z Reader . HT. Readci then waits a few seconds before trying the connection again. In this manna . E Z Reader is sciM1tnort4oling and maintains maximum up·irrnc

91- Reader enrorted consistency of responses to customers. Befoic deployment of FZ Rcadcr, C7ascDiseet workers ahernafed the daily responsibility of reviewing cuai-s. Each day, a different wotkcrtypically scent the wboir work day manua4 y, ltadi:% and n:spondbig to email . Cousistcncy or scsponse was an important business consideration for ChaseDlrecb and since quality orsesponsewas a function orknowledge and expericnee , responses frequently had to be checked by a srspcrvisot . IZ Reader assured consistency of response because n aatomatieally assigned ptepwrd text depending on its singularinteryretarioa Ora message. simplified proeess Rsadcr the business CZ substaullttlly. EZ Readci enabled Ch=Dircct to seduce the number of manual steps and the cfrort needed to ptnccss Its incoming email. A comparison or processing steps el-aniatated and modifled by xZ Rcadcr is ousl'med in -fable 2 below.

Applfc-alion Benefits
M Reader ployed a nitieal tole In establishing Lhasal>irecl 's ability io psnvldc and teaimOrs a tcsponsivt Marketing and .online The service channel, implememation or automiucd icasoning enabled pnoccss simplificas4on, speed and consistency or responses, as described below. F-Z Reader incseased the speed or response to she eusiomci . E'L Render cRminatcd . mxnusl mcrvcmion for a Percentage or message? and mol ihan halved the time e in pinaess messogcs iequising manual intervention. -f7te reduction in Inanua ) intervcnsinn allowcd OntzeDireet ro turn wound email responses fasicito eustomeis. With a knowledge base psoeesstng speed or I message per second plus 2-5 seconds rot Lotus Notes communications, IZ Reader reduced overall processing i1mc by 6-8 minutes i l Pestles 4 . Another factor contributing to lassies Customer response was that the PrIplieation was available on n 24 · houi basis, allowing continuous processing of automasie resporne types or messages over, a wcefccnd when many customers. tendedlo be 3clivc Online.

Dies in tp eraail sysgm Rcadimdnns se
Print and-noWe,

SetcalAdjust response Send email /trChiveoriginal DGaeftomroldar Falter into C415

one min pct acws ruteroinmewmon; tooncmin0c live minces ut more
up 10onetntnule

etlminNed

erimioatai

clim;ri au
ctimirut of! mudificd mndirxd muditkd modiacd no change

up to one minlse uprnnntmintse avg: five tnintncs

Table;. Proc=ChangesEnnbled by CZReadcr

Application Maintenance
Curicngy, knowledge-base maintenance requires edning of cases, rules, and actions, alt or which we in ART*Fnwrprise syntax . One proposed technical enhancement to CZ Readci is to build a framework fa maintenance in which business uses could add gad modiry cost objects, action objects, and even ruk; thiour.h a GUt inltsfaee where the undvslyirs; ART'Eruerpriso syntax or the objects is invisible to the maintainer. Althou?h spceiartrA knowledge and skill sets are rrquised to maintain EZ Reader today, the appiitaiion is hsgitlp roodtdar and object- oriented . · Ilse case-base and rule-base arc indepeadently · suvetured . In addition, the .t

I

x around 5 percent of au messages see Asrrwtrafic Tf rougfspnr i'trae+rmpt for a discussion of the elIen ofinessagc content on avamsc speed of response

a Contact managtxaem syslcm'ltscd to track cottiactswith customers and psosPC=

I

]514

LkA1-96

J

BR 001259

design is general cnOU0 10 bt· CUiiy adapmd to otheJ domains, and code-levc) maintenance ptomilutes art straightforward. T Icadci mainiciruicc is icquircd .nc^ncvci.cc'tain business W technical eavirorvnem changes octna. The "ronairing tyfrrs of busincas changes usually require ET Raadcf nnaintanunco: A new oi revised association between a type of mu,sagc and its prepared i ponsc is rcquhcd. Fof instance, it entails conccrmng a former Trarkering piomorion nerd to be answered diffenemty than thcl were originally , EZ Rcaden mainicriancc is performed . (Modifications .no picpamd tesponst wording arc pcnfarmcd dnough a Lotus Nocs art vitwJ A new Type of message needs to be associated -kith a pmpared , esponse . For instituter. iremoilsconceminjt a new marketing promotion need to be ptoccsstd automatically, EZ Rcade' mainienancr rs per farnrrd. A new inlurmatiunal mes"at: needs to be generated by EZ Rrudn when it detects it. paniculea Type of invamh g message , icgmdicss or the response. Pot insotnee, it rmails rrom current customers nerd to be flagged , M Rea de mairitcnance is pcrfomrcd. In addition to these know ) cdgc·basc maimcnanee seeaarios , whenere a chwrEa in the technical crivuonmcni around EZ Rcadcr is planned. the application is assessed rot nequircd modifrearicats and tems , ed in the target tnvitonmena . For example. because EZ Rcuder interfaces with Lorus Notes, if a new vcrteion or Lotus Notes is planned for iWirallmion, ncc EZ Reader I Lotus Notes API is rested and assessed rot any rclcvanr maintenance. )n this cast: because the connection To Lotus Notes is independent of ncc knawlcdgc - basc, ' only )be AN portion of la Reader requires me sting, · One person from the icsidcm X13 train at Chase is responsible rot maintaining IM Rcadcr, among onhet Busies. The team is naincd in ART- Ehierprire as well as C language and I.arus Noics dcvcldpmcni . Thcy also understand the Wsiness Tet unancnts or E-7- Rmdci and arc obk To irartslase change tEgdb3fS',`nTE ' L7. Reticles knowlcdoc · base modifications. No manual iniervcnY'wn is necessary ion du; day-to-day oyt-rarion of EZ Reader. A lure] replica of the l.otus Nola databases icsides on the dt;velopmcm yoaehine; thtse databases arc used lot testing and maintenance of the system . In addition. M Reader cars be run in strictly tnamal mode in which it bypasses the API connection. input data in this mndc is retrieved from an cxicwU text tilt and . elow3 for quick resting of the knowlcdgt base independent oftbc APL ·

Frvieci 5lislor-1
Chase Manhadw Bank, Pe innal Pankirt& Taumhcd en initiative in 1995 to csplorc haw artillclal irvell'.gerrec (A 1) could help Tried its business chollcgcs. At that time, i3 applications were idemifted to leverage Al at Chase and EZ Reader is one of those implumenimions. EZ Reader started with a concept papa wniitcn by members of Chase' s Knowledge Unsc (KB) technology [cant. including Uriyhrw·arr consultnots, to May 1995. ruithet development of fnnctitmal spccirwerions, cost timlysis and preservations To mono&ement saved to get approval for promtypc development, which began in July T995 and was fnWicd fn August . DcvOnparcrtr of a production system was gnmcd approval in September 1995. BcSinnfng September 1995 . EZ Reader pnodueliun' application development ptoccedcd with one full-lime: technical developer and several part.lirre staff including four business analysis , rhrcc lasting specialists, and numerous technical support pnsonnct who spcciafrtcd in Lotus Notes and the Chase email ncrwoik. The knowledge " base, consisting of the ralobasc and case . basc, were completed fnsr, enabling uset testing while the Lotus Notes API was b6M, developed . Infrial knowledgc · base hosting war perforated in Novcmbet, resulting in accuracy very clust: To uut target level with thnu s lIPM of one message pct second further ntihicmcntT incicand the accuracy to our goal in Dceemlxr. Although the knowledge -base could hive been deployed at This point in a semi-mange! mndc, managemcm single a imcgrarcd decided for bnplcmeniation after completion of the Lotus Notes API. The Lotus Nocc API was eor,rplctcd. oonsoUdotod and tested with the knowlcdgc · batc a few weeks later. EZ Reader was dcvclupcd using ART*Emerprim version Z.Oh with the bichtded Microsoft Win32s library, Visual CACi4 1.5 and Lotus Notes version 3AC all running on Windower" 3.1'. 71x application runs on a PC with an lnlcl-l;ascd 486166MIIzpraccssor. 5DOXZ orhord disk space and 32MU of RAM. 9hc Lotus Notes server is an t8M 9595 running OW vcnion 23.1. with an lmclbased Pcniium lzzW1z ptoccuor, 2 hard disk drives or 5DOMB and J GA tout 64MR ofRAM.

R

Mcasurilig XZ Reader Porfor m axee.
EZ Reader was evaluated before implementation to csstxc that Ooisc w ptarluclict" appxoval Criteria write, rite'. To evaluate EZ Reader ' s performance, tic team analyzed Three rorasures: speed, accamey and urnomatre Ihroughputpacctttagc . EZ Iteadec"s petCsinnaace Tat tack measure is dcsvibcdbelow.

. Ca30514474ts

IM5

OR 001250

Speed, One csscrrlalsoceessciheriert·For LIZ Rradcrw·as :hat it reeruee the oral amuunt of anc spent praccst ig emaits. The contest rests of EZ Rr;arl^r krmx' Icdge bocc demonstrated speed of one second or !W for interpieranon functions that manually tuuk unc minute or more. After I.Mus Noses imegrariun. an ad6itiurral $-5 seconds was icWired for each mcssorm rZ Reader's Lmos Noses inTCgrmion teduced the amount of rime nettled so attach and send respunscs where manual inrervemion was SIM required. AeeuT aey. ChascDbcc; was concerned Thai any cuftware would be able To inlerptvl messages with she required level of accuracy. 'f lit vaimaMd accoTacy Icva) for manual processing was 9M The scam sci EZ RcadcP's accuracy goal at a rate equal to That or 95% ur 1bc accuTxy of manual pr cccssing. The case·bascd reasoning Intic in ncc EZ Reader knowledge-base was able To deliver a high level of accuracy, )TS eapabWry so rank the degree of similarity between incoming messages and previous messages in its ease-base, combined with Rtes, iesulrs In a high level of accuracy, Auromarie Tbroughput PereenTarga Amnnraric )hrOrrghprrr jrareanrubrr w·as a measure h5Tablished b} The rcam to evaroase the egailibdom between incwhiiit mcssa $e eumeris and EZ Reader ' s knowledge base. It refers To The percentage of messages Thar can Ire pmemscd wiihoui manual iniervenhun As move eases and rules are added su n Treader , The scam cxpeers in achieve an outomaTie Throunhpur percentage of RVI* or more, Daring The early manors of initial tcsr )ng, The aasomaric rhiuoghpur peicenraggc varied hum 200A To got/% It fell when high -volumes of messages with new, unexpected eonrem were Tcccived, IT also thupped when business segnllrncnts were implemcmed To Tries addirionat types nrmessaoes. Based on a cranparism ofmanual processing Time wirh U RL 0 pTUCC%Siag rime, ncC aurorgrdtie Throughput pcrccntate translates sigTihcam To productivity gains for cult pcrerhsagc puint gained in aumotarie rhruoghpuT, Measures within Lotus Nules ea)euluic and track the volome of messages by category (aurumaric or Tcfcr ), dare and sender cmwl address . When a dcneusint percentage of aorumatfe throughput is deiated · it alms the business to the need to add new roles or examples to FZ Rratlcr to criAle it to recugaNx new Types of messvgts sera by custumcm and prospects,

:hrava)z iSZ Reader. -the results frarn cacti were complied to za-,css she seauincss of the ET, Reader knuw)cdgc.t:asc far psuduadom 6usinez^ analyst eullee : ed cleenurtic copies of die aerial incoming! messages received during o two-wcck period as tcsl bed daTa fur The parallet test. U Reader developers oscd Denial messages received by ChascDircc) as input data for icsiurg EZ Reader. The Tuttrs fed the lest bed messages eleenuniwlly emu F.Z. Reader in its native A1tT4&fcrprisa devclopmem mode andpsinicd the Tesubs . The psimucrs contained she incnming messages. The EZ Reader on", elessiftemions and she amuunt of stmt b Taut, GZ Rc+dcT To process the messuvr. The primuots also euruaincd a blank furmancd area Thar was used To record evaluation remarks. Two reviewers from ChaseVocci who wcic nor involved in the original manual processing sragc analyzed the prmroess. Where EZ Reader produced an incurred carcgury or response, the tevicwer nuied the expececd icspnnsvMand she error was Tcviewcd by The EZ Rcadcr team. l:ehncrnems m the easobasc and roles writ made and vw1fied in subsequent abbreviated tests before The opplicwr)on was appruved for pruduction. The Lunrs Nuics API and cnd·to·cnd network communicatluns wee Thoroughly rested over a period of weeks using eurrvemiunal systems icaing Techniques, Summary t;'L. Reader is an Al application that pTUVidm marry Tangibic and imangibla bencfns lu Chase Maohanan Bank as Ti seeks su maximize upponunints in The intamer mat ket. EZ Reod.. enabled ChmcDlact so eliminate ncc cows of wenime for email pruccssing , helped mcer customer expcenvions fin service standards (such as response timeliness), piuvided for smoorbm implemcmar)on of GhaseDircct maikering programs and onrbled unattended Ix ocessing of email on weekends. ,-Since EZ Reader's Pnuwlcdgc base was devclnped for it Cbasel)bccct, i ts iniria) urlliry was timn ed so email pivxssing Cur ihas depwwasens, TM fullowirrg enhaneamems for EZ Reader arc eurremly being considered: · Add husiae sknowledge ftuother business areasw)or choose To sake advantage of World Wide Web communic lion with cusmmcis. Enable EZ Rcadcr output its be addressed nummaricaliy 10 rcep*nls rhraugboot Chase via Lutes Notes. Automam !tic business knowledge maintenance fono[rons of she application , I.e, enable ChaseDhO

Y

F

·

ILnowkdgye Dwe Testing Method
To test EZ Reader's knowledge, base, the ecwu pcrlwmcd paruilel resting ro compare E!. Reader with manual processing, using the same messages both manually and ·

151b

EAA)-96

a

$R DOI261

M

business users to imcruct with rz RCD&T :n t ivngc how hS know)edgc base interprets armsaga anc inks them is rusponscs. Novidc the abillty io auwmnticaliv process irmorning me;sagcs in Spanish or etlrel languages, providing a poicrrtfol Slobal tnoAtling advantage. Enablc F,Z header use of`histmical anal and profile data to personalize EZ Readul processinp.. Link fbn eomaot matagumem iyslem To v process that adds customct knowlcdgc from snails processed by IEZ Reader. AcknowIcdgmcnls ARM"Icrrrriredi is a registered orad0mark of Brighmwr , hm.. Novato. California. Lotus NotCS450i5 a fegincred trademark of Lotus Nuocs Development Corporation . MieiosogO is a aegistered uaderrnrk and Mkmsofl Word for wl"dows7s` and Windows"' are Iradamarks of Microsoft Corporation. OtTtcr ftwts mentioned in this pope' Wray he oodcmaAs and ate wed for idemTcation purposes only.
I2 [fCYCn c e5

I

Compton, M. and Wolfe, S. 2993. hnelligcnt Volidetlon and Routing of Mccuonic Fonns In a Diskibuicd Wvrfi Flow Environment. Technical Rcport. FJA-93.31: NASA Ames Research Center, Anificial Inic)lipence Rescrtth Branch. Goodman, M. Prism. 1991. A Cn!o-l3ased Telex CiassiTci. ' In Innovative Applico6ons of An7ocial Initifigenee . Vol. 2: AAAI Press. $ahhr, Kacrd Sawyer. X. 1989. The Inctftcm Bauking System: Naturat LaDguagc Pttv%semg for FinancW Communicutions . In InnovativrAl plicatians ofAr'ifrdat. f Iniclligcnce: AAA] Press. ,Taylor, D.; Mchta , B.; and Wurster, T. 1996, Online Delivery & the Infomrration Superhighway; Searching far Retail Strategic . &mk Managcmcnr 72(I): 7Z-79.

i

C+SC Siw^ r+

t517

SR 001262