Polaris IP, LLC v. Google Inc. et al

Filing 465

REPLY to Response to Motion re 391 SEALED MOTION for Summary Judgment on Google's Affirmative Defenses and Counterclaims, 389 SEALED MOTION for Summary Judgment on Defendant AOL's Affirmative Defenses and Counterclaims, 390 SEALED MOTION for Summary Judgment on Defendant Yahoo filed by Bright Response LLC. (Attachments: # 1 Affidavit Wiley Declaration, # 2 Exhibit A, # 3 Exhibit B, # 4 Exhibit C, # 5 Exhibit D, # 6 Exhibit E, # 7 Exhibit F, # 8 Exhibit G, # 9 Exhibit H)(Wiley, Elizabeth)

Download PDF
Polaris IP, LLC v. Google Inc. et al Doc. 465 Att. 3 EXHIBIT B Dockets.Justia.com Proceedings of the 11. hirteenth National Conference on Artificial Intelligence and the Eighth Innovative Applications of Artificial Intelligence Conference Volume Two Sponsored by the American Association for Artificial Intelligence AMI Press / The MIT Press Menlo Park Cambridge London BR 001250 Copyright 0 1996 American Association for Artificial Intelligence 445 Burgess Drive Menlo Park, California 94025 USA All Rights Reserved. No pan of this book may be reproduced in any form by any electronic or mechanical means (including photocopying, recording, or information storage and retrieval) without permission in writing from the publisher. The trademarked terms that appear throughout this book are used in an editorial nature only, to the benefit of the trademark owner. Nu infringement of any tradernarked term is intended. Copublished and distributed by The MIT Press, Massachusetts Institute of Technology Cambridge, Massachusetts and London, F.ngland ISBN 0-262-5109I-X Manufactured in the United States of America on acid-free paper. BR 001251 EZ Reader: Einbetided M for Automatic Electronic Mad Interpretation and Routin2 Amy Rice and Julie Hsu Brighrware, Anthony Angotti and Rosanna Picco/o Chase Manhattan Bank, N.A. Abstrart EZ Reader is an intelligent electronic mail (email) tender that employs a unique combination of rulebased Bank N.A.. Regional Bank, began lo provide electronic banking services using phone and personal computer technology in 1995. Marketing campaigns advertised that email conld be used ro request information and services, opening a new electronic channel of communication with customers and prospects. parsing and case-based reasoning to automaticalty and with a high level of accuracy c/assify and respond to large vo/umes of incoming EZ Reader reduces the time and human email. resources required to handle incoming email by se/cuing responses and adding attachments and advice tu each incoming message based on how similar messages were handled. The application, developed for Chase Manhattan Bank using Brightware. Inc.'s ART* Enterprise® tool, previous answers Phase I The success of its marketing campaigns created a challenge for ChaseDirect from the beginning to quickly nod cost-effectively process emai/ from multiple sources, including the Internet, Microsoft Money email, and another internal DOS-based money manager program with email capability. In addition to ChaseDirect's commimient emails automatically and decreases to provide excellent, timely service to its customers, electronic commerce laws required the bank to respond to certain types of electronic correspondence within specific processing time for those requinng manual review. of EZ Reader was deployed in the Ilist quarter of 1996, and handles up to 80% of incoming mail automatically, depending on mmssage content_ Lain phases will enable BOONTIalk prOCCSSing Of 3 wider variety of messages. By dramatically reducing time frames. Although niore than 80% of incoming messages were simple requests for product information, the staff often got backlogged and worked after hours and the effort associated with manual processing, EZ Reader will pay its own development costs within six months and will mutt in substantial, recurring dollar savings each year. This paper describes EZ Reader in detail, including its Al-based design, testing, iinplementation and development history. Problem Description Like other businesses that sought to expand access to their on weekends to keep up with the required analysis and responses. Faced with the huge projected 'mercase in Internet email volume due to the planned innoduction of a new World Wide Web server, as illustrated in Figure I below, ChascDirect aggressively sought cost-effective, high-quality ways to process emalls. This urgent business problem attracted attention from Chase's Regional Barak Knowledge Base (KB) technology team. The team's general charge was to apply artificial intelligence (Al) technology in key areas of the Regional Bank where appropriate to optimize operationai decisions. To address ChaseDirect's business problem, the Knowledge Base team created El Reader, an embedded AI application operating as an invisible layer between the The Lotus Notes® email system and ChaseDirect. application continuously retrieves incoming Internet email from Chase prospects and customers through an interface to Lotus Notes, and aLso acts as a filtering and routing products and services through the internet and other online channels', ChaseDirect, a unit of Chase Manhattan Contact information follows. Amy Rice: 301 Tresser B/vd., 13th floor, Stamford, CT 06901, rice@ Julie Hsu: 2010 Corporate Ridge, brighrware_com. Suite 700, McLean, VA 22102. hsu@brightwaresom. Anthony Angolti: One Chase Manhattan Plaza, 19th floor, New York, NY 10081, anthony.angoni@chase.com. Rosanna Piccolo: 15 Er 26th St. New York NY 10010, rosanna.piccolo@chase.com. Banks lace significant threats to the retail banking franchise from advances 3/ online banking (Taylor, Mehta & Wurster 1996) Case Studies 1507 BR 001252 ArCP Enterprise, a commercially successful knowledgebased application development tool. As described below, the Kaowletige Base team considered and rejected several applications and tools for alternative software imp/ementing the reasoning component of EZ Reader. In-house procedural application languages such as COBOL cou/d not handle the reasoning component of the email process because the input to the process was so complex and unpredictable. An approach of coding 1Tre.. 9f Feb-96 each potential input using procedural methods alone would have been impractically complex. branches for Jun-95 Oct-95 Figure I. Projected Genwth of Incoming Email by Source "fhe Rete algorithm used by ART'Enterprises panera matcher addressed this input complexity issue for EZ Reader. ARTEnterprise's integrated nde scripting language, case-based and object modeling capabilities enabled automated reasoning to reverse OV change conclusions drawn throughout sequential parsing and ro disambiguate text while ignoring irrelevant portions, such as signature lines, without coding explicit subroutines. Although input was in the form of natural language, the output was a simple classification based on the application of heuristic rides and experience. For this 'reason, syntactic analysis and interpretation, discourse anz/ysis and pragmatic inferencing in commercial natural language processing (NLP) products such as SRA or Logicon, augmented transition networks in custom developed applications such as the Intelligent Banking System (Sahin 8: Sawyer 1989) were vievved as providing only an pre-processor function to the classification reasoning task, and not 'providing a value-added output per se. Tins is illustrated by the fact that the ChaseDireet human email reviewers were able to classify a /arge majority of messages by using a relatively limited set of key linguistic clues that could be expressed as simple mies. Furthermore, when human reviewers had trouble mechanism, either replying to the email automatically or attaching a suggested response and refening the message for manual review_ The KB team's overall business goal for EZ Reader was to reduce the lumber of emails that needed manual processing by more than 80%, with an accuracy rate of 95%. Other goals were to provide rapid turnaround lime for return messages to custorners and prospects and consistency in response>. EZ Reader's automated reasoning capabilities enabled Chaseffirect to reach ;hese goals and significantly reduce the manual effon needed for email processing. El Reader Design The ChaseDirect email processing problem offered several clear opportunities for the implementation of automated reasoning. Al in El. Reader The email review process involved complex reasoning. The process was distinctly knowledge-intensive; __specialists applied dornain-specifie knowledge using identifying common linguistic clues in a particular message, they relied upon temn experience io help them classify- the message and configure or synthesize a response. Compared with the time lo create roles and heuristics based on know/edge of Chasc products and services. The reliance on knowledge as a process component was evident as Junior team members routinely senior case structures in ART'Enterprise, the time frame to fully analyze the semantics of the text interpretation domain and deveop a comprehensive NLP module vvas deemed relied upon the experience of their supervisor and other team members to explain how to classify ambiguous messages and forrnulate responses for new classes of messages. Finally, common reasoning tasks were reflected in the process output classrwation of the email and configuration oía response from a limited set of prepared text modules. too long to meet business needs, and would have provided, at best, secondary functionality al greatly increased cost and technical complexity. Fundamental technical requirements for ama automated solution included compatibility with existing hardware and standards for run-time and network performance stability; scalability, vendor support and COMOICICiai Wide/y-available shareware, EZ Reader's Al reasoning component is a data-driven forward-chaining rule parser operating in concert with case-based reasoning written in Brightwart, Inc.'s proprietary intelligent programs specifically written for electronic forms processing (Compton & Wolfe 1993) and packaged email ronterhesponder software such as Prolog-based 1508 /AA1-96 BR 001253 -^1 :deiltor these reasons. (Daimon Laboratories, inc.) were re¿ecleci for in contras!, die ART' Enter prtse tool had 6 L.t CUSP:M.1QT itttiVeS ChaseDireer's email reply airrady been deployed in another application at Chase where broad functionality and customizability, product reliability and vendor support necessary for fie:ding commercial applications was previously validated. Since the content, number and ordering of concepts in incoming emails was unpredictable, automatic processing coulti not easily be accomplished by conventional procedural programming techniques. Thc full-featured Al capabilities of the in-house knowledge-based development tool afforded a quickly implementable, one-toot technique Custarme Message Rerponre Domain Smte r for transforming the linguistic dues in emails to output classifications. ChaseDirect Lotus Notes Seam Process Flow Figure 2_ illustrates the flow pian email throng)] the EZ Reader system as described below: 1. The customer sends an email to Chase Manhattan Bank's Internet address_ Oulbox helms 2. Chase's corporate email router passes the message Traci the domain server to ChaseDirect's Lotus Notes server_ 3. EZ Reader periodically chccks the inbox (a Lotus Notes mail database) for new mail. When a new email arrives in the inbox, EZ Reader retrieves the message and "interprets" ir by performing rule-based parsing and case-based retrieval. The outcome of its imerprelation is one of two possibi/itics: EZ Reader can respond to the email automatically. An automatic response, which is routed directly to the ChaseDirect outbox, consists of the original email and one or more attachments, or prepared replies, that are retrieved from a Lotus Notes repository of standard responses. EZ Reader cannot respond to automatically. the It refers the email email to Figure 2_ Email Path through Cturscrirrecr Chaseriireri _for human review and response Before placing tht email in the manual review inbox, EZ Reader assigns a category and priority to the message and suggests one or more standard replies based on message (Categories and priorities are content. described in more detail later.) ChaseDirect specialists review and write responses to all messages referred by EZ Reader and place these electronic replies in the outbox_ C_hasc's corporate email router routes the message EZ Reader Hybrid IC nowledge Base Approach EZ Reader's hybrid reasoning approach reflects the actual interpretation process used by human email reviewers in 4. 5. ChaseDirect. The application combines pre-processing rules for parsing and case-based retrieval with a domainspecific knowledge-base. Other text inteapretadon applicatioas have successfully used -a hybrid approach (Sabin & Sawyer 1989) (Goodman 1991). A hybrid A/ from amseDirect's Lotus Notes mail management system and places it in Chase's domain server for reply back through thc internet. design provided both a functional and manageable programmatic representation of the business knowledge and rules for email interpretation. CaseStudies isoF BR 001254 'I he combined-re/es and case-based approach was fiat evaluated after iatt team knowledge engineer observed and entails within the Lotus Notes 3ystem. so EZ Reader does analyzed the ChaseDirect email interpretation pocess. Human email reviewers read each message from not iraerfere with Lotus Notes as the standard Chase platform for entail-related word processing, archivin:!, reference., and reporting functions. beginning lo end while continuously evolving a final interpretation. The email EtVieWtt ICCUTSIVely applied business knowledge to message content throughout the review_ Reviewers modified their conclusions throughout I I CANCEL the review, since an email contained any number of concepts in unpredictable order. START t Break IMT/ALIZE 1 The application emulates the recursive nature of evolving interpretation by firs/ detecting combinations of continuous processing anytime prominent words and patterns of text in any order throughout an incoming message, then sening object anribme values that both trigger and influence the casebased reasoning process. The application's case-based reasoning process then provides data to the rule-base lo infer a classification by comparing the message content against the repository of mas.sages in the case-base. processing NO Internal Processing Flow Within El Reader, program flow is controlled through the firing of declarative mies which trigger, monitor and control proccssing in the application programming interface (AP/), rule-base and case-base. Figure 3 depicts the knowledge-base processing flow, described in detail below. Lotus Notes is Chase Manhattan Bank's corporate email standard; Chase's corporate email router mutes emails to and from the Internet domain and Lotus Notes databases. Accordingly, EZ. Reader was built to operate continuously and automatically in conjunction with Lotus Notes mail functions. The standard replies avaitaige for selection by EZ Reader are stored in a separate Lotus NotCs database. El Reader input and output is perforrned automatically through its connection to Lotus Notes, which was programmed via a Windows*" 3.1 API using the Vendor Independent Messaging (VIM) protocol and the API provided by Lotus Notes. The EZ Reader API performs three important tasks: I. It retrieves an email from the Lotus Notes inbox and returns it lo El Reader. Ir writes EZ Reader's output to either the manual review inbox (referral) or the outbox (auto-reply). It marks the current email in the Lotus Notes inbox as read. Figure 3. EZ Reader internal Processing Flow The AP/ enables El Reader to send its output (the original message and El Reader's chosen response) to targeted Lotus Notes databases that can be vkwed and edited by business users through e customized Lotus Notes interface that lists outputs by category. ChaseDirtet business users have access to the Lotus Notes databases and al/ incoming The API also contains functions to CIISUre tranSaCtiOn integrity in case of connectivity problems. Ordinary Lotus Notes processing tags a message as "read" when an unread mad message is opened for the fust time and then closed. Typically, an API program that retrieves mail will mimic this action by marking a message as read immediately 1510 BR 001255 after retrieval. However, in EZ Reader, message :narking is defened until after the response has been sent to ensure Rule-base. EZ Reader uses mks to represent ChaseUirect dim no data will be lest For eaamp/c, if the EZ Reader client machine loses its connection to the Lotus .eotes server in die middle of processing an email, upon application testart, EZ Reader will again reniese that email and attempt to complete processing. Al Enables Email Classification EZ Reader uses classification rules and case-based reasoning to assign a business category and priority to each incoming email. EZ Reader then uses the infested dassitication to select and attach a standard response from the Lotus Notes database of standard responses. business knowledge about how email content should be inlet-pried and handled. EL Rrader rules siso control application processing flow. EL Reader's mies observe standard ART'Erverprue syntax. These rules are represented in an II uTIIEN format with a left hand side containing a set of conditions and a right hand side containing conc/usions. The rule.s fire whenever the conditions set forth in the left hand side of the rule are met regardless of the sequencing of conditions. The general syntax oía rule follows. RULE RI)/ E-NAMEconehdon, Categories. Using Al techniques described, in the following sections, EZ Reader classifies each incoming email based on total message content into any of the following three categories: Automatic Responsa EZ Reader assigns a category of Automatic Response to items that can be associated with a condition, actions performed if all conditions are true action, actiort2 response from the Lotus Notes repository of standard responses and directly mailed hack to the sender without manual review or revision. Referred EZ Reader assigns a category of Rgerrol messages that cannot be processed sokly as automatic EZ Render then assigns a further subresponses. classification to the message to assist ChaseDireci staff with interprMation later. The sub-classifications reflect ChaseDirect's organization and operations, and are expticted to change over rime. CUITClaiY, tiltre are tWO sub-categories for referred emails: Sales and Service, and An simple example oía rule for detecting foreign phone numbers is shown below. In this example, if the typical fommt of a foreig,n phone number is found in the message body (by calling a ffinction called masked-member) or if other specified keywords are present, then all actions in the right-hand side of the nile will occur, including the printing of the text "Foreign Phone Number detected." 4 /evels of priority. Detected EZ Reader assigns a crnegory of Detected to emails that contain phrases Ot Panents that imPlY 3 It RULE fmgig&-,hone (or (masked-memberS "+99' ?message-body) (masked-memberS 499." ?message-body) (masked-memberS "(+99)" ?message-body) etc. particular manual handling procedure or interpretative aid for a refened message. In these cases EL Reada selects he appropriate remark to attach to incoming mail for manual review. Examples of Detected remarks include: 'detected a phone number", "detected a foreign addres-s4. any other conditions_ a> (printout "Foreign phone number detected.") any other actions .. The left hand sides of the business knowledge rutes in -.-- , , . I Fraud I Ion cards 2 Sensitive info (e.g., occormt number included) Promotional content (e.g., Microsoft Network) Scnd Sign-Up Kit EZ Reader represent key linguistic clues that directly imply interpretive conclusions, including literals, wild card panems, variables and segments, or choices of pattern sets. Fos example, one wild card pattern rule infers the presence of a foreign phone number by looking for patterns of text that resemble a phone number with a preceding plus sign. The inference of a foreign phone number is then used by the case-based search procas to trigger an output Classification. Miscellaneousservice COITIBICOM FYI Multiplequestions or lengthy messages /none/ rabic 1. Referrals - Categories and Priorities Case Studies tsti BR 001256 Case-base. EZ Readet contains a case-base cornpontm iba. enables the applicanon to emulate the reasoning of ChascDate; staff when they use experience to determine .ART'Enrerprise's default algorithm: feature-we-JO/1(Di= Ingram characte matching how so handle an ambiguous email. When rule-base processing fails to clearly identify a classification for an ambiguous incoming email, EZ Reader attempts to find cases that close/y resemble it. If a similar previous email is found, EZ Reader infers that the response used ((tt-Im)/tx) 'mismatch, + OndLx)f match; Where eteviously can be used (or adapted) for the incoming email. Technically, the EZ Reader case-base is a searchabie tm is the number of trigrams in common between the presented case and the stored MSC IX database of emails associated with specific actions and object instances stored within the EZ Reader application. is the total number of trigrams in the presented case realm e It consists of an ARTEnterprise object model of an email, called a Case. The Case object class contains attributes, or slots, for the important I-COMICS Of 'Quails 95 defined by the knowledge used by ChaseDitect to interlacel and respond to the messages. Initial attributes of the Case j mismatch; is the mismatch weight of feature/for the i-th case match; is file match weight of feature f for the MI) case. object include references to addresses, specific types of computers, investment options, etc. The Case object also includes control attributes such as a title for the case instance. Another attribute of the Case object lists the resutrs associated with the case, serving to link the casebase with the rule-base and also to direct Lotus Notes to retrieve specific standard responses. Each actual email samp/e in thc case-base 's defined as an object instance. The standard algorithm works as follows: if the value in a feature of the stored email matches the value in the corresponding Intuit of thc incoming email, the feature's match weight is add to the stored email's score. If the feature's value mismatches, the feature's mismatch weight, typically a negative value, is added to the score. Within the ART'Enterprise development environment, an optimized case-base is prepared for searching by developer functions that create a case-base index, a highly optimized internal data structure that enables stored case and feature values to be matched very quickly with a an input case, called a presented case. EZ Reader searches the case-base assigning relative scores to each stored case based on the number of features, the mismatch of feature values and the absence of features as compared with the presented case using customizable case-based reasoning components supplied in the ARTEnterprise tool. -Character matching with trigrams was chosen to drive case-base scoring in EL Reader. A trigram is a 3character sequence. For example, the word "CY/ASE" C; Cu; CHA; generates 7 consecutive trigrams: HAS; ASE; SE_; E_ _. When character matching is used, In CZ Reader, each attribute, or feanne, used by the case-base was assigned a defau/t match-weight and a customized mismatch-weight of 7.C50. In EZ Reader, the mismatch-weight of zero leads to bener differentiation of scores,, because of the incidence of misspellings in incoming emails, combined with the well-bounded knowledge domain. The actual weight that any feature contributes is meaningful only within the context of a particular case and relative to the weights of other features. Since stored cases can contain different numbers of feanwes, a presented case's raw score is normalized by dividing thc raw score by the maximum possible match score for the case. Mismatches are not entirely ismared by El Reader. Another facton in scoring-cases in El Reader is that a global absence weight is assigned to selected stored cases throughout the case-base. The total contribution of the absence weight to a stored case's match score is calculated the va/ue of the character feature is btoken up into consecutive trigrams, and the Irigrams of a stored case are matched against the trigrams of the presented case. The degree of partial matching is based on the proportion of the trigmms in the presented case that match trigrams in the by multiplying the value by the number of features in the presented case which are not in the stored case. The total absence weight is then added into the raw match score for the case prior to normalization. l'he default absence weight is -I; EZ Reader utilizes an absence weight of Oto reduce the impact of missing feahiseS_ The case-base process is- dependent upon rules to derive stored case. The trigram matching technique minimizes the importance of the order of the individual words in the incoming message. Standard case-base scoring for tht message text of an email (as for all text type features) is driven by its presented case femme values. In El Reader, roles fire before the case-based reasoning process to extract features or characteristics of the email that help distinguish the content of the message. Depending on the content of the 1512 lAA.1-96 BR 001257 message. any of the case-base search features inav be set 'a the pre-processing rule phase. Any features set win Meg atfcc: the storing calculations performed by ar Enterprise 't c.ise.based reasoning engine. Drawing out salient characteristics of the message content using rifes combined with inexact case-based :ctrieval ailows for more powerfu/ and precise email interpretation than simple keyword parsing or case-based retrieval based on message body only. A rule for rietecring an address will fire_ resulting in setting ¡he case attribute address? to -Yes." Next, EZ Reader will perform a search against the ease-base. ranking CASE001 with a score higher than CASE002 because of the match on address?. The emai/ will he referred because the sien-up kit must be scat out via postal mail, and the sender will receive an demonic acknowledgment that their request has been received and that it is being processed. The detected action simply aids FOT exam*, if EZ Reader infers from incoming email test that the sender does not want ro be telephoned by ChaseDirect, the ride for do-not-calietistomea tres and sets that attribute in the case to "Yes'. Features set to "Yes" then contribute lo the case-based search by adding weight for similar stored cases during case-base retrieval. the ChaseDireet staff in quickly determining imponant contents in Me email. Next suppose another person requests the kit but does not include his postal address in the email, in which case the request cannot be fulfilled. The case-base search will result in CASE002 scoring higher and being selected over CA SE001. The sender will then TOCCiVC an automatic standard ChaseDirect response with instructions on how to receive the sigas-up kit. A sample of EZ Reader hybrid processing flow, including the interaction between ink firings and casebase matching, is set forth bc/ow. The importance of set attributes for the case-base search is clearly illustrated in DICSC MO examples. One of the main benefits of case-based retrieval is that Two sample instances of the object class.case are shown below: the cases retrieved from the case-base do not have to match the criteria exactly (as i» the Dat.SSaPP text auribute), but the desired precision of a match can be easily specified. This quality is imponant TO die success of email interpretation. Recatase EZ Reader processes CASE00 title "Sign-Up Kit request; Refer.' subject = "chase direct' mes.sage text "Please send me a Chase/Niter sign.up kit My addiess is" address? - "Yes" action = refer:sign-up-kit. detededmddress, free-format text, it cannot simpty rely on an exact match between tbe incoming message body and the amageleal attribute. As a consequence, some suPerfluous such as the actual address of the message sender, were removed from rnessage text attribute values during the case creation process. Since the casc-based retrieval algorithm performs Ingram matching on the me.ssage text feature, litera/s such as addresses can unintentionally affect a case-base match score. auto:sign-up-8a CA SE002: In addition to the attribute-setting rules described above, EZ Reader's rule-base cc:insists of several "action- fine = Kit request/no address: Auto Respond." subject "clase direct': masage text -Please send me a ChaseDirect sign-up kit." ndion ebase-diremotd setting" tules. The ru/es can detect informatitm that a human readiv may overlook. Some aspects of the customer's email reveal valuable information for ChaseDirect but do not necessarily contribute to the reply. For example, ChaseDirect keeps track-of-prospects' and customers' phone numbers. This information is important to ChaseDireet, but ChoseDirect does not necessarily want Suppose ChaseDireet receives an email with the body of the naessagc as fo/lows: to respond to the customer in a different manner. EZ Reader can tag this email with a message about detecting a phone number, which will consequently be easily seen by staff members when manually reviewing the email. Tbe case-base currently contains over 300 cases; the introduction of more sample emails over time will enable EZ Reader to interpret a wider variety of messages, and precision ['trough further casi feature increase refinement's. Dear ChaseDirect, P/case send the C_ImseDireet Sign-Up Kit to my borne address. Thanks, Jo/ut Doe 123 Elm St. NY, NY 10001 Case Studies 1513 BR 001258 Exeernion Message Handling. There are serne MC5SageS ne.lt EZ Reader b not able te interpret', eg., exception mesvages. The content of a ey.ception message is ar..bigucus even for a knowledgeable person to interpret. eustomer.s. EZ Reader enforced consistency of responses to Before deployment of El Reader, CbaseDirect workers alternated the daily responsibility of reviewing enes_ Each day, a different worker typically "Test". for example, a Menage that cernfaills only the wozd The EX Reader application contains case is able to forward them and all other uninserpretable for manual eXaMpleS er previously zeceived exception messages, so it the whole work day manually leading and responding to email. Consistency of response WaS an important business consideration for ChaseDirect, and spent messages ro the Lotus Notes database evaluation. since quality of response was a function of know/edge and experience, responses frequently had ba be checked by a supervisor. El Reader assured consistency of response because it automatically assigned prepared text depending on its singular interpretation of a message. Error Processing_ ET Reader checks for errors both in its knowledge-base poets-sing and in the API connection to Lotus Notes. Errors generated by the Lotus Notes API do not cause EZ Reader to terminate; instead, EZ Reader will continue trying to access Lotus Notes until a connection is made. For instance, if the Lotus Notes server is El Reader simplified the business process substantially. El Reader enab/ed ChaseDirect to reduce the number of manual steps and the effort needed to process its incoming email. A comparison of processing steps eliminated and modified by EZ Reader is outlined in Table 2 below. temporarily down, thc API will send the relevant error message back to EZ Reader. EZ Reader then waits a few seconds before trying the connection again. In this manner, EZ Reader is self-inonitoring and maintains maximum up-time Tr,aW Dial in to email system Read and analy2e Print and annotate Select/Adjust response litilit OX/t Et1tittitetri'.4:t.._,. , ., itod...9. onc min per access diminatert one minute or more criminate° up to one minute five minutes or more up 10 one minute up co one minute up to one minute aVg: Application Benefits El Reader establishing ChaseDirect's ability lo provide and maintain a responsive online marketing and service channel. The imp/ementation of automated reasoning enabled process a p/ayed critical role in eliminated eliminated/ modified modified modified modified no change Send email simplification, speed and consistency of responses, as described below. El Reader increased the speed of response to the customer. EZ Reader eliminated manual intervention for a percentage of messages' and more than halved the time ro prOCeSS messages requiring manual intervention. The reduction in manual intervention al/owed ChaseDirect to tum around email responses fastet to customers. With a knowledge base processing speed of I message per second Archive original Delete from folder Enter ¡tito CMS' fief minutes Table 2. Process Changes Enabled by EL Reader Application Maintenance Currently, knowledge-base maintenance requires editing of cases mies, and actions, al/ of vvhich are in ART* Enterprise syntax. One proposed technical plus 2-5 seconds for Lotus Notes communications, El Reader reduced overall processing time by 6-8 minutes per mess*. -Another factor contributing to faster customer response was that the application was available on a 29-hour basis, allowing continuous processing of automatic response typcs of messages over a weekend when many customers tended to be active online. enhancement to El Reader is to build a frarneworlc for maintenance in which business usen could add and modify case objects, action objects, and even rules, throupjt a GUI interface whew the underlying ART* Etuerprise syntax of the objects is invisible to the mainbiner. Although specialized knowkdge and skill sets are required to maintain El Reader today, the application is highly modular and object-oriented. The case-base and rule-base are independendy structured. In addition, the 2 around 5 percent of all messages 4 a contact management system used to track contacts with 3 see Automatic Throughput Percentage for a discussion of the effect of message content on average speed of response customers and prospects 1514 IAAI-96 BR 001259 design is general enough to be easily adapted to ether domains, and code-level maintenance procedures are straightfonvard. F_Z Reader maintertance is required whenever certain business et technical environment changes occur. Tbe Project History Chase Manhattan Bank. R egiona! Banking, launched an initiative in 199$ so explore how artificial intell/Renee (Al) could help meet its business char leng,es. Al that time, following types of business changes usually termite EZ Reader maintenance: 17 applications were identified to leverage AI at Chase and El Reader is one of hose implementations. . A new or revised association between a type of message and its prepared response is required. For El Reader started with a concept paper written by members of Chase's Knowledge Base (KB) technology instance, if emails concerning a forrner marketing promotion need to be answered differently than the) team, including Britfitware consultants. bl May 1995. Further development of functional specifications, cost analysis and presentations to management served to get approval for prototype development, which began in July 1995 and was finished in August. Development of a production system was granted approval in September 1995. WM originally, performed. El (Modifications Reader maintenance is to prepared response wording are performed through a Lows Notes edit view.) A new type of message needs to be associated with a prepared response. For instance, if emails concerning a new marketing promotion need to be processed ir automaticany, El Reader maintenance is performed. A new informational message needs to be generated Beginning September 1995, EZ Reader production application development proceeded with one full-time technical devr/oper and several part-lime staff including four business analysts, three testing specialists, and numerous Technical support personae/ who specialized in Lotus Notes and the Chase email network. by EZ. Readtr when it detects a particular type of incomir g message, regardless of the response. For instance, if mails from cement customers need to be Bagged, EZ /leader maintenance is performed. In addition to these knowledge-base maintenance scenarios, whenever a change in thz technical environment around EZ Reader as planned, the application is assessed for required modifications and retested in the target environment. For example, because EZ Reader interfaces with Lotus Notes, if a new version of Lotus Notes is planned for insta/lation, the EZ Reader / Lotus Notes API is tested and assessed for any relevant maintenance. In this case, because the connection to Lotus Notes is independent of the knowledge-base, only the AP/ portion of FZ Reader requires retesting.' The knowledge-base, consisting of the rule-base and case-base, were completed first, enabling user testing while the Lotus Notes API was being developed. Initial knowledge-base testing was performed in November, resulting in accuracy very close to our target level with Miring/min of OTIC ITICSSagt per second. arnbet telinements increased the accuracy to our goal in December. Although the knowledge-base could have been deployed at this point in a semi-manual mode, management decided for a single integrated implementation after completion of the Lotus Notes API. The Lotus Notes API was completed, consolidated and tested with the knowledge-base a few weeks lates. One person from the resident KB teatn at Chase is responsible for maintaining, El Reader, among other duties. The team is trained in ART' Enterprise as. well as EZ Reader was developed using ARTEnterprise version 20h with the included Microsoft Win32s library, C language and Lotus Notes devehipment. They also understand the business requirements of EZ Reader and Visual C/C-4 1.5 atad Lotus Notes version 3.0C all mnning on Windows174 3.1. The application runs on a PC with an Intel-based 486166M1-lz processor, 500MB of hard disk space and 32M13 of RAM. The Lotus Notes server is are able to translate change Mr:teats IMO El Reader know/edge-base modifications. No manual intervention is necessary for the day-to-day operation of El Reader. A local replica of the Lotus Notes databases resides on dm development machine; these databases are used for testing and maintenance of the system. In addition, El Reader can be run in strictly manual mode in which ir bypasses the API connection. Input data in this mode is retrieved from an external text file and allows for quick testing of the knowledge-base independent of the API. an /BM 9595 running 0S12 version 2.1.1. with an Intel.based Pentium 122Mliz processor, 2 hard disk t/rives of 500MB and I GB, and 64MB of RAM. Measuring El Reader Performance EZ Reader was evaluated before implementation to ensure that ChaseDirect production approval criteria were met. To evaluate El Reader's performance, the team analyzed three pleasures: speed, accurracy and automatic throughput percentage. EZ Reader's performance on each measure is described below. CaseStudies 1515 -4 BR 001260 Speed. One esserrial success criterion for 1:2 RzaCer was throu. h EZ Reader. The resit from each were compared lo as6-ess the reaainess of the EZ Reader know/edge-base for production. that it reduce ihe rotal amount of time spent pracess..12 emails. The earliest tests of £2 RCACT blOWiedge base demonstrated speed of one second or less ibi interpretation functions that manually took one minute or more. After Lotus Notes integration, an additittna/ 3-5 seconds was requited for each message. EZ Reader's Business- analysts collect(' electronic copies of the actual incoming messages received during a two-week period as test bed data for the parallel test. EZ Reader developers used actual messages received by ChaseDirect as input data fot testing EZ. Reader. The testers fed the test bed messages electronically into F.7. Reader in its native ART*Enterprise development mode and printed the results. The printouts contained the incoming messages, the EZ. Reader output classifications and the amount of time it took EZ Reader to process the message. The printouts also contained a blank formatted area that was used to record evaluation remarks. TWO reviewers from ChaseDirect who were not involved in the original manual processing stage analyzed the printouts. Where EZ Reader procluced an incorrect Lotus Notes integration reduced the amount of time needed co attach and send responses where manual intervention was still required. Accuracy. ChaseDirect was concerned that any software would be able to interpret messages with the required level of acc-uracy. The estimated accuracy level for manual processing was 98%. The team set El Reader's accuracy goal at a tate equal to that of 95% of the accuracy of manual processing. The case-based reasoning /ogic in the EZ Reader knowledge-base was able to deliver a high level of accuracy. Its capability to rank the degree of similarity between incoming messages and previous messages in its category or response, the reviewer noted the expected response, and the error was reviewed by the EZ Reader team. Itefinemerns to the case-base and rules were made case-base, combined with rules, results in a high level uf accuracy. and verified in subsequent abbreviated tests before the appildation was approved for production. The Lotus Notes API and end-to-end network communications were thoroughly tested over a period of weeks using conventional systems testing techniques. Automatic Percentage. Automatic throughput percentage was a measure established by the team to evaluate the equilibrium between inconiing message content and EZ Reader's knovvledge base. It refers to the percentage of messages that can be processed without manual intervention. As more cases and rules are Throughput added to EZ Reader, the team expects to achieve an automatic throughput percentage of 80% or more. During the early months of initial testing, thc automatic Summary - EL Reader is an M application that provides many tangible and intangible benefits to Chase Manhattan Bank as throughput percentage varied from 20% to 80%. It fell when high .volumes of messages with new, unexpected content were received. It also dropped when business requirements WCTC implemented 10 rckr additional types of messages. Based on a comparison of manual processing it seeks to maximize opportunities in the Internet EZ Reader enab/ed ChascOireei to eliminate the cost of mar k et_ time with El Reader processing time, the automatic throughput percentage translates overtime for email processing, helped meet customer expectations for service standards (such as response timeliness), provided for smoother implementation of ChaseDirect marketing prograrns and enabled unattended processing of email on weekends. to significant productivity gains for each percentage point gained in automatic throughput. Since EZ Reader's knowledge base was developed for ChaseDirect, its initia/ utility was limited to email processing for that department. The following enhancements for EZ Reader are currently being Measures within Lotus Notes ca/culate and track the volume of messages by category (automatic or trienal), date and sender email address. When a decreasing percentage of automatic throughput is detected, it alerts thc business to the need to add new mks or examples to EZ Reader to enable it tu recognize new types of messages sent by customers and prospects. considered: Add business knowledge for other business areas who choose to take advantage of World Wide Web communication with customers. Enable EZ Reader output to Lotus Notes. be addressed Knowledge Base Testing Method To test El Reader's knowledge base, the team perforated automatically to recipients throughout Chase via Automate the business knowledge maintenance functions of the application, i.e.. enable ChaseDired parallel testing to compare F7_ Reader with manual processing, using the same messages both manually and 1516 1AA1-96 BR 001261 business users te interact with EZ Reader :o emerge how irs knowledge base interprets messages anc links them to responses. Provide the ability to auromaticaily process int.-mine messages in Spanish or other languages, providing a potential global marketing advantage. Enable EZ Reader use of historical email and profile data to personalize EZ Reader processinp. Link the contact management system to a process ihat adds customer knowledge from emails processed by EZ Reader. Acknowledgments ART* EnterprzseW is a registered trademark of Brightwar e, Int.. Novato, California. Lotus Nores® is a registered trademark of Lotus Noles Development Corporation. Microsoft® is a registered trademark and Microsoft Word for Windows'," and Windowsm are trademarks of Microsoft Corporation. Other manes rnentioned in this paper may he trademarks and are used for identification purposes only. References Compton, M. and Wo/fe. S. 1993. Intelligent Validation and Routing of Electronic Forms in a Distributed Work F/ow Environment. Technical Repon, FIA-93-31. NASA Ames Research Center, Artificial Intelligence Research Branch. Goodman, M. Prism. 1991. A Case-Based Telex Classifier. hi Innovative Applications of Artificial Intelligence, Vol. 2: AAA' Press. Sahin, K., and Sawyer, K. 1989. The Inte/ligent Banking System: Natural Language Processing for Financial Communications. In Innovative App/ications of Artificial Intelligence; AAAI Press. Taylor, a; Mehra, B.; and Wurster, T. 1996. Online Defivery & the Information Superhighway; Searching for Retail Strategies. Bank Management 72(1): 22-29. Cast Studies 1517 BR 001262

Disclaimer: Justia Dockets & Filings provides public litigation records from the federal appellate and district courts. These filings and docket sheets should not be considered findings of fact or liability, nor do they necessarily reflect the view of Justia.


Why Is My Information Online?