Polaris IP, LLC v. Google Inc. et al

Filing 465

REPLY to Response to Motion re 391 SEALED MOTION for Summary Judgment on Google's Affirmative Defenses and Counterclaims, 389 SEALED MOTION for Summary Judgment on Defendant AOL's Affirmative Defenses and Counterclaims, 390 SEALED MOTION for Summary Judgment on Defendant Yahoo filed by Bright Response LLC. (Attachments: # 1 Affidavit Wiley Declaration, # 2 Exhibit A, # 3 Exhibit B, # 4 Exhibit C, # 5 Exhibit D, # 6 Exhibit E, # 7 Exhibit F, # 8 Exhibit G, # 9 Exhibit H)(Wiley, Elizabeth)

Polaris IP, LLC v. Google Inc. et al

Doc. 465 Att. 3

EXHIBIT B

Dockets.Justia.com

Proceedings of the 11. hirteenth National Conference on Artificial Intelligence and the

Eighth Innovative Applications of Artificial Intelligence Conference

Volume Two

Sponsored by the American Association for Artificial Intelligence

AMI Press / The MIT Press
Menlo Park Cambridge

London

BR 001250

Copyright 0 1996 American Association for Artificial Intelligence 445 Burgess Drive Menlo Park, California 94025
USA

All Rights Reserved.

No pan of this book may be reproduced in any form by any electronic or mechanical means (including photocopying, recording, or information storage and retrieval) without permission in writing from the publisher.

The trademarked terms that appear throughout this book are used in an editorial nature only, to the benefit of the trademark owner. Nu infringement of any tradernarked term is intended.
Copublished and distributed by The MIT Press, Massachusetts Institute of Technology Cambridge, Massachusetts and London, F.ngland
ISBN 0-262-5109I-X

Manufactured in the United States of America on acid-free paper.

BR 001251

EZ Reader: Einbetided M for Automatic Electronic Mad Interpretation and Routin2
Amy Rice and Julie Hsu
Brighrware,

Anthony Angotti and Rosanna Picco/o
Chase Manhattan Bank, N.A.

Abstrart EZ Reader is an intelligent electronic mail (email) tender that employs a unique combination of rulebased

Bank N.A.. Regional Bank, began lo provide electronic banking services using phone and personal computer
technology in 1995. Marketing campaigns advertised that email conld be used ro request information and services, opening a new electronic channel of communication with
customers and prospects.

parsing

and

case-based

reasoning

to

automaticalty and with a high level of accuracy
c/assify and respond to large vo/umes of incoming EZ Reader reduces the time and human email.

resources required to handle incoming email by se/cuing responses and adding attachments and advice tu each incoming message based on how
similar messages were handled. The application, developed for Chase Manhattan Bank using Brightware. Inc.'s ART* Enterprise® tool,
previous
answers
Phase I

The success of its marketing campaigns created a
challenge for ChaseDirect from the beginning to quickly nod cost-effectively process emai/ from multiple sources,

including the Internet, Microsoft Money email, and
another internal DOS-based money manager program with email capability. In addition to ChaseDirect's commimient

emails

automatically

and

decreases

to provide excellent, timely service to its customers,
electronic commerce laws required the bank to respond to certain types of electronic correspondence within specific

processing time for those requinng manual review.

of EZ Reader was deployed in the Ilist

quarter of 1996, and handles up to 80% of incoming mail automatically, depending on mmssage content_ Lain phases will enable BOONTIalk prOCCSSing Of 3 wider variety of messages. By dramatically reducing

time frames. Although niore than 80% of incoming messages were simple requests for product information,
the staff often got backlogged and worked after hours and

the effort associated with manual processing, EZ
Reader will pay its own development costs within six months and will mutt in substantial, recurring dollar savings each year. This paper describes EZ Reader in detail, including its Al-based design, testing, iinplementation and development history.

Problem Description
Like other businesses that sought to expand access to their

on weekends to keep up with the required analysis and responses. Faced with the huge projected 'mercase in Internet email volume due to the planned innoduction of a new World Wide Web server, as illustrated in Figure I below, ChascDirect aggressively sought cost-effective, high-quality ways to process emalls. This urgent business problem attracted attention from Chase's Regional Barak Knowledge Base (KB) technology team. The team's

general charge was to apply artificial intelligence (Al) technology in key areas of the Regional Bank where
appropriate to optimize operationai decisions. To address ChaseDirect's business problem, the Knowledge Base team created El Reader, an embedded AI application operating as an invisible layer between the The Lotus Notes® email system and ChaseDirect. application continuously retrieves incoming Internet email from Chase prospects and customers through an interface to Lotus Notes, and aLso acts as a filtering and routing

products and services through the internet and other
online channels', ChaseDirect, a unit of Chase Manhattan

Contact information follows. Amy Rice: 301 Tresser B/vd., 13th floor, Stamford, CT 06901, rice@ Julie Hsu: 2010 Corporate Ridge, brighrware_com. Suite 700, McLean, VA 22102. hsu@brightwaresom.
Anthony Angolti: One Chase Manhattan Plaza, 19th floor, New York, NY 10081, [e-mail]. Rosanna Piccolo: 15 Er 26th St. New York NY 10010,
[e-mail].

Banks lace significant threats to the retail banking franchise from advances 3/ online banking (Taylor, Mehta & Wurster 1996)

Case Studies

1507

BR 001252

ArCP Enterprise, a commercially successful knowledgebased application development tool. As described below, the Kaowletige Base team considered and rejected several applications and tools for alternative software imp/ementing the reasoning component of EZ Reader. In-house procedural application languages such as COBOL cou/d not handle the reasoning component of the email process because the input to the process was so

complex and unpredictable. An approach of coding
1Tre..

9f
Feb-96

each potential input using procedural methods alone would have been impractically complex.
branches for

Jun-95

Oct-95

Figure I. Projected Genwth of Incoming Email by Source

"fhe Rete algorithm used by ART'Enterprises panera matcher addressed this input complexity issue for EZ Reader. ARTEnterprise's integrated nde scripting language, case-based and object modeling capabilities enabled automated reasoning to reverse OV change
conclusions drawn throughout sequential parsing and ro disambiguate text while ignoring irrelevant portions, such as signature lines, without coding explicit subroutines. Although input was in the form of natural language, the output was a simple classification based on the application of heuristic rides and experience. For this 'reason, syntactic analysis and interpretation, discourse anz/ysis and pragmatic inferencing in commercial natural language processing (NLP) products such as SRA or Logicon, augmented transition networks in custom developed applications such as the Intelligent Banking System (Sahin 8: Sawyer 1989) were vievved as providing only an pre-processor function to the classification reasoning task, and not 'providing a value-added output per se. Tins is illustrated by the fact that the ChaseDireet human email reviewers were able to classify a /arge majority of messages by using a relatively limited set of key linguistic clues that could be expressed as simple mies. Furthermore, when human reviewers had trouble

mechanism, either replying to the email automatically or
attaching a suggested response and refening the message for manual review_ The KB team's overall business goal

for EZ Reader was to reduce the lumber of emails that needed manual processing by more than 80%, with an accuracy rate of 95%. Other goals were to provide rapid turnaround lime for return messages to custorners and prospects and consistency in response>. EZ Reader's
automated reasoning capabilities enabled Chaseffirect to

reach ;hese goals and significantly reduce the manual
effon needed for email processing.

El Reader Design
The ChaseDirect email processing problem offered several

clear opportunities for the implementation of automated
reasoning.

Al in El. Reader
The email review process involved complex reasoning. The process was distinctly knowledge-intensive; __specialists applied dornain-specifie knowledge using

identifying common linguistic clues in a particular
message, they relied upon temn experience io help them
classify- the message and configure or synthesize a response. Compared with the time lo create roles and

heuristics based on know/edge of Chasc products and services. The reliance on knowledge as a process
component was evident as Junior team members routinely
senior

case structures in ART'Enterprise, the time frame to fully analyze the semantics of the text interpretation domain and deveop a comprehensive NLP module vvas deemed

relied upon the experience of their supervisor and other team members to explain how to classify ambiguous messages and forrnulate responses for new classes of messages. Finally, common reasoning tasks were reflected in the process output classrwation of the email and configuration oía response from a limited set of
prepared text modules.

too long to meet business needs, and would have
provided, at best, secondary functionality
al greatly increased cost and technical complexity. Fundamental technical requirements for ama automated

solution included compatibility with existing hardware and standards for run-time and network performance stability; scalability, vendor support and COMOICICiai
Wide/y-available
shareware,

EZ Reader's Al reasoning component is a data-driven forward-chaining rule parser operating in concert with case-based reasoning written in Brightwart, Inc.'s

proprietary

intelligent programs specifically written for electronic
forms processing (Compton & Wolfe 1993) and packaged

email ronterhesponder software such as Prolog-based

1508

/AA1-96

BR 001253

-^1

:deiltor

these reasons.

(Daimon Laboratories, inc.) were re¿ecleci for in contras!, die ART' Enter prtse tool had

6

L.t CUSP:M.1QT itttiVeS ChaseDireer's email reply

airrady been deployed in another application at Chase where broad functionality and customizability, product reliability and vendor support necessary for fie:ding
commercial applications was previously validated. Since the content, number and ordering of concepts in incoming emails was unpredictable, automatic processing coulti not easily be accomplished by conventional procedural programming techniques. Thc full-featured Al capabilities of the in-house knowledge-based development tool afforded a quickly implementable, one-toot technique

Custarme

Message

Rerponre

Domain
Smte r

for transforming the linguistic dues in emails to output
classifications.
ChaseDirect Lotus Notes Seam

Process Flow
Figure 2_ illustrates the flow pian email throng)] the EZ Reader system as described below:
1.

The customer sends an email to Chase Manhattan
Bank's Internet address_

Oulbox

helms

2.

Chase's corporate email router passes the message Traci the domain server to ChaseDirect's Lotus Notes
server_

3.

EZ Reader periodically chccks the inbox (a Lotus Notes mail database) for new mail. When a new email arrives in the inbox, EZ Reader retrieves the
message and "interprets" ir by performing rule-based parsing and case-based retrieval. The outcome of its imerprelation is one of two possibi/itics: EZ Reader can respond to the email automatically. An automatic response, which is routed directly to the ChaseDirect outbox,

consists of the original email and one or more attachments, or prepared replies, that are

retrieved from a Lotus Notes repository of
standard responses.

EZ Reader cannot respond to
automatically.

the

It refers

the

email email to
Figure 2_ Email Path through Cturscrirrecr

Chaseriireri _for human review and response Before placing tht email in the manual review

inbox, EZ Reader assigns a category and
priority to the message and suggests one or more standard replies based on message
(Categories and priorities are content. described in more detail later.) ChaseDirect specialists review and write responses to all messages referred by EZ Reader and place these electronic replies in the outbox_ C_hasc's corporate email router routes the message

EZ Reader Hybrid IC nowledge Base Approach
EZ Reader's hybrid reasoning approach reflects the actual interpretation process used by human email reviewers in

4.

5.

ChaseDirect. The application combines pre-processing rules for parsing and case-based retrieval with a domainspecific knowledge-base. Other text inteapretadon applicatioas have successfully used -a hybrid approach (Sabin & Sawyer 1989) (Goodman 1991). A hybrid A/

from amseDirect's Lotus Notes mail management system and places it in Chase's domain server for
reply back through thc internet.

design provided both a functional and manageable
programmatic representation of the business knowledge and rules for email interpretation.

CaseStudies

isoF

BR 001254

'I he combined-re/es and case-based approach was fiat evaluated after iatt team knowledge engineer observed and

entails within the Lotus Notes 3ystem. so EZ Reader does

analyzed the ChaseDirect email interpretation pocess. Human email reviewers read each message from

not iraerfere with Lotus Notes as the standard Chase platform for entail-related word processing, archivin:!,
reference., and reporting functions.

beginning lo end while continuously evolving a final
interpretation. The email EtVieWtt ICCUTSIVely applied business knowledge to message content throughout the review_ Reviewers modified their conclusions throughout
I
I

CANCEL

the review, since an email contained any number of
concepts in unpredictable order.

START

t Break
IMT/ALIZE
1

The application emulates the recursive nature of evolving interpretation by firs/ detecting combinations of

continuous processing anytime

prominent words and patterns of text in any order
throughout an incoming message, then sening object anribme values that both trigger and influence the casebased reasoning process. The application's case-based
reasoning process then provides data to the rule-base lo infer a classification by comparing the message content
against the repository of mas.sages in the case-base.

processing
NO

Internal Processing Flow
Within El Reader, program flow is controlled through the

firing of declarative mies which trigger, monitor and
control proccssing in the application programming interface (AP/), rule-base and case-base. Figure 3 depicts

the knowledge-base processing flow, described in detail
below. Lotus Notes is Chase Manhattan Bank's corporate email

standard; Chase's corporate email router mutes emails to and from the Internet domain and Lotus Notes databases. Accordingly, EZ. Reader was built to operate continuously

and automatically in conjunction with Lotus Notes mail functions. The standard replies avaitaige for selection by
EZ Reader are stored in a separate Lotus NotCs database.

El Reader input and output is perforrned automatically through its connection to Lotus Notes, which was

programmed via a Windows*" 3.1 API using the Vendor

Independent Messaging (VIM) protocol and the API
provided by Lotus Notes. The EZ Reader API performs three important tasks:

I. It retrieves an email from the Lotus Notes inbox and
returns it lo El Reader.

Ir writes EZ Reader's output to either the manual
review inbox (referral) or the outbox (auto-reply). It marks the current email in the Lotus Notes inbox as
read.

Figure 3. EZ Reader internal Processing Flow

The AP/ enables El Reader to send its output (the original message and El Reader's chosen response) to targeted Lotus Notes databases that can be vkwed and edited by business users through e customized Lotus Notes interface that lists outputs by category. ChaseDirtet business users have access to the Lotus Notes databases and al/ incoming

The API also contains functions to CIISUre tranSaCtiOn integrity in case of connectivity problems. Ordinary Lotus
Notes processing tags a message as "read" when an unread mad message is opened for the fust time and then closed.

Typically, an API program that retrieves mail will mimic this action by marking a message as read immediately

1510

BR 001255

after retrieval. However, in EZ Reader, message :narking is defened until after the response has been sent to ensure

Rule-base. EZ Reader uses mks to represent ChaseUirect

dim no data will be lest For eaamp/c, if the EZ Reader client machine loses its connection to the Lotus .eotes server in die middle of processing an email, upon application testart, EZ Reader will again reniese that
email and attempt to complete processing.

Al Enables Email Classification
EZ Reader uses classification rules and case-based reasoning to assign a business category and priority to each incoming email. EZ Reader then uses the infested
dassitication to select and attach a standard response from the Lotus Notes database of standard responses.

business knowledge about how email content should be inlet-pried and handled. EL Rrader rules siso control application processing flow. EL Reader's mies observe standard ART'Erverprue syntax. These rules are represented in an II uTIIEN format with a left hand side containing a set of conditions and a right hand side containing conc/usions. The rule.s fire whenever the conditions set forth in the left hand side of
the

rule are met

regardless

of the sequencing of

conditions. The general syntax oía rule follows.

RULE RI)/ E-NAMEconehdon,

Categories. Using Al techniques described, in the following sections, EZ Reader classifies each incoming email based on total message content into any of the following three categories: Automatic Responsa EZ Reader assigns a category of
Automatic Response to items that can be associated with a

condition,
actions performed if all conditions are true action,
actiort2

response from the Lotus Notes repository of standard
responses and directly mailed hack to the sender without

manual review or revision. Referred EZ Reader assigns a category of Rgerrol messages that cannot be processed sokly as automatic EZ Render then assigns a further subresponses. classification to the message to assist ChaseDireci staff with interprMation later. The sub-classifications reflect ChaseDirect's organization and operations, and are expticted to change over rime. CUITClaiY, tiltre are tWO
sub-categories for referred emails: Sales and Service, and

An simple example oía rule for detecting foreign phone numbers is shown below. In this example, if the typical
fommt of a foreig,n phone number is found in the message

body (by calling a ffinction called masked-member) or if other specified keywords are present, then all actions in the right-hand side of the nile will occur, including the
printing of the text "Foreign Phone Number detected."

4 /evels of priority.
Detected EZ Reader assigns a crnegory of Detected to

emails that contain phrases Ot Panents that imPlY 3
It

RULE fmgig&-,hone (or (masked-memberS "+99' ?message-body) (masked-memberS 499." ?message-body) (masked-memberS "(+99)" ?message-body)
etc.

particular manual handling procedure or interpretative aid for a refened message. In these cases EL Reada selects

he appropriate remark to attach to incoming mail for manual review. Examples of Detected remarks include:
'detected a phone number", "detected a foreign addres-s4.

any other conditions_
a>

(printout "Foreign phone number detected.") any other actions ..

The left hand sides of the business knowledge rutes in
-.--

,

,

.

I

Fraud I Ion cards

2

Sensitive info (e.g., occormt number included)

Promotional content (e.g., Microsoft Network) Scnd Sign-Up Kit

EZ Reader represent key linguistic clues that directly imply interpretive conclusions, including literals, wild card panems, variables and segments, or choices of pattern sets. Fos example, one wild card pattern rule
infers the presence of a foreign phone number by looking for patterns of text that resemble a phone number with a preceding plus sign. The inference of a foreign phone number is then used by the case-based search procas to trigger an output Classification.

Miscellaneousservice
COITIBICOM FYI

Multiplequestions
or lengthy messages

/none/

rabic 1. Referrals - Categories and Priorities

Case Studies

tsti

BR 001256

Case-base.

EZ Readet contains a case-base cornpontm

iba. enables the applicanon to emulate the reasoning of ChascDate; staff when they use experience to determine

.ART'Enrerprise's default algorithm:
feature-we-JO/1(Di=

Ingram characte matching

how so handle an ambiguous email. When rule-base processing fails to clearly identify a classification for an ambiguous incoming email, EZ Reader attempts to find
cases that close/y resemble it. If a similar previous email is found, EZ Reader infers that the response used

((tt-Im)/tx)

'mismatch, + OndLx)f match;
Where

eteviously can be used (or adapted) for the incoming
email. Technically, the EZ Reader case-base is a searchabie

tm is the number of trigrams in common
between the presented case and the stored
MSC
IX

database of emails associated with specific actions and object instances stored within the EZ Reader application.

is the total number of trigrams in the

presented case realm e

It consists of an ARTEnterprise object model of an
email, called a Case.
The Case object class contains
attributes, or slots, for the important I-COMICS Of 'Quails 95 defined by the knowledge used by ChaseDitect to interlacel and respond to the messages. Initial attributes of the Case

j mismatch;

is the mismatch weight of feature/for the i-th case match; is file match weight of feature f

for the MI) case.

object include references to addresses, specific types of computers, investment options, etc. The Case object also

includes control attributes such as a title for the case instance. Another attribute of the Case object lists the
resutrs associated with the case, serving to link the casebase with the rule-base and also to direct Lotus Notes to retrieve specific standard responses. Each actual email
samp/e in thc case-base 's defined as an object instance.

The standard algorithm works as follows: if the value in a feature of the stored email matches the value in the corresponding Intuit of thc incoming email, the feature's match weight is add to the stored email's score. If the feature's value mismatches, the feature's mismatch
weight, typically a negative value, is added to the score.

Within the ART'Enterprise development environment, an optimized case-base is prepared for searching by
developer functions that create a case-base index, a highly optimized internal data structure that enables stored case and feature values to be matched very quickly with a an input case, called a presented case. EZ Reader searches the case-base assigning relative scores to each stored case based on the number of features, the mismatch of feature values and the absence of features as compared with the presented case using customizable case-based reasoning components supplied in the ARTEnterprise tool. -Character matching with trigrams was chosen to drive case-base scoring in EL Reader. A trigram is a 3character sequence. For example, the word "CY/ASE" C; Cu; CHA; generates 7 consecutive trigrams: HAS; ASE; SE_; E_ _. When character matching is used,

In CZ Reader, each attribute, or feanne, used by the case-base was assigned a defau/t match-weight and a customized mismatch-weight of 7.C50. In EZ Reader, the mismatch-weight of zero leads to bener differentiation of scores,, because of the incidence of misspellings in incoming emails, combined with the well-bounded knowledge domain. The actual weight that any feature contributes is meaningful only within the context of a

particular case and relative to the weights of other
features. Since stored cases can contain different numbers of feanwes, a presented case's raw score is normalized by

dividing thc raw score by the maximum possible match
score for the case.

Mismatches are not entirely ismared by El Reader. Another facton in scoring-cases in El Reader is that a
global absence weight is assigned to selected stored cases throughout the case-base. The total contribution of the absence weight to a stored case's match score is calculated

the va/ue of the character feature is btoken up into
consecutive trigrams, and the Irigrams of a stored case are matched against the trigrams of the presented case. The

degree of partial matching is based on the proportion of the trigmms in the presented case that match trigrams in
the

by multiplying the value by the number of features in the presented case which are not in the stored case. The total absence weight is then added into the raw match score for the case prior to normalization. l'he default absence weight is -I; EZ Reader utilizes an absence weight of Oto
reduce the impact of missing feahiseS_ The case-base process is- dependent upon rules to derive

stored

case. The trigram matching technique

minimizes the importance of the order of the individual
words in the incoming message. Standard case-base scoring for tht message text of an email (as for all text type features) is driven by

its presented case femme values. In El Reader, roles fire
before the case-based reasoning process to extract features

or characteristics of the email that help distinguish the
content of the message. Depending on the content of the

1512

lAA.1-96

BR 001257

message. any of the case-base search features inav be set 'a the pre-processing rule phase. Any features set win Meg atfcc: the storing calculations performed by

ar Enterprise 't c.ise.based reasoning engine. Drawing out salient characteristics of the message content using rifes combined with inexact case-based :ctrieval ailows for more powerfu/ and precise email interpretation than simple keyword parsing or case-based retrieval based on
message body only.

A rule for rietecring an address will fire_ resulting in setting ¡he case attribute address? to -Yes." Next, EZ Reader will perform a search against the ease-base. ranking CASE001 with a score higher than CASE002 because of the match on address?. The emai/ will he
referred because the sien-up kit must be scat out via postal mail, and the sender will receive an demonic acknowledgment that their request has been received and that it is being processed. The detected action simply aids

FOT exam*, if EZ Reader infers from incoming email test that the sender does not want ro be telephoned by ChaseDirect, the ride for do-not-calietistomea tres and sets that attribute in the case to "Yes'. Features set to "Yes" then contribute lo the case-based search by adding
weight for similar stored cases during case-base retrieval.

the ChaseDireet staff in quickly determining imponant contents in Me email. Next suppose another person
requests the kit but does not include his postal address in the email, in which case the request cannot be fulfilled.

The case-base search will result in CASE002 scoring
higher and being selected over CA SE001. The sender will then TOCCiVC an automatic standard ChaseDirect response with instructions on how to receive the sigas-up kit.

A sample of EZ Reader hybrid processing flow, including the interaction between ink firings and casebase matching, is set forth bc/ow. The importance of set attributes for the case-base search is clearly illustrated in
DICSC MO examples.

One of the main benefits of case-based retrieval is that

Two sample instances of the object class.case are shown below:

the cases retrieved from the case-base do not have to match the criteria exactly (as i» the Dat.SSaPP text auribute), but the desired precision of a match can be
easily specified. This quality is imponant TO die success of email interpretation. Recatase EZ Reader processes

CASE00 title "Sign-Up Kit request; Refer.'
subject = "chase direct'
mes.sage text

"Please send me a

Chase/Niter sign.up kit My addiess is" address? - "Yes" action = refer:sign-up-kit. detededmddress,

free-format text, it cannot simpty rely on an exact match between tbe incoming message body and the amageleal attribute. As a consequence, some suPerfluous such as the actual address of the message sender, were removed from rnessage text attribute values during the case creation process. Since the casc-based retrieval algorithm performs Ingram matching on the me.ssage text
feature, litera/s such as addresses can unintentionally affect a case-base match score.

auto:sign-up-8a
CA SE002:

In addition to the attribute-setting rules described
above, EZ Reader's rule-base cc:insists of several "action-

fine =

Kit request/no address:

Auto Respond." subject "clase direct': masage text -Please send me a ChaseDirect sign-up kit." ndion ebase-diremotd

setting" tules. The ru/es can detect informatitm that a human readiv may overlook. Some aspects of the customer's email reveal valuable information for
ChaseDirect but do not necessarily contribute to the reply. For example, ChaseDirect keeps track-of-prospects' and customers' phone numbers. This information is important to ChaseDireet, but ChoseDirect does not necessarily want

Suppose ChaseDireet receives an email with the body of
the naessagc as fo/lows:

to respond to the customer in a different manner. EZ
Reader can tag this email with a message about detecting a phone number, which will consequently be easily seen by staff members when manually reviewing the email. Tbe case-base currently contains over 300 cases; the introduction of more sample emails over time will enable EZ Reader to interpret a wider variety of messages, and precision ['trough further casi feature increase
refinement's.

Dear ChaseDirect, P/case send the C_ImseDireet Sign-Up Kit to my borne address. Thanks, Jo/ut Doe 123 Elm St.

NY, NY 10001

Case Studies

1513

BR 001258

Exeernion Message Handling. There are serne MC5SageS

ne.lt EZ Reader b not able te interpret', eg., exception mesvages. The content of a ey.ception message is
ar..bigucus even for a knowledgeable person to interpret.

eustomer.s.

EZ Reader enforced consistency of responses to Before deployment of El Reader,

CbaseDirect workers alternated the daily responsibility of reviewing enes_ Each day, a different worker typically

"Test".

for example, a Menage that cernfaills only the wozd The EX Reader application contains case
is able to forward them and all other uninserpretable for manual

eXaMpleS er previously zeceived exception messages, so it

the whole work day manually leading and responding to email. Consistency of response WaS an important business consideration for ChaseDirect, and
spent

messages ro the Lotus Notes database evaluation.

since quality of response was a function of know/edge and experience, responses frequently had ba be checked by a

supervisor. El Reader assured consistency of response
because it automatically assigned prepared text depending on its singular interpretation of a message.

Error Processing_ ET Reader checks for errors both in its knowledge-base poets-sing and in the API connection to Lotus Notes. Errors generated by the Lotus Notes API do not cause EZ Reader to terminate; instead, EZ Reader will continue trying to access Lotus Notes until a connection is made. For instance, if the Lotus Notes server is

El

Reader simplified the business process substantially. El Reader enab/ed ChaseDirect to reduce

the number of manual steps and the effort needed to process its incoming email. A comparison of processing
steps eliminated and modified by EZ Reader is outlined in Table 2 below.

temporarily down, thc API will send the relevant error
message back to EZ Reader. EZ Reader then waits a few

seconds before trying the connection again. In this manner, EZ Reader is self-inonitoring and maintains
maximum up-time

Tr,aW
Dial in to email system
Read and analy2e Print and annotate Select/Adjust response

litilit OX/t
Et1tittitetri'.4:t.._,. , ., itod...9. onc min per access diminatert one minute or more criminate°
up to one minute five minutes or more up 10 one minute up co one minute up to one minute
aVg:

Application Benefits
El Reader
establishing ChaseDirect's ability lo provide and maintain a responsive online marketing and service channel. The imp/ementation of automated reasoning enabled process
a

p/ayed

critical role

in

eliminated
eliminated/ modified modified modified modified
no change

Send email

simplification, speed and consistency of responses, as
described below.

El Reader increased the speed of response to the
customer. EZ Reader eliminated manual intervention for a percentage of messages' and more than halved the time ro prOCeSS messages requiring manual intervention. The reduction in manual intervention al/owed ChaseDirect to tum around email responses fastet to customers. With a knowledge base processing speed of I message per second

Archive original Delete from folder Enter ¡tito CMS'

fief minutes

Table 2. Process Changes Enabled by EL Reader

Application Maintenance
Currently, knowledge-base maintenance requires editing of cases mies, and actions, al/ of vvhich are in ART* Enterprise syntax. One proposed technical

plus 2-5 seconds for Lotus Notes communications, El
Reader reduced overall processing time by 6-8 minutes per mess*. -Another factor contributing to faster customer response was that the application was available on a 29-hour basis, allowing continuous processing of automatic response typcs of messages over a weekend when many customers
tended to be active online.

enhancement to El Reader is to build a frarneworlc for maintenance in which business usen could add and

modify case objects, action objects, and even rules,
throupjt a GUI interface whew the underlying ART* Etuerprise syntax of the objects is invisible to the

mainbiner. Although specialized knowkdge and skill sets are
required to maintain El Reader today, the application is highly modular and object-oriented. The case-base and rule-base are independendy structured. In addition, the

2

around 5 percent of all messages
4 a contact management system used to track contacts with

3 see Automatic Throughput Percentage for a discussion

of the effect of message content on average speed of

response

customers and prospects

1514

IAAI-96

BR 001259

design is general enough to be easily adapted to ether domains, and code-level maintenance procedures are
straightfonvard. F_Z Reader maintertance is required whenever certain business et technical environment changes occur. Tbe

Project History
Chase Manhattan Bank. R egiona! Banking, launched an

initiative in 199$ so explore how artificial intell/Renee
(Al) could help meet its business char leng,es. Al that time,

following types of business changes usually termite EZ
Reader maintenance:

17 applications were identified to leverage AI at Chase
and El Reader is one of hose implementations.

.

A new or revised association between a type of
message and its prepared response is required.
For

El Reader started with a concept paper written by
members of Chase's Knowledge Base (KB) technology

instance, if emails concerning a forrner marketing
promotion need to be answered differently than the)

team, including Britfitware consultants. bl May 1995. Further development of functional specifications, cost
analysis and presentations to management served to get approval for prototype development, which began in July 1995 and was finished in August. Development of a production system was granted approval in September
1995.

WM originally,
performed.

El

(Modifications

Reader maintenance is to prepared response

wording are performed through a Lows Notes edit view.)
A new type of message needs to be associated with a prepared response. For instance, if emails concerning

a new marketing promotion need to be processed
ir

automaticany, El Reader maintenance is performed. A new informational message needs to be generated

Beginning September 1995, EZ Reader production application development proceeded with one full-time technical devr/oper and several part-lime staff including four business analysts, three testing specialists, and
numerous Technical support personae/ who specialized in Lotus Notes and the Chase email network.

by EZ. Readtr when it detects a particular type of
incomir g message, regardless of the response. For instance, if mails from cement customers need to be Bagged, EZ /leader maintenance is performed. In addition to these knowledge-base maintenance scenarios, whenever a change in thz technical environment around EZ Reader as planned, the application is assessed

for required modifications and retested in the target
environment. For example, because EZ Reader interfaces

with Lotus Notes, if a new version of Lotus Notes is
planned for insta/lation, the EZ Reader / Lotus Notes API is tested and assessed for any relevant maintenance. In this case, because the connection to Lotus Notes is independent of the knowledge-base, only the AP/ portion of FZ Reader requires retesting.'

The knowledge-base, consisting of the rule-base and case-base, were completed first, enabling user testing while the Lotus Notes API was being developed. Initial knowledge-base testing was performed in November, resulting in accuracy very close to our target level with Miring/min of OTIC ITICSSagt per second. arnbet telinements increased the accuracy to our goal in December. Although the knowledge-base could have been deployed at this point in a semi-manual mode, management decided for a single integrated implementation after completion of the Lotus Notes API. The Lotus Notes API was completed, consolidated and
tested with the knowledge-base a few weeks lates.

One person from the resident KB teatn at Chase is responsible for maintaining, El Reader, among other
duties. The team is trained in ART' Enterprise as. well as

EZ Reader was developed using ARTEnterprise
version 20h with the included Microsoft Win32s library,

C language and Lotus Notes devehipment. They also
understand the business requirements of EZ Reader and

Visual C/C-4 1.5 atad Lotus Notes version 3.0C all
mnning on Windows174 3.1. The application runs on a PC with an Intel-based 486166M1-lz processor, 500MB of hard disk space and 32M13 of RAM. The Lotus Notes server is

are able to translate change Mr:teats IMO El Reader
know/edge-base modifications. No manual intervention is necessary for the day-to-day

operation of El Reader. A local replica of the Lotus
Notes databases resides on dm development machine; these databases are used for testing and maintenance of the system. In addition, El Reader can be run in strictly manual mode in which ir bypasses the API connection. Input data in this mode is retrieved from an external text file and allows for quick testing of the knowledge-base
independent of the API.

an /BM 9595 running 0S12 version 2.1.1. with an Intel.based Pentium 122Mliz processor, 2 hard disk t/rives of 500MB and I GB, and 64MB of RAM.

Measuring El Reader Performance
EZ Reader was evaluated before implementation to ensure

that ChaseDirect production approval criteria were met. To evaluate El Reader's performance, the team analyzed three pleasures: speed, accurracy and automatic throughput percentage. EZ Reader's performance on each
measure is described below.

CaseStudies

1515

-4

BR 001260

Speed. One esserrial success criterion for 1:2 RzaCer was

throu. h EZ Reader. The resit from each were compared
lo as6-ess the reaainess of the EZ Reader know/edge-base for production.

that it reduce ihe rotal amount of time spent pracess..12 emails. The earliest tests of £2 RCACT blOWiedge base demonstrated speed of one second or less ibi interpretation functions that manually took one minute or more. After Lotus Notes integration, an additittna/ 3-5 seconds was requited for each message. EZ Reader's

Business- analysts collect(' electronic copies of the actual incoming messages received during a two-week period as test bed data for the parallel test. EZ Reader
developers used actual messages received by ChaseDirect as input data fot testing EZ. Reader. The testers fed the test bed messages electronically into F.7. Reader in its native ART*Enterprise development mode and printed the results. The printouts contained the incoming messages, the EZ. Reader output classifications and the amount of time it took EZ Reader to process the message. The printouts also contained a blank formatted area that was used to record evaluation remarks. TWO reviewers from ChaseDirect who were not involved in the original manual processing stage analyzed the printouts. Where EZ Reader procluced an incorrect

Lotus Notes integration reduced the amount of time needed co attach and send responses where manual
intervention was still required.

Accuracy. ChaseDirect was concerned that any software

would be able to interpret messages with the required level of acc-uracy. The estimated accuracy level for manual processing was 98%. The team set El Reader's

accuracy goal at a tate equal to that of 95% of the
accuracy of manual processing.

The case-based reasoning /ogic in the EZ Reader knowledge-base was able to deliver a high level of accuracy. Its capability to rank the degree of similarity
between incoming messages and previous messages in its

category or response, the reviewer noted the expected response, and the error was reviewed by the EZ Reader
team. Itefinemerns to the case-base and rules were made

case-base, combined with rules, results in a high level uf
accuracy.

and verified in subsequent abbreviated tests before the
appildation was approved for production. The Lotus Notes API and end-to-end network communications were thoroughly tested over a period of weeks using conventional systems testing techniques.

Automatic

Percentage. Automatic throughput percentage was a measure established by the team to evaluate the equilibrium between inconiing message content and EZ Reader's knovvledge base. It refers to the percentage of messages that can be processed without manual intervention. As more cases and rules are

Throughput

added to EZ Reader, the team expects to achieve an
automatic throughput percentage of 80% or more. During the early months of initial testing, thc automatic

Summary
-

EL Reader is an M application that provides many
tangible and intangible benefits to Chase Manhattan Bank
as

throughput percentage varied from 20% to 80%. It fell when high .volumes of messages with new, unexpected content were received. It also dropped when business
requirements WCTC implemented 10 rckr additional types of messages. Based on a comparison of manual processing

it seeks to maximize opportunities in the Internet
EZ Reader enab/ed ChascOireei to eliminate the cost of

mar k et_

time with El Reader processing time, the automatic
throughput
percentage
translates

overtime for email processing, helped meet customer expectations for service standards (such as response timeliness), provided for smoother implementation of
ChaseDirect marketing prograrns and enabled unattended processing of email on weekends.

to

significant

productivity gains for each percentage point gained in
automatic throughput.

Since EZ Reader's knowledge base was developed for
ChaseDirect, its initia/ utility was limited to email processing for that department. The following enhancements for EZ Reader are currently being

Measures within Lotus Notes ca/culate and track the volume of messages by category (automatic or trienal), date and sender email address. When a decreasing percentage of automatic throughput is detected, it alerts thc business to the need to add new mks or examples to
EZ Reader to enable it tu recognize new types of messages sent by customers and prospects.

considered:
Add business knowledge for other business areas who

choose to take advantage of World Wide Web
communication with customers. Enable EZ Reader output to
Lotus Notes.

be

addressed

Knowledge Base Testing Method
To test El Reader's knowledge base, the team perforated

automatically to recipients throughout Chase via
Automate the business knowledge maintenance functions of the application, i.e.. enable ChaseDired

parallel testing to compare F7_ Reader with manual
processing, using the same messages both manually and

1516

1AA1-96

BR 001261

business users te interact with EZ Reader :o emerge how irs knowledge base interprets messages anc links
them to responses.

Provide the ability to auromaticaily process int.-mine messages in Spanish or other languages, providing a potential global marketing advantage. Enable EZ Reader use of historical email and profile
data to personalize EZ Reader processinp. Link the contact management system to a process ihat adds customer knowledge from emails processed by EZ Reader.

Acknowledgments
ART* EnterprzseW
is
a

registered

trademark

of

Brightwar e, Int.. Novato, California. Lotus Nores® is a registered trademark of Lotus Noles Development Corporation. Microsoft® is a registered trademark and

Microsoft Word for Windows'," and Windowsm are
trademarks of Microsoft Corporation.
Other manes rnentioned in this paper may he trademarks and are used for identification purposes only.

References
Compton, M. and Wo/fe. S. 1993. Intelligent Validation and Routing of Electronic Forms in a Distributed Work F/ow Environment. Technical Repon, FIA-93-31. NASA Ames Research Center, Artificial Intelligence Research
Branch.

Goodman, M. Prism. 1991. A Case-Based Telex Classifier. hi Innovative Applications of Artificial Intelligence, Vol. 2: AAA' Press.
Sahin, K., and Sawyer, K. 1989. The Inte/ligent Banking System: Natural Language Processing for Financial Communications. In Innovative App/ications of Artificial
Intelligence; AAAI Press.

Taylor, a; Mehra, B.; and Wurster, T.

1996.

Online

Defivery & the Information Superhighway; Searching for Retail Strategies. Bank Management 72(1): 22-29.

Cast Studies

1517

BR 001262