Apple Inc. v. Samsung Electronics Co. Ltd. et al

Filing 563

EXHIBITS re 559 Declaration in Support, filed byApple Inc.. (Attachments: # 1 Exhibit 5.02, # 2 Exhibit 5.03, # 3 Exhibit 5.04, # 4 Exhibit 5.05, # 5 Exhibit 5.06, # 6 Exhibit 5.07, # 7 Exhibit 5.08, # 8 Exhibit 5.09, # 9 Exhibit 5.10, # 10 Exhibit 5.11, # 11 Exhibit 6.1, # 12 Exhibit 6.2, # 13 Exhibit 6.3, # 14 Exhibit 6.4, # 15 Exhibit 6.5)(Related document(s) 559 ) (Jacobs, Michael) (Filed on 12/29/2011)

Download PDF
EXHIBIT 5.03 claims 8-14 are directed to statutory subject matter and request the removal of the rejection of these claims under 35 U.S.C. §101. Allowable Subiect Matter The Office Action indicates that claims 1-7 are allowed. Applicants thank the Examiner for the allowance of claims 1-7. New independent claim 89 includes similar limitations as independent claim 1. Thus, applicants request the allowance of independent claim 89 and associated dependent claims 90-95. Conclusion Applicant respectfully submits that in view of the amendments and discussion set forth herein, the applicable rejections have been overcome. Accordingly, the present and amended claims should be found to be in condition for allowance. If a telephone interview would expedite the prosecution of this application, the Examiner is invited to contact the undersigned at (408) 720-8300, if there are any additional charges/credits, please charge/credit our deposit account no. 02-2666. Respectfully submitted, BLAKELY, SOKOLOFF, TAYLOR & ZAFMAN LLP Dated: ' J R . Scheller, Jr. . No. 31,195 Customer No. 45217 1279 Oakmead Parkway Sunnyvale, CA 94085 (408) 720-8300 Ser No.: 11/620,717 Page 9 of 9 Dkt No: 4860P4895 APLNDC00027256 Electronic Acknowledgement Receipt EFS ID: 6232207 Application Number: 11620717 International Application Number: Confirmation Number: 9801 Title of Invention: Application Programming Interfaces for Scrolling Operations First Named Inventor/Applicant Name: Andrew Platzer Customer Number: 45217 Filer: James Scheller Jr./Connie Thayer Filer Authorized By: James Scheller Jr. Attorney Docket Number: 04860.P4895 Receipt Date: 08-OCT-2009 Filing Date: 07-JAN-2007 Time Stamp: 22:01:41 Application Type: Utility under 35 USC 111(a) Payment information: Submitted w th Payment no File Listing: Document Number . Document Description i File Name File Size(Bytes)/ . Message Digest Multi Pages Part /.zip (if appl.) 262432 1 04860P4895_amendment.PDF yes 9 b8175fe7b82267f3bf788f78f30f3d568dec4 3d9 APLNDC00027257 Multipart Description/PDF files in .zip description Document Description Start End Amendment/Req. Reconsideration-After Non-Final Reject 1 1 Specification 2 2 Claims 3 7 Applicant Arguments/Remarks Made in an Amendment 8 9 Warnings: Information: Total Files Size (in bytes) 262432 This Acknowledgement Receipt evidences receipt on the noted date by the USPTO of the indicated documents, characterized by the applicant, and including page counts, where applicable. It serves as evidence of receipt similar to a Post Card, as described in MPEP 503. New Annlications Under 35 U.S.C. 111 If a new application is being filed and the application includes the necessary components for a filing date (see 37 CFR 1.53(b)-(d) and MPEP 506), a Filing Receipt (37 CFR 1.54) will be issued in due course and the date shown on this Acknowledgement Receipt will establish the filing date of the application. National Staae of an International Annlication under 35 U.S.C. 371 If a timely submission to enter the national stage of an international application is compliant with the conditions of 35 U.S.C. 371 and other applicable requirements a Form PCT/DO/EO/903 indicating acceptance of the application as a national stage submission under 35 U.S.C. 371 will be issued in addition to the Filing Receipt, in due course. New International Annlication I¯iled with the USPTO as a Receivina Office If a new international application is being filed and the international application includes the necessary components for an international filing date (see PCT Article 11 and MPEP 1810), a Notification of the International Application Number and of the International Filing Date (Form PCT/RO/105) will be issued in due course, subject to prescriptions concerning national security, and the date shown on this Acknowledgement Receipt will establish the international filing date of the application. APLNDC00027258 PTO/SBIO6 (07-06) Approved for use through 1/31/2007. OMB 0651-0032 U.S. Patent and Trademark Office; U.S. DEPARTMENT OF COMMERCE Under the Paperwork Reduction Act of 1995 no persons are required to respond to a collection of information unless it displavs a valid OMB control number. PATENT APPLICATION FEE DETERMINATION RECORD Application or Docket Number Filing Date Substitute for Form PTO-875 11/620,717 01/07/2007 APPLICATION AS FILED - PART I To be Mailed OTHER THAN (Column 1) (Column 2) NUMBER FILED NUMBER EXTRA RATE ($) N/A N/A N/A N/A SEARCH FEE N/A N/A N/A N/A EXAMINATION FEE N/A N/A N/A N/A FOR BASIC FEE (37 CFR 1.16 a b), or (c)) (37 CFR 1.16(k), (i), or (m)) SMALL ENTITY Q OR FEE ($) SMALL ENTITY RATE ($) FEE ($) (37 CFR 1.16(o), (p), or (q)) TOTAL CLAIMS (37 CFR 1.16(i)) minus 20 = (37 CFR 1.16(h)) APPLICATION SIZE FEE (37 CFR 1.16(s)) * X $ = minus 3 = INDEPENDENT CLAIMS * X $ OR = X $ = X $ = If the specification an, drawings exceed 100 sheets of paper, the application size fee due is $250 ($125 for small entity) for each additional 50 sheets or fraction thereof. See 35 U.S.C. 41(a 1 G) and 37 CFR 1.16(s) MULTIPLE DEPENDENT CLAIM PRESENT (37 CFR 1.16(j)) * If the difference in column 1 is less than zero, enter "0" in column 2. TOTAL TOTAL APPLICATION AS AMENDED - PART II OTHER THAN (Column 1) (Column 2) I CLAIMS REMAINING 10/08/2009 AFTER AMENDMENT (Column 3) HIGHEST NUMBER PREVIOUSLY PAID FOR PRESENT EXTRA SMALL ENTITY RATE ($) OR ADDITIONAL FEE ($) SMALL ENTITY RATE ($) x o= ADDITIONAL FEE ($) 0 Application Size Fee (37 CFR 1.16(s)) FIRST PRESENTATION OF MULTIPLE DEPENDENT CLAIM (37 CFR 1.16(j)) OR TOTAL ADD'L FEE (Column 1) (Column 2) HIGHEST NUMBER PREVIOUSLY PAID FOR PRESENT EXTRA TOTAL ADD'L FEE 0 (Column 3) CLAIMS REMAINING AFTER AMENDMENT OR RATE ($) ADDITIONAL FEE ($) RATE ($) x$ = OR x$ = x$ = OR x$ ADDITIONAL FEE ($) = Application Size Fee (37 CFR 1.16(s)) < OR O FIRST PRESENTATION OF MULTIPLE DEPENDENT CLAIM (37 CFR 1.16(j)) TOTAL ADD'L FEE * If the entry in column 1 is less than the entry in column 2, write "0" in column 3. OR TOTAL ADD'L FEE Legal Instrument Examiner: ** If the "Highest Number Previously Paid For" IN THIS SPACE is less than 20, enter "20". /PATSY ZIMMERMAN/ *** If the "Highest Number Previously Paid For" IN THIS SPACE is less than 3, enter"3". The "Highest Number Previously Paid For" (Total or Independent) is the highest number found in the appropriate box in column 1. This collection of information is required by 37 CFR 1.16. The information is required to obtain or retain a benefit by the public which is to file (and by the USPTO to process) an application. Confidentiality is governed by 35 U.S.C. 122 and 37 CFR 1.14. This collection is estimated to take 12 minutes to complete, including gathering, preparing, and submitting the completed application form to the USPTO. Time will vary depending upon the individual case. Any comments on the amount of time you require to complete this form and/or suggestions for reducing this burden, should be sent to the Chief Information Officer, U.S. Patent and Trademark Office, U.S. Department of Commerce, P.O. Box 1450, Alexandria, VA 22313-1450. DO NOT SEND FEES OR COMPLETED FORMS TO THIS ADDRESS. SEND TO: Commissioner for Patents, P.O. Box 1450, Alexandria, VA 22313-1450. If you need assistance in completing the form, call 1-800-PTO-9199 and select option 2. APLNDC00027259 PTO/SBIO6 (07-06) Approved for use through 1/31/2007. OMB 0651-0032 U.S. Patent and Trademark Office; U.S. DEPARTMENT OF COMMERCE Under the Paperwork Reduction Act of 1995 no persons are required to respond to a collection of information unless it displavs a valid OMB control number. PATENT APPLICATION FEE DETERMINATION RECORD Application or Docket Number Filing Date Substitute for Form PTO-875 11/620,717 01/07/2007 APPLICATION AS FILED - PART I To be Mailed OTHER THAN (Column 1) (Column 2) NUMBER FILED NUMBER EXTRA RATE ($) N/A N/A N/A N/A SEARCH FEE N/A N/A N/A N/A EXAMINATION FEE N/A N/A N/A N/A FOR BASIC FEE (37 CFR 1.16 a b), or (c)) (37 CFR 1.16(k), (i), or (m)) SMALL ENTITY Q OR FEE ($) SMALL ENTITY RATE ($) FEE ($) (37 CFR 1.16(o), (p), or (q)) TOTAL CLAIMS (37 CFR 1.16(i)) minus 20 = (37 CFR 1.16(h)) APPLICATION SIZE FEE (37 CFR 1.16(s)) * X $ = minus 3 = INDEPENDENT CLAIMS * X $ OR = X $ = X $ = If the specification an, drawings exceed 100 sheets of paper, the application size fee due is $250 ($125 for small entity) for each additional 50 sheets or fraction thereof. See 35 U.S.C. 41(a 1 G) and 37 CFR 1.16(s) MULTIPLE DEPENDENT CLAIM PRESENT (37 CFR 1.16(j)) * If the difference in column 1 is less than zero, enter "0" in column 2. TOTAL TOTAL APPLICATION AS AMENDED - PART II OTHER THAN (Column 1) (Column 2) I CLAIMS REMAINING 10/08/2009 AFTER AMENDMENT (Column 3) HIGHEST NUMBER PREVIOUSLY PAID FOR PRESENT EXTRA SMALL ENTITY RATE ($) OR ADDITIONAL FEE ($) SMALL ENTITY RATE ($) x o= ADDITIONAL FEE ($) 0 Application Size Fee (37 CFR 1.16(s)) FIRST PRESENTATION OF MULTIPLE DEPENDENT CLAIM (37 CFR 1.16(j)) OR TOTAL ADD'L FEE (Column 1) (Column 2) HIGHEST NUMBER PREVIOUSLY PAID FOR PRESENT EXTRA TOTAL ADD'L FEE 584 (Column 3) CLAIMS REMAINING AFTER AMENDMENT OR RATE ($) ADDITIONAL FEE ($) RATE ($) x$ = OR x$ = x$ = OR x$ ADDITIONAL FEE ($) = Application Size Fee (37 CFR 1.16(s)) < OR O FIRST PRESENTATION OF MULTIPLE DEPENDENT CLAIM (37 CFR 1.16(j)) TOTAL ADD'L FEE * If the entry in column 1 is less than the entry in column 2, write "0" in column 3. OR TOTAL ADD'L FEE Legal Instrument Examiner: ** If the "Highest Number Previously Paid For" IN THIS SPACE is less than 20, enter "20". /ANNETTE SMITH/ *** If the "Highest Number Previously Paid For" IN THIS SPACE is less than 3, enter"3". The "Highest Number Previously Paid For" (Total or Independent) is the highest number found in the appropriate box in column 1. This collection of information is required by 37 CFR 1.16. The information is required to obtain or retain a benefit by the public which is to file (and by the USPTO to process) an application. Confidentiality is governed by 35 U.S.C. 122 and 37 CFR 1.14. This collection is estimated to take 12 minutes to complete, including gathering, preparing, and submitting the completed application form to the USPTO. Time will vary depending upon the individual case. Any comments on the amount of time you require to complete this form and/or suggestions for reducing this burden, should be sent to the Chief Information Officer, U.S. Patent and Trademark Office, U.S. Department of Commerce, P.O. Box 1450, Alexandria, VA 22313-1450. DO NOT SEND FEES OR COMPLETED FORMS TO THIS ADDRESS. SEND TO: Commissioner for Patents, P.O. Box 1450, Alexandria, VA 22313-1450. If you need assistance in completing the form, call 1-800-PTO-9199 and select option 2. APLNDC00027260 Document code: WFEE United States Patent and Trademark Office Sales Receipt for Accounting Date: 04/20/2010 ASMITH3 SALE #00000001 Mailroom Dt: 10/08/2009 022666 01 FC : 1201 220.00 DA 02 FC : 1202 364.00 DA 11620717 APLNDC00027261 Attorney's Docket No. 4860P4895 PATENT IN THE UNITED STATES PATENT AND TRADEMARK OFFICE In Re Application of: Andrew Platzer Examiner: Bautista, Xiomara L Application No.: 11/620,717 Art Unit: Filed: Confirmation No.: 9801 January 7, 2007 2179 For: APPLICATION PROGRAMMING INTERFACES FOR SCROLLING OPERATIONS Mail Stop Amendment Commissioner for Patents P.O. Box 1450 Alexandria, Virginia 22313-1450 INFORMATION DISCLOSURE STATEMENT Sir: Enclosed is a copy of Information Disclosure Citation Form PTO-1449 or PTO/SB/08 together with copies of the documents cited on that form, except for copies not required to be submitted (e.g., copies of U.S. patents and U.S. published patent applications need not be enclosed for applications filed after June 30, 2003). It is respectfully requested that the cited documents be considered and that the enclosed copy of Information Disclosure Citation Form PTO-1449 or PTO/SB/08 be initialed by the Examiner to indicate such consideration and a copy thereof returned to applicant(s). I hereby certify that this correspondence is being submitted electronically via EFS Web to the United States Patent and Trademark Office on the date shown below. (Date of Deposit) Jeremy A Schwegert (Typed or printed name of person mailing correspondence) /JeremyA. Schweig_ert/ (Signature of person mailing correspondence) Application No.: 11/620,717 - 1 - 10-13-2009 (Date) Docket No.: 4860P4895 APLNDC00027262 Pursuant to 37 C.F.R. § 1.97, the submission of this Information Disclosure Statement is not to be construed as a representation that a search has been made and is not to be construed as an admission that the information cited in this statement is material to patentability. Pursuant to 37 C.F.R. § 1.97, this Information Disclosure Statement is being submitted under one of the following (as indicated by an "X" to the left of the appropriate paragraph): 37 C.F.R. §1.97(b). X 37 C.F.R. §1.97(c). If so, then enclosed with this Information Disclosure Statement is _o_n_e of the following: A statement pursuant to 37 C.F.R. §1.97(e) Œ X The Director is Authorized to charge our Deposit Acct. No. 022666 in the amount of $180.00 for the fee under 37 C.F.R. § 1.17(p). 37 C.F.R. §1.97(d). If so, then enclosed with this Information Disclosure Statement are the following: (1) A statement pursuant to 37 C.F.R. §1.97(e); and (2) Authorization to charge our Deposit Acct. No. 02-2666 in the amount of $180.00 for the fee under 37 C.F.R. §1.17(p) for submission of the Information Disclosure Statement. If there are any additional charges, please charge Deposit Account No. 02-2666. Respectfully submitted, BLAKELY, SOKOLOFF, TAYLOR & ZAFMAN LLP Dated: nos 12 onn Jeremy A. Schwegrt Jeremy A. Schweigert Reg. No. 56,244 1279 Oakmead Parkway Sunnyvale, CA 94085-4040 (408) 720-8300 Application No.: 11/620,717 -2- Docket No.: 4860P4895 APLNDC00027263 t Europäisches Patentamt ¿ (19) European Patent Office Office européen des brevets (12) (11) EP 1 517 228 A2 EUROPEAN PATENT APPLICATION (43) Date of pubücation: 23.03.2005 Buitetin 2005/12 (51) int Cli: GO6F 3/033 (21) Appucation number: 04255568.0 (22) Date of finng: 15.09.2004 (84) Designated Contracting States: AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT Li LU MC NL PL PT RO SE Si SK TR Designated Extension States: AL HR LT LV MK • Morrison, Gerald D. Calgary, Alberta T3G 4T6 (CA) (30) Priority: 16.09.2003 US 662813 (74) Representative: Naismith, Robert Stewart et at Marks & Clerk Scotland 19 Royal Exchange Square (71} Applicant: Smart Technologies, Inc. Calgary, Alberta T3C OMS (CA) (54) (72) Inventors: • Hill, Douglas B. Calgary, Alberta T2K 1Z2 (CA) Glasgow, G1 3AE Scotland (GB) Gesture recognition method and touch system incorporating the same (57) A gesture recognition method includes detecting multiple pointers in close proximity to a touch surface to determine if the multiple pointers are being used to are being used to perform a known gesture, acommand associated with the gesture is executed. A touch system incorporating the gesture recognition method is also perform a known gesture, When the multiple pointers provided, 60 FlG. 4a co CL LLI Panted ey Jowe soo ens em (Cont next page) APLNDC00027264 EP 1 517 228 A2 FIG 4b FIG. 4c o 60 FIG. 4e 2 )OCID:<EP (Cont. next page) 1517228A2 1 > APLNDC00027265 EP 1 517 228 A2 FIG. 4f 3 APLNDC00027266 1 EP 1 517 228 A2 Description sociated with the digital cameras to determine if a pointer exists in the captured image data. When it is determined that a pointer exists in the captured image data, Field of the invention [0001] The present invention relates generally to touch systems and in particular to a gesture recognition s method and touch system incorporating the same. Background of the Invention 10 [0002) Touch systems are well known in the art and typically include a touch screen having a touch surface on which contacts are made using a pointer in order to generate user input. Pointer contacts with the touch surface are detected and are used to generate correspond- ing output depending on areas of the contact surface where the contacts are made. There are basicany two general types of touch systems available and they can be broadly classified as "active" touch systems and "passive" touch systems, [0003) Active touch systems allow a user to generate user input by contacting the touch surface with a special 2 15 the digital signal processors generate pointer information packets (PIPs) and convey the PIPs to a mâster controffer. Each PiP includes a header portion, a data portion and a checksum. The data portion includes a pointer ID field that stores a pointer identif a allow multiple pointers to be tracked The data p n also includes a pointer location parameter that dies a pointer x-position and a pointer tip parameter that ídentifies a pointer z-position. A contact state field stores a value indicating whetherthe pointer is in or out of contact with the touch sudace allowing pointer hover to be detected. [0007] Upon receipt of the PIPs, the master controßer processes the PIPs using triangulation to determine the location of each pointer in the captured images relative power source, typically batteries. The special pointer emits signals such as infrared light, visible light, ultra- to the touch surface in (x,y) coordinates, in this manner, 20 as PIPs are generated in response to captured images, the position and movement of pointers over the touch surface can be tracked. The pointer location data generated by the master controller is conveyed to a computer executing one or more application programs. The 25 computer uses the pointer location data to update the sonic frequencies, electromagnetic frequencies, etc. computer-generated image that is presented on the that activate the touch surface. [0004) Passive touch systems allow a user to generate user input by contacting the touch surface with a touch surface. Pointer contacts on and pointer movement over the touch surface can therefore be recorded as writing or drawing or used to control execution of ap30 plication programs executed by the computer. pointer that usually requires some form of on-board passive pointer and do not require the use of a special pointer in orderto activate the touch surface. The pointer (0008] can be a finger, a cylinder of some material, or any suitable object that can be used to contact some predetermined area of interest on the touch surface. [DOOS) Passive touch systems provide advantages over active touch systems in that any suitable pointing device, including a user's finger, can be used as a point- As will be appreciated, since digital cameras at the comers of the bezeln are used to capture image data, the touch system is saa to determine when mul- tiple pointers contact and move across the touch sur35 face. This of course provides for enhanced functionality as compared to analog resistive touch systems that are only able to track a single pointer. Although enhanced functionality is provided by the above-described camera-based touch system, to-date, this enhanced functionally has not been fully exploited. It is therefore an object of the present invention to provide a novel gesture recognition method and touch system incorporating the er to contact the touch surface. As a result, user input can easily be generated, Also. since special active pointers are not necessary in passive touch systems, 40 battery power levels and/or pointer damage, theft, or misplacement are of no concern to users. [0006] For example, U.S. Patent Application No. same. 09/610481 filed on July 5, 2000 and Intemational PCT Application No. PCT/CA01/00980 filed on July 5, 2001 45 Summarv of the Invention and published under No, WO 02/03316 on January 10, 2002, assigned to SMART Technologies Inc., assignee [0009] According to one aspect of the present invenof the present invention, disclose a camera-based touch tion there is provided a gesture recognition method comsystem comprising a touch screen that includes a pasprising the steps of: sive touch surface on which a computer-generated im- 50 age is presented. A rectangular bezei or frame surdisplaying an image on a touch surface; rounds the touch surface and supports digital cameras detecting pointer contacts on said touch surface at its comers. The digital cameras have overlapping and examining said pointer contacts to recognize fields of view that encomcass and look across the touch multiple pointer contacts representing a gesture; surface. The digital cameras acquire images looking 55 and across the touch surface from different locations and when multiple pointer contacts representing a gesgenerate image data, image data acquired by the digital ture occur updating the displayed image in accordcameras is processed by digital signal processors asance with said gesture. 4 APLNDC00027267 3 (0010) EP 1 517 228 A2 4 Multiple pointer contacts representing a ges- at least one processor communicating with said at ture include multiple finger contacts on the touch surface, a fingercontact on the touch surface and an object least one imaging device and analyzing images acquired by said at least one imaging device to deter- contact on the touch surface and multiple object con- minethelocationonsaidtouchsurfacewherepoint- tacts on the touch sudace, (0011] in one aspect, the gesture is a right-click event 5 and is represented by a first pointer contact on a displayed applicatiort and a subsequent second pointer contact that occurs within athreshold distance of the first pointer contact and while the first pointer contact is maintained. In another aspect, the gesture is a scroll event and is represented by simultaneous pointer contacts on the touch surface. (0012) According to another aspect of the present invention there is provided a gesture recognition method comprising the steps of: to is 20 ciated with said gesture. [0013] Preferably, during the detecting, pointer contacts with or close pointer hovers overthe touch surface amining said multiple pointer contacts to determine if said multiple poinîer contacts represent a gesture and when said multiple pointer contacts represent agesture,saidprocessorexecutingacommandassociated with said gesture [0016] According to still yet another aspect of the present invention there is provided an interactive input system comorising: at least one imaging device having an input region within its field of view into which one or more pointers is moved to generate user input; and detecting multiple pointers in close proximity to a touch surface to determine if said multiple pointers are being used to perform a known gesture; and when said multiple pointers are being used to perform a known gesture, executing a command asso- er contacts are made. when said touch surface is contacted by multiple pointers, said processor ex- at least one processor cornrnunicatirig with said at least one imaging device and analyzing each image acquired by said at least one imaging demce to determine the action of pointers in said input region, 25 said at least one processor determining when multiple pointer actions within said input region represent a gesture, when said multiple pointer actions are detected to determine if a known gesture is being represent a gesture, said at least one processor experformed and specifically if one of a number of known ecuting a command corresponding to said gesture. gestures is being performed, each known gesture being associated with a different command, in a preferred em- 30 [0017) According to still yet another aspect of the bodiment, the movement of the multiple pointers relative present invention there is provided in an interactive to the touch sudace and/or the pointer type determines touch system, a method of simulating a right-click the gesture being performed. mouse event comprising the steps of: [0014] According to yet another aspect of the present invention there is provided an input detection method in an interactive system capable of detecting movement of multiple pointers generally simultaneously within an input region, said method comprising the steps of: 35 capturing images looking generally across said input region; analyzing said images to detect multiple pointers within said input region; when multiple pointers are detected, examining dala associated with said multiple pointers to determine if the data represents an input gesture; and 40 45 when the data represents an input gesture, executing acommand corresponding to the recognized in- detecting a first pointer contact on a touch surface over a displayed application that represents a leftclick mouse event: detecting a second pointer contact on said touch surface that occurs within a threshold period of time fonowing said first pointer contact and within a threshold distance of said first pointer contact; and generating a right-click mouse event in response to said detected second pointer contact, (0018] The present invention provides advantages in that since gestures representedbymultiplepointercon- tacts on and/or movement over the touch surface are detected and corresponding commands generated, en- put gesture. hanced touch system functionality is provided. 50 [0015) According to still yet another aspect of the present invention there is provided a touch system compasing a touch surface to be contacted by at îeast one pointer; at least one imaging device having a field of view looking generany across said touch surface: and Brief Description of the Drawings SS [0019] Embodiments of the present invention will now be described more fully with reference to the accompanying drawings in which: Figure 1 is a schematicdiagram of acamera-based touch system in accordance with the present inven- 5 APLNDC00027268 5 EP 1 517 228 A2 6 essors of the digital cameras 70 generate pointer information packets(PIPs) and convey the PIPs to the digital signal processor (DSP) based master controller 54. tion: Figure 2 is a front elevation view of a touch screen forming part of the touch system of Figure 1; Figures 3a to 3d are front perspective views of the touch screen of Figure 2 showing an input right-click gesture; and Figures 4a to 4f are front perspective views of the touch screen of Figure 2 showing input up/down and left/right scroli gestures. 5 to Detaned Descriotion of the Preferred Embodiments Each PIP includes a header portion, a data portion and a checksum. The data portion includes a pointer (D field that stores a pointer identifier to anow multiple pointers lo be tracked. The data porbon also includes a pointer location parameter that identifies a pointer x-position and a pointer tip parameter that identifies a pointer zpositîon. A contact state field stores a value indicating whether the pointer is in ðr out of contact with the touch surface 60 allowing pointer hover to be detected. (0020] Turning now to Figure i, a camera-based touch system in accordance with the present invention is shown and is generally identifËed by reference numeral 50. Camera-based touch system 50 is similar to that disclosed in International PCT Application Serial No. WO 02/03316, assigned to SMART Technologies Inc. assignee of the present invention, the contents of which is [0023] Upon receipt of the PîPs, the master controller 54 processes the PIPs using triangulation to determine the location of each pointer in the captured images relative to the touch surface 60 in (x,y) coordinates. In this manner, as PfPs are generated in response to captured images, the position and movement of pointers over the touch surface 60 can be tracked. Since image data is are incorporated herein by reference. As can be seen, 20 processed to detect the existence of one or more point. touch system 50 includes a touch screen 52 coupled to ers, the pointers may take any suitable form such as for a digitai signal processor (DSP) based master controller example, a user's finger, a cylinder of material, a passive 54. Master controller 54 is also coupied to a computer or active pen tool or erase tool or other appropriate ob56. Computer 56 executes one or more application project. Specifics of the marmer by which the image data is grams and provides computer-generated image output 2s acquired by the digital cameras 70 and processed by that is displayed on the touch screen 52. The coordinate the master controller 54 are described in Intemational systern of the touch system 52 is mapped to the coordiPCT Application No. PCT/CA01/00980 fued on July 5, nate system of the computer. The touch screen 52, mas2001 and published under No, WO 02/03316 on Januter controller 54 and computer 56 form a closeddoop so ary 10, 2002, the contents of which are incorporated that pointer hover or contacts with and pointer move- so herein by reference. Accordingly, such specifies will not ment overor above the touch screen 52 can be recorded be described further herein. as writing or drawing or used to control execution of ap[0024] The master controller 54 outputs generated pHcation programs executed by the computer 56. pointer data to the computer 56 that identifies the loca[002¶ Figure 2 better illustrates the touch screen 52. tion of each pointer relative to the touch surface as each Touch screen 52 in the present embodiment includes a 35 pointer approaches and/or contacts and moves over the high-resolution display device such as a plasma display touch surface 60, A driver loaded on the computer 56 58, the front sudace of which defines a touch surface 60. The touch surface 60 is bordered by an inuminated receives the pointer data and examines the pointer data to determine if the pointer data has been generated in bezel or frame 62 coupled to the display device, illuminated bezel 62 is of the type disciosed in U,S. Patent Application No. 10/354,168 filed on January 30, 2003 and includes elongate side frame assemblies 64 that are coupled to the sides of the plasma display 58. Each side frame assembly 64 accommodates a light source (not shown) that projects infrared backlighting across the touch surface 60. The ends of the side irame assemblies 64 are joinedbycomerpieces 68 that house DSP-based CMOS digital cameras 70. Each digital camera 70 Ís mounted within its res: comer piece 68 so that its field of view encompa ad looks across the entire touch surface 60, [0022] During operato .ne digital cameras 70 acquire images of the touch surface 60 and generate image data. The acquired image data is processed by digital signal processors associated with the digital cameras 70 to determine if a pointer exists in the captured images. When it is determined that one or more pointers exist in the acquired irnage data, the digital signal proc- response to a known input gesture stored in a gesture 40 45 library. Specifically, the driver examines the pointer data to detect the existence of multiple pointers in captured images and then examines the nature of the multiple pointers to determine if a known gesture has been performed such as forexample a right-click gesture, a scroll gesture, a rotate gesture etc. When a gesture has been performed, the driver generates a comrnand (event) that is associated with the determined gesture and conveys the command to the active application program being executed by the computer 56, so [0025] Turning now to Figures 3a to 4f, examples of gestures that can be recognized by the touch system andthe resulting actions that are performed in response to the recognized gestures are shown. 56 intuitive Right-Click Gesture [0026] Figures 3a to 3d illustrate an intuitive right-elick gesture. When a user contacts the touch surface 60 with 6 )O <EP 151722BA2 i > APLNDC00027269 7 EP i 517 228 A2 5 closely and generally horizontally spaced, the driver recognizesthesimultaneousfingercontactsasapagegesture and injects a page event into the application. Pointer position data conveyed to the application by the driver in response to subsequent vertical movement of the fin- 10 gers is interpreted by theapplication as page up or page down commands depending on the direction of the verBeat movement. 10 response to the page up or page down commands, the application moves information displayed within the window in the appropriate direction. a finger over an application displayed on the touch surface, the driver recognizes the contact as a left-click mouse event and injects the left-click mouse event into tne appHcation. If the user subsequently contacts the touch surface 60 with another finger while rnaintaining the contacl with the one finger and the subsequent con- tact is to the right. of and close to the initial contact, the driver recognizes the second touch surface contact as a right-click gesture and injects a right-click event into the application. In response to the right-cGck event. the application opens and displays a drop down menu (not shown). As will be appreciated, this enables a user to invoke a right-click action using a hand gesture that is similar to the action performed when invoking a rightclick action using a mouse. Although, Figures 3a to 3d show the irituitive right-click gesture being performed using two fingers on the same hand, it will be appreci- Rotate Gesture is (0027] Figures 4a to 4e illustrate up/down and left/ right scroli gestures. If the user contacts the touch sur- face 60 with a pair of fingers simultaneously over an application window displayed on the touch surface and the fingers are closely and generally horizontally spaced, the driver recognizes the simultaneous finger contacts as a scroll gesture and injects a scroll event into the application. Pointer position data conveyed to the applica- tion by the driver in response to subsequent vertical movement of the fingers is interpreted by the application either as scroll up or scron down commands, in response to the scron up or down comrnands, the appilcation moves informatíon presented within the application window in the direction of the vedical .movement. Pointer position data conveyed to the application by the driver in response to subsequent horizontal movement of the fingers is interpreted by the application as scroll to side commands. In response to the scroll to side commands, the application moves infonnation displayed within the application window to the side corresponding to the direction of the horizontal movement, Although Figures 4a to 4f show the scroli gestures being performed using two fingers on the same hand. it will be appreciated that the scroll gestures can be perfonmed using a finger on different hands. [0028) Although not illustrated, anumberof other gestures can be recognized by the driver and used to generate commands to control an application being executed by the computer56, Examples of such other gestures will now be described. Page Up/Down Gesture (0029) if the user contacts the touch surface 60 with three fingers simultaneously overan application window dísplayed on the touch surface and the threetingers are (0030) If the user contacts the touch surface 60 over an object displayed within an application window with one finger and then subsequently contacts the touch surface with another finger and moves that other finger in an arc whue maintaining the touch surface contact ated that the right-click gesture can be performed using a finger on different hands Scroll Un/Down and Left/Riciht Gesture 8 with the one finger. the driver recognizes the arcuate 20 movement of the second finger as a rotate gesture. The driver in turn injects a rotate command into the application causing the application to rotate the object about the contact point defined by the first finger in the direction of the arc and by an amount equivalent to the path 25 of the arc. Zoom Gesture (0031] If the user contacts the touch surface 60 with so a pair of closely spaced fingers simultaneously over an application window and expands the distance between the fingers in a generally horizontal direction, the driver recognizes the finger movement as a zoom-out gesture, The driver in tum injects a zoom-out command into the 35 application causing the application to expand the size of the information presented in the application window. If the user contacts the touch surface 60 with a pair of spaced fingers simultaneously over an application window and moves the fingers in a generally horizontal di4o rection towards one another, the driver recognizes the finger movement as a zoom-in gesture. The driver in turn injects a zoom-in command into the application causing the application to reduce the size of the information presented in the application window. 45 [0032] Afternatively, the zoom-out and zoom-in commands may be represented by other gestures. For example, if the user contacts the touch surface 60 with a clawed hand having its fingers bunched together over an appücation window and expands the hand by extendso ing the fingers outwardly. the driver recognizes the finger movement as the zoom-out esture. [0033] If the user contacts the touch surface 60 with a generally flat hand having its fingers extended over an application window and contracts the hand by clawing 55 the fingers inwardly to bunch them together the driver recognizes the finger movement as the zoom-in gesture. 7 APLNDC00027270 9 EP 1 517 228 A2 Expand Gesture 10 scrolled. Pointers contacting the touch surface 60 at a steep angle may represent a slow scroll rate whereas [0034] If the user contacts the touch surface 60 with a pair of closely spaced fingers simultaneously over an application wîndow and expands the distance between pointers contacting the touch surface 60 at a shattow angle may represent a fast scroll rate. 5 (0040) if the touch system is able to differentiate between the type of pointers used to contact the touch surface 60 as is described in co-pending U.S. Patent Application Serial No. 10/384,783 filed on March 11, 2003 and/or is able to determine pointer characteristics as is described in co-pending U.S. Patent Application Serial No. 10/294,917, fited on November 15, 2002, the contents of which are incorporated herein by reference, different functionalíty can be assigned to similar gestures that are performed using different pointers. For exampie, in the case of the rotate gesture described above. If the same gesture is carried out using a fingerto initially the fingers in a generally diagonal direction, the driver recognizes the fînger rnovement as an increase window size gesture. The driver in turn injects an increase window size command into the application causing the application to expand the size of the application window, if the user contacts the touch surface 60 with a pair of spacod fingers simultaneously over an application window and moves the fingers in a generally diagonal direction towards one another, the driver recognizes the finger movement as a decrease window size gesture, The driver in turn injects a decrease window size command into the application causing the application to re- 10 15 contact an object within the application window and a pen tool to describe the are, the driver recognizes the finger contact and pen movement as a pattern fill gesIcon Select and Open Gesture 20 ture rather than a rotate gesture. A finger contact and subsequent closely spaced pen tool contact may repre[0035] If the user contacts the touch surface 60 with sent a draw circle gesture rather than a scroli gesture two closely spaced fingers simultaneously over an icon, and a finger contact and subsequent closely spaced the driver recognizes the finger contact as a double-click eraser contact may represent an erase page gesture, gesture. The driver in turn generates an open applica- 25 As will be appreciated, being able to differentiate betion command causing the desktop application running tween multiple pointers brought into proximity with the on the computer 56 to open the selected application. touch surface 60 significantly increases the functions that may be invoked by performing the same gestures Obiect/Window Move Gesture simply by using discrete pointers that can be differenti30 ated. [0036] If the user moves a pair of closely spaced fin(0041] Although the driver is described as examming gers above the touch surface 60 and over an object disthe pointer data to determine if the pointer data is genplayed within an application window, the driver recogerated in response to a known gesture, it will be apprenizes the hovering finger movement as a translate obciated by those of skill in the art that if the active appliject gesture. The driver in tum injects a translate object as cation being executed by the computer has the capabilcommand into the application causing the application to ity of recognizing gestures, the pointer data may be contranslate the displayed object in the direction of and by veyed to the active application for gesture recognition, an amount equal to the distance the fingers are moved. [0042] If desired the touch surface 60 may be partä [0037] ff the user moves three closely spaced fingers tioned into multiple regions to enable multiple users to above the touch surface 60 and over an application win- 40 interact with the touch surface simultaneously without dow, the driver recognizes the hovering finger moveambiguity between user input. In this case multiple conment as a translate window gesture. The driver in turn tacts on or hovers over the touch surface that are beduce the size of the application window, generates a translate window command causing the desktop application running on the computer 56 to trans- late the application window in the direction of and by an amount equal to the distance the fingers are moved, (0038] 45 Although the above gestures are described as being recognized in responseto multiple finger contacts or hovers, the same gestures can be recognized if other objects are used to perform the gestures. For example, 50 multiple pen tools can be used to pedorm the gestures or alternatively a finger and a pen tool can be used to perform the gestures. [OC 9] Also, recognized gestures may be enhanced U p different pointer characteristics. For e×ample, in 55 the case of scroll gestures, the angle at which the point- yond a threshold distance are treated as multiple user inputs. Multiple contacts on or hovers over the touch surface that are within the threshold distance are treated as multiple contacts made by a single user and are examined to determine if the multiple contacts represent a gesture. [0043] Although preferred embodiments of the present invention have been described, those of skill in the art will appreciate that variations and modifications may be made without departing from the spirit and scope thereof as defined by the appended claims. Claims ers contact the touch surface 60 may be used to deter- mine the rate at which the displayed inforrnation is i A gesture recognition method comprising the steps 8 CO <EP 1517228A2 i > APLNDC00027271 11 EP 1 517 228 A2 of: 12 on said touch surface determining the direction of scroll displaying an image on a touch surface: detecting pointer contacts on said touch surface and examining said pointer contacts to recognize multiple pointer contacts represent- 5 10 ing a gesture; and when multiple pointer contacts representing a gesture occur. updating the displayed image in accordance with said gesture. 12. Agesturerecognitionmethodcomprisingthesteps of: detecting multiple pointers in close proximity to a touch surface to determine if said multiple pointers are being used to perform a known gesture; and when said multiple pointers are being used to 2. The method of claim 1 wherein multiple pointer con- perform a known gesture, executing a com- tacts representing a gesture include multiple finger mand associated with said gesture, contacts on said touch surface. 15 3. The method of claim 1 wherein multiple pointer contacts representinga gesture include a finger contact on said touch surface and an object contact on said touch surface if a known gesture is being performed, 20 4. The method of claim 1 wherein multiple pointer contacts representing a gesture include multiple object contacts on said touch surface. 5. The methodof claim 1 wherein multiple pointer contacts representing a gesture include multiple finger contacts on said touch surface, a finger contact on said touch surface and an object contact on said touch surface, and/or multiple object contacts on said touch surface. 13. The method of claim 12 wherein duríng said detecting, pointer contacts with or close pointer hovers over said touch surface are detected to determine 14. The method of claim 13 wherein said muîtÍple pointers include multiple fingers, at least one finger and at least one object, and multiple objects in close proximity to said touch surface. 25 15. The method of claim 13 wherein during said detecting the multiple pointers are examined to determine if one of a number of known gestures is being performed, each known gesture being associated with a different command, 30 16. The method of claim 15 wherein the movement cf 6. The method of claim 1 wherein said gesture represents a right-click event. 7. the multiple pointers relative to the touch surface determines the gesture being performed. Themethodolciaim6whereinsaidright-clickevent .35 is represented by a first pointer contact on a dis- 17. The method of claim 15 wherein the pointer type determines the gesture being performed. played application, and a subsequent second pointer contact adjacent said first pointer contact. 8. The methodof claim 6 wherein said right-click event is represented by a first pointer contact on a dis- 18. The method of claim 15 wherein the movement of the multiple pointers relative to the touch surface 40 pîayed application, and a subsequent second pointer contact that occurs within a threshold distance of said first pointer contact and whBe said first pointer contact is maintained. 9. 19. An input detection method in an interactive system capable of detecting movement of multiple pointers 45 The method of claim i wherein said gesture is represented by simultaneous pointer contacts on said touch surface, generaHy simultaneously within an input region, said method comprising the steps of: capturing images looking generally across said input region; 50 10. The method of claim 9 wherein said simultaneous pointer contacts are simultaneous finger contacts on said touch surface. 11. The method of claim 9 wherein said simultaneous pointer contacts on said touch surface represent a scroll event, the direcdon of movement of the pointers over said touch surface subsequent to-contact and the pointer type determines the gesture being performed. 55 anaîyzing said images to detect multiple pointers within said input region; when multiple pointers are detected, exarnining data associated with said multiple pointers to determine if the data represents an input gesture; and when the data represents an input gesture, executing a command corresponding to the recognized input gesture, 9 APLNDC00027272 13 EP 1 517 228 A2 14 lating a right-click mouse event comprising the steps of: 20. A touch system comprising: a touch surface to be contacted by at least one pointer; at least one imaging device having a field of view looking generally across said touch surface; and 5 at least one processorcommunicating with said detecting a first pointer contact on a touch surface that represents a left-click mouse event; deteclog when a subsequent second pointer contact on said touch surface occurs within a threshold distance of said first pointer contact: at least one imaging device and analyzing images acquired by said at least one imaging device to determine the location on said touch sur- 70 face where pointer contacts are made, when said touch surface is contacted by multiple pointers, said processor examining said multi- and generating a right-clîck rnouse event in response to said detected second pointer contact- pie pointercontacts to determine if said multiple pointer contacts represent a gesture and when is said multiple pointer contacts represent a ges- 28. The method of claim 27 wherein said second pointer contact must also occur during said first pointer contact in order for said right-click mouse event to be generated. ture, said processor executing a command associated with said gesture. 20 21 A touch system according to claim 20 wherein a first pointer contact followed by a subsequent second pointer contact adjacent said first pointer contact represents a gesture. 29. The method of claim 28 wherein said second pointer contact must also occur to the right of said first poínter contact in order for said right-click mouse event to be generated. 25 22. A touch system according to claim 20 wherein at least two simultaneous pointer contacts represents a gesture. 23. A touch system according to claim 20 wherein each gesture is represented by specified multiple pointer actions. 24. A touch system according to claim 23 wherein each gesture is further represented by pointer type. 30 35 25. An interactive input system comprising: at least one imaging device having an input region within its field of view into which one or more pointers is moved to generate user input; 4 and at least one processor communicating.with said at least one imaging device and analyzing each image acquired by said at least one imaging device to determine the action of pointers in said 45 input region, said at least one processor determiningwhenmultiplepointeractionswithinsaid input region represent a gesture, when said multiple pointer actions represent a gesture, 50 said at least one processor executing a command corresponding to said gesture. 26. An interactive input system according to claim 25 wherein each gesture is further represented by pointer type. 55 21 in an interachve touch system, a method of simo 10 XJCID <EP TS1722BA2 E > APLNDC00027273 EP 1 517 228 A2 52 CAMERA 1 70 CAMERA 2 119 CAMERA 0 70 70 58 CAMERA 3 70 54 MASTER DSP COMPUTER se FIG. 1 52 sa 84 68 88 . 64 FIG. 2 11 EP 1517228A2 APLNDC00027274 EP 1 517 228 A2 62 60 60 FIG. 3a FIG. 3b 60 FIG. 3c FIG. 3d 12 30010: <EP 1517228A > APLNDC00027275 EP 1 517 228 A2 so FIG. 4a FIG. 4b FIG. 4c FIG. 4¢ ©o FIG. 4e FIG. 4f 13 APLNDC00027276 APLNDC00027277 lHI lililliMIllIII (11) (12) EP 1 517 228 A3 EUROPEAN PATENT APPLICATION (88) Date of publication A3: 02.04.2008 Bulletin 2008/14 (51) Int CL: GD6F3/033f2°°6 11 (43) Date of publication A2: 23.03.2005 Bultetin 2005/12 (21) Application number: 04255568.0 (22) Date of filing: 15.09.2004 (84) Designated Contracting States: AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT Li LU MC NL PL PT RO SE Si SK TR Designated Extension States: AL HR LT LV MK • Morrison, Gerald D. Calgary, Alberta T3G 4T6 (CA) (30) Priority: 16.09.2003 US 662813 (74) Representative: Naismith, Robert Stewart et al Marks & Clerk Scotland 19 Royal Exchange Squate Glasgow, G1 3AE Scotland (GB) (71) Applicant: Smart Technologies, Inc. Calgary, Alberte T3c OMS (CA) (54) (72) inventors: • Hiti, Douglas B. Calgary, Alberta T2K 1Z2 (CA) Gesture recognition method and touch system incorporating the same (57) A gesture recognition method includes detecting multiple pointers in close proximity to a touch surface to being used to perform a known gesture, a command associated with the gesture is executed. A touch system determine if the multiple pointers are being used to perform a known gesture. When the multiple pointers are incorporating the gesture recognition method is also provided. 60 FIG. 4a co CM CM LU F teny somessoo: PAnis (m (Cont next page) APLNDC00027278 EP 1 517 228 A3 60 F / 60 FIG. 4c 60 FIG. 4d FIG. 4e 2 YMPr <F 1FØ799RA3 (Cont. next page) > APLNDC00027279 EP 1 517 228 A3 F G. 4f 3 APLNDC00027280 EP 1 517228A3 European Patent EUROPEAN SEARCH REPORT Appucanonnemee onde EP 04 25 5568 DOCUMENTS CONSIDERED TO BE RELEVANT cate citation of clocument with ndication, where appropriate, Aeievant CLAsswicATioNOF THE tocoxim ofrelevantpassages APrucATsoN UPci X US 4 746 770 A (MCAVINNEY PAUL [US]) 24 May 1988 (1988-65-24) * column 3, line 15 - column 4. line 49; figure 1 * * column 7, line 17 - column 8, line 18; figures 10-16 * 1-29 X WO 98/07112 A (SYMBIOS LOGIC INC [US]; GILL DAVID ALAN [GB) SYMBIOS LOGIC INC INV. GO6F3/033 1-29 [US); G) 19 February 1998 (1998-02-19) * page 4, lines 1-9 * * page 6, line 22 - page 9, line 28 * X WO 99/28812 A (NORTHERN TELECOM LTD (CA]) 10 June 1999 (1999-06-10) * the whole document * 1-29 X WO 02/08881 A (QINETIQ LTD [GB]; WORTHINGTON CHRISTOPHER [GB]; ANDREW IAN DUNCAN CAMP) 31 January 2002 (2002-01-31) * page 13, line 11 - page 14, line 9; 1-29 WO 03/054681 A (NOKIA CORP [FI]; PIHLAJA 11 TECHNICALFlEUX seaneneo nec 606F figures 4,5 * A A PEKKA [FI)) 3 July 2003 (2003-07-03) * the whole document * US 6 278 443 B1 (AMRO HATIM YOUSEF [US) ET 24,26 AL) 21 August 2001 (2001-08-21) * page 10, last paragraph - page 11, paragraph 1; figures 13,14 * Berlin CATEGORY OF OfTED DOCUMENTS X : particulady relevant 4 taken alone Y : particulariy redevant à combined with another document of the same category A : techociogicalinsch:grotmd O: non-writien disodesure P ntermediate document 25 February 2008 Mouton, Benjamin T : tfory or pencipfe underlying the invemon E : aarlier patent document, but publehedon, or after the filing date O doeurnenteiteg n the applionijon L : document cited for other reasons & : member of the sono patent family, correspondag dCCUmenT 4 APLNDC00027281 EPT 517228A3 ANNEX TO THE EUROPEAN SEARCH REPORT ON EUROPEAN PATENT APPLIGATION NO. EP 04 25 5566 This annex fets the patent famdy members twielmg to the patent documents cited in the acove-mentioned European searcluegott The members are as contened in the Eurooean Patent Office EDP fWe an The Europes.n Patent Office is in no wav liable for mese particulars which are merely given for the purpose cHnformation. 25-02-2008 Patent document Publication aned in search leport date US 4746770 A A 19-02-1998 Paterd family AU Pub¾eation memimris) 24-05-1988 40 9807112 . date 603643 B2 22-11-1990 AU EP JP 1173288 A 0279652 A2 63223819 A 18-08- 988 24-08- 988 19-09- 988 AU EP US 3857697 A 0919023 A2 06-03-1998 02-06-1999 6208329 81 27-03-2001 40 9928812 A 10-06-1999 US 6310610 B1 30-10-2001 WO 0208881 A 31-01-2002 AU AU 7080801 A 2001270808 A1 05-02-2002 05-02-2002 40 03054681 A 03-07-2003 AU EP FI 2002366877 A1 1456740 Al 20012540 A 09-07-2003 15-09-2004 21-06-2003 US US 2003146905 Al 2006267956 A1 07-08-2003 30-11-2006 US 6278443 B1 21-08-2001 NONE o o 16 For more details abord this anner : see Official Joumal of the European Patent Office, No 12/82 5 APLNDC00027282 L APLNDC00027283 PATENT COOPERATION TREATY PCT From the INTERNATIONAL SEARCHING AUTHORITY To: BLAKELY Attn. SOKOLOFF - Scheller, NOTIFICATION OF TRANSMITTAL OF THE INTERNATIONAL SEARCH REPORT AND THE WRITTEN OPINION OF THE INTERNATIONAL SEARCHING AUTHORITY, OR THE DECLARATION TAYLOR & 2APMAN LLP Jr James C. 1279 Oakmead Parkwav Sunnyvale, California 94085 4040 ETATS-UNIS D'AM2RIQUE (PCT Rule 44.1) Dâte of mailing (day/rnonthlyear) , , , Applicants or agents file reference 4860P4895PCT intemational application No· JUN 0 9 200Û FOR FURTHER ACTION ELAK PCT/US2008/000089 O 4 / O 6 / 2 OO 8 See paragraphe t and 4 below intemational filing date e (day/month/year) 04/01/2008 Applicant APPLE INC. L The applicant is hereby notified that the international search report and the written opinion of the Intemational Searching Authority have been established and are transmitted herewith, Filing of amendments and statement under Article 19: The applicant is entitled, if he so wishes, to amend the claims of the Intemational Application (see Rule 46): When? The time Nmu for filing such amendments is normauy two months from the date of transmittal of the intemational Search Report Where? Directly to the intemational Bureau of WIPO, 34 chemin des Colombettes 1211 Geneva 20. Switzerland, Fascimile No.: (41-22) 338.82.70 For more detailed instructions, see the notes on the accompanying sheet. 2. The applicant is hereby notified that no intemational search report will be established and that the declaration under Article 17(2)(a) to that effect and the written opinion of the Intemational Searching Authority are transmitted herewith. 3. With regard to the protest against payment of (an) additional fee(s) under Rule 402, the applicant is notified that: O the protest together with the decision thereon has been transtnitted to the Intemational Bureau together with the applicanfs request to forward the texts of both the protest and the decision thereon to the designated Offices, no decision has been made yet on the protest; the applicant will be notified as soon as a decision is made. 4. Reminders Shortly after the expiration of 18 months from the priority date, the international appilcation will be published by the international Bureau. If the applicant wishes to avoid or postpone publication, a notice of withdrawal of the international application, or of the priority claim, must reach the international Bureau as provided in Rules 90bis.1 and 90bis.3, respectively, before the completîon of the technicâl preparations for intemational publication. The applicant may submit comments on an informal basis on the written opinion of the international Searching Authority to the intemational Bureau. The intemational Bureau will send a copy of such comments to all designated Offices unless an intemational preliminary examination report has been or is to be established, These comments would also be made avaNable to the public but not before the expiration of 30 months from the priority date. Within 19 months from the priority dates but only in respect of some designated Offices, a demand for international preliminary examination must be fited if the applicant wishes to postpone the entry into the national phase until 30 months from the priority date (in some Offices even later); otherwise, the applicant must, within 20 months from the priority date, perform the prescribed acts for entry into the national phase before those designated Offices. In respect of other designated Offices, the time limit of 30 months (or later) will apply even if no demand is filed within 19 months. See the Annex to Form PCT/tB/301 and, for details about the applicable time limits, Office by Office, see the PCT Applicants Guide, Volume 11, National Chapters and the WIPO Internet site. Name and mening address of the internationai Searching Authority Authodzed officet European Patent Office. PB 5818 Patentiaan 2 NL-2280 HV R4ew k Tet (+31-70) 340-2040, Tx, 31 651 epo nL Pax: (÷31-7ö} 340-3016 Form PCT/ISA/220 (October 2005) . Date in to SV F Ma rj a B romdSW4b d by: W - Reviewed by: Sant MthyJW Ry 00 rtir (See notes on accompanying speet) APLNDC00027284 NOTES TO FORM PCTIIS"' These Notes are intended to give the basic instructions conceming the filing of amendments under article 19. The Notes are based on the requirements of the Patent Cooperation Treaty, the Regulations and the Administrative instructions under that Treaty. In case of discrepancy between these Notes and those requirements, the latter are apolicable. For more detailed information, see also the PCT Applicants Guide, a publication of WIPO. In these Notes, "Article", "Rule", and "Section" refer to the provisions of the PCT, the PCT Regulations and the PCT Administrative instructions, respectively. INSTRUCTIONS CONCERNING AMENDMENTS UNDER ARTICLE 19 The applicant has, after having received the intemational search report and the written opinion of the intemationaî Searching Authority, one opportunity to amend the claims of the intemational application, it should however be emphasized that, since a!! parts of the intemational application (cíaims.description and drawings) may be amended during the intemationaî prenminary examination procedure, there is usually no need to file amendments of the claims under Article 19 except where, e.g. the applicant wants the latter to be published for the purposes of provisional protection or has another reason for amending the claims before intemational publication. Furthermore, it should be emphasized that provisional protection is available in some States only (see PCT Applicants Guide, Volume VA, Annexes BT and B2) The âttention of the applicant is drawn to the fact that amendments to the claims under Article 19 are not allowed where the International Searching Authority has declared, under Article 17(2), that no intemational search report would be established (see PCTApplicant's Guide, Volume t/A, paragraph 296). What parts of the international application may be amended? Under Article 19, only the claims may be amended. During the international phase, the claims may also be amended (or further amended) under Article 34 before the Irtternational Preliminary Examining Authority. The description and drawings may only be amended under Article 34 before the International Examining Authority. Upon entry into the national phase, all parts of the intemational application may be amended under Article 28 or, where applicable, Article 4L When? Within 2 months from the date of transmittal of the intemational search report or 16 months from the priority date, whichever time limit expires later. It should be noted, however, that the amendments will be considered as having been received on time if they are received by the laternational Bureau after the e×piration of the applicable time limit but before the completion of the technical preparations for intemational publication (Aule 46.1). Where not to fue the amendments? The amendments may only be filed with the fritemational Bureau and not with the receiving Office or the Intemationâl Searching Authority (Rule 462), Where a demand for înternational preliminary e×amination has beenlis filed, see below. How? Either by cancelling one or more entire claims, by adding one or more new claims or by amending the text of one or more of the claims as filed. A replacement sheet must be submitted for each sheet of the claims which, on account of an amendment or amendments, differs from the sheet originally filed. All the claims appearing on a replacement sheet must be numbered in Arabic numerals. Where a claim is cancelled, no renumbering of the other claims is required. In all cases where claims are renumbered, they must be renumbered consecutively (Section 205(b)). The amendments must be made in the language in which the international application is to be published. What documents must/may accompany the amendments? Letter (Section 205(b)): The amendments must be submitted with a letter. The letter will not be published with the international application and the amended claims. It should not be confused with the "Statement under Article 19(1)" (see below, under "Statement under Article 19(t)"). The letter must be in English or French, at the choice of the applicant However, if the language of the international applicatiore is Enghsh, the letter raust be in EngHah; if the language of the international appiscation is French, the letter must be in French. Notes to Form PCT/ISA/220 (first sheet) (October 2005) APLNDC00027285 PATENT COOPERATION TREATY PCT INTERNATIONAL SEARCH REPORT (PCT Article 18 and Rules 43 and 44) Applicânt's or agenfs file reference 4 860 P4 8 95 PCT FOR FURTHER see Form PCT/ISA/220 ACTION înternational application No- as wen as, where appucable, item 5 below. Intemational f¾ing date (day/month/year} PCT/Us2008/000089 Applicant \ (Earliest) PriorÑ|ÕÄÏ 04/01/2008 ÑÃy- 07/01/2007 APPLE INC. This intemational search report has been prepared by this latemational Searching Authority and is transmitted to the applicant according to Article 18. A copy is being transmitted to the intemational Bureau.. This international search report consists of a total of 3 sheets. It is also accompanied by a copy of each prior art document cited in this report, 1. Basis of the report a, With regard to the language, the international search was carried out on the basis of: the intemational application in the language in which it was filed O a imosiason ome satsmasona, avereason into . walors me lanossos of a translation fumished for the purposes of intemational search (Rules 12.3(a) and 23.1(b)) b. This intemationaî search report has been established taking into account the rectification of an obvious mistake authorized by or notified to this Authority under Rule 91 (Rule 43.6bis(â)}. c. With regard to any nucleotide and/or amino acid sequence discîosed in the intemational application, see Box No. L 2. Conan clans were found unseardable (See Box No. 0) 3. Unity of invention is lacking (see Bo× No 10) 4. With regard to the title, O 5 the text is approved as submitted by the applicant the text has been established by this Authority to read as follows: With regard to the abstract, the text is approved as submitted by the applicant the text has been established, according to Rule 38.2(b), by thie Authority as it appears in Box No. IV. The applicant may, within one month from the date of maliing of this intemational search report, submit comments to this Authority 6. With regard to the drawings, a. the figure of the drawings to be published with the abstract is Figure No. 4 as suggested by the app¾cant as selected by this Authority, becâuse the appticant faned to suggest a figure b. as selected by this Authority. because this figure better characterizes the invemion none of the figures is to be published with the abstract Form PCT/ISA/210 (first sheet) (April2007) APLNDC00027286 INTERNATIONAL SEARCH REPORT international app¾cation No PCT/Us2008/000089 A. CLASSIFICATI INV. SUBAGT MATTER GO6F3 048 According to intomational Patent Classification (IPC) of to both national classification and IPC B. FlELDS SEARCHED Minimum documentation searched (classitication System followed by classification symbols) GO6F Documentanon searched other than rmnimum documentation to the extent that such documents are included in the fields searched Electronic data base consulted danng the miemationai search (name of data base and. where practical search terms used) EPO-Internal C. DOCUMENTS CONSIDERED TO BE RELEVANT Category' Cilation of document with indicatiort, where appropriate, of the relevant passages X EP 1 517 228 A (SMART TECHNOLOGIES INC [CA)) 23 March 2005 (2005-03-23) paragraphs [0009], [0031] [0024], Relevant to claim No 1-51, 74-77, 80-88 [0027], X US 2003/174149 Al (FUDISAKI HITOMI [JP] ET AL) 18 September 2003 (2003-09-18) paragraphs [0001] - [0017] 52-73, 78,79 A US 6 958 749 B1 (MATSUSHITA NOBUYUKI (JP] ET AL) 25 October 2005 (2005-10-25) 1-88 O Further documents are listed in the continuation of Box C Special categones of cited documents : *A" document defining the general state of the art which is not considered to be of particular relevance 'E' earlier document but published on or after the intemational nRag date 'L' document whîch my throw doubts ora priority claim(s} or which is cited to estaMsh the publication date of another citation or other special reason (as specified) *O' document referring to an oral disclosure, use, exhibdion or other means "P' document published prior to the intomational filing date but :aler than the priority date claimed Date of the actual completion of the international search 27 May 2008 'T' later document published after the international fding date or priority date and not in conflict with the application but cited to understand the principle or theory underlying the mvention •X' document of particular relevance; the claimed invention cannot be considered novel or cannot be considered to involve an inventive step when the document is taken alone 'Y' document of parlicular relevance; the claimed invention canoc4 be considered to involve an inventive step when the document is combîned with one or more other such documents, such combina1îcn being obvious to a person skilled in the art *&' document member of the same patent family Date of mailing of the intemational search repon 04/06/2008 Nam and rradirra arkirm of me ISA Europm Palent See patent family annex. Auth<:>rized officer k:B, PB 5618 Paîentiaan 2 NL - 2¾ HV R4ewgk Tel (+31-70) 340-2040, TX 37 651 epo nL Fax: (+3 70}340-N16 Davenport, Kevin Fomi PCTllSAl210 (second shee0 (ApriE 2004) APLNDC00027287 INTERNATIONAL SEARCH REPORT information on patent family members Patent document cited in search report PuNication date intemational application No PCT/Us2008/ooooa9 Patent family member(s) PubHcation date EP 1517228 A 23-03-2005 CA JP US 2481396 Al 2005108211 A 2005057524 Al 16-03-2005 21-04-2005 17-03-2005 US 2003174149 Al 18-09-2003 JP JP 3951727 B2 2003233455 A 01-08-2007 22-08-2003 US 6958749 B1 25-10-2005 JP 2001134382 A 18-05-2001 i Form PCT/ISAl210 (patent family annex) (AprB2005) APLNDC00027288 PATENT COOPERATION TREATY From the 1NTERNATIONAL SEARCHiNG AUTHORlTY To: PCT WRITTEN OPINION OF THE INTERNATIONAL SEARCHING AUTHORITY (PCT Rule 43bis.1) see form PCTASA220 Date of manmg any tronth yea i Applicanfs or agents file reference see form PCTASA220 see form PCTASA210 (second sheet) FOR FURTHER ACTION See paragraph ? below intemational application No nÑnational filing date (daymonthyear) Priority date (dayänonthyear) PCTAJS2OO8DOOOS9 04.OL2008 07.002007 intemational Pateni Classification (IPC}or both national classification and IPC INV, GO6F3D48 Applicant APPLE INC. L This opinion contains indications relating to the following items: Bo× O Box O Box O Box Box O Box No. Vl O Box No. Vil Basis of the opinion Priority Non-establishment of opinion with regard to novelty, inventive step and industrial applicability Lack of unity of invention Reasoned statement under Rule 43bis.1(a)(i) with regard to novelty, inventive step or industrial applicability; citations and explanations supporting such statement Certain documents cited Certain defects in the international application O Box No. Vill 2. No. No. No. No. No. I Il til IV V Certain observations on the international application FURTHER ACTION If a demand for international preliminary examination is made, this opinion will usually be considered to be a written opinion of the International Preliminary Examining Authority ("lPEA ) except that this does not apply where the applicant chooses an Authority other than this one to be the IPEA and the chosen IPEA has notifed the Intemational Bureau under Rule 66.1bis(b) that written opinions of this International Searching Authority will not be so considered, if this opinion is, as provided above, considered to be a written opinion of the IPEA, the applicant is invited to submit to the IPEA a written reply together, where appropriate, with amendments, before the expiration of 3 months from the date of mailing of Form PCTASA220 or before the expiration of 22 months from the priority date, whichever expires later. For further options, see Form PCTASA/220. ! 3. For further details, see notes to Form PCTASA220, - n Date of completion of Authorized Officer this opinion European Patent O¾iee - P,B.. 5818 Patent! a6Ñe form NL-2280 HV R swijk - Pays Bas PCTASARTO TeE ÷31 70 340 - 2O4O TX 31 651 apo ni Fax: ÷31 70 340 - 3016 Telephone No +31 70 340-2191 Form (PCTASA237) (Cover Sheet} (April 2005) APLNDC00027289 WRITTEN OPINION OF THE INTERNATIONAL SEARCHING AUTHORITY Box No. I International application No. PCT/US2008DOOOS9 Basis of the opinion 1. With regard to the language, this opinion has been established on the basis of: the international application in the language in which it was filed O a translation of the intemational application into , which is the language of a transtation tumished for the purposes of intemational search (Rules 12.3(a) and 23.1 (b)) 2. O This opinion has been established taking into account the rectification of an obvious mistake authorized by or notified to this Authority under Rule 91 (Rule 43bis.1(a)) 3. With regard to any nucleotide andhr amino acid sequence disclosed in the international application and necessary to the claimed invention, this opinion has been established on the basis of: a. type of material: O a sequence listing O table(s) related to the sequence listing b. format of material: O on paper O in electronic form c. time of filing/fumishing: O O filed together with the international application in electronic form. O 4. O contained in the intemational application as filed. furnished subsequently to this Authority for the purposes of search. in addition, in the case that more than one version or copy of a sequence listing and/or table relating thereto has been filed or fumished, the required statements that the information in the subsequent or additional copies is identical to that in the application as filed or does not go beyond the application as filed, as appropriate, were fumished. 5. Additional comments: Form PCTASA/237 (April 2007) APLNDC00027290 WRITTEN OPINION OF THE INTERNATIONAL SEARCHING AUTHORITY intemational application No PCT/US2008DOOO89 Box No. V Reasoned statement under Rule 43bis.1(a)(i) with regard to novelty, inventive step or industrial appHcability citations and explanations supporting such statement 1. Statement Novelty (N} Yes: Clâlms No: Claims imi 7L77 anon 1-14529178Je Inventive step (IS) Yes: Claims No: Claims 1-88 Yes: Claims No: Claims 1-88 Industrial applicability (IA} 2. Citations and explanations s_ee yearate eet Form PCTASA/237 (April 2007) APLNDC00027291 WRITTEN OPINION OF THE INTERNATIONAL SEARCHING AUTHORITY (SEPARATE SHEET) Intemational apptication No, PCT/US2008/000089 Ra Itam V Reasoned statement with regard to novelty, inventivo step or industrial applicability; citations and explanations supporting such statement 1 Reference is made to the following documents: D1: EP-A-1 517 228 (SMART TECHNOLOGIES INC (CA]) 23 March 2005 (200503-23) D2: US 2003/174149 A1 (FUJlSAKI HITOMI (JP] ET AL) 18 September 2003 (200309-18) D3: US-81-6 958 749 (MATSUSHITA NOBUYUKI [JP] ET AL) 25 October 2005 (2005-10-25) 2 2.1 The present application does not meet the criteria of Article 33(1) PCT, because the subject-matter of claims 1,52 is not new in the sense of Article 33(2) PCT. Regarding claim 1, the document D1 discloses a machine implemented method for scrolling on a display of a portable data processing device comprising: receiving a user input (§9); creating an event object in response to the user input (the very nature of data processing devices means that this will always occur); determining whether the event object invokes a scroll or gesture operation (§24 - Di discloses that both scrolling and scaling are two different types of gesture; while claim 1 does not define a scroll call as being initiated by a gesture, it may be seen from the description that a scroll call is initiated by a movement of a finger across the input device, which indeed constitutes a gesture; what is significant is that there is a differentiation between two different gestures;); issuing at least one scroll or gesture call based on invoking the scroll or gesture operation (the very nature of data processing devices means that this will always occur); Form PCT/ISA/237 (Separate SheetBSheet i HER Aprú 2005) APLNDC00027292 WRITTEN OPINION OF THE INTERNATIONAL SEARCHING AUTHORITY (SEPARATE SHEET) intemational application No. PCT/US2008/000089 responding to at least one scroll call, if issued, by scrolling a window having a view associated with the event object based on an amount of a scroll with the scrofi stopped at a predetermined position in relation to the user input (§27 - " If the user contacts the touch surface 60 with a pair of fíngers simultaneously over an application window displayed on the touch surface and the fingers are closely and generally horizontally spaced, the driver recognizes the sirnultaneous finger contacts as a scroll gesture and injects a scroll event into the application."); and responding to at least one gesture call, if issued, by scaling the view associated with the event object based on receiving a plurality of input points in the form of the user input (§31- "If the usercontacts the touch surface 60 with a pair of closely spaced fingers simultaneously over an application window and expands the distance between the fingers in a generally horizontal direction, the driver recognizes the finger rnovernent as a zoorn-out gesture."). The subject-matter of claim 1 is therefore not new. The same reasoning applies, mutatis mutandis, to the subject-matter of the corresponding independent claim 8, which therefore is also considered not new. 2.2 2.2.1 2.3 Regarding claim 52, D2 discloses a method for operating through an application programming interface (API) in an environment with user interface software interacting with a software application and a user input contacting a view of a display of a device, comprising transferring a directional scroll call to determine if directional scrolling is enabled. See D2, §11-§17. With regard to the method of operating through an API, the use of an application programming interface is considered to be implicitly disclosed in the closest prior art for each case because it is standard practice to use an APt to interface between and input device and the software which it is manipulating. See e.g. D3, §7-§13, §33. The same reasoning applies, mutatis mutandis, to the subject-matter of the Form PCT/ISAl237 (SeparateShæîMSheet 2) (EP Aprä 2005) APLNDC00027293 WRITTEN OPINION OF THE INTERNATIONAL SEARCHING AUTHORITY (SEPARATE SHEET) intemational application No. PCT/US2008/000089 corresponding independent claims 58,64,65,69,73,78 which therefore are also considered not new. 3 3.1 The present application does not meet the criteria of Article 33(1) PCT, because the subject-matter of claim 15 does not involve an inventive step in the sense of Article 33(3) PCT. The document D1 is regarded as being the closest prior art to the subject-matter of claim 15, and discloses a method for operating through an application programming interface in an environment with user interface software interacting with a software application (§9). See also §2.2.1 supra. The subject-matter of claim 15 therefore differs herefrom in that a set bounce call is transferred. 3.2 The problem to be solved by the present invention may therefore be rece. ded as how to present a more visually intuitive graphical user interface. However, this problem is merely a problem of visual design nature, and thus cannot be used in the assessment of inventive step. The result achieved by solving this problem, namely the simulation of the natural phenomenon of bouncing of one object against another before eventually coming to rest has no technical effect on the usability of the interface by the user, and does not lead to an improved user - machine interaction. The claims merely define a solution which does not lead to technical effects once they are applied to a real world situation. 3.3 The ISA does not dispute the fact that technical means may be involved in the implementation of a more visually intuitive graphical user interface. However, a technical problem solved by this combination of technical means is not apparent. Therefore, the technical problem has to be seen as how to implement a method Forrn PCTASAl237 (Separate Sheet) (Sheet 3) (EPC>Apúl 2005) APLNDC00027294 WRITTEN OPINION OF THE INTERNATIONAL SEARCHING AUTHORITY (SEPARATE SHEET) Intemational application No. PCT/US2008/000089 of presenting a more visually intuitive graphical user interface. The inventive activity is then assessed using the general technical knowledge of the skilled person, in the sense of Article 33(3) PCT, which, in the present case is a programmer of graphical user interfaces, acting on instructions (see §2.8 supra) received from the person in charge of defining the visual design of the interface. Starting from this knowledge, the skilled person, aware of the design requirements of the person in charge of defining the visual design of the interlace, would arrive at the proposed solution without the exercise of an inventive step. The implementation details described in the application, are well known algorithms used in the field of programming for looping through sequential lists. In general, said implementation does not require any technical considerations that go beyond the skilled person's normal technical abilities. The same reasoning applies, mutatis mutandis, to the subject-matter of the corresponding independent claims 20,25,26,29,32,74,80,86 which therefore are also considered not inventive. 4 The document D1 is regarded as being the closest prior art to the subject-matter of claim 33, and discloses a method for operating through an application programming interface in an environment with user interface kit interacting with a software application (§9). See also §2.2.1 supra. 4.1 The subject-matter of claim 33 therefore differs herefrom in that a rubberband call to cause a scrolled region displayed within a region of a device to rubberband is transferred. 4.2 The problem to be solved by the present invention may therefore be regarded as how to present a more visually intuitive graphical user interface. However, this problem is merely a problem of visual design nature, and thus cannot be used in the assessment of inventive step. The result achieved by APLNDC00027295 WRITTEN OPINION OF THE INTERNATIONAL SEARCHING AUTHORITY (SEPARATE SHEET) International application No. PCT/US2008/000089 solving this problem, namely the simulation of the natural phenomenon of the behaviour of two objects elastically bound to each other has no technical effect on the usability of the interface by the user, and does not lead to an improved user - machine interaction. The claims merely define a solution which does not lead to technical effects once they are applied to a reat world situation. 4.3 A similar argumentation to that of §3.3 supra applies. The same reasoning applies, mutatis mutandis, to the subject-matter of the corresponding independent claims 39,44,45,48,51,76 which therefore are also considered not inventive. 5 The dependent claims do not contain any features which, in combination with the features of any claim to which they refer, meet the requirements of the PCT in respect of novelty and/or inventive step. See documents D1-D3 and the corresponding passages cited in the search report. Form PCTilSA/237 Separate Sheet} (Sheef 5) (EP AprR 2005) APLNDC00027296 Possible steps after receipt of the inte<national search report (ISR) and written opinion of the International Searching Authority (WO-ISA) General For all international applications filed on or after 01/01/2004 the competent information ISA will establish an ISR It is accompanied by the WO-ISA. Línlike the former written opinion of the IPEA (Rule 66.2 PCT), the WO-ISA is not meant to be responded to, but to be taken into consideration for further procedural steps. This document explains about the possibilities. Amending claims Within 2 months after the date of mailing of the ISA and the WO-ISA the under Art. 19 PCT applicant may file amended claims under Art. 19 PCT directly with the I nternational Bureau of WI PO. The PCT reform of 2004 did not change this procedure. For further information please see Rule 46 PCT as well as form PCT/ISN220 and the corresponding Notes to form PCT/tSN220. Filing a demand for international preliminary in principle, the WO-ISA will be considered as the written opinion of the iPEA. This should, in many cases, make it unnecessary to file a demand for international preliminary examination. If the applicant nevertheless wishes examination to file a demand this must be done before expiry of 3 months after the date of mailing of the ISR/WO-ISA or 22 months after priority date, whichever expires later (Rule 54bis PCT). Amendments under Art. 34 PCT can be filed with the IPEA as before, normally at the same time as filing the demand (Rule 66.1 (b) FCT). If a demand for international preliminary examination is filed and no comments/amendments have been received the WO-ISA will be transformed by the IPEA into an IPRP (International Preliminary Report on Patentability) which would merely reflect the content of the WO-ISA. The demand can still be withdrawn (Art. 37 PCT). Filing informal comments After receipt of the ISFVWO-ISA the applicant may file informal comments on the WO-ISA directly with the I nternational Bureau of WI PO. These will be communicated to the designated Offices together with the I PRP (international Preliminary Report on Patentability) at 30 months from the priority date. Rease also refer to the next box. End of the international phase Relevant PCT Ru les and more inf orm at son 1000 At the end of the international phase the International Bureau of WIPO will transform the WO-fŠA or, if a demand was filed, the written opinion of the IPEA into the IPRP, which will then be transmitted together with possible informal comments to the designated Offices. The IPRP replaces the former IPER (international preliminary examination report). Rule 43 PCT, Rule 43bis PCT, Rule 44 PCT, Rule 44bis PCT, PCT Newsletter 12/2003, Œ11/2003, Œ12/2003 <XS 2U70&M K i > APLNDC00027297 PATENT COOPERATION TREATY From the INTERNATIONAL SEARCHING AUTHOAITY To: NOTIFICATION OF TRANSMITTAL OF BLAKELY, SOK DFF, TATL R & ZAFMAN THE INTERNATIONAL SEARCH REPORT AND Acca, schener, or James tro oakmaae Parkway 3untry,7ale, Ca.1.ifornia THE WAtTTEN OPINiON OF THE!NTEANATIONAL SEARCHING AUTHORITY, OR THE DECLARA.TION 94085 4040 (PCT Rule 44.1) Date of mainng Applicants or agents file reference 4860 P5040PCT ' FOR FURTHER ACTION intemational application No. PCT/US2008/000060 See paragraphs 1 and 4 below inter ational fliing date . . (day/month eâr) 03/01 2008 Applicant APPLE INO. L The applicant is hereby notified that the intemational search report and the written opinion of the intemational Searching Authoríty have been established and are transmitted herewith. Filing of amendments and statement under Article 19: The applicant is entuied, if he so wishes, to amend the claims of the intemational Application (see Rule 46); When? The time limit for filing such amendments is normally two months íïom the date of transmittal of the Intemational Search Report Where? Directly to the Intemational Bureau of WIPO, 34 chemin des Colombettes 1211 Geneva 20, Switzerland, Fascimile No.: (41-22) 338.82.70 For more detailed instructions, see the notes on the accompanying sheet. 2. The applicant is hereby notified that no internatonal search report will be established and that the declaration under Article 17(2)(a) to that effect and the written opinion of the Intemational Searching Authority are transmitted herewith. 3. With regard to the protest against payment of (an) additional fee(s) under Rule 40.2, the applicant is notified that the protest together with the decision thereon has been transmitted to the international Bureau together with the applicanfs request to forward the texts of both the protest and the decision thereon to the designated Offices. no decision has been made yet on the protest; the applicant will be notified as soon as a decision is made, 4. Aeminders Shortly after the expiration of 18 months from the priority date, the intemational application witi be published by the Intemationai Bureau if the applicant wishes to avoid or postpone publication, a notice of withdrawal of the íntemâtional applicatiore or of the priority claim, must reach the international Bureau as provided in Aules 90Dis.i and 90bis 3, respectively, before the completion of the technical preparations for intemâtfonal publicatiort The appheant may submit comments on an informal basis on the written opinion of the intemational Searching Authority to the international Bureau The intemational Bureau will send a copy of such comments to âll designated Offices unless an international preliminary examination report has been of is to be estab¾shed. These comments would also be made avahabte to the pubha but not before the expiration of 30 months from the priority date. Within 19 months from the priority date, but only in respect of some designated Offices. a demand for intemational preliminary exarnination must be filed if the applicant wishes to postpone the entry into the national phase until 30 months from the priority date On some Offices even later); otherwise, the appúcant must, within 20 months from the priority date, perform the prescribed acts for entry into tne national phase befoæ those designated Ottices. In respect of other designated Offices, the time ilmit of 30 months for later) win apply even if no demand is TRed within 19 months. See the Annex to Form PCT/IS/301 and, for details about the applicable time umits, Office by Office, see the ROT Applicants Gui<ie. Volume IL National Chapters and the WIPO intemet site. Form PCT/ISAl220 (October 2005) APLNDC00027298 NOTES TO FORM PCTIISA/220 These Notes are interided to give the basic instructions conceming the filing of amendments under article 19. The Notes are based on the requirements of the Patent Cooperation Treaty, the Reguíations and the Administrative lastructions under that Treaty. in case of discrepancy between these Notes and those requirements, the latter are aopticable. For more detailed information. see also the PCT Applicants Guide, a pubücation of WOO. In these Notes. "Article". "Rule", and "Section" refer to the provisions of the PCT. the PCT Regulations and the PCT Administrative instructions, respectively. INSTRUCTIONS CONCERNING AMENDMENTS UNDER ARTICLE 19 The appBcant has, after having received the intemational search report and the written opinion of the intematíonal Searching Authority, one opportunity to amend the claims of the international application, it shouid however be emphasized that since an parts of the international application (clairns,description and drawings) may be amended durireg the international preliminary examination procedure, there is usually no need to file amendments of the claims under Article 19 except where, e.g. the applicant wants the latter to be pubüsted for the purposes of provisional protection or has another reason for amending the claims before international pubücation Furthermore, it should be emphasized that provisional protection is availabîe in some States only (see PCT AppRcanfs Guíde, Volume I/A, Annexes 81 and 82). The attention of the applicant is drawn to the fact that ameridments to the claims under Article 19 are not allowed where the International Searching Authority has declared, under Article 17(2), that no internâtional Search report would be estâbiished (see POT Applicanfs Guide, Volume I/A, paragraph 29ô). What parts of the international application may be amended? Under Article 19, only the claims may be amended. During the intemational phase, the claims may also be amertded (or further amended) under Article 34 before the international Preliminary Examining Authority. The description and drawings may only be amended under Article 34 before the International Examirung Authority. Upon entry into the national phase, ad parts of the intamational application may be amended under Article 28 or, where applicable, Article 41. When? Within 2 months from the date of transmittai of the intemational search report or 16 months from the priority date, whichever time Mmit expires later. It should be noted, however, that the amendments win be considered as having been received on tirne if they are received by the Intemational Bureau after the expiration of the applicable time limit but before the completion of the technical preparations for intemational publication (Rute 46-1¾ Where not to file the amendments? The amendments may only be filed with the totemational Bureau and not with the receiving Office or the intemational Searching Authority (Rule 46 2) Where a demand for international preliminary examination has beenlis filed. see below. How? Either by cancelling one or more entire claims, by adding one or more new claims or by amending the text of one or more of the claims âs filed. A replacement sheet must be submitted for each sheet of the claims which, on account of an amendment or amendments, differs from the sheet originally filed, All the claims appearing on a replacement sheet must be numbered in Arabic numerals. Where a claim is canceled, no renumberíng of the other claims is required, in aM cases where claims are renumbered. they must be renumbered consecutively (Section 205(b)). The amendments must be made in the language in which the international application is to be pubHshed. What documents must/may accompany the amendments? Letter (Section 205(b)): The amendments must be submítted with a letter. The tetter wili not be pubBshed with the international application and the amended claims. It should not be confused with the "Statement under Article 19(1 (see below, under "Statement under Article 19(if) The letter must be in EngHsh or French, at the choice of the appücant However, if the language of the international appdcation is English, the letter must be in Engnan; if the language of the intamational appneation is French, the letter must be in French. Notes to Form PCT/ISA/220 (first sheet) (October 2005) APLNDC00027299 PATENT COOPERATION TREATY PCT INTERNATIONAL SEARCH REPORT (PCT Adicle 18 and Rules 43 and 44) AppäCanfSof agenfs fue reference FOR FURTHER 486OP504 OPCT see Form PCT/ISA/220 ACTION Intemational appucation No. as wen as. where appucaele, item a below. Intemâtional finng date Iday/month/year) ecT/us2ooa/ooooso (Eartiest) Priority Date (day/month/pea:) 03/01/2008 07/01/2007 AppWeant APPLE INC. This intemational search report has been prepared by this intemational Searching Authority and is transrnitted to the appleant according to Article 18. A copy is being transmitted to the latemational Bureau. This intemational search report consists of a total of L . sheets. t is also accompanied by a copy of each prior art document cited in this report 1. Basis of the report a. With regard to the language, the internâtional search was carried out on the basis of: the intemational applícation in the îanguage in which it was filed a translation of the intemational app¾catíon into , which is the language of a translation furnished for the purposes of international search (Rules 12.3(a) and 23.1(b)) b. This internationaf search report has been estabushed taking into account the rectification of an obvious mistake authorized by or notified to this Authority under Rule 91 (Rule 43.6bis(a)). c. With regard to any nucleotide and/or amino acid sequence disclosed in the intomational application, see Box No. L 2, Certain claims were found unsearchable (See Box No. 11) 3. Unity of invention is lacking (see Box No !!!} 4_ With regard to the title, the text is approved as submitted by the appitcant the text has been established by this Authority to read as follows: 5.. With regard to the abstract, the text is approved as submRied by the applicant the text has been established, according to Rule 38,2(b), by tNs Authority as it appears in Box No. IV. The appleant may, wdhin one month from the date of mening of this intemational search report, submit comments to this Authority a 9 cy * -1: * - drawmgs -: 'odrawmgsi a ---, s -t, ,- eN 4 ,tion APLNDC00027300 INTERNATIONAL SEARCH REPORT toternational application No PCT/US2008/000060 A. CLASSfFICATION INV. F SUBJECT MATTER GO6F3 048. According lo înternational Patent Glassnication (IPCs or to tWXh national classificadon and IPC B. FIELDS SEA ACHED Minimum documentation searched (ciassWicadon system tonowed by classifica¾on symbols) GO6F Eîectronic data base consuned during the intemational search (name of data base and, where pract cat search terras used} EPO-Internal C. DOCUMENTS CONSIDERED TO BE RELEVANT Category* Citation of document with indication, where appropriate. onhe relevant passages X Reîevanno claim No US 6 958 749 B1 (MATSUSHITA NOBUYUKI [JP] 1-33, 35-96 US 5 534 893 A (HANSEN JR DANIEL J LUS] ET AL) 9 July 1996 (1996-07-09) abstract column 2, line 33 - line 59 column 5, line 1 - column 9, line 46; figures 1-4 1-33, 35-96 ET AL) 25 October 2005 (2005-10-25) the whole document X O Fuither documents are foted in the continuation of Box C Special categories of cited documents : *A' document defining the general state of the art «Noi, is not considered to be of parucular relevance 'E' earher document out published on or after the irdemahonal filing date *U documed which may throw doubtson priority claim(s) or wMoh is cited to establish the pub¾eation date of another citation or other special reason las specified) 'O' document referNog to an oral disclosure. Use. exhibdion or other means "P' document published prior to the intemahond filing date but later than the priorny date claimed Date of the actual completion of the internahonal search See patent family annex. T' later document published after the internabonai tWing date or priority date and not in conflict with the appúcanort but cited to Emderstand the principle of theory underlying the invention X' document of pardeular relevanæ: the cialmed invenhon cannot be considered novel or cannot be considered to involve an inventive step when the documenOs taken alone Y' document of padicular relevance; the claimed invention cannot be considered to involve an invenhve step when the documenns combinerJ WRh one or rnere other such documents. such combinahan being obvious to a. person skmed in the att &* document member of the same patent famdy Date of malfing of the intemaßenal search report 22/04/2008 Authorized omcer 2 Vieira, L Form PCTñSA/2 iD isecond sneeg (Apr 2005) APLNDC00027301 INTERNATIONAL SEARCH REPORT information on patent famNy members International appheation No PcT/us2008/000060 Patent document Publication Patent famuy Pubi!Cation cited in search report date memberís) date US 6958749 B1 25-10-2005 JP US 5534893 A 09-07-1996 2001134382 A 18-05-2001 NONE APLNDC00027302 PATENT COOPERATION TREATY From the INTERNATIONAL SEARCHING AUTHORITY PCT WRITTEN OPINION OF THE INTERNATIONAL SEARCHING AUTHORITY (PCT Rule 43bis.1) see form PCTilSA;220 Appucants or agent's file referene" FOR FURTHER ACTION see form PCTASA/220 international application No. PCTAJS2008:000060 : see paragraph 2 oelow întemational filing date (daymonthyear) i 03,01 2008 Priority date idayhonthgear) 07.01 2007 intemational Patent Classification (IPC) or both national classification and IPC INV. GO6F3/048 Applicant APPLE INC. L This opinion contains indications relating to the following items: Box No. I Basis of the opinion O Box No. Il Priority Box No. til O Box No, IV Non-establishment of opinion with regard to novelty, inventive step and industrial applicability Lack of unity of invention Box No. V Bo× No. Vi Box No. Vil O Box No. VUI 2, Reasoned statement under Ruie 43bis..1(a)(i) with regard to novelty, inventive step or índustrial applicability; citations and explanations supporting such statement certain documents cited Certain defects in the intemational application Certain observations on the international application FURTHER ACTION If a demand for intemational preliminary e×amination is made, this opinion will usually be considered to be a written opinion of the intemational Preliminary Examining Authority ("iPEA") except that this does not apply where the applicant chooses an Authority other than this one to be the IPEA and the onosen IPEA has notifed the Intemational Bureau under Rule 66 bis(b) that written opinions of this intemational Searching Authority will not be so considered. If this opinion is, as provided above, considered to be a written opinion of the iPEA, the applicant is invitaci to submit to the iPEA a written reply together, where appropriate, with amendments, before the expiration of 3 months from the date of mailing of Form PCT/ISA220 or before the expiration of 22 months from the priority date, whichever expires later For further options. see Form POTASA220 3. For further detals, see notes to Form PCTASA220. Name and mailing address of the ISA APLNDC00027303 WRITTEN OPINION OF THE intemational application No. INTERNATIONAL SEARCHING AUTHORITY PCT/US2008;DOOO60 Box No, I Basis of the opinion 1. With regard to the language, this opinion has been established on the basis of: O the intemational application in the language in which it was filed O a translation of the intemational application into , which is the language of a translation fumished for the 2. O purposes of international search (Rules 12,3(a) and 23.1 (b)). This opinion has been established taking into account the rectification of an obvious mistake authorized by or notified to this Authority under Rule 91 (Rule 43bis.1(a)) 3. With regard to any nucleotide and/or amino acid sequence disclosed in the intemational application and necessary to the claimed inventiort this opinion has been established on the basis of: a. type of material: O a sequence listing O table(s) related to the sequence listing b. format of material: O on paper O in electronic form c, time of filingßumishing: O O filed together with the intemational application in electronic form. O 4. O contained in the intemational application as fited. fumished subsequently to this Authority for the purposes of search. In addition, in the case that more than one version or copy of a sequence listing ander table relating thereto has been filed or furnished, the required statements that the information in the subsequent or additional copies is identical to that in the application as filed or does not go beyond the application as filed, as appropriate, were furnished. 5. Addítional comments: i orm PCTASA/237 (April 2007) APLNDC00027304

Disclaimer: Justia Dockets & Filings provides public litigation records from the federal appellate and district courts. These filings and docket sheets should not be considered findings of fact or liability, nor do they necessarily reflect the view of Justia.


Why Is My Information Online?