Apple Inc. v. Samsung Electronics Co. Ltd. et al

Filing 946

EXHIBITS re #943 Declaration in Support,, Exhibits to Arnold Declaration (Ex. 84 (Part 1)) filed bySamsung Electronics America, Inc., Samsung Electronics Co. Ltd., Samsung Telecommunications America, LLC. (Attachments: #1 Ex. 84 (Part 2), #2 Ex. 84 (Part 3), #3 Ex. 84 (Part 4), #4 Ex. 84 (Part 5), #5 Ex. 85 (Part 1), #6 Ex. 85 (Part 2), #7 Ex. 85 (Part 3), #8 Ex. 86 (Part 1), #9 Ex. 86 (Part 2), #10 Ex. 86 (Part 3), #11 Ex. 86 (Part 4))(Related document(s) #943 ) (Maroulis, Victoria) (Filed on 5/18/2012)

EXHIBIT 86

U 7302677

UNITED STATES DEPAR· -- OF COMMERCE
United States Patent and Trademark Office
June 28, 2011
THIS IS TO CERTIFY THAT ANNEXED HERETO IS A TRUE COPY FROM
THE RECORDS OF THIS OFFICE OF:

U.S. PATENT: 7,864,163
ISSUE DATE: January 04, 2011

By Authority of the
Under Secretary of Commerce for Intellectual Property
and Director of the United States Patent and Trademark Office

.
LLIAMS
Certifying Officer

APLNDC00027870

IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIll l l l l l l l l l l l l l l l l l l l l
(12) United States Patent

(10) Patent No.:

Ording et al.
(54) PORTABLE ELECTRONIC DEVICE,
METHOD, AND G APRICAL USER
INTERFACE FOR DISv· --G
STRUC··-··ELECTRONICD··· ··-NTS
(75)

(56)

References Cited

U.S. PATENT DOCUMENTS
6,025,842 A

FOREIGN P
EP

Milic-Frayling, N. et al., "Smartview: Enhanced Document Viewer
for Mobile Devices," Microsoft Technical Report, Nov. 15, 2002,
URL:
ftp://ftp.research.microsoft.com/pub/tr/tr-2002-114.pdf,

retrieved Dec. 17, 2007.

(Continued)

Primary Examiner-Stephen G Sherman
(74) Attorney, Agent, or Firm-Morgan, Lewis & Bockius
LLP

Sep. 4, 2007
Prior Publication Data

(57)

Apr. 24, 2008

ABSTD M'T

A computer-implemented method, for use in conjunction
with a portable electronic device with a touch screen display,
comprises displaying at least a portion of a structured electronic document on the touch screen display, wherein the
structuredelectronic documentcomprises a plurality ofboxes
of content, and detecting a first gesture at a location on the
displayed portion of the structured electronic document. A
first box in the plurality of boxes at the location of the first

Related U.S. Application Data
(60) Provisional application No. 60/937,993, filed on Jun.
29, 2007, provisional application No. 60/946,715,
filed on Jun. 27, 2007, provisional application No.
60/879,469, filed on Jan. 8, 2007, provisional application No. 60/879,253, filed on Jan. 7, 2007, provisional
application No. 60/824,769, filed on Sep. 6, 2006.
(51) Int. Cl.
GO6F 3/041

3/1992

(Continued)

(21) Appl. No.: 11/850,013

US 2008/0094368 Al

- DOCUMENTS

0476972 A2

OTHER PUBLICATIONS

Subject to any disclaimer, the term ofthis
patent is extended or adjusted under 35
U.S.C. 154(b) by 688 days.

(65)

2/2000 Filetto et al. ................ 345/345

(Continued)

(73) Assignee: Apple Inc., Cupertino, CA (US)

(22) Filed:

Jan. 4, 2011

(58) Field of Classification Search ......... 345/173-178;
178/18.01-18.09, 18.11; 715/810, 828-831,
715/234, 781, 700
See application file for complete search history.

Inventors: Bas Ording, San Francisco, CA (US);
Scott Forstall, Mountain View, CA
(US); Greg Christie, San Jose, CA (US);
Stephen O. Lemay, San Francisco, CA
(US); Imran Chaudhri, San Francisco,
CA (US); Richard Williamson, Los
Gatos, CA (US); Chris Blumenberg,
San Francisco, CA (US); Marcel Van
Os, San Francisco, CA (US)

( * ) Notice:

US 7,864,163 B2

(45) Date of Patent:

gesture is determineçl. The first box on the touch screen display is enlarged and substantially centered.

(2006.01)

(52) U.S. Cl. ........................ 345/173; 715/234; 715/781

de

61 Claims, 29 Drawing Sheets

lht p I)www.company.com/start agga |
WeD page

Block 3
Block i

21¾

Block 7
Block4
Block 5

Block 2

Block 8
Block 6

3B18

1118

1120

3822

Copy provided by USPTO from the PIRS Image Database on 06/20/2011

APLNDC00027871

US 7,864,163 B2
Page 2

U.S. PATENT DO= -6,073,036 A

6/2000 Heikkinen et al. .......... 455/575

6,177,936 B
6,199,082 B
6,243,080 B

1/200 Cragun ....................... 345/340
3/200 Ferrel et al. ................. 707/522
6/200 Molne ........................ 345/173

6,262,732 B

7/200

Coleman et al. ............ 345/348

6,326,970 B
6,349,410 B

12/200 Mott et al. .................. 345/439
2/2002 Lortz ......................... 725/110

6,359,615
6,411,283
6,466,198
6,466,203
6,489,975
6,570,583

3/2002
6/2002
10/2002
10/2002
12/2002
5/2003

6,613,100
6,639,584
6,771,250
6,928,461
7,054,965
7,075,512
7,149,549

Bl
B1
B1
B2
B1
B1

B2
B1
B1
B2
B2
Bl
B1

9/2003
10/2003
8/2004
8/2005
5/2006
7/2006
12/2006

Singh .........................
Murphy ......................
Feinstein ....................
Van Ee .......................
Patil et al. ...................
Kung et al. .................

345/173
345/173
345/158
345/173
345/781
345/661

Miller ........................ 715/526
Li .............................. 345/173
Oh ............................. 345/156
Tuli ........................... 709/203
Bell et al. ..................... 710/72
Fabre et al. ................. 345/156
Ortiz et al. .................. 455/566

7,166,791
7,346,855
7,461,353
2002/0152283
2003/0033331
2003/0095135
2003/0095155
2003/0164861
2003/0193524

B2
B2
B2
A
A
A
A
A
A

1/2007
3/2008
12/2008
10/2002
2/2003
5/2003
5/2003
9/2003
10/2003

Robbin et al. ................. 84/477
Hellyar et al. .............. 715/783
Rohrabaugh et al. ........ 715/815
Dutta et al. ................. 709/218
Sena et al. .................. 707/513
Kaasila et al. .............. 345/613
Johnson ...................... 345/864
Barbanson et al. .......... 345/815
Bates et al. ................. 345/786

2004/0143796
2004/0169674
2004/0201595
2005/0071364

Al
Al
Al
Al

7/2004
9/2004
10/2004
3/2005

Lerner et al. ................
Linjama .....................
Manchester .................
Xie et al. ....................

2004/0103371 Al

5/2004 Chen et al. .................. 715/513

2005/0071778 Al

715/538
345/702
345/649
707/102

3/2005 Tokkonen ................... 715/822

2005/0079896 A

4/2005 Kokko et al. ................ 345/702

2005/0114788
2005/0166232
2005/0177783
2005/0285880
2006/0026521
2006/0033761
2006/0064647
2006/0085743
2006/0097991
2006/0101354
2006/0 25799
2006/0 43574
2006/0 46016
2006/0 46038
2006/0 97753
2006/0277588
2006/0284852
2007/0038612
2007/0152984
2007/0155434
2007/0157228
2007/0250768

5/2005
7/2005
8/2005
12/2005
2/2006
2/2006
3/2006
4/2006
5/2006
5/2006
6/2006
6/2006
7/2006
7/2006
9/2006
12/2006
12/2006
2/2007
7/2007
7/2007
7/2007
10/2007

2005/0093826 A

5/2005 Huh ........................... 345/168

A
A
A
A
A
A
A1
A
A
A
A *
A
A
A
A
A
A *
A
A
A
A
Al *

FOREIGN P
EP
EP
EP
EP
EP
EP
EP
EP
JP

0 651 544
0 701 220
0880090
0 990 202
I 049 305
1 517 228
1 632 874
1 752 880
11 143604

Fabritius ..................... 715/767
Lamkin et al. ................ 725/43
Agrawala et al. ........... 715/512
Lai et al. .................... 345/660
Hotelling et al. ............ 715/702
Suen et al. .................. 345/660
Tapuska et al. ............. 715/800
Baudisch et al. ............ 715/526
Hotelling et al. ............ 345/173
Hashimoto et al. .......... 715/863
Hillis et al. ................. 345/173
Ito et al. ..................... 715/800
Chan et al. .................. 345/ 56
Park et al. ................... 345/ 73
Hotelling .................... 345/ 73
Harrington et al. .......... 725/ 35
Hofmeister et al. ......... 345/173
Sull et al. ...................... 707/3
Ording et al. ............... 345/ 73
Jobs et al. ................... 455/ 65
Bayer et al. ................... 725 34
Funakami et al. ........... 715/ 21

DO
A2
A1
A2
A1
Al
A2
A2
Al

5/1995
3/1996
11/1998
4/2000
11/2000
/2005
/2006
/2007
5/1999

S

JP
JP
WO
WO
WO
WO
WO
WO
WO
WO
WO
WO
WO
WO

2000181436
2005 267049
WO 99/54807
WO 02/46903
WO 02/082418
WO 02/093542
WO 02093542
WO 03/052626
WO 2004/021166
WO 2004/040481
WO 2005/036416
WO 2005/074268
WO 2005/106684
WO 2006/003591

6/2000
9/2005
A1 10/1999
A1
6/2002
A2 10/2002
Al
11/2002
Al * 11/2002
A1
6/2003
Al
3/2004
Al
5/2004
A2
4/2005
Al
8/2005
Al
11/2005
A2
1/2006

- - PUBLICATIONS
Holmquist, L., "The Zoom Browser Showing Simultaneous Detail
and Overview in Large Documents," Human IT, 1998, URL : http://
www.hb.selbhs/ith/3-98/leh.htm, retrieved Dec. 17, 2007.
Khella,A. et al., "Pocket PhotoMesa: AZoomable Image Browserfor
PDAs," Proceedings of the 3rd International Conference on Mobile
and Ubiquitous Multimedia, Oct. 29, 2004, pp. 19-24, URL: http://
delivery.aem.org/10.1145/1060000/1052384/p19-khella.
pdf?keyl=1052384&key2=2419987911&co11=GUIDE
&d1=GUIDE&CFID-47073625- - · ·KEN=65767142, retrieved
Dec. 17, 2007.
Invitation to Pay Additional Fees for International Application PCT/
US2007/077644, dated Jan. 23, 2008.
Cooper, A., "The Inmates Are Running the Asylum," Sams Publishing, 1999, pp. 138-147.
European Search Report dated Jan. 26, 2010, received in European
Application No. 09171787.6-2212, which corresponds to U.S. Appl.
No. 11/850,013.
Versiontracker, "Photogather-7.2.6. Hi-res Image Viewer & Editor
for Palm," printed Jun. 12, 2006, 5 pages, http://www.versiontracker.
com/dyn/moreinfo/palm/4624.
Office Action dated Sep. 28, 2009, received in Australian Patent
Application No. 2009100760, which corresponds to U.S. Appl. No.

11/850,013.
Agarwal, A., "iTunesInlineVideo," Digital Inspiration-The Tech
Guide,
27
pages,
httpl/:labnol.blogspot.com/

2006_09_17_labnol_archive.html.
Ahmad I. et al., "Content-Based Image Retrieval on Mobile
Devices," Proc. Of SPIE-IS&T Electronic Imaging, vol. 5684, 2005,
10 pages.
Alam, H. et al., "Web Document Manipulation for Small Screen
Devices: A Review," 4 pages, http://www.csc.liv.ac.uk/~wda2003/
Papers/Section_II/Paper_8.pdf, 2003.
Apparao, V. et al., "Level 1 Document Object Model Specification,"
W3C Working Draft Jul. 20, 1998, 3 pages, http://www.w3.org/TR/
WD-DOM/.
Bos, B. et al., "3 Conformance: Requirements and Recommendations," Cascading Style Sheets, level 2 CSS2 Specification, W3C
Recommendation, May 12, 1998, 6 pages, http://www.w3.org/TR/
CSS21/conform.html#doctree.
Chen, L. et al., "DRESS: A Slicing Tree Based Web Representation
for Various Display Sizes," Microsoft Researach, Technical Report
MSR-TR-2002-126, Nov. 16, 2002, 9 pages.
Eyemodule, "TurnYour Handspring= Handheld into a Digital Camera," User's Manual, www.eyemodule.com, 9 pages, 2000.
Fling, B., "Designing for Mobile, Bringing Design Down to Size,"
Copyright© 2006 Blue Flavor.
, Hart, K., "Rewriting the Web for Mobile Phones," washingtonpost.
com, Jul. 26, 2006, 2 pages, http://www.washingtonpost.com/wpdyn/content/article/2006/07/25/AR2006072501517_pf.html.
Hinc1dey et al., "Input/Output Devices and Interaction Techniques,"
Microsoft Research, 79 pages, 2003.
Laakko, T. et al., "Adapting Web Content to Mobile User Agents,"
IEEE Internet Computing, vol. 9, Issue 2, lylar./Apr. 2005, 8 pages.
Opera Software, "Downloadthe Opera Mobile= Browser," 5 pages,
http://www.opera.com/products/mobile/products/,2006.

Copy provided by USPTO from the PIRS Image Database on 06/20/2011

APLNDC00027872

US 7,864,163 B2
Page 3
Opera Software, "Opera7.60 for Series 60 Mobile," http://jp.opera.
com/support/tutorials/s60/760/O760manual.pdf 2009.
Palme, J. et al., "MIME Encapsulation of Aggregate Documents,
such as HTML," The Internet Society, 1999, 24 pages.
Raman, B. et al., "Application-specific Workload Shaping in Multimedia-enabled Personal Mobile Devices," CODES+ ISSS' 06, Oct.
22-25, 2006, Seoul, Korea, Copyright 2006 ACM, 6 pages.
Robie, J., "What is the Document Object Model?" Texcel Research,
5 pages, http://www.w3.orgTR-DOM/introduction.html,2006.
Rohrer, T., "Metaphors We Compute by: Bringing Magic into Interface Design," http://www.uoregon.edu/-uophillmetaphor/gui4web.
htm, printed Jun. 13, 2006, 7 pages.
Roto, V. et al. "Minimap-A Web Page Visualization Method for
Mobile Phones," CHI 2006, Nokia Research Center, Apr. 22-27,

2006, 10 pages.
Salmre, I., "Chapter 2, Characteristics of Mobile Applications,"
Salme_02.fm, pp. 19-36, Dec. 20, 2004.
Schreiner, T., "High DPI in IE: Tip & Mystery Solved " Tony
Schreiner's Weblog, May 2004, 2 pages, http://blogs.msdn.com/
tonyschr/archive/2004/05/05/126305.aspx.
Stampfli, T., "Exploring Full-Screen Mode in Flash Player 9," Jan. 5,
2007,
http://web.archive.org/web20070105231635/http://www.
adobe.com/devnet/flashplayer/articles/fullis screen_mode.html.
Stanek, W. et al., "Chapter 9, Video and Animation Plug-Ins," Web
Publishing Professional Reference Edition, copyright 1997 by Sams.
net Publishing, http://www.ssuet edu.pk/taimoor/books/1-57521198-X/index.htm.
Stanek, W. et al., "Chapter 22, Adding MultimediatoYourWeb Site,"
Web Publishing Professional Reference Edition, copyright 1997 by
Sams.net Publishing, http://www.ssuet.edu.pk/taimoor/books/1-

57521-198-X/index.htm.
Surfin'Safari, "XUL," 7 pages, Oct. 2003, http://weblogs.mozillazine.org/hyatt/archives/2003_10.html.
w3 schools.com, "Multimedia Video Formats," www.w3sschools.
com/medialmedia videoformats.asp?output=print,2006.
w3schools.com, "Playing QuickTime Movies;' http://www.
w3schools.com/media/media quicktime.asp?output=print,2006.
w3schools.com, "Playing Videos On The Web," www.w3schools.
com/medialmedia_browservideos.asp?out=print, 2006.
Wave Technologies, "Certified Internet Webmaster Foundations
Study Guide," Wave Technologies Internation, Inc., a Thomson

Learning company, copyright 1988-2000.
Warabino, T. et al., "Video Transcoding Proxy for 3GwirelessMobile
InternetAccess," IEEE CommunicationsMagazine,vol. 38, Issue 10,
Oct. 2000, 6 pages.
weblogs, "Chapter 1: Downloading and Building WebCore,"
WebCore documentation, 2 pages, http://weblogs.mozillazine.org/
hyatt/WebCore/chapter1.html, 2006.
weblogs, "Chapter 2: An Overview of WebCore," WebCore documentation, 3 pages, http://weblogs.mozillazine.org/hyatt/WebCore/
chapter2.html, 2006.
webmasterworld.com, "Page Zooming with IE," webmasterworld.
com, Jul. 2004, 7 pages, http://www.webmasterworld.com/forum83/

4179.htm.
Wikipedia, "Comparison of Layout Engines," Wikipedia, the free
encyclopedia, 3 pages, http://en.wikipedia.org/wiki/Comparison
of_loyout_engines, 2005.
Wikipedia, "KDE,"Wikipedia, the free encyclopedia, 9 pages, http://

en.wikipedia.org/wiki/KDE, 2004.
Wikipedia, "

," Wikipedia, the free encyclopedia, 3 pages,

http://en.wikipedia.org/wiki/KDHTML,2004.
Wikipedia, "List ofLayout Engines," 1 page, http://en.wikipedia.org/
wiki/List_of_layout_engines, 2005.
Wobbrock, J. et al., "WebThumb: Interaction Techniques for SmallScreen Browsers," Human Computer Interaction Institute and School
of Design, Carnegie Mellon University, {jrock, forlizzi, scott.
hudson, bam}@cs.cmu.edu, Oct. 27-30, 2002, 4 pages.
Yin, X. et al., "Using Link Analysis to Improve Layout on Mobile
Devices," ww w2004, May 17-22, 2004, 7 pages, http://www.
iw3c2.org/WWW2004/docs/1p338.pdf.
Xiao, X. et al., "Slicing*-Tree Based Web Page Transformation for
Small Displays," CIKM'05, Oct. 31-Nov. 5, 2005, Bremen, Gero
many, 2 pages.

Xie, X. et al., "Efficient Browsing ofWeb Search Results on Mobile
Devices Based on Block Importance Model;' Microsoft Research
Asia, mgx03@ mails.tsinghua.edu.cn, 10 pages, 2005.
Zhiwei et al., "Zoom Selector: A Pen-basedInteractionTechnique for
Small Target Selection," Transactions of the Information Processing
Society of Japan, Aug. 2004, vol. 45, No. 8, pp. 2087-2097, Inf.
Process. Soc. Japan, ISSN 0387-5806.
International Search Report and Written Opinion for International
Application No. PCT/lJS2007/088893, mailed Jul. 11, 2008.
Designing Interfaces.com, "Animated Transition," http://
designinginterfaces.com/Animated_Transition, printed Oct. 16,
2006, 2 pages.
Alejandre, S., "Graphing Linear Equations," http://mathforum.org/
alejandre/palm/times.palm.html, printed Jun. 12, 2006, 3 pages.
Baudisch, P., "Collapse-to-Zoom: Viewing Web Pages on Small
Screen Devices by Interactively Removing Irrelevant Content," Oct.
24-27, 2004, 4 pages.
Bitstream®, ""--Hawk Pocket PC Edition for End Users,"
http://www.bitstream.com/wireless/products/pocketpc/faq_using.
html, printed Jun. 12, 2006, 4 pages.
Buyukkokten, O. et al., "Power Browser: Efficient Web Browsing for
PDAs," Digital Libraries Lab (InfoLab), Stanford University,
Stanford, CA, 8 pages, 2002.
Chen, H. et al, "A Novel Navigation and Transmission Technique for
Mobile Handheld Devices," 8 pages, 2002.
Chen, Y., "Detecting Web Pages Structure for Adaptive Viewing on
Small
Form
Factor
Devices,"
Microsoft
Research,
[e-mail], May 20-24, 2003, 9 pages.
Coolsmartphone, "Orange SPV C600 Review," http://www
coolsmartphone.com/articleS69.html, Apr. 14, 2006, 58 pages.
Getgreg, "JeffHan's Multiple Touch Point Display, the StuffDreams
are
Made
of,"
http://www.theyshoulddothat.com2006/08/
jeff_hanns multiple_touch_poin.html>,Aug. 16, 2006, 2 pages.
Han, J., "Talks Jeff Han: Unveiling the Genius of Multi-touch Interface Design," Ted Ideas Worth Spreading, http://www.ted.com/
inhttp://www.ted.com/index.php/talks/view/id/65>,Aug. 6, 2006, 1
page.
Karlson et al., "AppLens and Lunch Tile: Two Designs for Onehanded Thumb Use on Small Devices," http://heil.cs.umd.edu/trs/
2004-37.html, printed Jun. 12, 2006, 11 pages.
Landragin, F., "The Role of Gesture in Multimodal Referring
Actions," Proceedings ofthe Fourth IEEE Internaational Conference
on Multimodal Intèrfaces, http://ieeexplore.iee.org/ie15/8346il
26309/01166988pdf?arnumber=116i6988>,2002, 6 pages.I.
Lie, H., "Cascading Style Sheets," http://people.opera.com/
howcome/2006/phd/css.pdf 2005, pp. 243-247.
Milic-Frayling, N. et al., "Smartview: FlexibleViewing ofWeb Page
Contents," The Eleventh International World Wide Web Conference,
http://www2002.org/CDROM/poster/172/>,May 11, 2002, 4 pages.
Opera Software, "Opera for Mobile, The Full Web Anytime, Anywhere," www.opera.com/mobile, Jan. 2006, 7 pages.
Opera Software, "Opera for S60 Tutorial," http://www.opera.com/
support/tutorials/260/, Apr. 5, 2006, 5 pages.
Opera-Press Releases Database, "The New Opera Browser for
Series 60 Features Zoom and Password Manager," Nov. 14, 2005, 3
pages.
Opera Software, "Opera for Windows Mobile Smartphone 2003
Tutorial," http://www.opera.com/support/tutorials/winmobile, Apr.
5, 2005, 4 pages.
Opera Software, "Opera 8.5 Beta2 forWindows Mobile, Pocket PC,"
http://www.opera.com/products/mobile/products/winmobileppc,
Apr. 5, 2006, 2 pages.
Opera, "Opera 8.5 for S60 Phones-Get the Full Internet Experience
onYourMobilePhone,"http://www.symbian-freak.com/news/1105/
opera.htm, Apr. 5, 2006, 3 pages.
International Search Report and Written Opinion for International
Application No. PCT/US2007/077644, mailed May 30, 2008.
International Search Report andWritten Opinion dated Jun. 30, 2008,
received in International Application PCT/US2007/088879 which
corresponds to U.S. Appl. No. 11/620,647.

Copy provided by USPTO from the PIRS Image Database on 06/20/2011

APLNDC00027873

US 7,864,163 B2
Page 4
Office Action dated Nov. 17, 2009, received in U.S. Appl. No.
11/620,647.
Office Action dated Jun. 24, 2010, received in U.S. Appl. No.
11/620,647.
Examiner's ReportdatedMar. 24, 2010, receivedinAustralian Patent
Application No. 2007292383, which corresponds to U.S. Appl. No.
11/850,013, 2 pages.
Office Action dated Jun. 7, 2010, received in German Patent Application No. 11 2007 002 107.1-53, which corresponds to U.S. Appl.
No. 11/850,013.

Office Action dated Jun. 21, 2010, received in European Application
No. 07 814 690.9-2212, which corresponds to U.S. Appl. No.
11/850,013.
Office Action dated Oct. 13, 2010, received in Chinese Patent Application No. 20078004122.6, which corresponds to U.S. Appl. No.
11/850,013.
Office Action dated Oct. 19, 2010, received in European Application
No. 07 814 690.9, which corresponds to U.S. Appl. No. 11/850,013.

* cited by examiner

Co PY P rovided bY USPTO from the PIRS Ima9 e Database on 06/20/2011

APLNDC00027874

U.S. Patent

Jan. 4, 2011

Sheet 1 of 29

Memory

Portable Multifunction Device

Operating System
Communication Module
Contact/Motion Module
Graphics Module
Text Input Module
GPS Module
Applications
Contacts Module

/ 126 Applications (continued
; 128
Calendar Module
7130
Widget Modules
132
/
Weather Widget
7134
Stocks Widget
135
136
Calculator Widget
137
Alarm Clock Widget
138
Dictionary Widget

Telephone Module
Video Conference Module
E-mail Client Module
Instant Messaging Module
Biogging Module
Camera Module
Image Management Module
Video Player Module
Music Player Module
Browsing Module
103
104
122

US 7,864,163 B2

139
140
141
142
143
144
145
146
147
118

1

136
148
149
149-1

149-2
149-3
149-4
149-5

•
User-Created Widget(s)
-Widget Creator Module
Search Module
$

149-6
150
151

Power iy 162
System /
External ;124
^103

103

|
RF Circuitry
Speaker

111
Controller
4103

‡

C rc try
Pericherals

103

Interface

110

Micro hone

Proximity
Sensor

120

Processor(s)

113
166

103

Accelerometer(s)

Display
Controller

103
I/O Subsystem
.
Optical
sensor(s)
Controller

103

103

106 J

156

112

Touch-Sensitive
Display System

Optical
Sensor(s)
.1§&

158
Other input
Controller(s)

160

103
Other Input
Control
Devices

7116

Figure 1A

Conv nrovided by USPTO from the PIRS Image Database on 06/20/2011

APLNDC00027875

U.S. Patent

Jan. 4, 2011

Sheet 2 of 29

Memory E

Portable Multifunction Device

Operating System

126

Communication Module
Contact/Motion Module
Graphics Module
Text Input Module
GPS Module

m

128 Applications (continued)
V 130
Calendar Module
I
132
134
Widget Modules
Weather Widget(s)
135
136
Stocks Widget

Applications

137

Contacts Module
Telephone Module
Video Conference Module
E-mail Client Module
Instant Messaging Module

138
139
140
141
142
143
144
152

Biogging Module
Camera Module
Image Management Module
Video and Music Player Module

Notes Module

I

Map Module
Browsing Module
103

104
122

US 7,864,163 B2

153
154
147
118

149
149-1
149-2
149-3
149-4

Calculator Widget

Alarm Clock Widget
Dictionary Widget
•
-•
- -User-Created Widget(s)
Widget Creator Module

149-5
149-6
150
151

Search Module
Power , 162
System /
External
Port
103

103

124

RF Circuitry

Speaker

111

Controller
103

120

Peripherals
Interface

103

Cr ry
110

Proximity
103 Sensor

Micro hone
113
166

Processor(s)

†
(103
106
Display

Controller

I/O Subsystem
Optical
sensor(s)

Controller

Accelerometer(s) I

m

|

158
Other Input

,160

Controller(s)

156

103

112

.
Touch-Sensitive
Display System

103

103

Optical
Sensor(s)

Other Input
Control

§.4

Devices

;116

Figure 1B

Copy provided by USPTO fmm the PIRS Immaa Database on 06/20/2011

APLNDC00027876

U.S. Patent

Jan. 4, 2011

Sheet 3 of 29

US 7,864,163 B2

Portable Multifunction Device 100
| 206

SIM Card Slot 210
\

Speaker 1_1_1

Optical
Sensor B

Proximity a
Sensor y

212 |
head- '
phone

jack

200

202

Touch Screen 1_1_2

' Microphone ' / Home

|

External Port 1B

Accelerometer(s)

|

Figure 2

Copy provided by USPTO from the PIRS Image Database on 06/20/2011

APLNDC00027877

U.S. Patent

Jan. 4, 2011

Sheet 4 of 29

US 7,864,163 B2

Portable Multifunction Device
100
Optical >
Sensor 164 ;|

Speaker 111

300

Proximity
Sensor 1_66

Current Time
310 - Day and Date

308
312

Wallpaper image

304

slide to unlock

302

306

Touch Screen 112
Microphone '

Home

Accelerometer(s)>

Figure 3

Copy provided by USPTO from the PIRS Image Database on 06/20/2011

APLNDC00027878

U.S. Patent

Jan. 4, 2011

Sheet 5 of 29

US 7,864,163 B2

Portable Multifunction Device
100

2_Qg

Optical

Speaker 111
400A

02

Proximity

Sensor1§A

Sensor1.

Current Time

n dos

IM

Text

Photos

Camera

Videos

141

144

143

145

Jan
75.

15

Weather

Stocks

Blog

Calendar

140-1

149-2

142

1AZ

. .

User-

+-x÷

i. ..·

ABC

Created
Widget

Calculator

Alarm

Dictionary

Widget

jg9-3

1 9-4

149-5

149-8

410
6

Phone
138

'

Mail
140

Browser
147

Music
146
Touch Screen 112

' Microphone >

Home

Accelerometer(s) '

110.

204

1ññ

Figure 4A

Copy orovided bv USPTO from the PIRS Imaae Database on 06/20/2011

APLNDC00027879

U.S. Patent

Jan. 4, 2011

US 7,864,163 B2

Portable Multifunction Device
100

2_Qg

400B

Sheet 6 of 29

Optical
Sensor 1ß_4

Speaker 11î

Q2 o

Proximity
Sensor 1§.

Current Time o

Text

m. 10.6'

Calendar

Photos

Camera

148

144

g_g

+-X

75°

Calculator

Stocks

Map

Weather

140-3

14_R-2

1p_4

140-1

Notes

Clock

Settings

1 3

149-4

412

414

Phone

,

Mail

410

Browser

iPod

Touch Screen 112

' Microphone

Home "

Accelerometer(s)

îî

B i

168

Figure 4B

Copy provided by USPTO from the PIRS Imacle Database on 06/20/2011

APLNDC00027880

U.S. Patent

Jan. 4, 2011

Sheet 7 of 29

US 7,864,163 B2

Portable Multifunction Device
206

100

Optical
Sensor 164

Speaker --111

3900A

1 E2

' Proximity '
. Sensor j_6 |

Current Time saa

m. son'

http:llwww.company.com/staÀ 3908

3910

Web page
Block 3
3914-3

Block 1
3914-1

Block 7

Block 4

" *¯

3914-4

,·a g,

Block 5

''

3914-5

Block 2

'ai

Block 8

3914-2

3914-8

Block 6

3916

918

Microphone

3920

/ Home

3922

Accelerometer(s)

Figure 5A

Copy provided by USPTO from the PIRS Immae Database on 06/20/2011

APLNDC00027881

U.S. Patent

Jan. 4, 2011

Sheet 8 of 29

Portable Multifunction Device
100

206

'
3900 B

'

US 7,864,163 B2

~

Speaker

'

Optical

/

'

' Proximity '

Sensor B

Sensor 166
¯¯

2-1 m

Current Time m

Cancel

o m'

Welcome 3904

http://www.company.com| 3926

Clear 3928

Search 39_3_o

Go to URL 3932

QWERTYU

A

S

D

I

J

OP

F

G

H

K

xo

V

L

BNM

.?123
Microphone ' ' Home

Accelerometer(s)

Figure 5B

Conv nrovided bv USPTO from the PIRS Image Database on 06/20/2011

APLNDC00027882

U.S. Patent

Jan. 4, 2011

US 7,864,163 B2

Portable Multifunction Device
100

| 2M

Optical '|
Sensor 164

Speaker 111
-

Sheet 9 of 29

-

3900C

402

Proximity
Sensor 16_6

Current Time aos

gang

ch i

Welcome 3901

3906

http://www.company.com/start 3908
.'3 fg

BIOck 4 3914-4

s'3929's

Block 5
3_914-5
s'3941 i

4
< 27° /
/

3939
> 27°,'
,/

..
e3943i

Block 6 3914-6

3916

3918

Microphone '

*

3920

Hom

3922

Accelerometer(s)

Figure 5C

Copy proyided by USPTO from the PIRS Image Database on 06/20/2011

APLNDC00027883

U.S. Patent

Jan. 4, 2011

3900D

US 7,864,163 B2

Portable Multifunction Device
100

Qg

-

Sheet 10 of 29

Speaker -111

02

Optical
Sensor 1.§§

' Proximity '
Sensor îQû

Current Time aos

Microphone ' / Home

a> soi

Accelerometer(s)'

Figure 5D

Copy proyided by USPTO from the PIRS Image Database on 06/20/2011

APLNDC00027884

U.S. Patent

Jan. 4, 2011

3900E

US 7,864,163 B2

Portable Multifunction Device
100

206

-

Sheet 11 of 29

Optical '
Sensor 164

Speaker 111

1 402

Proximity '
Sensor 166

ÛUrrent Time a

Em

o

×

o
o

O

O

Microphone

Home

2

O

' Accelerometer(s) '

Figure 5E

Copy provided by USPTO from the PIRS Image Database on 06/20/2011

APLNDC00027885