Freshub, Inc. et al v. Amazon.Com Inc. et al

Filing 1

COMPLAINT ( Filing fee $ 400 receipt number 0542-12267080), filed by Freshub, Inc., Freshub, Ltd.. (Attachments: # 1 Exhibit 1, # 2 Exhibit 2, # 3 Exhibit 3, # 4 Exhibit 4, # 5 Exhibit 5, # 6 Exhibit 6, # 7 Exhibit 7, # 8 Exhibit 8, # 9 Exhibit 9, # 10 Exhibit 10, # 11 Exhibit 11, # 12 Exhibit 12, # 13 Exhibit 13, # 14 Exhibit 14, # 15 Exhibit 15, # 16 Exhibit 16, # 17 Exhibit 17, # 18 Exhibit 18, # 19 Exhibit 19, # 20 Exhibit 20, # 21 Exhibit 21, # 22 Exhibit 22, # 23 Exhibit 23, # 24 Exhibit 24, # 25 Exhibit 25, # 26 Exhibit 26, # 27 Exhibit 27, # 28 Exhibit 28, # 29 Exhibit 29, # 30 Exhibit 30, # 31 Exhibit 31, # 32 Exhibit 32, # 33 Exhibit 33, # 34 Exhibit 34, # 35 Exhibit 35, # 36 Exhibit 36, # 37 Exhibit 37, # 38 Exhibit 38, # 39 Exhibit 39, # 40 Civil Cover Sheet)(Palmer, John)

IN THE UNITED STATES DISTRICT COURT
WESTERN DISTRICT OF TEXAS
WACO DIVISION
)
)
)
)
)
)
)
)
)
)
)
)
)
)
)
)

FRESHUB, INC., a Delaware Corporation, and
FRESHUB, LTD., an Israeli Limited Liability
Company,
Plaintiffs,
vs.
AMAZON.COM, INC., a Delaware
Corporation, AMAZON DIGITAL SERVICES,
LLC, a Delaware Limited Liability Company,
PRIME NOW, LLC, a Delaware Limited
Liability Company, and WHOLE FOODS
MARKET, INC., a Texas Corporation,
Defendants.

Case No. 6:19-cv-388
DEMAND FOR JURY TRIAL

COMPLAINT FOR PATENT INFRINGEMENT
Plaintiffs Freshub, Inc. and Freshub, Ltd. (together “Plaintiffs” or “Freshub”) jointly
file this Complaint for Patent Infringement and Demand for Jury Trial against Amazon.com,
Inc., Amazon Digital Services, LLC, AmazonFresh, LLC, Prime Now, LLC, and Whole Foods
Market, Inc. (collectively “Amazon” or “Defendants”) and allege as follows:
THE PARTIES
1.

Freshub, Inc. is a Delaware corporation with its principal place of business at 29

South Third Street, Suite 1C, in Brooklyn, New York. Freshub, Inc. is a wholly owned
subsidiary of Freshub, Ltd. Freshub, Ltd. is an Israeli limited liability company with its
principal place of business in Giv’at Shmuel, Israel. Freshub, Ltd. is a subsidiary of Ikan
Holdings, LLC (“Ikan”).
2.

Amazon.com, Inc. is a Delaware corporation which maintains its headquarters

at 410 Terry Avenue North in Seattle, Washington. Amazon.com, Inc. may be served through
1

its agent for service of process, Corporation Service Company, 251 Little Falls Drive in
Wilmington, Delaware.
3.

Amazon Digital Services, LLC is a Delaware limited liability company

registered to transact business in Texas with the Texas Secretary of State. Amazon Digital
Services, LLC maintains its headquarters at 410 Terry Avenue North in Seattle, Washington.
Amazon Digital Services, LLC may be served through its agent for service of process,
Corporation Service Company, 251 Little Falls Drive in Wilmington, Delaware. On
information and belief, Amazon Digital Services, LLC is a wholly owned subsidiary of
Amazon.com, Inc.
4.

Prime Now, LLC is a Delaware limited liability company registered to transact

business in Texas under the assumed name “AmazonFresh” with the Texas Secretary of State.
Prime Now, LLC maintains its headquarters at 410 Terry Avenue North in Seattle,
Washington. Prime Now, LLC may be served through its agent for service of process,
Corporation Service Company, 251 Little Falls Drive in Wilmington, Delaware, 19808. On
information and belief, Prime Now, LLC is a wholly owned subsidiary of Amazon.com, Inc.
5.

Whole Foods Market, Inc. is a Texas corporation. Whole Foods Market, Inc.

maintains its headquarters at 550 Bowie Street in Austin, Texas. Whole Foods Market, Inc.
may be served through its agent for service of process, CT Corporation System, 1999 Bryan
Street, Suite 900 in Dallas, Texas, 75201. On information and belief, Whole Foods Market,
Inc. is a wholly owned subsidiary of Amazon.com, Inc.

2

JURISDICTION AND VENUE
6.

This action for patent infringement arises under the patent laws of the United

States, 35 U.S.C. § 101 et seq. This court has original jurisdiction over this controversy
pursuant to 28 U.S.C. §§ 1331 and 1338.
7.

Venue is proper in this Court pursuant to 28 U.S.C. §§ 1391(b), (c), and (d),

and/or 1400(b).
8.

This Court has personal jurisdiction over Amazon because Amazon regularly

and continuously does business in this District and has infringed or induced infringement, and
continues to do so, in this District. On information and belief, Amazon specifically targets its
products and/or services to residents of this District in and around Waco, Texas since at least as
early as 2014. Exhibit 1. On information and belief, Amazon maintains a large office
employing thousands of workers, including management-level and engineering employees, in
this District. Exhibit 4. On information and belief, Amazon maintains around ten Fulfillment
Centers in Texas, including a Fulfillment Center to service its customers in this District in or
around West, Texas, to provide products and/or services to residents in this District and others.
See Exhibits 2–3. Further, on information and belief, Amazon maintains other Fulfillment
Centers in this District, including in San Marcos and Schertz, Texas. Exhibit 4. Upon
information and belief, Amazon’s locations in this District are regular and established places of
business. In addition, the Court has personal jurisdiction over Amazon because minimum
contacts have been established with the forum and the exercise of jurisdiction would not offend
traditional notions of fair play and substantial justice. For example, Amazon advertises active
job listings in this District and makes, uses, offers for sale, and sells products or services that
infringe the Patents-in-Suit in this District, as further described below.

3

THE PATENTED INNOVATIONS
9.

Ikan developed the technology that enables users to use natural voice commands

to build and maintain lists and purchase retail items as early as 2005, and filed patent
applications on its inventions related to, inter alia, voice processing and voice interpretation
technology. The four patents involved in this case resulted from these inventions.
10.

On information and belief, Amazon is and has been aware of the patented

technologies at issue since at least as early as 2010. For example, Ikan developed a wireless
scanner product using related patented technologies in the “smart home” and in-home
commerce space which was featured in The New York Times in 2008 and Forbes as one of
“The 10 Best Time-Saving Home Appliances” in 2010. Exhibits 5–6. Only five days later, an
Amazon representative working on AmazonFresh requested a copy of the wireless scanner,
citing a potential opportunity with Amazon and that he had seen the feature in Forbes.
Amazon confirmed receipt of the product in November 2010 and stated it intended to try it. At
the time, these scanners were shipped in boxes marked with four relevant patents and the
further statement that the product within was also covered by “other patents pending.” As
such, on information and belief, Amazon was aware that it had received and would be testing
patented technology with additional patents pending in the “smart home” and in-home
commerce technology space.
11.

On information and belief, Amazon is also aware of U.S. Patent No. 9,821,344

(the “’344 Patent”), filed on December 12, 2005. The ‘344 Patent is an earlier patent in the
priority chain shared by all the Asserted Patents, such that it and all the Asserted Patents are
directly related. All the Asserted Patents claim priority back through the ‘344 Patent to
Provisional Application Serial No. 60/635,122. The ‘344 Patent is a cited reference on the face

4

of Amazon’s U.S. Patent No. 9,165,320. Exhibit 7 (citing the publication number of the ‘344
Patent, which is US 2006-0178947 A1). In fact, Amazon through its prosecution counsel cited
the published application for the ‘344 Patent for the patent examiner’s consideration. Exhibit
8. Thus, on information and belief, Amazon is and has been aware of at least the earliest
issued patent in the priority chain of the Asserted Patents and its relevance to Amazon’s smart
home and in-home commerce technology.
12.

Freshub is a leading technology provider and systems integrator in the smart

kitchen commerce ecosystem. Freshub brings grocery retailers and appliance manufacturers
together to make the smart kitchen a reality, offering consumers full access to online
supermarkets via kitchen appliances such as connected microwaves, kitchen music players and
smart bins. Consumers can add products to their digital shopping cart by simply waving the
package in front of the appliance or using voice commands. Freshub’s groundbreaking Smart
Kitchen Commerce solution, powered by technology enabling any grocery retailer to connect
directly to the customer’s kitchen via IoT-driven smart appliances, enables consumers to easily
add any grocery item to their carts or shopping lists, and to quickly order or replenish products
at any time. The Freshub application allows them to do so in the comfort of their own homes,
right from their kitchen, via natural voice commands or with a simple wave of the hand.
Freshub additionally helps consumers maximize savings by providing them with purchase
history-driven personalized offers and discounts, and enabling them to easily calculate their
potential savings.
13.

Ikan merged with Freshub, Ltd. and granted all rights, title, and interest in the

patents in this case to Freshub, Ltd. Freshub, Ltd. granted a license to the patents to Freshub,
Inc. Freshub’s patented technologies include software and cloud-based solutions to problems

5

encountered by consumers, retailers, and electronic shopping platform providers alike. The
ability to manage information and order items by voice removes barriers to entry for in-home
shopping by increasing consumer convenience and flexibility. In turn, this greater convenience
promotes adoption of the technology that facilitates in-home shopping and enables retailers and
electronic shopping platform providers to capture economic activity that was previously
unavailable to them. Freshub owns a number of domestic patents and continues to innovate in
the Internet of Things (“IoT”) and voice processing space.
14.

Amazon is and has been aware of Freshub since at least as early as 2015. From

2015 until as recently as about May 2019, Freshub and Amazon have had ongoing discussions
about a potential partnership or collaboration. Amazon and Freshub representatives have even
twice participated on a panel together about in-home commerce models and technology at the
annual Smart Kitchen Summit (“SKS”), in 2015
(https://www.smartkitchensummit.com/videos2015/2015/11/7/food-commerce-and-the-smartkitchen) and 2018 (https://www.smartkitchensummit.com/schedule). Exhibits 9 and 10.
FACTUAL BACKGROUND
15.

On March 6, 2018, the United States Patent and Trademark Office (“PTO”)

issued U.S. Patent No. 9,908,153 (the “’153 Patent”) titled SYSTEMS AND METHODS FOR
SCANNING INFORMATION FROM STORAGE AREA CONTENTS. The ‘153 Patent lists
Fabio Zsigmond, Sion Elie Douer, Geraldo Yoshizawa, and Frederico Wagner as its inventors
and states that it was assigned to Ikan Holdings LLC. Attached hereto as Exhibit 11 is a true
and correct copy of the ‘153 Patent.
16.

Ikan was the sole owner of the ‘153 Patent at the time that it issued. Ikan

merged with Freshub, Ltd. and assigned all rights to the ‘153 Patent to Freshub, Ltd. Freshub,

6

Ltd. conveyed rights to the ‘153 Patent to Freshub, Inc., including the rights to sue, assert,
exclude, assign, and license the ‘153 Patent.
17.

The ‘153 Patent is generally directed toward a voice processing system to

collect, manage, and display information about items described by a user.
18.

On February 26, 2019, the PTO issued U.S. Patent No. 10,213,810 (the “’810

Patent”) titled SYSTEMS AND METHODS FOR SCANNING INFORMATION FROM
STORAGE AREA CONTENTS. The ‘810 Patent lists Fabio Zsigmond, Sion Elie Douer,
Geraldo Yoshizawa, and Frederico Wagner as its inventors and states that it was assigned to
Ikan Holdings LLC. Attached hereto as Exhibit 12 is a true and correct copy of the ‘810
Patent.
19.

Ikan was the sole owner of the ‘810 Patent at the time that it issued. Ikan

merged with Freshub, Ltd. and assigned all rights to the ‘810 Patent to Freshub, Ltd. Freshub,
Ltd. conveyed rights to the ‘810 Patent to Freshub, Inc., including the rights to sue, assert,
exclude, assign, and license the ‘810 Patent.
20.

The ‘810 Patent is generally directed toward a voice processing system to

collect, manage, and display items described by a user and provide a set of items to an item
provider.
21.

On March 19, 2019, the PTO issued U.S. Patent No. 10,232,408 (the “’408

Patent”) titled SYSTEMS AND METHODS FOR SCANNING INFORMATION FROM
STORAGE AREA CONTENTS. The ‘408 Patent lists Fabio Zsigmond, Sion Elie Douer,
Geraldo Yoshizawa, and Frederico Wagner as its inventors and states that it was assigned to
Ikan Holdings LLC. Attached hereto as Exhibit 13 is a true and correct copy of the ‘408
Patent.

7

22.

Ikan was the sole owner of the ‘408 Patent at the time that it issued. Ikan

merged with Freshub, Ltd. and assigned all rights to the ‘408 Patent to Freshub, Ltd. Freshub,
Ltd. conveyed rights to the ‘408 Patent to Freshub, Inc., including the rights to sue, assert,
exclude, assign, and license the ‘408 Patent.
23.

The ‘408 Patent is generally directed toward a voice processing system to

collect, manage, and display items described by a user and identified within a database.
24.

On March 26, 2019, the PTO issued U.S. Patent No. 10,239,094 (the “’094

Patent”) titled SYSTEMS AND METHODS FOR SCANNING INFORMATION FROM
STORAGE AREA CONTENTS. The ‘094 Patent lists Fabio Zsigmond, Sion Elie Douer,
Geraldo Yoshizawa, and Frederico Wagner as its inventors and states that it was assigned to
Ikan Holdings LLC. Attached hereto as Exhibit 14 is a true and correct copy of the ‘094
Patent.
25.

Ikan was the sole owner of the ‘094 Patent at the time that it issued. Ikan

merged with Freshub, Ltd. and assigned all rights to the ‘094 Patent to Freshub, Ltd. Freshub,
Ltd. conveyed rights to the ‘094 Patent to Freshub, Inc., including the rights to sue, assert,
exclude, assign, and license the ‘094 Patent.
26.

The ‘094 Patent is generally directed toward a voice processing system to

collect, manage, and display items described by a user and provide a set of items to an item
provider.
27.

The ‘153 Patent, ‘810 Patent, ‘408 Patent, and ‘094 Patent are collectively

referred to herein as the “Asserted Patents.”

8

DEFENDANTS’ PRODUCTS AND SERVICES
28.

Amazon makes, uses, sells, offers for sale, and/or imports into the United States

and this District products and services. Amazon’s sells products with Alexa integrated in four
product categories which are as follows: Echo, Echo Dot, Echo Spot, Echo Show, Echo Input,
and Echo Auto (collectively “Echo Products”); Fire TV Stick, Fire TV Stick 4k, Fire TV Cube,
and the Alexa Voice Remote (collectively “Fire TV Products”); Fire 7, Fire HD 8, and Fire HD
10 (collectively “Fire Tablet Products”); Amazon App and Alexa App (collectively “Amazon
App Products”). The Echo Products, Fire TV Products, Fire Tablet Products, and Amazon
App Products will be collectively referred to as the “Accused Products” and are described in
further detail below.
Amazon’s Products with Alexa
29.

Amazon Alexa is a virtual assistant developed by Amazon and used in products

sold by Amazon. Alexa is available on Amazon’s Echo Products, Fire TV Products, Fire
Tablet Products, and the Amazon Apps. The products with Alexa integrated will listen for and
send speech to Alexa’s Voice Service in the cloud. The Alexa Voice Service turns the voice
into commands such as adding items to a shopping cart or shopping list. As shown below,
Amazon Alexa with the Alexa Voice Service is used by Amazon’s Echo Products, Fire TV
Products, Fire Tablet Products, the Amazon App and the Alexa App.

9

Exhibit 15 - https://amazon.jobs/en/business_categories/alexa.
Shopping with Alexa
30.

Alexa includes Alexa’s Voice Purchasing system which allows customers to use

Alexa enabled devices to purchase products from Amazon. A voice command is issued to
Alexa such as “add [product] to my shopping cart,” and the product is then added to the
customer’s Amazon shopping cart. Additionally if a voice command is issued to Alexa such
as “add [product] to my list”, the product is then added to the customer’s Alex Shopping List.
Alexa is used for adding items to an Amazon Cart, Whole Foods Cart, Fresh Cart, Prime Now
Cart, and Alexa Shopping List. As shown below, Shopping with Alexa on Amazon is
achieved through Voice Purchasing.

10

Exhibit 16 - https://www.amazon.com/gp/help/customer/display.html?nodeId=202083830.
31.

As further shown below, Alexa Voice Purchasing is used to add groceries to

one’s Whole Foods Cart, Fresh Cart, and Prime Now Cart.

Exhibit 17 https://www.amazon.com/b/ref=s9_acss_bw_cg_VANS_7a1_w?node=17919794011&pf_rd_
m=ATVPDKIKX0DER&pf_rd_s=merchandised-search6&pf_rd_r=ZB042FZTNT88HEJDMRW5&pf_rd_t=101&pf_rd_p=70f1dbbb-fb5b-485a-bbfdaf68d0e2a7f3&pf_rd_i=14552177011.
32.

As shown below, Alexa is used to add items to the Alexa Shopping List.

11

Exhibit 17 https://www.amazon.com/b/ref=s9_acss_bw_cg_VANS_7a1_w?node=17919794011&pf_rd_
m=ATVPDKIKX0DER&pf_rd_s=merchandised-search6&pf_rd_r=ZB042FZTNT88HEJDMRW5&pf_rd_t=101&pf_rd_p=70f1dbbb-fb5b-485a-bbfdaf68d0e2a7f3&pf_rd_i=14552177011.
33.

As shown below, the Amazon App uses Alexa for adding items to a user’s

shopping cart.

Exhibit 17 https://www.amazon.com/b/ref=s9_acss_bw_cg_VANS_7a1_w?node=17919794011&pf_rd_
m=ATVPDKIKX0DER&pf_rd_s=merchandised-search6&pf_rd_r=ZB042FZTNT88HEJDMRW5&pf_rd_t=101&pf_rd_p=70f1dbbb-fb5b-485a-bbfdaf68d0e2a7f3&pf_rd_i=14552177011.
Amazon’s Echo Products
34.

Amazon develops and sells the Echo Products. The Echo Products are a brand

of hands-free smart speakers which use the voice-controlled Alexa service. Echo devices
require a wireless internet connection to connect to the cloud component of Alexa and include
microphones to capture voice commands. The Echo was the first product to include an Alexa
integration. As shown in the two images below, the Echo Products utilize Alexa and Alexa
Voice Service for Voice Shopping on Amazon, which includes adding items to a cart and/or a
shopping list.

12

Exhibit 19 - https://www.amazon.com/dp/B06XCM9LJ4?ref=ODS_v2_FS_AUCC_rr.

Exhibit 20 - https://www.amazon.com/dp/B06XCM9LJ4?ref=ODS_v2_FS_AUCC_rr&th=1.
Amazon’s Fire TV Products
35.

Amazon develops and sells the Fire TV Products. The Fire TV Products are

digital media players developed by Amazon. The Fire TV Stick and Fire TV Stick 4k also
include a micro console remote to input voice commands, also called an Alexa Voice Remote.
The Fire TV Products integrate the Alexa software into its platform. As shown below, Fire
13

TV Products utilize Alexa and Alexa Voice Service for Voice Shopping on Amazon, which
includes adding items to a cart and/or a shopping list.

Exhibit 21 - https://www.amazon.com/dp/B0791T9CV7?ref=ODS_v2_FS_SMP_TVCube.
Amazon’s Fire Table Products
36.

Amazon develops and sells Fire Tablet Products. The Fire Tablet Products are

tablet computer devices developed by Amazon. The Fire Tablet Products utilize Amazon Fire
OS which includes the integrated Alexa software. As shown below, Fire Tablet Products
utilize Alexa and Alexa Voice Service for Voice Shopping on Amazon, which includes adding
items to a cart and/or a shopping list.

14

Exhibits 23–24 - https://www.amazon.com/dp/B0794RHPZD?ref=ODS_v2_FS_TAB_FHD8.
Amazon App Products
37.

Amazon develops and provides the Amazon App and Alexa App (collectively

“Amazon App Products”), which utilize Alexa for Voice Shopping. The Alexa App is
available on Fire OS, Android, iOS, and Windows 10 operating systems, and thus is available
as both a mobile and desktop application. As shown in the two pictures below, the Amazon
App Products utilize Alexa and Alexa Voice Service for Voice Shopping on Amazon, which
includes adding items to a cart and/or a shopping list.

15

Exhibit 18 https://www.amazon.com/b/ref=s9_acss_bw_cg_VANS_6a1_w?node=17691568011&pf_rd_
m=ATVPDKIKX0DER&pf_rd_s=merchandised-search-6&p%E2%80%A6.

16

Exhibit 25 - https://www.amazon.com/gp/help/customer/display.html?nodeId=202202210.
Alexa Voice Service
38.

Alexa Voice Service (“AVS”) is Amazon’s suite of services built around its

voice-controlled artificial intelligence (“AI”) assistant Alexa. AVS is tightly integrated with
Amazon’s e-commerce environment in order to make purchases fast and simple. AVS is used
to power Alexa in the Echo Products, Fire TV Products, Fire Tablet Products, Amazon App
Products, and may also be utilized by other third-party hardware makers. The hardware
required to utilize AVS includes a microphone array for audio pickup, a processor to run the
AVS Client, internet connectivity to send audio to the cloud, and a speaker to play back audio
for the customer. The AVS Client program will take the audio picked up by the microphone
and send it to AVS in the cloud over the internet for processing. This includes processing
requests to add items to a shopping cart or Alexa Shopping List. As shown in the two images
and video below, AVS powers Amazon’s Alexa enabled devices.

https://www.youtube.com/watch?time_continue=700&v=t58fUp15YB4.

17

Exhibit 26 - https://developer.amazon.com/blogs/post/Tx1IWOJG191LB8X/Announcing-theAlexa-Voice-Service-AVS.
DEFENDANTS’ INFRINGEMENT OF THE PATENTS
39.

Defendants have been and are now infringing, and will continue to infringe, the

Asserted Patents in this District and elsewhere in the United States by, among other things,
making, using, importing, selling, and/or offering for sale its Echo Products, Fire TV Products,
Fire Tablet Products, and Amazon App Products.
40.

In addition to directly infringing the Asserted Patents pursuant to 35 U.S.C. §

271(a), either literally or under the doctrine of equivalents, or both, Defendants also indirectly
infringe the ‘810 Patent, ‘408 Patent, and ‘094 Patent pursuant to 35 U.S.C. § 271(b) by
instructing, directing, and/or requiring others, including their customers, purchasers, users, and
developers, to perform all or some of the steps of the method claims, either literally or under
the doctrine of equivalents, or both, of the ‘810 Patent, ‘408 Patent, and ‘094 Patent.
COUNT I
(Direct Infringement of the ‘153 Patent)
41.

Freshub repeats, realleges, and incorporates by reference, as if fully set forth

herein, the allegations of the preceding paragraphs, as set forth above.
42.

Defendants have infringed and continue to infringe Claims 1-11 of the ‘153

Patent in violation of 35 U.S.C. § 271(a).

18

43.

Defendants’ infringement is based upon literal infringement or infringement

under the doctrine of equivalents, or both.
44.

Defendants’ acts of making, using, importing, selling, and/or offering for sale

infringing products and services have been without the permission, consent, authorization, or
license of Freshub.
45.

Defendants’ infringement includes, but is not limited to, the manufacture, use,

sale, importation and/or offer for sale of Defendants’ products and services, including the Echo
Products, Fire TV Products, Fire Tablet Products, and Amazon App Products (collectively, the
“’153 Accused Products”).
46.

The ‘153 Accused Products infringe the ‘153 Patent because they have a voice

processing system comprising: a first system configured to receive user spoken words
comprising: a microphone, a wireless network interface, a digitizer coupled to the microphone,
wherein the digitizer is configured to convert spoken words into a digital representation; a first
computer; non-transitory memory that stores instructions that when executed by the first
computer cause the first system to perform operations comprising: receive via the digitizer a
verbal order, comprising at least one item, from a user, wherein the verbal order was captured
by the microphone and digitized by the digitizer; immediately transmit, using the wireless
network interface, the digitized order to a computer system remote from the first system; the
computer system, the computer system comprising: a networks interface; a second computer;
non-transitory memory that stores instructions that when executed by the second computer
cause the computer system to perform operations comprising: receive, using the network
interface, the digitized order from the first system; translate at least a portion of the digitized
order to text; identify an item corresponding to the text; add the identified item to a list

19

associated with the user; enable the list, including the identified item, to be displayed via a user
display.
47.

Using an Echo Device as an example, as shown below, the ‘153 Accused

Products include a microphone (shown by the microphone array) and wireless network
interface (shown by the Wi-Fi antenna).

Exhibit 20 - https://www.amazon.com/dp/B06XCM9LJ4?ref=ODS_v2_FS_AUCC_rr&th=1.
48.

Using a Fire Tablet as an example, as shown below, the ‘153 Accused Products

include a microphone and wireless network interface (shown by the Wi-Fi antenna).

20

Exhibit 27 - https://www.amazon.com/All-New-Fire-Tablet-DisplayBlack/dp/B0794RHPZD/ref=sr_1_1?keywords=fire+tablets&qid=1559846204&s=gateway&sr
=8-1.
49.

Using a Fire TV Cube as an example, as shown below, the ‘153 Accused

Products include a microphone and wireless network interface (shown by the Wi-Fi antenna).

Exhibit 22 - https://www.amazon.com/dp/B0791T9CV7?ref=ODS_v2_FS_SMP_TVCube.
50.

Further, as shown below, AVS requires the use of a microphone and an internet

connection. The ‘153 Accused Products all utilize AVS.

Exhibit 28 - https://developer.amazon.com/alexa-voice-service/learn.
51.

The ‘153 Accused Products, as shown below, also include a digitizer coupled to

the microphone which digitized the captured words into pulse code modulation (“PCM”)
format and utilizes a recognition event to send the captured spoken words to AVS.

21

Exhibit 29 - https://developer.amazon.com/docs/alexa-voice-service/speechrecognizer.html.
52.

The ‘153 Accused Products, as shown below and in the video, utilize the AVS

Client software to process a verbal request from a user, such as a request to add an item to an
Alexa Shopping List, and sends that request via WiFi to Alexa’s remote cloud component.

22

https://www.youtube.com/watch?v=omPvBOJGw_U&list=PL2KJmkHeYQTMFumv03WLeG
Ux3l_Ub5NwJ&index=4&t=493s.
53.

The ‘153 Accused Products, as shown in the below image and in the video

cited, utilize Alexa Voice Service computers/servers in the cloud (a second computer) to
receive a user’s spoken words and will convert those words into an action associated with the
user.

23

https://www.youtube.com/watch?time_continue=700&v=t58fUp15YB4.
54.

The ‘153 Accused Products, as shown below, utilize Alexa’s Automatic Speech

Recognition technology in the cloud, which converts the user’s speech into text and then takes
action (such as adding an item to a shopping list).

Exhibit 30 - https://developer.amazon.com/fr/docs/alexa-voice-service/audio-hardwareconfigurations.html.
55.

The ‘153 Accused Products, as shown below, utilize Alexa to take spoken

words and translate those words to add items to an Alexa Shopping List associated with the
user’s Amazon account. The Amazon Shopping List(s) is viewable to the user via the Amazon
App Products, Echo Device with a display, or the Amazon website.

Exhibit 17 https://www.amazon.com/b/ref=s9_acss_bw_cg_VANS_7a1_w?node=17919794011&pf_rd_
24

m=ATVPDKIKX0DER&pf_rd_s=merchandised-search6&pf_rd_r=ZB042FZTNT88HEJDMRW5&pf_rd_t=101&pf_rd_p=70f1dbbb-fb5b-485a-bbfdaf68d0e2a7f3&pf_rd_i=14552177011.
56.

Defendants’ infringement of the ‘153 Patent has injured and continues to injure

Freshub in an amount to be proven at trial, but not less than a reasonable royalty.
57.

On information and belief, Defendants are aware of Freshub’s patent portfolio,

including the ‘153 Patent. Amazon knew of and cited the published application for the ‘344
Patent to a patent examiner for one of Amazon’s own patents. All of the Asserted Patents
claim priority through the ‘344 Patent and are thus directly related, such that Amazon was
aware of the patent family containing the Asserted Patents. In 2010, Amazon requested and
received a copy of a wireless scanner product. The product packaging was marked with four
patents and the statement “and other patents pending,” such that on information and belief,
Amazon was aware of relevant patents and other pending applications in the smart home and
in-home commerce technology space. Further, Amazon is and has been aware of Freshub for
years, and the two companies have discussed collaborating. Freshub and Amazon have also
twice participated together in public discussions regarding smart home and in-home commerce
technology and the valuable opportunity that it represents. As a result, on information and
belief, Amazon has known of and possessed information regarding the Asserted Patents.
58.

Despite knowledge of Freshub’s patents, Defendants have sold, used, offered

for sale, and continue to sell, use, and offer for sale, the ‘154 Accused Products, disregarding
Freshub’s patent rights. Defendants have therefore acted recklessly, and continue to willfully,
wantonly, and deliberately engage in acts of infringement of the ‘153 Patent, justifying an
award to Freshub of increased damages under 35 U.S.C. § 284 and attorneys’ fees and costs
incurred under 35 U.S.C. § 285.

25

59.

Defendants’ infringement has caused and is continuing to cause damage and

irreparable injury to Freshub, and Freshub will continue to suffer damage and irreparable
injury unless and until that infringement is enjoined by this Court.
60.

Freshub is entitled to injunctive relief, damages and any other relief in

accordance with 35 U.S.C. §§ 283, 284 and 285.
COUNT II
(Direct Infringement of the ‘810 Patent)
61.

Freshub repeats, realleges, and incorporates by reference, as if fully set forth

herein, the allegations of the preceding paragraphs, as set forth above.
62.

Defendants have infringed and continue to infringe Claims 1-29 of the ‘810

Patent in violation of 35 U.S.C. § 271(a).
63.

Defendants’ infringement is based upon literal infringement or infringement

under the doctrine of equivalents, or both.
64.

Defendants’ acts of making, using, importing, selling, and/or offering for sale

infringing products and services have been without the permission, consent, authorization, or
license of Freshub.
65.

Defendants’ infringement includes, but is not limited to, the manufacture, use,

sale, importation and/or offer for sale of Defendants’ products and services, including the Echo
Products, Fire TV Products, Fire Tablet Products, and Amazon App Products (collectively, the
“’810 Accused Products”).
66.

The ‘810 Accused Products infringe the ‘810 Patent because they have a voice

processing system comprising: a networks interface; a computer; non-transitory memory that
stores instructions that when executed by the computer cause the computer to perform
operations comprising: associate a unique identifier with a remote system configured to receive

26

user spoken words, the remote system comprising: a microphone, a wireless network interface,
a voice output system, and a digitizer coupled to the microphone, wherein the digitizer is
configured to convert spoken words into a digital representation; download configuration data
to the remote system; receive, using the network interface, a digitized order of a user from the
remote system; translate at least a portion of the digitized order to text; use the text, translated
from the digitized order, to identify an item corresponding to the text description; include the
identified item in a set of one or more items associated with the user; enable the set of items,
including the identified item, to be displayed to the user via a user device different than the
remote system; and enable the set of items, including the identified item, to be provided to an
item provider.
67.

For example, as shown below using an Echo Device, the ‘810 Accused Products

include a microphone (shown by the microphone array) and wireless network interface (shown
by the Wi-Fi antenna).

27

Exhibit 20 - https://www.amazon.com/dp/B06XCM9LJ4?ref=ODS_v2_FS_AUCC_rr&th=1.
68.

Using a Fire Tablet as an example and as shown below, the ‘810 Accused

Products include a microphone and wireless network interface (shown by the Wi-Fi antenna).

Exhibit 27 - https://www.amazon.com/All-New-Fire-Tablet-DisplayBlack/dp/B0794RHPZD/ref=sr_1_1?keywords=fire+tablets&qid=1559846204&s=gateway&sr
=8-1.
69.

Using a Fire TV Cube as an example and as shown below, the ‘810 Accused

Products include a microphone and wireless network interface (shown by the Wi-Fi antenna).

Exhibit 22 - https://www.amazon.com/dp/B0791T9CV7?ref=ODS_v2_FS_SMP_TVCube.
70.

Further, as shown below, AVS requires the use of a microphone and an internet

connection. The ‘810 Accused Products all utilize AVS.

28

Exhibit 28 - https://developer.amazon.com/alexa-voice-service/learn.
71.

As shown below, the ‘810 Accused Products include a unique device serial

number.

Exhibit 31 - https://github.com/alexa/alexa-smarthome/wiki/How-to-get-a-device-serialnumber.

Exhibit 32 - https://www.amazon.com/gp/help/customer/display.html?nodeId=202143910.
72.

As shown below, the ‘810 Accused Products must also be associated with a

user’s unique account in order to place orders with Alexa (or use Alexa).

29

Exhibit 33 https://www.amazon.com/gp/help/customer/display.html?nodeId=201807210&ref=insider_ar_
tips_alexa.
73.

The ‘810 Accused Products, as shown below, also include a digitizer coupled to

the microphone which digitized the spoken words into PCM format and utilizes a recognition
event to send the captured spoken words to AVS.

30

Exhibit 29 - https://developer.amazon.com/docs/alexa-voice-service/speechrecognizer.html.
74.

The ‘810 Accused Products, as shown below and in the video, utilize the AVS

Client software to process a verbal order from a user, such as a request to add an item to a
shopping cart, and sends that request via WiFi to Alexa’s remote cloud component.

31

https://www.youtube.com/watch?v=omPvBOJGw_U&list=PL2KJmkHeYQTMFumv03WLeG
Ux3l_Ub5NwJ&index=4&t=493s.
75.

The ‘810 Accused Products, as shown in the below image and in the video

cited, utilize Alexa Voice Service computers/servers in the cloud to receive a user’s spoken
words and will convert those words into an action, such as having the item associated with the
user.

32

https://www.youtube.com/watch?time_continue=700&v=t58fUp15YB4.
76.

The ‘810 Accused Products, as shown below, utilize Alexa’s Automatic Speech

Recognition in the cloud which converts the customer speech into text and then takes action
(such as adding an item to a shopping cart).

Exhibit 30 - https://developer.amazon.com/fr/docs/alexa-voice-service/audio-hardwareconfigurations.html.
77.

As shown below, the ‘810 Accused Products settings are configured remotely

using the Alexa app and subsequently this configuration data is downloaded to the particular
device.

Exhibit 34 - https://www.amazon.com/gp/help/customer/display.html?nodeId=201601980.
78.

As shown below in the two images, the ‘810 Accused Products provide ordering

items and adding items to a shopping cart via Alexa. These items in the shopping cart can then
be viewed using a web browser or Amazon App Product.

33

Exhibit 16 - https://www.amazon.com/gp/help/customer/display.html?nodeId=202083830.

Exhibit 35 - https://www.amazon.com/gp/help/customer/display.html?nodeId=201889540.
79.

Defendants’ infringement of the ‘810 Patent has injured and continues to injure

Freshub in an amount to be proven at trial, but not less than a reasonable royalty.
80.

On information and belief, Defendants are aware of Freshub’s patent portfolio,

including the ‘810 Patent. Amazon knew of and cited the published application for the ‘344
Patent to a patent examiner for one of Amazon’s own patents. All of the Asserted Patents
claim priority through the ‘344 Patent and are thus directly related, such that Amazon was
aware of the patent family containing the Asserted Patents. In 2010, Amazon requested and
received a copy of a wireless scanner product. The product packaging was marked with four
patents and the statement “and other patents pending,” such that on information and belief,
Amazon was aware of relevant patents and other pending applications in the smart home and
in-home commerce technology space. Further, Amazon is and has been aware of Freshub for
years, and the two companies have discussed collaborating. Freshub and Amazon have also
34

twice participated together in public discussions regarding smart home and in-home commerce
technology and the valuable opportunity that it represents. As a result, on information and
belief, Amazon has known of and possessed information regarding the Asserted Patents.
81.

Despite knowledge of Freshub’s patents, Defendants have sold, used, offered

for sale, and continue to sell, use, and offer for sale, the ‘810 Accused Products, disregarding
Freshub’s patent rights. Defendants have therefore acted recklessly, and continue to willfully,
wantonly, and deliberately engage in acts of infringement of the ‘810 Patent, justifying an
award to Freshub of increased damages under 35 U.S.C. § 284 and attorneys’ fees and costs
incurred under 35 U.S.C. § 285.
82.

Defendants’ infringement has caused and is continuing to cause damage and

irreparable injury to Freshub, and Freshub will continue to suffer damage and irreparable
injury unless and until that infringement is enjoined by this Court.
83.

Freshub is entitled to injunctive relief, damages and any other relief in

accordance with 35 U.S.C. §§ 283, 284 and 285.
COUNT III
(Indirect Infringement of the ‘810 Patent)
84.

Freshub repeats, realleges, and incorporates by reference, as if fully set forth

herein, the allegations of the preceding paragraphs.
85.

Defendants have induced infringement of at least Claims 17-27 of the ‘810

Patent under 35 U.S.C. § 271(b).
86.

In addition to directly infringing the ‘810 Patent, Defendants indirectly infringe

the ‘810 Patent pursuant to 35 U.S.C. § 271(b) by instructing, directing and/or requiring others,
including customers, purchasers, users and developers, to perform one or more of the steps of
the method claims, either literally or under the doctrine of equivalents, of the ‘810 Patent,

35

where all the steps of the method claims are performed by either Defendants, its customers,
purchasers, users, and developers, or some combination thereof. Defendants knew or were
willfully blind to the fact that it was inducing others, including customers, purchasers, users,
and developers, to infringe by practicing, either themselves or in conjunction with Defendants,
one or more method claims of the ‘810 Patent, including Claims 17-27.
87.

Defendants knowingly and actively aided and abetted the direct infringement of

the ‘810 Patent by instructing and encouraging its customers, purchasers, users, and developers
to use the ‘810 Accused Products. Such instructions and encouragement included, but is not
limited to, advising third parties to use the ‘810 Accused Products in an infringing manner,
providing a mechanism through which third parties may infringe the ‘810 Patent, and by
advertising and promoting the use of the ‘810 Accused Products in an infringing manner, and
distributing guidelines and instructions to third parties on how to use the ‘810 Accused
Products in an infringing manner.
88.

Amazon updates and maintains help and customer service sections on its

website which cover in-depth aspects of operating Alexa with the Accused Products in an
infringing manner. Further, Amazon advertises on its website the Accused Products infringing
Alexa features and instructs consumers on how to configure and use the Accused Products in
an infringing manner. See, e.g., Exhibits 36–38,
https://www.amazon.com/gp/help/customer/display.html/ref=hp_bc_nav?ie=UTF8&nodeId=2
01952240; https://www.amazon.com/gp/help/customer/display.html?nodeId=202009680;
https://www.amazon.com/gp/help/customer/display.html?nodeId=201602230.
89.

Defendants’ indirect infringement of the ‘810 Patent has injured and continues

to injure Freshub in an amount to be proven at trial, but not less than a reasonable royalty.

36

90.

Defendants’ infringement has caused and is continuing to cause damage and

irreparable injury to Freshub, and Freshub will continue to suffer damage and irreparable
injury unless and until that infringement is enjoined by this Court.
91.

Freshub is entitled to injunctive relief, damages and any other relief in

accordance with 35 U.S.C. §§ 283, 284 and 285.
COUNT IV
(Direct Infringement of the ‘408 Patent)
92.

Freshub repeats, realleges, and incorporates by reference, as if fully set forth

herein, the allegations of the preceding paragraphs, as set forth above.
93.

Defendants have infringed and continue to infringe Claims 1-30 of the ‘408

Patent in violation of 35 U.S.C. § 271(a).
94.

Defendants’ infringement is based upon literal infringement or infringement

under the doctrine of equivalents, or both.
95.

Defendants’ acts of making, using, importing, selling, and/or offering for sale

infringing products and services have been without the permission, consent, authorization, or
license of Freshub.
96.

Defendants’ infringement includes, but is not limited to, the manufacture, use,

sale, importation and/or offer for sale of Defendants’ products and services, including the Echo
Products, Fire TV Products, Fire Tablet Products, and Amazon App Products (collectively, the
“’408 Accused Products”).
97.

The ‘408 Accused Products infringe the ‘408 Patent because they have a voice

processing system comprising: a networks interface; a computer; non-transitory memory that
stores instructions that when executed by the computer cause the computer to perform
operations comprising: receive, using the network interface, a digitized order of a user from a

37

remote system configured to receive user spoken words, the remote system comprising a
microphone, a wireless network interface, and a digitizer coupled to the microphone, wherein
the digitizer is configured to convert spoken words into a digital representation; translate at
least a portion of the digitized order to text; match the text, translated from the digitized order,
to a text description stored in a database comprising text descriptions of items and associated
unique product identifiers; based at least in part on the identified match of the text translated
from the digitized order the text description stored in a database, identify an item
corresponding to the text; add the identified item to a set of items associated with the user; and
enable the set of items, including the identified item, to be displayed via a user display.
98.

For example, using an Echo Device as an example and as shown below, the

‘408 Accused Products include a microphone (shown by the microphone array) and wireless
network interface (shown by the Wi-Fi antenna).

Exhibit 20 - https://www.amazon.com/dp/B06XCM9LJ4?ref=ODS_v2_FS_AUCC_rr&th=1.

38

99.

Using a Fire Tablet as an example and as shown below, the ‘408 Accused

Products include a microphone and wireless network interface (shown by the Wi-Fi antenna).

Exhibit 27 - https://www.amazon.com/All-New-Fire-Tablet-DisplayBlack/dp/B0794RHPZD/ref=sr_1_1?keywords=fire+tablets&qid=1559846204&s=gateway&sr
=8-1.
100.

Using a Fire TV Cube as an example and as shown below, the ‘408 Accused

Products include a microphone and wireless network interface (shown by the Wi-Fi antenna).

Exhibit 22 - https://www.amazon.com/dp/B0791T9CV7?ref=ODS_v2_FS_SMP_TVCube.
101.

Further, as shown below, AVS requires the use of a microphone and an internet

connection. The ‘408 Accused Products all utilize AVS.

39

Exhibit 28 - https://developer.amazon.com/alexa-voice-service/learn.
102.

The ‘408 Accused Products, as shown below, also include a digitizer coupled to

the microphone which digitized the spoken words into PCM format and utilizes a recognition
event to send the captured spoken words to AVS.

Exhibit 29 - https://developer.amazon.com/docs/alexa-voice-service/speechrecognizer.html.
103.

The ‘408 Accused Products, as shown below and in the video, utilize the AVS

Client software to process a verbal order from a user, such as a request to add an item to a
shopping cart, and sends that request via WiFi to Alexa’s remote cloud component.
40

https://www.youtube.com/watch?v=omPvBOJGw_U&list=PL2KJmkHeYQTMFumv03WLeG
Ux3l_Ub5NwJ&index=4&t=493s.
104.

The ‘408 Accused Products, as shown in the below image and in the video

cited, utilize Alexa Voice Service computers/servers in the cloud to receive a user’s spoken
words and convert those words into an action, such as having the item associated with the user.
Further the action will involve using the text description derived from the Automatic Speech
Recognition component and matching that with a database of text descriptions of products and
their unique product identifiers. Alexa will use the unique product identifier to put the
appropriate item in the users shopping cart.

41

https://www.youtube.com/watch?time_continue=700&v=t58fUp15YB4.
105.

The ‘408 Accused Products, as shown below, utilize Alexa’s Automatic Speech

Recognition in the cloud which converts the customer speech into text and then takes (such as
adding an item to a shopping cart that has a unique identifier).

Exhibit 30 - https://developer.amazon.com/fr/docs/alexa-voice-service/audio-hardwareconfigurations.html.
106.

As shown below, Amazon utilizes unique product identifiers for the products

that are purchased from the site. These include Universal Product Codes (“UPCs”), European
Article Numbers (“EANs”), International Standard Book Numbers (“ISBNs”), and Amazon
Standard Identification Numbers (“ASINs”), which are used by the ‘408 Accused Products to
add items to a user’s shopping cart after a verbal order is placed.

42

Exhibit 39 - https://www.amazon.com/gp/seller/asin-upc-isbn-info.html.
107.

As shown below in the two images, the ‘408 Accused Products enable ordering

items and adding items to a shopping cart via Alexa. These items in the shopping cart can then
be viewed using a web browser or Amazon App Product.

Exhibit 16 - https://www.amazon.com/gp/help/customer/display.html?nodeId=202083830.

43

Exhibit 35 - https://www.amazon.com/gp/help/customer/display.html?nodeId=201889540.
108.

Defendants’ infringement of the ‘408 Patent has injured and continues to injure

Freshub in an amount to be proven at trial, but not less than a reasonable royalty.
109.

On information and belief, Defendants are aware of Freshub’s patent portfolio,

including the ‘408 Patent. Amazon knew of and cited the published application for the ‘344
Patent to a patent examiner for one of Amazon’s own patents. All of the Asserted Patents
claim priority through the ‘344 Patent and are thus directly related, such that Amazon was
aware of the patent family containing the Asserted Patents. In 2010, Amazon requested and
received a copy of a wireless scanner product. The product packaging was marked with four
patents and the statement “and other patents pending,” such that on information and belief,
Amazon was aware of relevant patents and other pending applications in the smart home and
in-home commerce technology space. Further, Amazon is and has been aware of Freshub for
years, and the two companies have discussed collaborating. Freshub and Amazon have also
twice participated together in public discussions regarding smart home and in-home commerce
technology and the valuable opportunity that it represents. As a result, on information and
belief, Amazon has known of and possessed information regarding the Asserted Patents.
110.

Despite knowledge of Freshub’s patents, Defendants have sold, used, offered

for sale, and continue to sell, use, and offer for sale, the ‘408 Accused Products, disregarding
Freshub’s patent rights. Defendants have therefore acted recklessly, and continue to willfully,
44

wantonly, and deliberately engage in acts of infringement of the ‘408 Patent, justifying an
award to Freshub of increased damages under 35 U.S.C. § 284 and attorneys’ fees and costs
incurred under 35 U.S.C. § 285.
111.

Defendants’ infringement has caused and is continuing to cause damage and

irreparable injury to Freshub, and Freshub will continue to suffer damage and irreparable
injury unless and until that infringement is enjoined by this Court.
112.

Freshub is entitled to injunctive relief, damages and any other relief in

accordance with 35 U.S.C. §§ 283, 284 and 285.
COUNT V
(Indirect Infringement of the ‘408 Patent)
113.

Freshub repeats, realleges, and incorporates by reference, as if fully set forth

herein, the allegations of the preceding paragraphs.
114.

Defendants have induced infringement of at least Claims 20-29 of the ‘408

Patent under 35 U.S.C. § 271(b).
115.

In addition to directly infringing the ‘408 Patent, Defendants indirectly infringe

the ‘408 Patent pursuant to 35 U.S.C. § 271(b) by instructing, directing and/or requiring others,
including customers, purchasers, users and developers, to perform one or more of the steps of
the method claims, either literally or under the doctrine of equivalents, of the ‘408 Patent,
where all the steps of the method claims are performed by either Defendants, its customers,
purchasers, users, and developers, or some combination thereof. Defendants knew or were
willfully blind to the fact that it was inducing others, including customers, purchasers, users,
and developers, to infringe by practicing, either themselves or in conjunction with Defendants,
one or more method claims of the ‘408 Patent, including Claims 20-29.

45

116.

Defendants knowingly and actively aided and abetted the direct infringement of

the ‘408 Patent by instructing and encouraging its customers, purchasers, users, and developers
to use the ‘408 Accused Products. Such instructions and encouragement included, but is not
limited to, advising third parties to use the ‘408 Accused Products in an infringing manner,
providing a mechanism through which third parties may infringe the ‘408 Patent, and by
advertising and promoting the use of the ‘408 Accused Products in an infringing manner, and
distributing guidelines and instructions to third parties on how to use the ‘408 Accused
Products in an infringing manner.
117.

Amazon updates and maintains help and customer service sections on its

website which cover in-depth aspects of operating Alexa with the Accused Products in an
infringing manner. Further, Amazon advertises on its website the Accused Products infringing
Alexa features and instructs consumers on how to configure and use the Accused Products in
an infringing manner. See, e.g., Exhibits 36–38,
https://www.amazon.com/gp/help/customer/display.html/ref=hp_bc_nav?ie=UTF8&nodeId=2
01952240; https://www.amazon.com/gp/help/customer/display.html?nodeId=202009680;
https://www.amazon.com/gp/help/customer/display.html?nodeId=201602230.
118.

Defendants’ indirect infringement of the ‘408 Patent has injured and continues

to injure Freshub in an amount to be proven at trial, but not less than a reasonable royalty.
119.

Defendants’ infringement has caused and is continuing to cause damage and

irreparable injury to Freshub, and Freshub will continue to suffer damage and irreparable
injury unless and until that infringement is enjoined by this Court.
120.

Freshub is entitled to injunctive relief, damages and any other relief in

accordance with 35 U.S.C. §§ 283, 284 and 285.

46

COUNT VI
(Direct Infringement of the ‘094 Patent)
121.

Freshub repeats, realleges, and incorporates by reference, as if fully set forth

herein, the allegations of the preceding paragraphs, as set forth above.
122.

Defendants have infringed and continue to infringe Claims 1-24 of the ‘094

Patent in violation of 35 U.S.C. § 271(a).
123.

Defendants’ infringement is based upon literal infringement or infringement

under the doctrine of equivalents, or both.
124.

Defendants’ acts of making, using, importing, selling, and/or offering for sale

infringing products and services have been without the permission, consent, authorization, or
license of Freshub.
125.

Defendants’ infringement includes, but is not limited to, the manufacture, use,

sale, importation and/or offer for sale of Defendants’ products and services, including the Echo
Products, Fire TV Products, Fire Tablet Products, and Amazon App Products (collectively, the
“’094 Accused Products”).
126.

The ‘094 Accused Products infringe the ‘094 Patent because it performs a

computer-implemented method, the method comprising: receiving over a network at a first
computer system, using a network interface, a digitized spoken user order from a second
computer system, the second computer system comprising: a microphone, a wireless network
interface, and a digitizer coupled to the microphone, wherein the digitizer is configured to
convert spoken words into a digital representation, and the second computer system is
configured to transmit the digital representation over the network to the first computer system;
translating, using a translation module executed by the second computer system, at least a
portion of the digitized spoken order to text; matching the text, translated from the digitized

47

spoken order, to text descriptions of items, wherein the text descriptions of items are stored in a
data store; based on at least an identified match, identifying a corresponding item; adding the
identified corresponding item to a list associated with the user; and enabling the list, including
the identified item, to be displayed to the user via a user display.
127.

Using an Echo Device as an example, as shown below, the ‘094 Accused

Products Accused Products include a microphone (shown by the microphone array) and
wireless network interface (shown by the Wi-Fi antenna).

Exhibit 20 - https://www.amazon.com/dp/B06XCM9LJ4?ref=ODS_v2_FS_AUCC_rr&th=1.
128.

Using a Fire Tablet as an example and as shown below, the ‘094 Accused

Products include a microphone and wireless network interface (shown by the Wi-Fi antenna).

48

Exhibit 27 - https://www.amazon.com/All-New-Fire-Tablet-DisplayBlack/dp/B0794RHPZD/ref=sr_1_1?keywords=fire+tablets&qid=1559846204&s=gateway&sr
=8-1.
129.

Using a Fire TV Cube as an example and as shown below, the ‘094 Accused

Products include a microphone and wireless network interface (shown by the Wi-Fi antenna).

Exhibit 22 - https://www.amazon.com/dp/B0791T9CV7?ref=ODS_v2_FS_SMP_TVCube.
130.

Further, as shown below, AVS requires the use of a microphone and an internet

connection. The ‘094 Accused Products all utilize AVS.

Exhibit 28 - https://developer.amazon.com/alexa-voice-service/learn.

49

131.

The ‘094 Accused Products, as shown below, also include a digitizer coupled to

the microphone which digitized the spoken words into PCM format and utilizes a recognition
event to send the captured spoken words to AVS.

Exhibit 29 - https://developer.amazon.com/docs/alexa-voice-service/speechrecognizer.html.
132.

The ‘094 Accused Products, as shown below and in the video, utilize the AVS

Client software to process a verbal order from a user, such as a request to add an item to a
shopping cart, and sends that request via WiFi to Alexa’s remote cloud component.

50

https://www.youtube.com/watch?v=omPvBOJGw_U&list=PL2KJmkHeYQTMFumv03WLeG
Ux3l_Ub5NwJ&index=4&t=493s.
133.

The ‘094 Accused Products, as shown in the below image and in the video

cited, utilize Alexa Voice Service computers/servers in the cloud to receive a user’s spoken
words and will convert those words into an action, such as having the item associated with the
user and placed in the user’s shopping cart. This process includes matching the text
description derived from the ASP component with descriptions in a data store that Amazon
maintains.

51

https://www.youtube.com/watch?time_continue=700&v=t58fUp15YB4.
134.

The ‘094 Accused Products, as shown below, utilize Alexa’s Automatic Speech

Recognition in the cloud which converts the customer speech into text and then takes action
(such as adding an item to a shopping cart).

Exhibit 30 - https://developer.amazon.com/fr/docs/alexa-voice-service/audio-hardwareconfigurations.html.
135.

As shown below in the two images, the ‘094 Accused Products provide ordering

items and adding items to a shopping cart via Alexa. These items in the shopping cart can then
be viewed using a web browser or Amazon App Product.

52

Exhibit 16 - https://www.amazon.com/gp/help/customer/display.html?nodeId=202083830.

Exhibit 35 - https://www.amazon.com/gp/help/customer/display.html?nodeId=201889540.
136.

Defendants’ infringement of the ‘094 Patent has injured and continues to injure

Freshub in an amount to be proven at trial, but not less than a reasonable royalty.
137.

On information and belief, Defendants are aware of Freshub’s patent portfolio,

including the ‘094 Patent. Amazon knew of and cited the published application for the ‘344
Patent to a patent examiner for one of Amazon’s own patents. All of the Asserted Patents
claim priority through the ‘344 Patent and are thus directly related, such that Amazon was
aware of the patent family containing the Asserted Patents. In 2010, Amazon requested and
received a copy of a wireless scanner product. The product packaging was marked with four
patents and the statement “and other patents pending,” such that on information and belief,
Amazon was aware of relevant patents and other pending applications in the smart home and
in-home commerce technology space. Further, Amazon is and has been aware of Freshub for
years, and the two companies have discussed collaborating. Freshub and Amazon have also
53

twice participated together in public discussions regarding smart home and in-home commerce
technology and the valuable opportunity that it represents. As a result, on information and
belief, Amazon has known of and possessed information regarding the Asserted Patents.
138.

Despite knowledge of Freshub’s patents, Defendants have sold, used, offered

for sale, and continue to sell, use, and offer for sale, the ‘094 Accused Products, disregarding
Freshub’s patent rights. Defendants have therefore acted recklessly, and continue to willfully,
wantonly, and deliberately engage in acts of infringement of the ‘094 Patent, justifying an
award to Freshub of increased damages under 35 U.S.C. § 284 and attorneys’ fees and costs
incurred under 35 U.S.C. § 285.
139.

Defendants’ infringement has caused and is continuing to cause damage and

irreparable injury to Freshub, and Freshub will continue to suffer damage and irreparable
injury unless and until that infringement is enjoined by this Court.
140.

Freshub is entitled to injunctive relief, damages and any other relief in

accordance with 35 U.S.C. §§ 283, 284 and 285.
COUNT VII
(Indirect Infringement of the ‘094 Patent)
141.

Freshub repeats, realleges, and incorporates by reference, as if fully set forth

herein, the allegations of the preceding paragraphs.
142.

Defendants have induced infringement of at least Claims 1-19 of the ‘094 Patent

under 35 U.S.C. § 271(b).
143.

In addition to directly infringing the ‘094 Patent, Defendants indirectly infringe

the ‘094 Patent pursuant to 35 U.S.C. § 271(b) by instructing, directing and/or requiring others,
including customers, purchasers, users and developers, to perform one or more of the steps of
the method claims, either literally or under the doctrine of equivalents, of the ‘094 Patent,

54

where all the steps of the method claims are performed by either Defendants, its customers,
purchasers, users, and developers, or some combination thereof. Defendants knew or were
willfully blind to the fact that it was inducing others, including customers, purchasers, users,
and developers, to infringe by practicing, either themselves or in conjunction with Defendants,
one or more method claims of the ‘094 Patent, including Claims 1-19.
144.

Defendants knowingly and actively aided and abetted the direct infringement of

the ‘094 Patent by instructing and encouraging its customers, purchasers, users, and developers
to use the ‘094 Accused Products. Such instructions and encouragement included, but is not
limited to, advising third parties to use the ‘094 Accused Products in an infringing manner,
providing a mechanism through which third parties may infringe the ‘094 Patent, and by
advertising and promoting the use of the ‘094 Accused Products in an infringing manner, and
distributing guidelines and instructions to third parties on how to use the ‘094 Accused
Products in an infringing manner.
145.

Amazon updates and maintains help and customer service sections on its

website which cover in-depth aspects of operating Alexa with the Accused Products in an
infringing manner. Further, Amazon advertises on its website the Accused Products infringing
Alexa features and instructs consumers on how to configure and use the Accused Products in
an infringing manner. See, e.g., Exhibits 36–38,
https://www.amazon.com/gp/help/customer/display.html/ref=hp_bc_nav?ie=UTF8&nodeId=2
01952240; https://www.amazon.com/gp/help/customer/display.html?nodeId=202009680;
https://www.amazon.com/gp/help/customer/display.html?nodeId=201602230.
146.

Defendants’ indirect infringement of the ‘094 Patent has injured and continues

to injure Freshub in an amount to be proven at trial, but not less than a reasonable royalty.

55

147.

Defendants’ infringement has caused and is continuing to cause damage and

irreparable injury to Freshub, and Freshub will continue to suffer damage and irreparable
injury unless and until that infringement is enjoined by this Court.
148.

Freshub is entitled to injunctive relief, damages and any other relief in

accordance with 35 U.S.C. §§ 283, 284 and 285.
PRAYER FOR RELIEF
WHEREFORE, Freshub prays for judgment and relief as follows:
A.

An entry of judgment holding that Defendants have infringed and are infringing

the ‘153 Patent, ‘810 Patent, ‘408 Patent, and ‘094 Patent; and have induced infringement and
are inducing infringement of the ‘810 Patent, ‘408 Patent, and ‘094 Patent;
B.

A preliminary and permanent injunction against Defendants and their officers,

employees, agents, servants, attorneys, instrumentalities, and/or those in privity with them,
from infringing, or inducing the infringement of the ‘153 Patent, ‘810 Patent, ‘408 Patent, and
‘094 Patent and for all further and proper injunctive relief pursuant to 35 U.S.C. § 283;
C.

An award to Freshub of such damages as it shall prove at trial against

Defendants that is adequate to fully compensate Freshub for Defendants’ infringement of the
‘153 Patent, ‘810 Patent, ‘408 Patent, and ‘094 Patent, said damages to be no less than a
reasonable royalty;
D.

A determination that Defendant’s infringement has been willful, wanton, and

deliberate and that the damages against it be increased up to treble on this basis or for any other
basis in accordance with the law;
E.

An award to Freshub of increased damages under 35 U.S.C. § 284;

56

F.

A finding that this case is “exceptional” and an award to Freshub of its costs and

reasonable attorneys’ fees, as provided by 35 U.S.C. § 285;
G.

An accounting of all infringing sales and revenues, together with post judgment

interest and prejudgment interest from the first date of infringement of the ‘153 Patent, ‘810
Patent, ‘408 Patent, and ‘094 Patent; and
H.

Such further and other relief as the Court may deem proper and just.

Respectfully submitted,
Dated: June 24, 2019

/s/ John P. Palmer
John Palmer
Texas Bar No. 15430600
[e-mail]
NAMAN HOWELL SMITH & LEE PLLC
400 Austin Ave., Suite 800, P.O. Box 1470
Waco, TX 76703
Telephone: [phone]
Fax: [phone]
Paul J. Andre (Pro Hac Vice Motion to be Filed)
California Bar No. 196585
Lisa Kobialka (Pro Hac Vice Motion to be Filed)
California Bar No. 191404
James Hannah (Pro Hac Vice Motion to be Filed)
California Bar No. 237978
KRAMER LEVIN NAFTALIS
& FRANKEL LLP
990 Marsh Road
Menlo Park, CA 94025
Telephone: [phone]
[e-mail]
[e-mail]
[e-mail]
Attorneys for Plaintiffs,
Freshub, Inc. and Freshub, Ltd.

57

DEMAND FOR JURY TRIAL
Freshub demands a jury trial on all issues so triable.

Respectfully submitted,
Dated: June 24, 2019

/s/ John P. Palmer
John Palmer
Texas Bar No. 15430600
[e-mail]
NAMAN HOWELL SMITH & LEE PLLC
400 Austin Ave., Suite 800, P.O. Box 1470
Waco, TX 76703
Telephone: [phone]
Fax: [phone]
Paul J. Andre (Pro Hac Vice Motion to be Filed)
California Bar No. 196585
Lisa Kobialka (Pro Hac Vice Motion to be Filed)
California Bar No. 191404
James Hannah (Pro Hac Vice Motion to be Filed)
California Bar No. 237978
KRAMER LEVIN NAFTALIS
& FRANKEL LLP
990 Marsh Road
Menlo Park, CA 94025
Telephone: [phone]
[e-mail]
[e-mail]
[e-mail]
Attorneys for Plaintiffs,
Freshub, Inc. and Freshub, Ltd.

58