Performance Pricing, Inc. v. Google Inc. et al

Filing 236

MEMORANDUM OPINION and ORDER granting in part and denying in part motion # 215 to compel documents. Google is ORDERED to make these documents available by September 11, 2009. Defendant Google is further ORDERED to re-produce a witness by September 25, 2009 for a limited Rule 30(b)(6) deposition, as outlined above. Prior to both dates, the parties are ORDERED to meet and confer regarding these issues and are strongly encouraged, going forward, to work together to resolve any remaining discovery disputes. Signed by Magistrate Judge John D. Love on 8/31/09. (ehs, )

IN THE UNITED STATES DISTRICT COURT FOR THE EASTERN DISTRICT OF TEXAS MARSHALL DIVISION PERFORMANCE PRICING, INC., Plaintiff, v. GOOGLE INC., et al., Defendants. § § § § § § § § § §

CIVIL ACTION NO. 2:07cv432

MEMORANDUM OPINION AND ORDER Before the Court is Plaintiff Performance Pricing, Inc.'s Motion to Compel Documents regarding Google's calculation of actual cost per click and Quality Score ("Motion") (Doc. No. 215). Defendant Google has filed an Opposition to Plaintiff's Motion to Compel Documents ("Response") (Doc. No. 228), as well as a Sur-Reply in Opposition to Plaintiff's Motion to Compel Documents ("Surreply") (Doc. No. 233). Plaintiff also filed a Reply to Motion to Compel Documents from Google ("Reply") (Doc. No. 229). Having fully considered the parties' arguments and for the reasons set forth herein, Plaintiff's Motion to Compel Production of Documents is GRANTED-IN-PART and DENIED-IN-PART. BACKGROUND On July 7, 2009, Plaintiff filed the instant Motion, requesting an order requiring Google to produce "all documents in its possession, custody, or control that show or describe the formulas that Google uses to calculate actual cost per click and Quality Score, including relevant source code." MOTION at 1. Plaintiff seeks all documents showing the formula for (1) the actual cost per click, and (2) determining Quality Score. Id. at 2 (citing Email from Cho to O'Brien, April 1, 2009). Plaintiff

asserts that it does not know what documents are in Defendant's possession, but presents the Court with email correspondence referring to a previous case where Google was the Defendant and counsel for Performance Pricing represented the Plaintiff in that suit. REPLY , EXH . 5 at 4. In the prior Bid For Position case, counsel for Performance Pricing alleges that Google engaged in limited production of technical documents, prompting Performance Pricing to express concern in the instant matter that Google not be permitted to limit disclosures during discovery. Id. It is Google's position that Performance Pricing has in their possession all the necessary documentation to appreciate the algorithms determining actual cost per click and Quality Score.1 It is undisputed that Google has produced the general formula2 for cost per click (dividing a variable numerator by a variable denominator), but it remains disputed whether Google has provided details as to how these numerators and denominators are calculated. MOTION at 3. Specifically, Performance Pricing argues that Google has not disclosed what factors affect the variables in the general formula and the precise way each of these factors is used in Google's Quality Score formula. Id. Google, however, maintains that it has produced documents describing the algorithms underlying actual cost per click and Quality score, but-- by their own acknowledgment--Performance Pricing does not understand these computations. RESPONSE at 4. Furthermore, it is undisputed that Performance Pricing has not hired a technical expert to review the Google disclosures and determine if they are sufficient to comprehend actual cost per click and Quality Score calculations. Id.

1

Google sells online advertising space to its customers and prices the advertisements on a "cost per click" basis. M O TIO N at 2. Plaintiff asserts that the actual cost per click is determined in part by the quality of the buyer's performance in creating high quality ad text and landing page content. Id. The buyer's performance is allegedly scored by Google using a computerized algorithm that Google calls the "Quality Score." Id.
2

CPC = (eCPI
n n+1

/ eCTR ), where CPC is the cost per click for an ad at position n. M O TIO N at 3.
n n

2

Google has also made a showing regarding Quality Score: "there is no static `formula' to produce" because Quality Score is computed by a machine learning algorithm.3 Similarly, in regards to actual cost per click, there is no formula to produce because the entire formula was previously disclosed as a four-page document in the prior suit. RESPONSE at 2­3 (citing Bid For Position, LLC v. AOL, LLC et al., Case No. 2:07-cv-582 (E.D. Vir. 2008) (Doc. No. 193 at 6­7)). Beyond the mathematical formulas, in a letter sent from counsel for Google (David Perlson) to counsel for Performance Pricing (Christin Cho), Google pointed out to Plaintiff that Google had produced a highly probative document entitled "SEPLA (A SmartASS Prediction Model)" ("the SEPLA document"), which describes the machine learning algorithm used to calculate the Quality Score in enough detail to be understood by a skilled technical person. REPONSE at 3; MOTION , EXH . 2 at 2­3 ("Perlson Letter"). Therefore, at the time this Motion was filed, Google argues that it had not only provided Plaintiff with price calculation details, but that it was Performance Pricing's inability to appreciate the disclosures that prevented them from instructing Google as to what further discovery they specifically sought. Id. DISCUSSION Performance Pricing devotes a substantial portion of its briefing to show that Google did not sufficiently make disclosures in accordance with this Court's Discovery Order and the Local Patent Rules. See MOTION at 2; REPLY at 1­2. The merits of that argument, however, are not properly before the Court because the weight of the arguments do not support Plaintiff's characterization that

3

"`Machine learning' refers to a type of computational algorithm which is derived by other algorithms. Since Quality Score is created by a constantly evolving method dervied by a computer for use by a computer, a human-readable static `formula' does not exist." R ESPO N SE at 3, n.2.

3

Google "has not produced the documents that show the complete formulas and algorithms that are used to calculate cost per click and, in particular, Quality Score." MOTION at 3. The discovery rules are accorded a broad and liberal treatment to effect their purpose of adequately informing litigants in civil trials. Herbert v. Lando, 441 U.S. 153, 177 (1979). Nevertheless, discovery does have "ultimate and necessary boundaries," Oppenheimer Fund, Inc. v. Sanders, 437 U.S. 340, 351 (1978) (quoting Hickman v. Taylor, 329 U.S. 495, 507 (1947)), and it is well established that the district court has wide discretion in establishing the confines of discovery. Alpine View Co. v. Atlas Copco AB, 205 F.3d 208, 220 (5th Cir. 2000). The Court's Discovery Order, Local Rules, and Patent Rules provide guidance as to relevant material, see Doc. No. 96, LOCAL RULE CV-26(d); LOCAL RULES, APPENDIX M. Specifically, Patent Rule 3-4(a) requires: "Source code, specifications, schematics, flow charts, artwork, formulas, or other documentation sufficient to show the operation of any aspects or elements of Accused Instrumentality." P. R. 3-4(a). Accordingly, while Performance Pricing accuses Google of only producing a "limited set" of documents and broadly demands "all documents in its possession, custody, or control that show or describe the formulas that Google uses for determining actual cost per click or Quality Score," MOTION at 2­3, the Court finds that Google has substantially provided documents showing the formulas and algorithms necessary to calculate cost per click and Quality Score. To date, Google represents that it has produced 1.8 million pages of documents in this matter, as well as several tutorial videos regarding Google AdWords. RESPONSE at 13. It is undisputed that Performance Pricing is in possession of the following: (1) Google interrogatory responses, see MOTION , EXH . 1; (2) cost per click (price) calculations, see PERLSON LETTER at 1; (3) eCPI/eCPM calculation, see 4

PERLSON LETTER at 2; (4) eCTR calculations, see PERLSON LETTER at 2; (5) a complete description of the mathematics behind the Google SmartASS algorithm, see PERLSON LETTER at 2­3 (referencing the SEPLA document); (6) the actual cost per click calculation, see RESPONSE at 2­3 (as disclosed in the Bid For Position case); and (7) the formula for computing AdRank, see RESPONSE at 15, n.7 (as disclosed in the Bid For Position case). Plaintiff has not adequately shown how these documents are insufficient to show cost per click and Quality Score. Nevertheless, Google has offered the following materials and the Court finds that these documents would enhance Performance Pricing's ability to (1) actually determine cost per click and Quality Score and (2) determine whether there is additional material it needs to perform the requisite calculations. The previously disclosed documents, coupled with further production of technical documents, as detailed herein, should provide Performance Pricing with adequate information and sufficient technical knowledge to satisfy Patent Rule 3-4. The request for "all documents relating to the calculation of actual cost per click and Quality Score, " MOTION at 5, is overbroad and does not provide adequate grounds for further relief. Therefore, having considered the parties' submissions and the weight of the arguments contained therein, Performance Pricing's Motion is DENIED to the extent that it seeks "all relevant documents" relating to the accused processes. I. Source Code Google has offered to make four categories of source code available for inspection, all of which are sought by Performance Pricing. These include: (1) source code describing the input factors to the Quality Score; (2) the operation of the machine learning algorithm which determines the importance of each input to the Quality Score; (3) the code that computes the Quality Score itself; and (4) the code for the actual cost per click. RESPONSE at 7 (listing the categories stated herein); 5

SURREPLY , EXH . Q at 1 (representing that Plaintiff has never "refused" the production of source code); SURREPLY at 2 (affirming that Performance Pricing explicitly requested source code on August 18, 2009). Google asserts that source code would merely show algorithms that are already disclosed in previous documents, and would not provide Performance Pricing with any further detail, MOTION , EXH . 2 at 2­3, but Performance Pricing expressly requests that this code be inspected so that Plaintiff has an independent opportunity to assess its value. See REPLY at 4. The Court is aware that Performance Pricing has characterized these categories of source code as only a "partial set of documents," REPLY at 3, but at the current juncture the Court is satisfied that Plaintiff will be in possession of both detailed algorithms and source code outlining formulas relevant to understand Google's process for determining price per click and Quality Score. Accordingly, the Court GRANTS Plaintiff's Motion that Google produce source code. The Court ORDERS that the relevant categories, supra, be made available. The Court, however, finds it unnecessary to consider whether the initial offer to produce the source code was timely and on what grounds it was offered, accepted and/or rejected. The parties are further ORDERED to meet and confer so that Google can make the above identified source code available for inspection by September 11, 2009. The Court encourages the parties to complete this production as soon as possible. II. Additional Documents From Previously-Identified Email Custodians Google has also proposed a search for "additional documents from previously-identified email custodians, including multiple employees with responsibility in Ads Quality." RESPONSE at 7. In responding to Google's Response, counsel for Performance Pricing has explicitly requested these same emails. SURREPLY , EXH . P at 1 (asking in email from Cho to O'Brien and Perlson on 6

August 18, 2009, for "emails from previously identified custodians that may relate to the computation of actual CPC or Quality Score"). Therefore, the Court GRANTS Plaintiff's Motion to the extent that it requests these emails and the parties are ORDERED to meet and confer regarding search terms and document exchange. Google is further ORDERED to produce any emails by September 11, 2009. III. 30(b)(6) Deposition As a result of the ongoing dispute between Performance Pricing and Google regarding the production of source code relating to calculations for actual cost per click and Quality Score the parties disagree as to whether Google is required to re-produce a 30(b)(6) witness for deposition on later-produced source code. SURREPLY at 3. In light of the Court's ruling, the Court GRANTS Plaintiff's request to take an appropriate deposition on those documents when they are produced. It is therefore ORDERED that Google produce a witness for a Rule 30(b)(6) deposition following the production of Google source code and emails as set forth in this Order. This deposition will be limited to three and a half hours in length and it is ORDERED that the witness only be questioned as to documents that are produced as a result of this Order, which were not available at the time of the initial deposition on August 20, 2009. Additionally, Google should make the witness available for this limited 30(b)(6) deposition on or before September 25, 2009. CONCLUSION For the reasons stated above, Plaintiff's Motion to Compel is DENIED-in-Part as to Performance Pricing's overbroad request for "all relevant documents," but GRANTED-in-part as to the categories identified in this Order. These categories include: source code describing the input factors to the Quality Score; the 7

operation of the machine learning algorithm which determines the importance of each input to the Quality Score; the code that computes the Quality Score itself; the code for the actual cost per click; and emails from previously identified custodians that may relate to the computation of actual CPC or Quality Score. Defendant Google is ORDERED to make these documents available by September 11, 2009. Defendant Google is further ORDERED to re-produce a witness by September 25, 2009 for a limited Rule 30(b)(6) deposition, as outlined above. Prior to both dates, the parties are ORDERED to meet and confer regarding these issues and are strongly encouraged, . going forward, to work together to resolve any remaining discovery disputes.

So ORDERED and SIGNED this 28th day of August, 2009.

___________________________________ JOHN D. LOVE UNITED STATES MAGISTRATE JUDGE

8

9