AMERICAN EDUCATIONAL RESEARCH ASSOCIATION, INC. et al v. PUBLIC.RESOURCE.ORG, INC.

Filing 134

MOTION for Summary Judgment , MOTION for Permanent Injunction by AMERICAN EDUCATIONAL RESEARCH ASSOCIATION, INC., AMERICAN PSYCHOLOGICAL ASSOCIATION, INC., NATIONAL COUNCIL ON MEASUREMENT IN EDUCATION, INC. (Attachments: # 1 Memorandum in Support, # 2 Statement of Facts, # 3 Declaration Hutter Decl., # 4 Exhibit 1, # 5 Exhibit 2, # 6 Exhibit 3, # 7 Exhibit 4, # 8 Declaration Hudis Decl., # 9 Exhibit A, # 10 Exhibit B, # 11 Exhibit C, # 12 Exhibit T, # 13 Exhibit U, # 14 Exhibit Z, # 15 Exhibit BB, # 16 Exhibit CC, # 17 Exhibit EE, # 18 Exhibit GG, # 19 Exhibit HH, # 20 Exhibit II, # 21 Exhibit JJ, # 22 Exhibit KK, # 23 Exhibit LL, # 24 Exhibit MM, # 25 Declaration Ernesto Decl., # 26 Exhibit NN, # 27 Exhibit OO, # 28 Exhibit PP, # 29 Exhibit QQ, # 30 Exhibit RR, # 31 Exhibit SS, # 32 Exhibit TT, # 33 Exhibit UU, # 34 Declaration Wise Decl., # 35 Exhibit KKK, # 36 Exhibit LLL, # 37 Declaration Camara Decl., # 38 Exhibit MMM, # 39 Declaration Levine Decl., # 40 Exhibit NNN, # 41 Exhibit PPP, # 42 Exhibit QQQ, # 43 Exhibit UUU, # 44 Declaration Geisinger Decl., # 45 Declaration Schneider Decl., # 46 Exhibit Levine Depo Tr., # 47 Exhibit No. 1207 to Levine Depo Tr., # 48 Exhibit No. 1308 to Levine Depo Tr., # 49 No. 1308 to Levine Depo Tr., # 50 Text of Proposed Order)(Elgarten, Clifton)

Case 1:14-cv-00857-TSC Document 84-1 Filed 02/12/16 Page 2 of 11

UNITED STATES DISTRICT COURT
FOR THE DISTRICT OF COLUMBIA
AMERICAN EDUCATIONAL RESEARCH
ASSOCIATION, INC., AMERICAN
PSYCHOLOGICAL ASSOCIATION, INC.,
and NATIONAL COUNCIL ON
MEASUREMENT IN EDUCATION, INC.,
Plaintiffs,

v.

)
)
)
)
)
)
)
)

)

PUBLIC.RESOURCE.ORG, INC.,

)
)
)
)

Civil Action No. 1: I 4-cv-00857-CRC

DECLARATION OF MARIANNE
ERNESTO IN SUPPORT OF
PLAINTIFFS' MOTION FOR
SUMMARY JUDGMENT AND ENTRY
OF A PERMANENT INJUNCTION

Defendant.
___________________________ )
I, MARIANNE ERNESTO, declare:
1.

I am the Director, Testing and Assessment, at the American Psychological

Association, Inc. ("APA"). I have been employed with the APA since May 2001. I submit this
Declaration in support of the motion of the American Educational Research Association, Inc.
("AERA"), the APA, and the National Council on Measurement in Education, Inc. ("NCME")
(collectively, "Plaintiffs" or the "Sponsoring Organizations") for a summary judgment and the
entry of a permanent injunction.
2.

In my role as Director, Testing and Assessment, I serve as APA's primary

authority on all matters that relate to testing and assessment.

This subject matter includes

educational testing, clinical assessment, forensic testing and employment testing. I advocate on
behalf of APA in matters involving federal or state legislative, regulatory or other policy issues
concerning testing and assessment. I coordinate APA's involvement in testing issues in matters
such as governance, executive boards, and managerial bodies. I also manage APA's responses to
internal, public, member and media inquiries regarding testing issues in a manner that is
consistent with the Standards for Educational and Psychological Testing (the "Standards").

- 1-

Case 1:14-cv-00857-TSC Document 84-1 Filed 02/12/16 Page 3 of 11

advise, counsel and oversee the activities ofthe APA's Science Directorate (and in particular its
Office of Testing and Assessment) on policy and governance issues related to testing and
assessments. I further serve as staff liaison to the APA's Committee on Psychological Tests and
Assessment ("CPTA"). Since 2001, I have served as APA's primary contact for infonnation
concerning the availability and interpretation of the Standards published in 1999, and more
recently I have done so regarding the updated Standards published in 2014.
3.

APA is a District of Columbia not-for-profit corporation.

4.

APA is the largest scientific and professional organization representing

psychology in the United States. APA is the world's largest association of psychologists and
counts a vast number of researchers, educators, clinicians, consultants and students among its
members.

APA's mission is to advance the creation, communication and application of

psychological knowledge to benefit society and improve people's lives.
5.

In 1954, APA prepared and published the "Technical Recommendations for

Psychological Tests and Diagnostic Techniques." It is my understanding that in 1955 AERA and
NCME prepared and published a companion document entitled, "Technical Recommendations
for Achievement Tests."
6.

Subsequently, a joint committee of the three organizations modified, revised and

consolidated the two documents into the first Joint Standards. Beginning with the 1966 revision,
the three organizations (AERA, APA and NCME) collaborated in developing the "Joint
Standards" (or simply, the "Standards"). Each subsequent revision of the Standards has been
careful to cite the previous Standards and note that it is a revision and update of that document.
7.

Beginning in the mid-1950s, AERA, APA, and NCME fonned and periodically

reconstituted a committee of experts in psychological and educational assessment, charged with

-2-

Case 1:14-cv-00857-TSC Document 84-1 Filed 02/12/16 Page 4 of 11

the initial development of the Technical Recommendations and then each subsequent revision of
the (renamed) Standards. These committees were fonned by the Plaintiffs' Presidents (or their
designees), who would meet and jointly agree on the membership. Often a chair or co-chairs of
these committees were selected by joint agreement. Beginning with the 1966 version of the
Standards, this committee became referred to as the "Joint Committee."
8.

Financial and operational oversight for the Standards' revisions, promotion,

distribution, and for the sale of the 1999 and 2014 Standards has been undertaken by a
periodically reconstituted Management Committee, comprised of designees of the three
Sponsoring Organizations.
9.

All members of the Joint Committee(s) and the Management Committee(s) are

unpaid volunteers. The expenses associated with the ongoing development and publication of
the Standards include travel and lodging expenses (for the Joint Committee and Management
Committee members), support staff time, printing and shipment of bound volumes, and
advertising costs.
I 0.

Many different fields of endeavor rely on assessments.

The Sponsoring

Organizations have ensured that the range of these fields of endeavor is represented in the Joint
Committee's membership - e.g., admissions, achievement, clinical counseling, educational,
licensing-credentialing, employment, policy, and program evaluation.

Similarly, the Joint

Committee's members, who are unpaid volunteers, represent expertise across major functional
assessment areas - e.g., validity, equating, reliability, test development, scoring, reporting,
interpretation, and large scale interpolation.
11.

From the time of their initial creation to the present, the preparation of and

periodic revisions to the Standards entail intensive labor and considerable cross-disciplinary

Case 1:14-cv-00857-TSC Document 84-1 Filed 02/12/16 Page 5 of 11

expertise. Each time the Standards are revised, the Sponsoring Organizations select and arrange
for meetings of the leading authorities in psychological and educational assessments (known as
the Joint Committee). During these meetings, certain Standards are combined, pared down,
and/or augmented, others are deleted altogether, and some are created as whole new individual
Standards. The 1999 version of the Standards is nearly 200 pages and took more than five years
to complete.
12.

The Standards, however, are not simply intended for members of the Sponsoring

Organizations, AERA, APA, and NCME. The intended audience of the Standards is broad and
cuts across audiences with varying backgrounds and different training.

For example, the

Standards also are intended to guide test developers, sponsors, publishers, and users by providing
criteria for the evaluation of tests, testing practices, and the effects of test use.

Test user

standards refer to those standards that help test users decide how to choose certain tests, interpret
scores, or make decisions based on tests results.

Test users include clinical or industrial

psychologists, research directors, school psychologists, counselors, employment supervisors,
teachers, and various administrators who select or interpret tests for their organizations. There is
no mechanism, however, to enforce compliance with the Standards on the part of the test
developer or test user.

The Standards, moreover, do not attempt to provide psychometric

answers to policy or legal questions.
I 3.

The Standards apply broadly to a wide range of standardized instruments and

procedures that sample an individual' s behavior, including tests, assessments, inventories, scales,
and other testing vehicles. The Standards apply equally to standardized multiple-choice tests,
performance assessments (including tests comprised of only open-ended essays}, and hands-on
assessments or simulations.

The main exceptions are that the Standards do not apply to

-4 -

Case 1:14-cv-00857-TSC Document 84-1 Filed 02/12/16 Page 6 of 11

unstandardized questionnaires (e.g., unstructured behavioral checklists or observational forms),
teacher-made tests, and subjective decision processes (e.g., a teacher's evaluation of students'
classroom participation over the course of a semester).
14.

The Standards have been used to develop testing guidelines for such activities as

college admissions, personnel selection, test translations, test user qualifications, and computerbased testing. The Standards also have been widely cited to address technical, professional, and
operational norms for all forms of assessments that are professionally developed and used in a
variety of settings. The Standards additionally provide a valuable public service to state and
federal governments as they voluntarily choose to use them. For instance, each testing company,
when submitting proposals for testing administration, instead of relying on a patchwork of local,
or even individual and proprietary, testing design and implementation criteria, may rely instead
on the Sponsoring Organizations' Standards to afford the best guidance for testing and
assessment practices.
15.

The Standards were not created or updated to serve as a legally binding document,

in response to an expressed governmental or regulatory need, nor in response to any legislative
action or judicial decision. However, the Standards have been cited in judicial decisions related
to the proper use and evidence for assessment, as well as by state and federal legislators. These
citations in judicial decisions and during legislative deliberations occurred without any lobbying
by the Plaintiffs.
16.

During the discovery phase of this litigation, APA located in its archives

correspondence relating to APA's support for proposed legislation sought to be introduced in
2001 by Senator Paul Wellstone (D-MN) on Fairness and Accuracy in High Stakes Educational
Decisions for Students- a suggested amendment to the Elementary and Secondary Education Act

-5-

Case 1:14-cv-00857-TSC Document 84-1 Filed 02/12/16 Page 7 of 11

(''No Child Left Behind Act") 147 Cong. Rec . S. 4,644 (daily ed. May 9, 2001).
17.

Accompanying this Declaration as Exhibit NN is a true copy of a signed

correspondence between Ellen Garrison Ph.D. and Patricia Kobor of APA and Ms. Jill
Morningstar, Legislative Assistant to U.S. Senator Paul Wellstone dated April 7, 2000, marked
as Exhibit 11 09 during my deposition.

J 8.

Accompanying this Declaration as Exhibit 00 is a true copy of an unsigned

correspondence between Ellen Garrison Ph.D. and Patricia Kobor of APA and Ms. Jill
Morningstar, Legislative Assistant to U.S. Senator Paul Wellstone dated April 7, 2000, marked
as Exhibit I J 10 during my deposition .

J9.

Accompanying this Declaration as Exhibit PP is a true copy of a signed

correspondence between Patricia Kobor and Ellen Garrison, Ph.D. of APA and Ms. Jill
Morningstar, Legislative Assistant to U.S. Senator Paul Wellstone dated April J 3, 2000, marked
as Exhibit 1111 during my deposition.
20.

Accompanying this Declaration as Exhibit

QQ is a true copy of an unsigned

correspondence between Raymond D. Fowler, Ph.D. of APA and an unnamed Senator dated
May 7, 200 J, marked as Exhibit 1114 during my deposition.
2 J.

Accompanying this Declaration as Exhibit RR is a true copy of an unsigned

correspondence between L. Michael Honaker, Ph.D. of APA and an unnamed Senator dated
March 6, 2001, marked as Exhibit 1 J 15 during my deposition.
22.

Accompanying this Declaration as Exhibit SS is a true copy of a document

containing "Highlights of APA's Involvement in Educational Testing Provisions of the 'No
Child Left Behind Act"' that also contains an unsigned correspondence to an unnamed Senator
dated May 7, 2001, marked as Exhibit 1 1 16 during my deposition.

-6-

Case 1:14-cv-00857-TSC Document 84-1 Filed 02/12/16 Page 8 of 11

23.

As noted above, many of these letters are unsigned and are not printed on APA

letterhead. Therefore, in accordance with APA practices and protocols, it is likely that the
unsigned letters (not printed on letterhead) were internal discussion drafts that were never sent.
24.

Regarding the signed letters that were printed on APA letterhead, they relate to

Senator Wellstone's proposed legislation that tests and assessments administered by the states
are of high quality and used appropriately for the benefit of test administrators and test takers.
These are goals that are consistent with APA policy as then reflected in the 1999 Standards.
Even though Senator Wellstone's amendments sought, in part, to mandate states' compliance
with the Standards, none of the Sponsoring Organizations actively advocated for this - and in
any event Senator Wellstone's proposed amendment including this language was never enacted
into law. Accompanying this Declaration as Exhibit TT is a true copy of 20 U.S.C. § 6301,
which is the current version of the legislation Senator Wellstone sought to amend.
25.

APA's search of its records did not disclose any further communications with

Congress relating to the Standards and, to the best of APA's knowledge, it has not engaged in
communications with Congress regarding citation of the Standards in legislation since 2001.
26.

APA has not solicited any government agency to incorporate the Standards into

the Code of Federal Regulations or other rules of Federal or State agencies.
27.

Rather, in the policymaking arena, APA believes the Standards should be treated

as guidelines informing the enactment of legislation and regulations consistent with best
practices in the development and use of tests - to insure that they are valid, reliable and fair.
28.

Plaintiffs promote and sell copies of the Standards via referrals to the AERA

website, at annual meetings, in public offerings to students, and to educational institution faculty.
Advertisements promoting the Standards have appeared in meeting brochures, in scholarly

-7-

Case 1:14-cv-00857-TSC Document 84-1 Filed 02/12/16 Page 9 of 11

journals, and in the hallways at professional meetings.

Accompanying this Declaration as

Exhibit UU is a true copy of an advertisement for the 1999 Standards that appeared in the
December 1999 issue of APA's Journal of Educational Psychology.
29.

Distribution of the Standards is closely monitored by the Sponsoring

Organizations.

AERA, the designated publisher of the Standards, sometimes does provide

promotional complementary print copies to students or professors.

Except for these few

complementary print copies, however, the Standards are not given away for free; and certainly
they are not made available to the public by any of the three organizations for anyone to copy
free of charge.
30.

To date, the Sponsoring Organizations have never posted, or authorized the

posting of, a digitized copy ofthe 1999 Standards on any publicly accessible website.
31.

The Sponsoring Organizations do not keep any of the revenues generated from the

sales of the Standards.

Rather, the income from these sales is used by the Sponsoring

Organizations to offset their development and production costs and to generate funds for
subsequent revisions.

This allows the Sponsoring Organizations to develop up-to-date, high

quality Standards that otherwise would not be developed due to the time and effort that goes into
producing them.
32.

Without receiving at least some moderate income from the sales of the Standards

to offset their production costs and to allow for further revisions, it is very likely that the
Sponsoring Organizations would no longer undertake to periodically update them, and it is
unknown who else would.
33.

Due to the relative minor portion of the membership of APA who devote their

careers to testing and assessment, it is highly unlikely that the members of APA will vote for a

- 8-

Case 1:14-cv-00857-TSC Document 84-1 Filed 02/12/16 Page 10 of 11

dues increase to fund future Standards revision efforts if Public Resource successfully defends
this case and is allowed to post the Standards online for the public to download or print for free.
As a result, the Sponsoring Organizations would likely abandon their practice of periodically
updating the Standards.
34.

The Joint Committee that authored the 1999 Standards comprised 16 members.

Except for Manfred Meier (who could not be located, nor could his heirs), work made-for-hire
letters were signed by 13 Joint Committee Members, and posthumous assignments were signed
by the heirs of 2 deceased Joint Committee Members, vesting ownership of the copyright to the
1999 Standards in the Sponsoring Organizations. Accompanying this Declaration as Exhibits
VV -HHH are the 13 work made-for-hire letters signed by Eva Baker, Lloyd Bond, Daniel Goh,
Bert Green, Edward Haertel, Jo-lda Hansen, Suzanne Lane, Sharon Johnson-Lewis, Joseph
Matarazzo, Pamela Moss, Esteban Olmedo, Diana Pullin, and Paul Sackett, marked as Exhibits
1065, 1069, 1071, 1072, 1075, 1078, 1082, 1085, 1086, 1089, 1090, 1091, and 1094 during my
deposition.

Accompanying this Declaration as Exhibits Ill and JJJ are the posthumous

assignments signed by the heirs of Leonard S. Feldt and Charlie Spielberger, marked as exhibits
1070 and 1097 during my deposition.
35.

Public Resource posted Plaintiffs' 1999 Standards to its website and the Internet

Archive website without the permission or authorization of any of the Sponsoring Organizations.
36.

Past harm from Public Resource's infringing activities includes misuse of

Plaintiffs' intellectual property without permission.
37.

Should Public Resource's infringement be allowed to continue, the harm to the

Sponsoring Organizations, and public at large who rely on the preparation and administration of
valid, fair and reliable tests, includes: (i) uncontrolled publication of the 1999 Standards without

-9-

Case 1:14-cv-00857-TSC Document 84-1 Filed 02/12/16 Page 11 of 11

any notice that those guidelines have been replaced by the 2014 Standards; (ii) future
unquantitiable loss of revenue from sales of authorized copies of the 1999 Standards (with
proper notice that they are no longer the current version) and the 2014 Standards; and (iii) lack of
funding for future revisions ofthe 2014 Standards and beyond.
Originally signed in identical form on December 8, 2015.
I declare under the penalty of perjury that the foregoing is true and correct. Executed on

\c::r:C~ c
Marianne Ernesto

- 10-

t

'

c

S:.--s--s ~ .