AMERICAN EDUCATIONAL RESEARCH ASSOCIATION, INC. et al v. PUBLIC.RESOURCE.ORG, INC.

Filing 136

Second MOTION for Summary Judgment by PUBLIC.RESOURCE.ORG, INC. (Attachments: # 1 [REDACTED] Public Resources Memorandum of Law, # 2 Declaration Carl Malamud, # 3 Declaration Matthew Becker, # 4 [REDACTED] Public Resources Statement of Material Facts, # 5 [REDACTED] Public Resources Statement of Disputed Facts, # 6 [REDACTED] Public Resources Objections to Plaintiffs Evidence, # 7 Public Resources Request for Judicial Notice in Opposition, # 8 Text of Proposed Order, # 9 Certificate of Service)(Bridges, Andrew)

UNITED STATES DISTRICT COURT
FOR THE DISTRICT OF COLUMBIA

AMERICAN EDUCATIONAL RESEARCH
ASSOCIATION, INC., AMERICAN
PSYCHOLOGICAL ASSOCIATION, INC., and
NATIONAL COUNCIL ON MEASUREMENT IN
EDUCATION, INC.,

Case No. 1:14-CV-00857-TSC

Plaintiffs-Counterdefendants,
v.
PUBLIC.RESOURCE.ORG,
Defendant-Counterclaimant.
PUBLIC RESOURCE’S STATEMENT OF DISPUTED FACTS IN OPPOSITION TO
[134] PLAINTIFFS’ MOTION FOR SUMMARY JUDGMENT AND FOR PERMANENT
INJUNCTION AND IN SUPPORT OF PUBLIC RESOURCE’S SECOND MOTION FOR
SUMMARY JUDGMENT
[PUBLIC REDACTED VERSION]

Pursuant to the Local Civil Rule 7(h), Defendant-Counterclaimant Public.Resource.Org
submits this statement of disputed facts in opposition to [134] Plaintiffs’ motion for summary
judgment and permanent injunction and in support of Public Resource’s motion for summary
judgment. Unless otherwise stated, citations are to Public Resource’s Supplemental Statement of
Material Facts in Opposition to [134] Plaintiffs’ Motion for Summary Judgment and Permanent
Injunction, and in Support of Public Resource’s Second Motion for Summary Judgment
(“SSMF”).
Plaintiffs’ Statement of Material Facts
1. Plaintiffs, the Sponsoring Organizations, are
each District of Columbia not-for-profit
corporations (Levine Decl., ¶ 4; Ernesto Decl.,
¶ 3; Wise Decl., ¶ 3). Each is an active
professional organization that engages in many
activities for many purposes other than to create
the
Standards
for
Educational
and
Psychological Testing at issue in this case.
2. American Education Resource Association
(AERA) is the major national scientific society
for research on education and learning. AERA’s
mission is to advance knowledge about
education, to encourage scholarly inquiry
related to education, and to promote the use of
research to improve education and serve the
public good (Levine Decl., ¶ 5).
3. The American Psychological Association
(APA) is the largest scientific and professional
organization representing psychology in the
United States. APA is the world’s largest
association of psychologists and counts a vast
number of researchers, educators, clinicians,
consultants and students among its members.
APA’s mission is to advance the creation,
communication,
and
application
of
psychological knowledge to benefit society and
improve people’s lives (Ernesto Decl., ¶ 4).

Defendant Public Resource’s Response

Plaintiffs’ Statement of Material Facts

Defendant Public Resource’s Response

4. The National Council on Measurement in
Education (NCME) is a professional
organization for individuals involved in
assessment, evaluation, testing, and other
aspects of educational measurement. NCME’s
members are involved in the construction and
use of standardized tests; new forms of
assessment, including performance- based
assessment; program design; and program
evaluation (Wise Decl., ¶ 4).
5. The Sponsoring Organizations have been
preparing and publishing versions of the
Standards for over fifty years. In 1954, APA
prepared and published the “Technical
Recommendations for Psychological Tests and
Diagnostic Techniques” (Camara Decl., ¶ 7;
Ernesto Decl., ¶ 5). In 1955, AERA and NCME
prepared and published a companion document
titled, “Technical Recommendations for
Achievement Tests” (Levine Decl., ¶ 6; Camara
Decl., ¶ 7; Wise Decl., ¶ 5). Subsequently, a
joint committee of the three organizations
modified, revised, and consolidated the two
documents into the first Joint Standards.
Beginning with the 1966 revision, the three
organizations collaborated in developing the
“Joint Standards” (or simply, the “Standards”).
Each subsequent revision of the Standards has
been careful to note that it is a revision and
update of the prior version (Levine Decl., ¶ 6;
Camara Decl., ¶ 7; Ernesto Decl., ¶ 6; Wise
Decl., ¶ 6).
6. Specifically, beginning in the mid-1950s,
Plaintiffs formed and periodically reconstituted
a committee of experienced experts in
psychological and educational assessment,
charged with the initial development of the
Technical Recommendations and then each
subsequent revision of the (renamed) Standards.
These committees were formed by the
Plaintiffs’ presidents (or designees), who would
meet and agree on the committees’

3

Plaintiffs’ Statement of Material Facts

Defendant Public Resource’s Response

membership. Beginning with the 1966 version
of the Standards, this committee has been called
the “Joint Committee” (Levine Decl., ¶ 7;
Camara Decl., ¶ 8; Ernesto Decl., ¶ 7; Wise
Decl., ¶ 7).
7. Financial and operational oversight for
revising, promoting, distributing, and selling the
1999 and 2014 Standards has been undertaken
by a periodically reconstituted “Management
Committee,” comprised of designees of the
three Plaintiffs (Levine Decl., ¶ 8; Camara
Decl., ¶ 9; Schneider Decl., ¶ 4; Ernesto Decl.,
¶ 8; Wise Decl., ¶ 8). All members of the Joint
Committee(s)
and
the
Management
Committee(s) are unpaid volunteers. The
expenses associated with the ongoing
development and publication of the Standards
include travel and lodging expenses (for the
Joint Committee and Management Committee
members), support staff time, printing and
shipment of bound volumes, and advertising
costs (Levine Decl., ¶ 9; Camara Decl., ¶ 10;
Schneider Decl., ¶ 5; Ernesto Decl., ¶ 9; Wise
Decl., ¶ 9).
8. Many different fields of endeavor rely on
assessments, and Plaintiffs seek to ensure that
the range of these fields of endeavor is
represented in the Joint Committees’
membership — e.g., admissions, achievement,
clinical counseling, educational, licensing
credentialing, employment, policy, and program
evaluation. Similarly, the Joint Committee’s
members represent expertise across major
functional assessment areas, including validity,
equating, reliability, test development, scoring,
reporting, interpretation, and large scale
interpolation (Levine Decl., ¶ 10; Ernesto Decl.,
¶ 10; Wise Decl., ¶ 10).
9. AERA currently serves as publisher of the Disputed to the extent that this statement
Standards, which are made available for implies that any edition of the Standards other
purchase through the AERA website. Levine than the 2014 Standards are currently available

4

Plaintiffs’ Statement of Material Facts

Defendant Public Resource’s Response

Tr. 29, 55. AERA is a small organization, and a for purchase through the AERA website.
purchaser may ordinarily contact some in SSMF ¶ 50.
person at AERA if they are having difficulty
placing an order. Levine Tr. 61-62. Sales are
made to individual entities, libraries across the
country, college bookstores and academic
institutions, with spikes in sales sometimes
associate with the academic calendar. See
Levine Tr. 49-50. AERA sells the volumes for
hard copy purchase, and one can also purchase
an electronic (read only) version of the 2014
Standards – or one can make a combined
purchase of both the electronic and hard copy
for a discounted price. Levine Tr. 52-54, 61,
Exh. 1308, 1309.
10. Plaintiffs promote and sell copies of the
Standards via a variety of referrals to the AERA
website, at annual meetings, in public offerings
to students, and to educational institution
faculty.
Advertisements
promoting
the
Standards have appeared in meeting brochures,
in scholarly journals, and in the hallways at
professional meetings (Levine Decl., ¶ 14, Exh.
NNN; Ernesto Decl., ¶ 28, Exh. UU; Wise
Decl., ¶ 21, Exh. KKK).

Disputed that Plaintiffs promote the 1999
Standards that are at issue in this case; to the
contrary, they discourage purchase or use of
the 1999 Standards. SSMF ¶ 49.

11. Distribution of the Standards is monitored
by the Sponsoring Organizations. AERA, the
now-designated publisher of the Standards,
sometimes
does
provide
promotional
complementary print copies to students or
professors. Except for these few complementary
print copies, however, the Standards are not
given away for free; they are not made available
to the public by any of the three organizations,
or given to anyone to copy free of charge
(Levine Decl., ¶ 16; Ernesto Decl., ¶ 29; Wise
Decl., ¶ 22). To date, Plaintiffs have never
posted, or authorized the posting of, a digitized
copy of the 1999 Standards on any publicly
accessible website (Levine Decl., ¶ 16; Ernesto
Decl., ¶ 30; Wise Decl., ¶ 23). Thus, the
Sponsoring Organizations do not make the

Disputed. Plaintiffs have agreed to allow the
State of New York to provide copies of the
1999 Standards to citizens who request it, as a
result of the incorporation of the 1999
Standards into New York law. SSMF ¶¶ 4344.

5

Disputed. Distribution of the Standards is not
closely monitored by the Sponsoring
Organizations. At deposition, the Director of
Testing and Assessment at APA and the
Executive Director of AERA both admitted
Becker Decl. ¶ 17, Ex. 50 (Ernesto Depo.) at
203:15–207:10, 208:20–209:11; Becker Decl.
¶ 19, Ex. 52 (Levine Depo. I) at 42:12–23.

Plaintiffs’ Statement of Material Facts

Defendant Public Resource’s Response

Standards accessible (except for purchase) online in read only format. (C.f. American Society
for Testing and Materials, et al. v.
Public.Resource.Org, Inc., 896 F.3d 437, 453
(2018) (“the SDOs, by their own admission,
make copies of their standards freely available
online in controlled reading rooms”)). Doing so
would logically reduce AERA’s ability to sell
the Standards, and thus reduce revenue for the
overall enterprise of maintaining the Standards.

Additionally Plaintiffs have allowed two
identical copies of the 1999 Standards to be
posted online in this case, and available for
free download. SSMF ¶¶ 52-3.

12. The 1999 Standards have historically been
sold at modest retail prices ranging from $25.95
to $49.95 per copy. (Levine Decl., ¶ 17).
Current pricing for members for either a hard
copy or e-book (single user) of the 2014 edition
is $49.95, or as a bundle (hard copy and single
user e-book) at $59.95. The non-member prices
for either a hard copy or single user ebook is
$69.95, with $79.95 for the bundle. The 1999
edition is available in hardcopy at $35.95 for
members and $45.95 for non-members, but the
advertisement/offer of the 1999 Standards
expressly sets forth the caution that the volume
has been superseded. Levine Tr. Exh. 1308,
1309.

Disputed that the Plaintiffs have historically
continuously sold the 1999 Standards. The
Plaintiffs previously took the 1999 Standards
off the market before temporarily returning
them to the market as a pretext in this lawsuit.
SSMF ¶ 50.

13. The 1999 Standards were first offered for
sale in mid-1999. Sales in the year 2000 were
estimated at 3797; for 2001, at 3755. They were
5592 in 2002; 3310 in 2003; 3218 in in 2004;
3803 in 2005; 3888 in 2006; 3077 in 2007;
3358 in 2008; 2590 in 2009; 3043 in 2010;
2132 in 2011; 1649 in 2012; 1732 in 2013; and
855 in 2014. Levine Tr. Exh. 1207. Declining
sales 2011-2014 are coincident with, and might
be attributed to, a combination of factors
including: PRO’s uploading of the 1999
Standards on-line beginning in May 2012 and
running through June 2014; saturation of parts
of the market; or anticipation of the impending
release of a new edition, the 2014 Standards,
which was introduced in the middle of 2014.

Disputed to the extent that Plaintiffs argue that
any portion of the decline in sales of the 1999
Standards was to Public Resource’s posting of
the 1999 Standards. SSMF ¶¶ 56-61.

6

Plaintiffs’ Statement of Material Facts

Defendant Public Resource’s Response

14. After the 2014 version of the Standards was
published in late summer of 2014, AERA for a
time discontinued sales of the 1999 Standards in
order to encourage sales of the newly-revised
edition — the 2014 Standards (Levine Decl., ¶
19, Exh. PPP). However, so long as purchasers
are aware that it is no longer the current edition
– which is made clear on the AERA website –
the 1999 Standards continue to have value for
those in the testing and assessment profession
who (i) need to know the state of best testing
practices as they existed between 1999 and
2014, (ii) believe they still may be held
accountable to the guidance of the 1999
Standards, or (iii) study changes in best testing
and assessment practices over time. In the
summer of 2015, AERA resumed sales of the
1999 Standards (Levine Decl., ¶ 20, Exh.
QQQ).
15. The production and updating of the
Standards is financed solely through sales
revenues. All revenue from the sale of the 1999
Standards above expenses is used to cover the
publishing costs of the Standards and for the
preparation of subsequent editions of the
Standards. The Sponsoring Organizations do
not distribute any proceeds from the sales of the
Standards to the Sponsoring Organizations.
Thus, generation of revenue allows the
Sponsoring Organizations to develop up-todate, high quality Standards that otherwise
would not be developed due to the time and
effort that goes into producing them (Levine
Decl., ¶ 21; Geisinger Decl., ¶ 22; Camara
Decl., ¶ 19; Ernesto Decl., ¶ 31). Without
income from the sales of the Standards to offset
production costs and to allow for further
revisions, it is very likely that the Sponsoring
Organizations would no longer undertake to
periodically update them (Levine Decl., ¶ 22;
Ernesto Decl., ¶ 32; Wise Decl., ¶ 24; Geisinger
Decl., ¶ 22).

7

Disputed to the extent that Plaintiffs assert the
Standards would not be developed if not for
the revenue from sale of earlier editions, or
that any further sales of the 1999 Standards are
necessary to finance the production of future
editions of the Standards now that the 2014
Standards have been on sale for over five
years, while unpaid volunteers bear the
majority of costs in developing the standards.
SSMF ¶ 47.

Plaintiffs’ Statement of Material Facts

Defendant Public Resource’s Response

16. Plaintiffs at one time considered soliciting
funding for the revision process from third party
sources
like
governmental
agencies,
foundations, and other associations interested in
testing and assessment issues. But Plaintiffs
rejected this option due to the difficulty of
procuring the funding as well as potential
conflicts of interest that could arise from such a
system. The Sponsoring Organizations therefore
concluded that revisions should be self-funding
— that is, from sale of prior editions of the
Standards (Levine Decl., ¶ 23; Camara Decl.,
¶ 20).
17. The Sponsoring Organizations do not, so far
as the record shows, market any associated
services for sale, such as seminars, as a way to
generate revenue or profit through collateral
businesses or related products. (Compare 896
F.3d at 453 (“[C]an the SDOs continue to make
money on derivative good such that they have
an adequate incentive to continue producing the
standards?”)). The revenue to finance the
important work of updating and revising the
Standards as appropriate is generated solely
through sales of the Standards.

8

Disputed to the extent that Plaintiffs claim
“[t]he revenue to finance the important work of
updating and revising the Standards as
appropriate is generated solely through sales of
the Standards.” Unpaid volunteers and their
employees bear the majority of costs in
developing the standards, while Plaintiffs
simply cover less significant logistical
expenses. SSMF ¶ 47.

Plaintiffs’ Statement of Material Facts

Defendant Public Resource’s Response

18. Due to the small membership size of
Plaintiff NCME, and the relative minor portion
of the membership of Plaintiffs AERA and
APA who devote their careers to testing and
assessment, it is highly unlikely that the
members of the Sponsoring Organizations
would vote for a dues increase to fund future
revisions if sales revenue is lost by virtue of an
entity like PRO allowing the Standards to be
accessed on-line and downloaded or printed for
free. As a result, the Sponsoring Organizations
would likely abandon their practice of
periodically updating the Standards (Levine
Decl., ¶ 24; Camara Decl., ¶ 24; Geisinger
Decl., ¶ 23; Ernesto Decl., ¶ 33).

Disputed. The current (2014) edition of the
Standards is not at issue in this litigation, and
Plaintiffs concede that the revenue from the
sale of the 1999 Standards began plummeting
prior to Public Resource’s posting of the 1999
Standards, and sales have been “near nil” ever
since Plaintiffs took the 1999 Standards off the
market and then restored them to the market
through a cumbersome sales channel. Because
sales of the 1999 Standards are “near nil,”
Plaintiffs have no basis for asserting that if
PRO were allowed to post that document
online, Plaintiffs “would likely abandon their
practice of periodically updating the
Standards” due to the absence of revenue.
SSMF ¶¶ 56-61.
Plaintiffs have failed to adduce admissible
evidence in support of these statements, which
are opinions, not facts. Dr. Levine, Mr.
Camara, and Ms. Ernesto are not qualified as
experts to opine on this subject. Dr. Geisinger
is not an expert on revenue models or
standards development and is not qualified to
opine on this subject. See Public Resource’s
Motion to Strike the Declaration of Dr.
Geisinger, Dkt. 67. Plaintiffs have provided no
evidence as to how Public Resource’s posting
of the 1999 Standards could harm Plaintiffs’
income from the 2014 Standards. Additionally,
Plaintiffs’ claims are not reasonable.
Becker Decl., ¶
21, Ex. 54 (Geisinger Depo.) at 183:15–
191:24.

Becker
Decl., ¶ 21, Ex. 54 (Geisinger Depo.) at
200:05–201:22.

9

Plaintiffs’ Statement of Material Facts

Defendant Public Resource’s Response

19. As described above, the 1999 Standards are
available for sale on-line through AERA.
However, the 1999 Standards have been widely
sold to professionals, academic institutions,
libraries, professionals, students and academics.
The 1999 Standards already in circulation are,
therefore, available for review and consultation
through those outlets.

Disputed. The 1999 Standards are not
available for sale on-line through AERA.
They do not appear in the AERA online store.
Someone who wishes to purchase the 1999
Standards must locate the AERA mail-order
form, which can be downloaded through the
AERA website, and then place a mail order for
the 1999 Standards. SSMF ¶ 50.

20. The Standards are published “to promote
the sound and ethical use of tests and to provide
a basis for evaluating the quality of testing
practices.” See 1999 Standards at 1 (quoted by
the Court of Appeals, 896 F.3d at 441). The
Standards set forth principles and guidelines,
designed to provide a set of best practices to
improve testing and assessment across multiple
settings, including education and various areas
of psychology. The Standards can and should
be used in the sound and ethical development
and use of tests, and also to evaluate the quality
of tests and testing practices (Geisinger Decl., ¶
18; Camara Decl., ¶ 13; Wise Decl., ¶ 12).
21. The Standards are not simply intended for
Plaintiffs’ members. They are intended for a
broad audience that cuts across professions,
backgrounds, and training. For example, they
can guide test developers, sponsors, publishers,
and users by providing criteria for the
evaluation of tests, testing practices, and the
effects of test use. Test user standards refer to
those standards that help test users decide how
to choose certain tests, interpret scores, or make
decisions based on tests results. Test users
include clinical or industrial psychologists,
research directors, school psychologists,
counselors, employment supervisors, teachers,
and various administrators who select or
interpret tests for their organizations. There is
no mechanism, however, to enforce compliance
with the Standards on the part of the test
developer or test user (Camara Decl., ¶ 14;
Wise Decl., ¶ 13; Geisinger Decl., ¶ 19; Ernesto

10

Plaintiffs’ Statement of Material Facts

Defendant Public Resource’s Response

Decl., ¶ 12).
22. The Standards promote the development of
high quality tests and the sound use of results
from such tests. Without such high quality
standards, tests might produce scores that are
not defensible or accurate, not an adequate
reflection of the characteristic they were
intended to measure, and not fair to the person
tested. Thus, the Standards help ensure that
measures of student achievement are relevant,
that admissions decisions are fair, that
employment
hiring
and
professional
credentialing results in qualified individuals
being selected, and patients with psychological
needs are diagnosed properly and treated
accordingly. Quality tests protect the public
from harmful decision-making and provide
opportunities for education and employment
that are fair to all who seek them (Camara
Decl., ¶ 15; Wise Decl., ¶ 14).
23. The Standards apply broadly to a wide
range of standardized vehicles and procedures
that sample an individual’s behavior, including
tests, assessments, inventories, scales, and other
testing vehicles. The Standards apply equally to
standardized multiple-choice tests, performance
assessments (including tests comprised of only
open-ended essays), and hands-on assessments
or simulations. The main exceptions are that the
Standards do not apply to unstandardized
questionnaires (e.g., unstructured behavioral
checklists or observational forms), teachermade tests, and subjective decision processes
(e.g., a teacher’s evaluation of students’
classroom participation over the course of a
semester) (Camara Decl., ¶ 16; Wise Decl., ¶
15; Geisinger Decl., ¶ 20; Ernesto Decl., ¶ 13).
24. The Standards have been used to develop
testing guidelines for such activities as college
admissions,
personnel
selection,
test
translations, test user qualifications, and

11

Plaintiffs’ Statement of Material Facts

Defendant Public Resource’s Response

computer-based testing. The Standards also
have been widely cited to address technical,
professional, and operational norms for all
forms of assessments that are professionally
developed and used in a variety of settings. The
Standards additionally provide a valuable public
service to state and federal governments that
choose to use them. For instance, each testing
company, when submitting proposals for testing
administration, instead of relying on a
patchwork of local, or even individual and
proprietary, testing design and implementation
criteria, may rely instead on the Standards to
afford the best guidance for testing and
assessment practices (Camara Decl., ¶ 17; Wise
Decl., ¶ 16; Geisinger Decl., ¶ 21; Ernesto
Decl.,¶ 14).
25. As described above, the Standards are Disputed. Plaintiffs explicitly identified
developed and designed for professional use. policymakers as key targets of the 1999
The Standards were not created or updated to Standards. SSMF ¶ 19.
address any governmental or regulatory need,
nor in response to any legislative action or
judicial decision. However, the Standards have
been cited in judicial decisions related to the
proper use and evidence for assessment, as well
as by state and federal legislators. These
citations in judicial decisions and during
legislative deliberations occurred without any
lobbying by the Plaintiffs (Levine Decl., ¶ 12;
Camara Decl., ¶ 18; Ernesto Decl., ¶ 15; Wise
Decl., ¶ 17).
26. During the first round of discovery in this
case, APA located some correspondence or
draft correspondence relating to APA’s support
for legislation proposed in 2001 by Senator
Paul Wellstone (D-MN) on Fairness and
Accuracy in High Stakes Educational Decisions
for Students — a suggested amendment to the
Elementary and Secondary Education Act (“No
Child Left Behind Act”) 147 Cong. Rec. S.
4,644 (daily ed. May 9, 2001) (Ernesto Decl.,
¶¶ 16–22, Exhs. NN-SS). Some of these letters

12

Plaintiffs have failed to adduce admissible
evidence in support of these alleged “facts,”
which are actually opinions. Plaintiffs provide
no source other than Ms. Ernesto’s conjectures
to support these statements, have not provided
any proof of “APA practices and protocols” as
they concern letters sent by APA’s lobbyists,
and Ms. Ernesto’s statements in her declaration
are contradicted by her statements at
deposition. At deposition, Ms. Ernesto
repeatedly stated

Plaintiffs’ Statement of Material Facts

Defendant Public Resource’s Response

are unsigned and are not printed on APA
letterhead. Therefore, in accordance with APA
practices, it is likely that the unsigned letters
(not printed on letterhead) were internal
discussion drafts that were never sent (Ernesto
Decl., ¶ 23).
Becker Decl., ¶ 17, Ex. 50 (Ernesto
Depo.) at 179:24–194:24. However, a
document produced by APA proves that at
least one such lobbying letter was sent: Exhibit
SS to Ms. Ernesto’s declaration is a 2002
memorandum APA produced titled
“Highlights of APA’s Involvement in
Educational Testing Provisions of the ‘No
Child Left Behind Act,’” that describes APA’s
lobbying work at the time. This memorandum
includes the full text of a letter that APA sent
on May 7, 2001 to U.S. Senators lobbying for
the mandating of the 1999 Standards through
an amendment by Senator Wellstone. At
deposition, Ms. Ernesto state
ecker Decl. ¶ 37,
Ex. 69; Becker Decl., ¶ 17, Ex. 50 (Ernesto
Depo.) at 189:06–190:11; SSMF ¶ 16.
27. Regarding the signed letters printed on APA
letterhead, they relate to Senator Wellstone’s
proposed legislation mandating that tests and
assessments administered by the states be of
high quality and used appropriately for the
benefit of test administrators and test takers.
These are goals that are consistent with APA
policy as then reflected in the 1999 Standards.
Even though Senator Wellstone’s amendments
sought, in part, to mandate States’ compliance
with
the
Standards,
the
Sponsoring
Organizations had not actively advocated for
this. In any event, Senator Wellstone’s
proposed amendment never became law
(Ernesto Decl., ¶ 24, Exh. TT).

13

Disputed. Plaintiff APA actively advocated
for a change in the law that would have
mandated the use of the 1999 Standards. See
the citations and explanation regarding
paragraph 26, above.

Plaintiffs’ Statement of Material Facts

Defendant Public Resource’s Response

28. APA’s search of its records did not disclose
any further communications with Congress
relating to the use of the Standards in legislation
or proposed legislation, and, to the best of
APA’s or the other Sponsoring Organizations’
knowledge, the Standards have not been
referenced in legislation since 2001 (Ernesto
Decl., ¶ 25). Moreover, neither AERA nor
NCME has ever communicated with Congress
for the purpose of encouraging reference to the
Standards in law (Levine Decl., ¶¶ 12-13; Wise
Decl., ¶ 18). None of the Plaintiffs has solicited
any government agency to incorporate the
Standards into the Code of Federal Regulations
or other rules of federal or state agencies
(Levine Decl., ¶ 13; Ernesto Decl., ¶ 26; Wise
Decl., ¶ 19).

Disputed. Plaintiffs produced an agreement
with New York consenting to the incorporation
by reference of the 1999 Standards into law.
SSMF ¶¶ 43-44.
Other documents produced by Plaintiffs
included
ee SSMF ¶ 16. All three plaintiff
organizations put on an event at the Russell
Senate Office Building on Capitol Hill about
the 2014 Standards. Dkt. 70-46 – 70-49,
Becker Decl. ¶ 19, Ex. 52 (Levine Dep.
185:21–189:12). Plaintiffs have no evidence
other than the conjecture of Dr. Levine and Dr.
Wise to support their statements that AERA
and NCME have never communicated with
Congress for the purpose of encouraging
reference to the Standards in law, and that
statement is directly refuted by the Capitol Hill
event described above.

29. Preparing and revising the Standards entails
intensive labor and considerable crossdisciplinary expertise. Each time the Standards
are revised, the Sponsoring Organizations select
and arrange for meetings of the Joint
Committee, composed of leading authorities in
psychological and educational assessments.
During these meetings, certain individual
standards are combined, pared down, or
augmented, while others are deleted altogether,
and some are created as whole new individual
standards.
30. The 1999 Standards took more than five
years to complete. It is the result of work put in
by the Joint Committee to generate a set of best
practices on educational and psychological
testing that are respected and relied upon by
leaders in their fields (Levine Decl., ¶ 11;
Camara Decl., ¶ 11; Wise Decl., ¶ 11). Draft
revisions of the 1985 Standards, which became
The Standards Are For Professional, Not

14

Disputed to the extent that Plaintiffs imply that
they performed any authorial function in the
creation of the 1999 Standards, which were
written by individual volunteers and members
of the public who were not employed by
Plaintiffs. SSMF ¶ 46. Also disputed to the
extent that Plaintiffs state the 1999 Standards
are for professional and not governmental use,
when they have explicitly targeted legislators

Plaintiffs’ Statement of Material Facts

Defendant Public Resource’s Response

Governmental, Use.

and policymakers in their advertisement of the
1999 Standards. SSMF ¶ 19.

31. Cover-to-cover, the 1999 Standards are 205
pages long. The vast bulk of those 205 pages is
explanatory and background material and
commentary, not the black-letter standards
themselves. For instance, the 1999 Standards
include:

Disputed to the extent that Plaintiffs’
subjective qualification of the contents of the
1999 Standards does not bear on the purposes,
reasoning, or actions of government agencies
that incorporated the 1999 Standards into law.



A front cover, a back cover, a cover
page, and a page showing copyright and
publisher information;



A table of contents showing how the
1999 Standards are organized;



A preface providing a short history of
the Standards and identifying the
people
and
organizations
that
participated in creating the 1999
Standards;



A five-plus page Introduction that gives
an overview of the testing process;
introduces the 1999 Standards; offers
some cautions to be exercised in using
the Standards; explains some updates
specific to the 1999 edition of the
Description of the 1999 Standards
(Hutter Decl. Exhibit 1).



Standards; clarifies how the 1999
Standards use the term “construct”; and
elaborates on how the 1999 Standards
are organized;



An index pointing users to the pages in
the 1999 Standards that refer to specific
terms; and



A glossary defining how certain terms
are specifically used in the 1999
Standards.

15

Plaintiffs’ Statement of Material Facts

Defendant Public Resource’s Response

32. The remaining 164 pages of the 1999
Standards consist of three substantive parts.
Part I is titled “Test Construction, Evaluation,
and Documentation.” Part II is titled “Fairness
in Testing.” Part III is titled “Testing
Applications.” Each of the three parts is broken
up into chapters. Part I, for example, contains
chapters on validity; reliability and errors of
measurement; test development and revision;
scaling, norming, and score comparability; test
administration, scoring, and reporting; and
supporting documentation for tests.
33. Each chapter in the 1999 Standards consists,
in turn, of two overall sections. First there is a
section of background and explanatory material
organized under various topical subheadings.
As the introduction to the 1999 Standards
makes clear, while the section of background
material opening each chapter may be helpful to
understanding the black-letter standards, “it
should not be interpreted as imposing additional
standards.”
34. The background material at the outset of
each chapter is followed in each chapter by a
section that includes the individual, black-letter
standards themselves. Each black-letter
standard is stated in bold-face type, one
paragraph long, and is written in a prescriptive
manner. Each individual standard is followed
by a Comment on which may help explain,
elaborate on or offer examples. The comments
are designed to assist in applying the blackletter standards, but they do not impose
additional requirements beyond the black-letter
standards. Even in the part of each chapter that
contains the standards (which is less than half
the chapter), more words tend to be devoted to
the commentary than to the black letter
standards themselves.

16

Plaintiffs’ Statement of Material Facts

Defendant Public Resource’s Response

35. The 1999 Standards, including all chapters,
contains 272 standards in total. The standards
take up less than 15% of 1999 Standards (as
determined by comparing the number of lines of
text devoted to standards in relation to the lines
in the text overall). See Hutter Decl. ¶ 6.
36. The National Technology Transfer and
Advancement Act of 1995 (“NTTAA”) requires
federal agencies to use privately-developed
standards to achieve federal objectives
whenever possible. Pub. L. No. 104-113 § 12,
110 Stat. 775, 782–83 (1996), codified at 15
U.S.C. § 272. Specifically it declares that “all
Federal agencies and departments shall use
technical standards that are developed or
adopted by voluntary consensus bodies, using
such technical standards as a means to carry out
policy objectives or activities.” Id.
37. One method through which federal agencies
avail themselves of privately-developed
standards is by incorporating the standards by
reference. The Code of Federal Regulations
currently contains more than 23,000
incorporations by reference, though not all of
the material incorporated are standards
developed by private standards-setting bodies.
See Bremer, Teaching Guide: Incorporation by
Reference, 2019 Administrative Law Review at
321; Bremer, Private Standards in Public Law,
63 Kansas L. Rev. 279.

17

Plaintiffs’ Statement of Material Facts

Defendant Public Resource’s Response

38. The Office of Management and Budget has
explained that incorporation by reference (i)
saves the government the cost of developing
standards on its own; (ii) provides incentives to
establish standards serving national needs; (iii)
promotes efficiency and economic competition
through harmonized standards; and (iv) furthers
the federal policy of relying on the private
sector to meet government needs for goods and
services. OMB Circular No. A-119, 63 Fed.
Reg. 8546 (Revised Feb. 10, 1998). See also
Final Revision of OMB Circular A-119, 81 Fed.
Reg. 4673 (Jan. 27, 2016).
39. OMB has recognized that it is essential to
not interfere with the ability of the standardsetting organizations to charge for the use of
their works: “If we required that all materials
IBR’d into the CFR be available for free, that
requirement would compromise the ability of
regulators to rely on voluntary consensus
standards, possibly requiring them to create
their own standards, which is contrary to the
NTTAA and the OMB Circular A-119.” See
Incorporation by Reference, Announcement of
Final Rule, Office of the Federal Register, 79
Fed. Reg. 66267, 66268 (Nov. 7, 2014),
available at https://federalregister.gov/a/201426445.

18

Plaintiffs’ Statement of Material Facts

Defendant Public Resource’s Response

40. The practice of incorporation by reference is Disputed to the extent that this is a legal
currently codified at 5 U.S.C. §552(a)(1), which interpretation and not a fact.
clarifies that no material incorporated by
reference can be deemed binding – in the sense
of adversely affecting any person – unless the
material is reasonably available to the class of
affected persons. Thus, any standard
incorporated by reference is self-limiting in its
application if not “reasonably available.”
Except to the extent that a person has
actual and timely notice of the terms
thereof, a person may not in any manner
be required to resort to, or be adversely
affected by, a matter required to be
published in the Federal Register and
not so published. For purposes of this
paragraph, matter reasonably available
to the class of persons affected thereby
is deemed published in the Federal
Register
when
incorporated
by
reference therein with the approval of
the Director of the Federal Register.
41. The Office of Federal Register in turn has
promulgated regulations governing the process
for agencies to follow in order to obtain
approval to incorporate materials by reference
in the CFR. 1 C.F.R. §51 (2014). Those
requirements interpret and implement the
statutory mandate that, in order for matters set
forth in the register to be binding, they must be
“reasonably available to the class of persons
affected.” 5 U.S.C. § 552(a)(1); 1 C.F.R.
§ 51.7(a)(3). They require that (i) a copy of the
incorporated material must be on file with the
Office of the Federal Register and (ii) that the
regulations incorporating such material must
state the ways those incorporated materials are
reasonably available to interested parties. 1
C.F.R. §§ 51.3, 51.5. There is no requirement
that such materials be available to the public at
no cost, or on the internet.

19

Disputed to the extent that Plaintiffs state that
placing a copy of an incorporated document on
file with the Office of the Federal Register and
stating the ways in which the incorporated
material is available to interested parties is
itself sufficient, and that “[t]here is no
requirement that such materials be available to
the public at no cost, or on the internet.” The
July 2018 edition of the Office of the Federal
Register’s “IBR Handbook” states that a caseby-case determination is necessary regarding
what kind of availability of the incorporated
material is required to meet the “reasonably
available” threshold, and that even providing
online read-only access may not be sufficient:
“Remember: Read-only access, on its own, may
not meet the reasonable availability requirement
at the final rule stage of rulemaking. If the
regulated parties aren’t able to use the material
(which may be different that simply reading or

Plaintiffs’ Statement of Material Facts

Defendant Public Resource’s Response
accessing it) throughout the life of the
rulemaking, this could lead to enforcement
issues.” Becker Decl., ¶ 11, Ex. 45 (IBR
Handbook) at p. 8 (emphasis in original).

42. Plaintiffs learned primarily during the
course of these proceedings that the Department
of Education has incorporated certain standards
set forth in the 1999 Standards in three sections
of the Code of Federal Regulations. As seen
below, none of these provisions apply to
conventional primary conduct or purport to
apply mandatory enforceable rules with
penalties attached. Rather, they address how
tests may qualify for purposes relevant to
Department of Education grant programs.

Disputed. Plaintiffs’ statements regarding
provisions of the Code of Federal Regulations
are legal arguments, not facts. Moreover, the
laws incorporating the 1999 Standards govern
the conduct of public officials who have a duty
to oversee industry actions and to enforce
compliance with the law, including compliance
with the 1999 Standards. The laws
incorporating the 1999 Standards further affect
whether low-income individuals are able to
obtain federal student aid necessary to attend
higher education programs. Schools and
administrators who incorrectly certify that they
comply with the regulations mandating
compliance with the 1999 Standards face
consequences. SSMF ¶ 33-43.

43. 34 C.F.R. §668.146(b)(6) is the only
regulation that was cited by PRO as the basis
for its incorporation-by-reference rationale for
posting the 1999 Standards. That regulation is
part of a subpart of regulations setting forth
provisions under which a student without a high
school diploma or its recognized equivalent
may become eligible to receive funds under
Title IV of the Higher Education Act. The
student may become eligible by obtaining a
diploma or its equivalent. Another way for a
student to become eligible for Title IV funds is
to pass a test that satisfies certain criteria
including “all standards for test construction
provided in the 1999 edition of the Standards
for Educational and Psychological Testing,
prepared by a joint committee of the American
Educational
Research
Association,
the
American Psychological Association, and the
National Council on Measurement in Education
incorporated by reference in this section.” The
regulation then goes on to note that the

Disputed. Public Resource identified
additional instances of incorporation of the
1999 standards in its filings in this litigation.
See, e.g., Dkt. 69-2 ¶ 33-36 (first Statement of
Material Facts). Also disputed because this
paragraph is composed of legal argument, not
facts, and disputed to the extent that Plaintiffs
imply through omission that anything less than
the full 1999 Standards document was
incorporated by reference at 34 C.F.R. §
668.146(b)(6).

20

Plaintiffs’ Statement of Material Facts

Defendant Public Resource’s Response

incorporation by reference was properly
approved, identifies where the material
referenced (the 1999 Standards) is on file, and
provides a phone number and website for the
National Archives and Records Administration
“[f]or information on the availability of this
material at NARA.” The regulation also
identifies the AERA website, noting that the
document may also be obtained from there.
44. The individual standards incorporated by
reference in 34 C.F.R. 668.146(b)(6) are set
forth in Part I of the 1999 Standards, entitled
“Test
Construction,
Evaluation
and
Documentation.” Part I actually consists of six
chapters, covering Validity, Reliability and
Errors of Measurement, Test Development and
Revision,
Scales,
Norms
and
Score
Comparability, Test Administration, Scoring,
and Reporting, and Supporting Documentation
for Tests. Even assuming that all six chapters of
Part I are being referenced by the regulation, the
actual bold-face “standards” set forth in these
chapters are very limited.
45. For example, the chapter within Part I on
“validity” runs from pp. 9-24, begins with
background and explanatory material, and
contains standards only on pages 17-24
(Standards 1.1-1.24), including comments on
each standard. The comments on each standard
are typically as long as, or longer than the
standards themselves.
46. The chapter within Part I on Reliability and
Errors is eleven pages long and begins with 6
pages of background and discussion of various
topics and considerations. Standards (2.1-2.20)
are set forth on 6 pages, and the comments to
the standards are again longer than the
referenced standards themselves.
47. The chapter on Test Development and
Revision runs from pages 37-48 and the first 6

21

Disputed to the extent that Plaintiffs state that
anything less than the full 1999 Standards
document was incorporated by reference at 34
C.F.R. § 668.146(b)(6), and disputed to the
extent that this paragraph is composed of legal
argument, not facts.

Plaintiffs’ Statement of Material Facts

Defendant Public Resource’s Response

pages describe background and discussions of
various related topics and considerations. The
actual standards are set forth within pages 43-48
(standards 3.1- 3.27), and again, most of that
appears to be commentary, rather than the
individual standards themselves (though some
of the standards have no immediately following
“comment” and some comments are as brief as
the standard).
48. The next three chapters in Part 1 follow a
similar format.
49. 34 C.F.R. §668.148(a)(1)(iv) is also a part
of the subpart of the regulations setting forth
provisions under which a student without a high
school diploma or its recognized equivalent
may become eligible to receive funds under
Title IV. This provision covers tests developed
for non-native speakers of English enrolled in a
program taught in their native language. Such
tests must, according to this provision, be
“[d]eveloped in accordance with guidelines
provided in the “’Testing Individuals of Diverse
Linguistic Backgrounds’ section of the” 1999
Standards.
50. That chapter of the 1999 Standards runs
from pages 91 to 100, with the first 6 pages
covering background and explanatory topics.
The actual standards are found at pages 97 to
100, and again, the actual standards themselves
appear to occupy less than half of those pages,
with comments taking up more space.

22

Disputed to the extent that Plaintiffs imply that
anything less than the full 1999 Standards
document was incorporated by reference at 34
C.F.R. § 668.148(a)(1)(iv), and disputed to the
extent that this paragraph is composed of legal
argument, not facts.

Plaintiffs’ Statement of Material Facts

Defendant Public Resource’s Response

51. 34 C.F.R. §462.13(c)(1) and (f)(1) are part
of the regulations establishing a process for
reviewing the suitability of tests, as submitted
by test publishers, for use in the National
Reporting System for Adult Education, which is
an outcome-based accountability system for
evaluating state-administered, federally-funded
adult education programs. Among the criteria
used to determine whether a test is suitable for
use in the that system is that “[t]he test must
meet all applicable and feasible standards for
test construction and validity provided in the
1999 edition of the Standards[.]” But a test or
publisher that seeks to qualify under this
program is specifically offered the opportunity
to demonstrate why a particular standard is not
feasible or applicable.

Disputed to the extent that Plaintiffs imply that
anything less than the full 1999 Standards
document was incorporated by reference at 34
C.F.R. § 462.13(c)(1) and ((f)(1), and disputed
to the extent that this paragraph is composed of
legal argument, not facts.

52. This
regulation,
like
34
C.F.R.
§668.146(b)(6), references Part I of the 1999
Standards. The regulation itself contains similar
information and directions as to how and where
to find the incorporated standards.

Disputed to the extent that Plaintiffs imply that
anything less than the full 1999 Standards
document was incorporated by reference at 34
C.F.R. § 462.13(c)(1) and ((f)(1), and disputed
to the extent that this paragraph is composed of
legal argument, not facts.

53. In addition, subsection (f)(1) provides that
“For a test that has been modified for
individuals with disabilities, the Error!
Hyperlink reference not valid [sic], must,”
among other things, “[p]rovide documentation
that it followed the guidelines provided in the
Testing Individuals With Disabilities section of
the 1999 edition of the Standards for
Educational and Psychological Testing.” That
is reference to a different chapter of the 1999
Standards, contained in Part 2. That chapter
comprises 8 pages, the first 5 of which provide
background and explanatory material, which is
followed by 3 pages containing the black letter
standards (10.1-10.12), many of which are
followed a Comment.

Disputed to the extent that Plaintiffs imply that
anything less than the full 1999 Standards
document was incorporated by reference at 34
C.F.R. § 462.13(c)(1) and ((f)(1), and disputed
to the extent that this paragraph is composed of
legal argument, not facts. Also disputed to the
extent that the quoted text contains errors that
are not in the original text (“Error! Hyperlink
reference not valid”).

54. Less than 6 per cent of the 1999 Standards Disputed. The full 1999 Standards document
is occupied by the standards incorporated by was incorporated by reference. See generally

23

Plaintiffs’ Statement of Material Facts

Defendant Public Resource’s Response

reference in the three regulation (as determined SSMF ¶ 33-43.
by comparing the number of lines of text in the
referenced standards with the number of lines in
the text overall). Hutter Decl. ¶ 6.
55. Plaintiffs are not aware that DOE has ever
had occasion to enforce the provisions of its
regulations referencing the 1999 Standards.
56. In addition, Plaintiffs are not specifically
aware of tests developed specifically to meet
these DOE regulations. Rather, tests meeting
these criteria may have been developed entirely
independent of the regulations because high
quality tests generally would, as a matter of
basic professionalism, be developed in a
manner consistent with the Standards, but
without regard to the fact that some of the
standards are cited in DOE regulations. The
exception could, of course, be for 34 C.F.R.
§462.13 (f)(1), which requires the test publisher
to provide documentation that it affirmatively
meets the requirements of the referenced
portion of the 1999 Standards. But plaintiffs are
not in possession of any information regarding
how or whether that provision has been applied.
57. Copies of the 1999 Standards are available, Disputed. The 1999 Standards are not
as identified in the CFR, in the reading room available for ordering on-line. See paragraph 9
and from AERA, either by ordering on-line or above.
over the telephone. In addition, existing copies
of the 1999 Standards are available and in
circulation throughout academia (including
schools of education and departments of
psychology), are in the possession of
organizations involved in testing, can be
accessed in public and university libraries, and
would expectably be accessible and referenced
in connection with the development and use of
tests irrespective of whether at some point the
test would be in any way subject to the DOE
regulations.

24

Plaintiffs’ Statement of Material Facts

Defendant Public Resource’s Response

58. Defendant Public.Resource.Org (PRO) is a
California non-profit corporation founded in
2007 by Mr. Carl Malamud, with the explicit
aim of making government information more
accessible, with particular emphasis on the law
(Hudis Decl., ¶ 2, Exh. A, pp. 77, 93–94, 163–
164). The identified purpose and objective of
PRO is to create and maintain so-called
informational “public works projects for the
Internet” (Hudis Decl., ¶ 2, Exh. A, pp. 94–95,
105–09, ¶ 3, Exh. B, Section II.B., ¶ 4, Exh. C.
Section 2.1).
59. PRO creates digital copies of standards
referenced or incorporated in regulations,
without the permission of the copyright owners.
It does so as a matter of principle. That is, PRO
does not purport to identify any specific need or
reason for such copying. Rather, PRO simply
believes, and acts on the belief, that all such
material should copied and made available for
copying on the internet. Its mission includes the
broad and indiscriminate objective “to make the
law and other government materials more
widely available.” 896 F.3d at 444 (quoting
Malamud’s Declaration in this case).

Disputed to the extent that Plaintiffs assert the
legal conclusions there are private parties that
“own” documents that are edicts of
government. Disputed to the extent that
Plaintiffs assert Public Resource “does not
purport to identify any specific need or reason
for such copying”. Disputed to the extent
Plaintiffs characterize Public Resource’s
mission as “broad and indiscriminate.” See
generally Malamud Decl.

60. In March 2012, PRO began copying
standards incorporated by reference into the
Code of Federal Regulations, and facilitating
copying by others. In May 2012, PRO began
the process of posting copies these standards to
its website. “Between 2012 and 2014, PRO
uploaded hundreds of technical standards,
which, collectively, were downloaded tens of
thousands of times.” Id.

Disputed to the extent that Plaintiffs assert
technical standards have been “downloaded”
tens of thousands of times. It is impossible on
the facts presented to know whether a visit
logged to a server is a “download” or simply
an access to the server, and it is likewise
impossible on the facts presented to determine
whether the visit was by a person or by an
automated computer function, such as a web
crawler or bot. SSMF ¶ 63.

61. Specifically, on May 17, 2012, PRO bought
a used hard copy of the 1999 Standards from an
Amazon re-seller (Hudis Decl., ¶ 2, Exh. A, pp.
232-240, ¶ 21, Exh. T, Int. Ans. 1, ¶¶ 22-23,
Exh. U). Upon receipt of the purchased paper
copy, Malamud disassembled the book,

Disputed to the extent that Plaintiffs assert
Public Resource created “a false semblance of
governmental imprimatur to the unauthorized
copying and online posting of the 1999
Standards” or that any authorization was
required for Public Resource’s activities. See

25

Plaintiffs’ Statement of Material Facts

Defendant Public Resource’s Response

removed the spine, trimmed the pages to give
them an even border, scanned the pages to
create a PDF, and named the PDF file
“aera.standards.1999.pdf.”
Malamud
then
appended a cover sheet, a self-made
“Certificate,” to the front of the PDF file giving
a false semblance of governmental imprimatur
to the unauthorized copying and online posting
of the 1999 Standards (Hudis Decl., ¶ 2, Exh.
A, pp. 257–59, 261–64, ¶ 21, Exh. T, Int. Ans.
3–4, ¶ 26. Malamud did nothing else to modify
or transform the PDF file or the Standards. He
simply copied and posted the 1999 Standards,
in its entirety, to his company’s website. He did
not provide for wordsearching, online
identification, or text-to-speech utilization for
the blind and visually impaired (Hudis Decl., ¶
27, Exh. Z, pp. 30, 122, 200–01, 206, 271–72,
315–16). PRO published the infringing digital
copy of the 1999 Standards on a website titled
https://law.resource.org.

62. The Internet Archive is a nonprofit
organization whose mission is to build and
maintain a digital library of the Internet. The
Internet Archive builds this internet library —
which it makes available for public use — by
scanning, digitally capturing, and saving
electronically scanned and captured third-party
websites, and by receiving submissions from
third parties who have user accounts enabling
them to upload content (Hudis Decl., ¶ 29, Exh.
BB, pp. 31–41). Malamud has such user
account access (Hudis Decl., ¶ 29, Exh. BB, pp.
51–56), and he uploaded the entirety of the
1999 Standards to the Internet Archive’s
website on May 26–27, 2012 (Hudis Decl., ¶
29, Exh. BB, pp. 59-112, ¶ 30, Exh. CC (¶¶ 3–
18 therein), ¶ 32, Exh. EE, ¶ 33). PRO posted
Plaintiffs’ 1999 Standards to its website and the
Internet Archive website without the permission
or authorization of any of the Sponsoring

26

generally Malamud Decl.
Disputed to the extent that Plaintiffs assert
“Malamud did nothing else to modify or
transform the PDF file or the Standards. He
simply copied and posted the 1999 Standards,
in its entirety, to his company’s website. He
did not provide for wordsearching, online
identification, or text-to-speech utilization for
the blind and visually impaired.” Public
Resource posted the 1999 Standards on the
Internet Archive website, which automatically
processed the document with optical character
recognition software to make the text
accessible to people with visual impairments,
or for word-searching and online
identification. Becker Decl. ¶ 30, Ex. 60
(Fruchterman expert report) at 11–12. Public
Resource also intended to rekey the 1999
Standards into HTML but was not able to
complete that project due to litigation. Becker
Decl. ¶ 32, Ex. 64 (2016 Malamud Decl.) ¶¶
16-17, 25-28;

Plaintiffs’ Statement of Material Facts

Defendant Public Resource’s Response

Organizations (Hudis Decl., ¶ 35, Exh. HH,
Admission Nos. 4–5; Levine Decl., ¶ 29;
Ernesto Decl., ¶ 35; Wise Decl., ¶ 26).
63. The copy of the Sponsoring Organizations’
1999 Standards that Malamud published to the
Internet Archive at https://archive.org/details/
gov.xlaw.aera.standards.1999 was in the same
format, using the same cover sheet or
“Certificate” employed by PRO in the posting
of the Sponsoring Organizations’ 1999
Standards to Defendant’s own website. All of
the surrounding text associated with the posting
to the Internet Archive website was inserted by
Malamud — including the insertion of
“Creative Commons License: CC0 1.0
Universal,” indicating that no rights are being
asserted over the item (Hudis Decl., ¶ 2, Exh.
A, pp. 275–84, ¶ 29, Exh. BB, pp. 57–63, ¶ 30,
Exh. CC (¶ 2 therein), ¶ 34, Exh. GG): [sic]

27

Plaintiffs’ Statement of Material Facts

Defendant Public Resource’s Response

64. Although based on incomplete reporting
(Hudis Decl., ¶ 2, Exh. A, p p. 272–74, 328–
36), during the near two-year period that the
Sponsoring Organizations’ 1999 Standards
were
first
posted
on
PRO’s
https://law.resource.org website, they were
accessed at least 4,164 times (Hudis Decl., ¶ 21,
Exh. T, Int. Ans. 2 and Amended Ans. 5
(labeled 6)). During that same period, the
Sponsoring Organizations’ 1999 Standards
were accessed on the Internet Archive
https://archive.org website 1,290 times (Hudis
Decl., ¶ 29, Exh. BB, pp. 124-132, ¶ 37,
Exh. II).

Disputed. Plaintiffs’ claims as to the number
of accesses of the 1999 Standards on the
Public Resource website is incorrect, as their
calculation appears to include access figures
for a stub page that replaced the 1999
Standards on the Public Resource website in
June 2014, when Public Resource had taken
down the 1999 Standards pending the
resolution of this litigation. See Dkt. 60-23,
Hudis Decl. Exh. T. Plaintiffs also fail to
explain that the number of “accesses” means
the number of access requests from a computer
to the Public Resource server, and does not
necessarily mean accesses by human beings,
but could instead be accesses by webcrawlers,
bots, or other automated programs. Dkt. 60-23,
Hudis Decl. Exh. T, at 7–8. Similarly disputed
to the extent that Plaintiffs imply these
“access” figures reflect actual page views or
downloads, as opposed to automated
processes, and that the access counts can in
any way be equated with lost sales. Also
disputed to the extent that the time period
pertinent to the access figures that Plaintiffs
cite as Exh. II is not established in the
documents Plaintiffs cite. See also SSMF
¶ 63.

65. The Internet Archive’s website is open to
the public and does not restrict an Internet
user’s ability to download or print the
Sponsoring Organizations’ 1999 Standards.
PRO also placed no such restrictions on its
website (Hudis Decl., ¶ 2, Exh. A, pp. 347–48).
There were no sign-up procedures to enter
PRO’s https://law.resource.org website, nor was
there any Digital Rights Management (or
“DRM”) plan to protect against, or identify,
further copying of the files accessed from
PRO’s site (Hudis Decl., ¶ 27, Exh. Z, pp. 324–
27, 167–73).
66. There is no way for Plaintiffs to calculate Undisputed that there is no evidence of any
the number of university/college professors, lost sales of the 1999 Standards as a

28

Plaintiffs’ Statement of Material Facts

Defendant Public Resource’s Response

students, testing companies and others who
would have purchased Plaintiffs’ Standards but
for their wholesale posting on PRO’s
https://law.resource.org website and the Internet
Archive http://archive.org website (Levine
Decl., ¶ 30; Geisinger Decl., ¶ 24).

consequence of Defendant’s postings of the
1999 Standards. Otherwise disputed. Plaintiffs
did not attempt to investigate or obtain
discovery on even a single instance in which
someone who would have purchased the 1999
Standards refrained from doing so because he
or she instead accessed or obtained a version
from the Public Resource or Internet Archive
website. Plaintiffs have not adduced
admissible evidence to support this contention.
This is opinion, not a fact. Dr. Levine is not
qualified as an expert, and Dr. Geisinger is not
qualified as an expert on the subject of
economic substitution. See Public Resource’s
Motion to Strike the Declaration of Dr.
Geisinger, Dkt. 67. Plaintiffs have the burden
of proof for establishing harm, and should not
invite the Court to speculate as to downstream
copying or distribution of which Plaintiffs
have failed to find any evidence. Notably, after
Public Resource took down the 1999 Standards
pending the outcome of this litigation, Mr.
Fruchterman looked for an electronic version
of the 1999 Standards online and could not
find one. Becker Decl. ¶ 30, Ex. 60
(Fruchterman expert report) at 5–6.

67. In late 2013 and early 2014, the Sponsoring
Organizations became aware that the 1999
Standards had been posted on the Internet
without their authorization, and that students
were obtaining free copies from the posting
source. Upon further investigation, they
discovered that PRO was the source of the
online posting (Camara Decl., ¶ 21, Exh.
MMM; Wise Decl., ¶¶ 27-28, Exh. LLL).

Disputed. Plaintiffs have not adduced
admissible evidence to support their contention
that students obtained free copies of the 1999
Standards from Public Resource. Plaintiffs
rely on a single hearsay-within-hearsay
statement, which Plaintiffs also overgeneralize
beyond the contents of that hearsay-withinhearsay statement.

68. In December 2013, Plaintiff AERA
requested in writing that PRO remove the 1999
Standards from its online postings (Levine
Decl., ¶ 31, Exh. UUU). Defendant refused
(Hudis Decl., ¶ 2, Exh. A, pp. 310–19, ¶ 38,
Exh. JJ, ¶ 39, Exh. KK). Once this lawsuit was
filed and the Sponsoring Organizations

29

Plaintiffs’ Statement of Material Facts

Defendant Public Resource’s Response

threatened to file a motion for a preliminary
injunction, PRO agreed in June 2014 to remove
its postings of the 1999 Standards from its
https://law.resource.org website and from
Internet Archive’s https://archive.org website,
pending a resolution of this litigation on the
merits. PRO’s undertaking included the promise
not to post any revision of the 1999 Standards
(i.e., the 2014 Standards) pending the outcome
of this litigation on the merits (Hudis Decl., ¶ 2,
Exh. A, pp. 322-28, ¶ 40, Exh. LL, ¶ 41, Exh.
MM).
69. Notwithstanding that undertaking, it is
currently posted on the Internet Archive
website, stating that it was uploaded on May 26,
2012. That posting includes the full volume of
the 1999 Standards, verbatim, covered with the
self-created “Certificate” stating “By Authority
of the United States of America … Legally
Binding Document,” and stating that the
document is posted by Public.Resource.Org.
Inc. The website cites a single section of the
CFR, 34 C.F.R. § 668.148(a)(2)(iv). The
website posting provides a variety of means by
which the 1999 Standards can be downloaded.
See Hutter Decl. Exh 3. The website reflected
that as of September 11, 2019, it was viewed
1445 times.
See https://archive.org/details/
gov.law.aera.standards.1999/page/n1.

Disputed. Plaintiffs fail to explain that the
number of “accesses” means the number of
access requests from a computer to the Internet
Archive server, and does not necessarily mean
accesses by human beings, but could instead
be accesses by webcrawlers, bots, or other
automated programs. Similarly disputed to the
extent that Plaintiffs imply these “access”
figures reflect actual page views or downloads,
as opposed to automated processes, and that
the access counts can in any way be equated
with lost sales. SSMF ¶ 63.

70. In 2017, this Court granted Plaintiffs’ Disputed to the extent this paragraph includes
motion for summary judgment on their legal argument instead of facts.
copyright claim. The Court held that Plaintiffs
own the copyright to the 1999 Standards; that
Plaintiffs’ copyright is valid; that nothing about
the 1999 Standards’ incorporation into federal
law negated the copyright; that PRO’s posting
on the internet was infringing; and that PRO’s
wholesale copying and posting of the entirety of
the 1999 Standards to the internet, from where
it could be downloaded, was not fair use.
Regarding the fair use issue, this Court’s
decision recognized that PRO had the burden on

30

Plaintiffs’ Statement of Material Facts

Defendant Public Resource’s Response

that issue, and the Court addressed the specific
arguments that PRO had made – arguments that
PRO advanced across the board, without
addressing the differences between the many
and varied standards at issue in the ASTM case,
or the differences between how those standards
were referenced in federal regulations. Based on
those holdings, the Court enjoined PRO from
continuing to post the 1999 Standards.
71. In 2018, the Court of Appeals for the Disputed to the extent this paragraph includes
District of Columbia Circuit vacated this legal argument instead of facts.
Court’s summary judgment and injunction order
and remanded the case for further proceedings.
The Court of Appeals’ remand order was
limited to a single issue: fair use. The Court of
Appeals noted that PRO had offered an
“undifferentiated” theory of why its publication
of any standards incorporated by reference in
federal regulations qualified as fair use (or
should be denied copyright protection at all).
This Court had therefore addressed those
arguments of PRO in the undifferentiated terms
under which they were presented.
72. But the Court of Appeals believed that Disputed to the extent this paragraph includes
approach “failed to account for the variation legal argument instead of facts.
among the standards at issue and … failed to
consider each fair use claim ‘on its own facts.’”
Thus, it noted the many and varied standards
and incorporations by reference at issue (at least
in the ASTM case), and directed that:
On remand, the district court will need
to develop a fuller record regarding the
nature of each of the standards at issue,
the way in which they are incorporated,
and the manner and extent to which they
were copied by PRO in order to resolve
this “mixed question of law and fact.”
73. In light of that direction, this Court allowed
PRO an extended period to take additional
discovery, which it did. Plaintiffs did not take

31

Plaintiffs’ Statement of Material Facts

Defendant Public Resource’s Response

any additional discovery.
Dated: November 8, 2019

Respectfully submitted,
/s/ Andrew P. Bridges
Andrew P. Bridges (USDC-DC AR0002)
[e-mail]
Matthew B. Becker (admitted pro hac vice)
[e-mail]
Armen N. Nercessian (pending pro hac vice)
[e-mail]
Shannon E. Turner (pending pro hac vice)
[e-mail]
FENWICK & WEST LLP
801 California Street
Mountain View, CA 94041
Telephone: [phone]
Facsimile: [phone]
Corynne McSherry (admitted pro hac vice)
[e-mail]
Mitchell L. Stoltz (D.C. Bar No. 978149)
[e-mail]
ELECTRONIC FRONTIER FOUNDATION
815 Eddy Street
San Francisco, CA 94109
Telephone: [phone]
Facsimile: [phone]
David Halperin (D.C. Bar No. 426078)
[e-mail]
1530 P Street NW
CSRL 2nd Floor
Washington, DC 20005
Telephone: [phone]
Attorneys for Defendant-Counterclaimant
Public.Resource.Org, Inc.

B9620/00404/FW/11106789.3

32