AMERICAN EDUCATIONAL RESEARCH ASSOCIATION, INC. et al v. PUBLIC.RESOURCE.ORG, INC.

Filing 89

REPLY to opposition to motion re #69 MOTION for Summary Judgment and Opposition to Plaintiffs' Motion for Summary Judgment and Permanent Injunction, #60 MOTION for Summary Judgment Filed MOTION for Permanent Injunction (Plaintiffs' Reply in Further Support of Plaintiffs' Motion for Summary Judgment and Permanent Injunction and Opposition to Defendant's Motion for Summary Judgment) filed by AMERICAN EDUCATIONAL RESEARCH ASSOCIATION, INC., AMERICAN PSYCHOLOGICAL ASSOCIATION, INC., NATIONAL COUNCIL ON MEASUREMENT IN EDUCATION, INC.. (Attachments: #1 Plaintiffs' Statement of Material Facts in Opposition to Defendant's Motion for Summary Judgment, #2 Plaintiffs' Reply to Defendants' Statement of Disputed Facts, #3 Plaintiffs' Response to Defendant's Objections to Plaintiffs' Evidence, #4 Declaration of Nikia L. Gray, #5 Exhibit VVV, #6 Exhibit WWW, #7 Exhibit XXX, #8 Exhibit YYY, #9 Exhibit ZZZ, #10 Exhibit AAAA, #11 Exhibit BBBB, #12 Exhibit CCCC, #13 Exhibit DDDD, #14 Exhibit EEEE, #15 Exhibit FFFF, #16 Exhibit GGGG, #17 Exhibit HHHH, #18 Exhibit IIII, #19 Exhibit JJJJ, #20 Exhibit KKKK, #21 Exhibit LLLL, #22 Exhibit MMMM, #23 Exhibit NNNN, #24 Exhibit OOOO, #25 Exhibit PPPP, #26 Exhibit QQQQ, #27 Exhibit RRRR, #28 Exhibit SSSS, #29 Exhibit TTTT, #30 Exhibit UUUU, #31 Exhibit VVVV, #32 Exhibit WWWW, #33 Exhibit XXXX, #34 Exhibit YYYY, #35 Exhibit ZZZZ, #36 Exhibit AAAAA, #37 Exhibit BBBBB, #38 Exhibit CCCCC, #39 Exhibit DDDDD, #40 Exhibit EEEEE, #41 Exhibit FFFFF, #42 Exhibit GGGGG, #43 Exhibit HHHHH, #44 Exhibit IIIII, #45 Exhibit JJJJJ, #46 Exhibit KKKKK, #47 Exhibit LLLLL, #48 Exhibit MMMMM, #49 Exhibit NNNNN, #50 Exhibit OOOOO, #51 Exhibit PPPPP, #52 Exhibit QQQQQ, #53 Exhibit RRRRR, #54 Exhibit SSSSS, #55 Exhibit TTTTT, #56 Exhibit UUUUU, #57 Exhibit VVVVV, #58 Exhibit WWWWW, #59 Exhibit XXXXX, #60 Exhibit YYYYY, #61 Exhibit ZZZZZ, #62 Exhibit AAAAAA, #63 Declaration of Wayne Camara, #64 Plaintiffs' Objections to Defendant's Evidence, #65 Certificate of Service)(Hudis, Jonathan)

UNITED STATES DISTRICT COURT
FOR THE DISTRICT OF COLUMBIA
AMERICAN EDUCATIONAL RESEARCH
ASSOCIATION, INC., AMERICAN
PSYCHOLOGICAL ASSOCIATION, INC.,
and NATIONAL COUNCIL ON
MEASUREMENT IN EDUCATION, INC.,

Plaintiffs/Counterclaim-Defendants,
v.
PUBLIC.RESOURCE.ORG, INC.,
Defendant/Counterclaim-Plaintiff.

)
)
)
)
)
)
)
)
)
)
)
)
)
)

Civil Action No. 1:14-cv-00857-TSC-DAR
REPLY DECLARATION OF WAYNE
CAMARA IN FURTHER SUPPORT
OF PLAINTIFFS’ MOTION FOR
SUMMARY JUDGMENT, AND IN
OPPOSITION TO DEFENDANT’S
CROSS-MOTION FOR SUMMARY
JUDGMENT

I, WAYNE J. CAMARA, declare:
1.

I submit this Reply Declaration in further support of Plaintiffs’ Motion for

Summary Judgment, and in Opposition to Public.Resource.Org, Inc.’s (“Defendant” or “Public
Resource”) Cross-Motion for Summary Judgment. Unless otherwise noted, I have knowledge of
all facts set forth in this Declaration and I would, and could, testify competently thereto if called
upon to do so.
2.

I am currently the Senior Vice President, Research at ACT, Inc. ACT produces

and publishes the ACT® college readiness assessment — a college admissions and placement test
taken by millions of high school graduates every year.

ACT also offers comprehensive

assessment, research, information, and program management services to support education and
workforce development. As the Senior Vice President of Research, I am responsible for all
research and evidence related to the design, development, use, and validation of our assessments
and programs. In my position, I serve on the Senior Leadership Team and manage over 110
researchers.
-1-

3.

Prior to working at ACT, I worked at The College Board, where I held the

positions of Vice President, Research and Development (July, 2000 – September, 2013),
Executive Director, Office of Research and Development (March, 1997 – June, 2000), and
Research Scientist (September, 1994 – February, 1997).
4.

Before working at The College Board, I worked for the American Psychological

Association, Inc. (“APA”), in the positions of Assistant Executive Director for Scientific Affairs
and Executive Director of Science (1992-1994), Director, Scientific Affairs (February, 1989 –
August, 1992), and Testing and Assessment Officer (November, 1987 – January, 1989). During
my employment at APA, I served as the Project Director for the revision of the 1985 version of
the Standards for Educational and Psychological Testing, the new product of which was
published in 1999 (the “1999 Standards”).

In 1997, I was elected to APA’s Council of

Representatives, and I served on the Council from 1997-2003. In April, 2012, I was elected to
the Council of the American Educational Research Association, Inc. (“AERA”), serving from
April, 2012 to April, 2015 as Vice President for Division D. I also was elected to the Board of
Directors of National Council On Measurement In Education, Inc. (“NCME”), serving on the
Board from 2002-2005 and 2009-2012, and served as NCME’s President from 2010-2011.
Additionally, I served on the Management Committee for the Standards from 2005-2015.
5.

I have written extensively on the Standards for Educational and Psychological

Testing, as well as other professional and technical guidelines which relate to educational and
industrial testing and assessment, including journal articles, book chapters, and paper
presentations at national conferences.
6.

I was asked to rephrase several of the standards recited in the 1999 Standards,

without changing their meaning.

-2-

7.

Standard 3.3, as recited in the 1999 Standards, states:

Those responsible for test development should include relevant subgroups in
validity, reliability/precision, and other preliminary studies used when
constructing the test.
8.

One of the many ways in which Standard 3.3 could be rephrased without

changing its meaning is as follows:
Studies collecting evidence for the interpretation and use of test scores,
quantifying the inconsistency in examinee performance, and of other topics
should be conducted during test construction by individuals and organizations
who mandate, sponsor, prepare and design, and market tests so that study results
will inform the discussion of the comparability of subgroup scores.
9.

Standard 4.4, as recited in the 1999 Standards, states:

If test developers prepare different versions of a test with some change to the test
specifications, they should document the content and psychometric specifications
of each version. The documentation should describe the impact of differences
among versions on the validity of score interpretations for intended uses and on
the precision and comparability of scores.
10.

One of the many ways in which Standard 4.4 could be rephrased without

changing its meaning is as follows:
Changes or augmentations to assessments which impact content, constructs, or
statistical properties of a test should be documented and made available to test
users. Documentation should address any effect on the overall reliability of the
test, the accuracy of scores, or the inferences which can be made from scores, as
well as the extent that scores across different versions are comparable.
11.

Standard 5.2, as recited in the 1999 Standards, states:

The procedures for constructing scales used for reporting scores and the rationale
for these procedures should be described clearly.
12.

One of the many ways in which Standard 5.2 could be rephrased without

changing its meaning is as follows:
Testing programs that use derived scale scores to enhance interpretation of
assessment results must report the justification and procedures used to create the
derived scores.
-3-

13.

Standard 5.3, as recited in the 1999 Standards, states:

If there is sound reason to believe that specific misinterpretation of a score scale
are likely, test users should be explicitly cautioned.
14.

One of the many ways in which Standard 5.3 could be rephrased without

changing its meaning is as follows:
When inaccurate interpretations of reported scores by users can be anticipated,
Test Publishers have the responsibility to articulate both the correct and the
possible incorrect interpretations for users.
15.

I believe the forgoing exercise could be done with any of the standards recited in

the 1999 Standards by a person who is sufficiently knowledgeable in psychometrics and/or
educational testing as well as the meaning and import of the standards contained within the 1999
Standards.
I DECLARE, under the penalty of perjury, that the foregoing is true and correct.

Dated: February 12, 2016
Wayne J. Camara

-4-