AMERICAN EDUCATIONAL RESEARCH ASSOCIATION, INC. et al v. PUBLIC.RESOURCE.ORG, INC.
Filing
70
LARGE ADDITIONAL ATTACHMENT(S) Index of Consolidated Exhibits In Support of Public.Resource.Org's Motion for Summary Judgment and Opposition to Plaintiffs' Motion for Summary Judgment and Permanent Injunction by PUBLIC.RESOURCE.ORG, INC. #69 MOTION for Summary Judgment and Opposition to Plaintiffs' Motion for Summary Judgment and Permanent Injunction filed by PUBLIC.RESOURCE.ORG, INC.. (Attachments: #1 Exhibit 1, #2 Exhibit 2 [Sealed], #3 Exhibit 3 [Sealed], #4 Exhibit 4 [Sealed], #5 Exhibit 5 [Sealed], #6 Exhibit 6 [Sealed], #7 Exhibit 7, #8 Exhibit 8 [Sealed], #9 Exhibit 9, #10 Exhibit 10, #11 Exhibit 11 [Sealed], #12 Exhibit 12 [Sealed], #13 Exhibit 13 [Sealed], #14 Exhibit 14 [Sealed], #15 Exhibit 15 [Sealed], #16 Exhibit 17 [Sealed], #17 Exhibit 18 [Sealed], #18 Exhibit 19 [Sealed], #19 Exhibit 20 [Sealed], #20 Exhibit 21 [Sealed], #21 Exhibit 22 [Sealed], #22 Exhibit 23 [Sealed], #23 Exhibit 24 [Sealed], #24 Exhibit 25 [Sealed], #25 Exhibit 26 [Sealed], #26 Exhibit 27 [Sealed], #27 Exhibit 28 [Sealed], #28 Exhibit 29 [Sealed], #29 Exhibit 30 [Sealed], #30 Exhibit 31, #31 Exhibit 32 [Sealed], #32 Exhibit 33 [Sealed], #33 Exhibit 34 [Sealed], #34 Exhibit 35, #35 Exhibit 36, #36 Exhibit 37, #37 Exhibit 38 [Sealed], #38 Exhibit 39, #39 Exhibit 40, #40 Exhibit 41 [Sealed], #41 Exhibit 42 [Sealed], #42 Exhibit 43 [Sealed], #43 Exhibit 44, #44 Exhibit 45, #45 Exhibit 46, #46 Exhibit 47, #47 Exhibit 48, #48 Exhibit 49, #49 Exhibit 50 [Sealed], #50 Exhibit 51, #51 Exhibit 52, #52 Exhibit 53, #53 Exhibit 54, #54 Exhibit 55, #55 Exhibit 56, #56 Exhibit 57, #57 Exhibit 58, #58 Exhibit 59, #59 Exhibit 60, #60 Exhibit 61, #61 Exhibit 62, #62 Exhibit 63, #63 Exhibit 64 [Sealed], #64 Exhibit 65, #65 Exhibit 66, #66 Exhibit 67, #67 Exhibit 68, #68 Exhibit 69, #69 Exhibit 70, #70 Exhibit 71, #71 Exhibit 72, #72 Exhibit 73, #73 Exhibit 74)(Bridges, Andrew)
EXHIBIT 52
OCR Issues Draft Guide on Disparate Impact in Educational Testing TIP October 1999
Home | A
Search by Keyword(s)
MEMBERSHIP
MEETINGS
SERVICES
PUBLICATIONS
JOBS
RESOURCES
FOUNDATION
PARTNE
OCR Issues Draft Guide on
Disparate Impact in Educational Testing
Wayne Camara
The College Board
What is I-O?
Industrial-organizational (I-O)
psychology is the scientific study of
the workplace. Rigor and methods of
psychology are applied to issues of
critical relevance to business,
including talent management,
coaching, assessment, selection,
training, organizational development,
performance, and work-life balance.
Find an I-O Job
I-O Graduate Programs
SIOP Social Media
The Guide may have limited direct impact on I-O psychologists, unless they
are involved in ed
However, the Guide may be of
interest for other reasons, since it interprets and applies both l
employment arena and professional testing standards to issues
of disparate impact in ways th
"overreaching" or
incorrect.
Test publishers, APA, and other educational institutions objected to the
proposed timing of the
agreed to revise the current document
with plans for a fall publication. OCR has stated the Gu
new federal guidelines or professional standards, but rather will provide a
meaningful interpre
tests in education. A number of
national media outlets (New York Times, Wall Street Journal,
Chronicle of Higher Education) have run
stories on the guidelines and op-ed pieces that have
emphasis on disparate impact being the sole determination of whether or not
a test should be
The Guide cites specific wording from the Standards for Educational and
Psychological Testin
40 occasions, leading APA,
AERA, and NCME to formally request that OCR delay revision of
has been revised and published (sometime around December 1999). Several
organizations h
comments on the OCR Guide.
My.SIOP Community
In May, the Department of Education's Office of Civil Rights (OCR) released a
draft Resource
Discrimination in High Stakes
Testing" that sought to provide an overview of federal standards
principles that should guide the use of tests for making
high stakes educational decisions (e.g
special
educational referrals, promotion, graduation, and scholarship awards). This
Resource
development for several years according to OCR,
but educational groups and test publishers
working days
before it was originally scheduled for release.
The Guide attempts to apply Title VII law, EEOC Guidelines, and professional
standards that a
to educational test use. It cites
several Supreme Court and lower courts decisions concerning
or transports decisions and standards to education. Major concerns
addressed by educationa
summarized in comments submitted by
the College Board (Camara, 6/21/1999):
First, the Resource Guide focuses exclusively on disparate impact
resulting from tests (or diff
ignores the level of
validity and utility offered by a test. Disparate impact cannot be considere
must be evaluated in terms of the overall validity and
utility of inferences associated with the p
Resource
Guide clearly elevates any measure, irrespective of validity, cost, or burden to
the e
lower disparate outcomes above any test having
greater disparate outcomes. We believe this
precedent that
has no legal or professional justification and the Guide will have a chilling
effec
educational tests.
Second, the Resource Guide offers no guidance on what level of disparate
impact would resu
there be substantial statistical
disparities or would any disparate outcome result in an investig
http://www.siop.org/tip/backissues/tipocto99/22Camara.aspx[1/20/2016 4:16:54 PM]
OCR Issues Draft Guide on Disparate Impact in Educational Testing TIP October 1999
should not be the primary statistical analysis used to determine if and
when an alternative me
A consistent pattern of ethnic
and racial disparities has been found across a variety of standa
National Assessment of Educational Progress (NAEP) and the National
Educational Longitud
educational measures used for
high-stakes decisions, such as high school grades, class rank
quality and rigor of courses completed, as well as educational outcomes
(e.g., college grades
(Camara and Schmidt, under
review). Disparities in test results reflect similar differences in ot
(e.g., job performance, college achievement, and grades)
and may be indicative of earlier diff
learn and
educational opportunities, not test bias or flaws with the test.
Third, professional and technical standards do not define tests so
narrowly that they exclude
assessments that are both used daily
to make high-stakes decisions about individual students
have similar levels of disparate impact against protected groups. Specifically,
the Test Standa
standardized ability (aptitude and
achievement) instruments, diagnostic and evaluative device
personality inventories, and projective instrumentsa more
appropriate choice among assessm
use will be
facilitated if there is a reasonable comparability in the kinds of information
availabl
three broad categories of test
instruments are covered [emphasis added]: constructed perfor
and to a lesser extent, structured behavioral samples (pages,
3_4)." Related to this comment
Resource Guide be
renamed to put added emphasis on Measures Used in Making High-Stak
Uniform Guidelines and Employment Selection Procedures),
rather than focus exclusively on
decision-making
process, testing.
Fourth, we applaud OCR's deference to the Test Standards. However, the
Resource Guide im
professional standards can be applied in a
rigid manner in evaluating tests. The Test Standar
rigid checklist approach, noting that specific circumstances affect the
relevance of standards
must be applied in evaluating
tests. Professional practice and standards are typically construe
other measures need not meet all standards to be appropriately used
within the bounds of pro
(Richardson, 729 F. Supp. At 821,
823). In addition, the three sponsoring educational associa
the Standards, which date back to 1985. We strongly endorse the
recommendations from AP
asking that issuance of this
Resource Guide be deferred until after publication and dissemina
Standards and requesting a standard 90-day review period for any
subsequent drafts of this d
publication of the revised Test
Standards.
Fifth, we would ask OCR to ensure that colleges and universities, school
districts, and state e
an opportunity to review and
comment on this proposed Resource Guide. The Resource Guid
disseminated or reviewed by colleges and secondary schools. These are the very
organizatio
affected by the Resource Guide once it
is issued and it seems appropriate that they be given
comment on the inferences and proposed standards.
Sixth, the distinction the Resource Guide makes between tests and other
assessment device
establishing a much lower
technical, professional, and legal standard for more subjective ass
applications, grades and GPA, recommendations, ratings or
evaluations of student work and
experiences and
honors, community service and involvement, samples of student work). In W
and Trust, 487 U.S. 977, the American
Psychological Association submitted an amicus curiae
argued there is no professional or scientific justification to treat subjective
and objective devic
validation requirements. In fact,
not imposing essentially the same legal and technical standa
and devices used in high-stakes individual decisions would provide a
sanctioned and covert m
APA further argued that
subjective procedures (in that case used for employment) are "amen
psychometric scrutiny" as objective procedures, citing the Test
Standards which address inte
(Camara, 1996). In deciding
Watson v. Fort Worth Bank and Trust, 487 U.S. 977, all eight of
t
O'Connor's opinion holding that the adverse impact
theory can be used in cases involving sub
was
concerned that an employer could combine an objective criteria (such as a test
or diplom
(such as interviews or ratings) and easily
insulate itself from the Griggs test. O'Connor noted
making systems could have "precisely the
same effects as a system pervaded by impermissib
(Opinion at 4926).
Seventh, professional and legal standards do not provide any support for
OCR's distinctions b
and other measures. We agree with
comments to an earlier draft of this Resource Guide subm
Testing and Assessment (Shavelson, June 10, 1996), stating that "OCR's
inquiry is not to pro
validity of inferences and decisions
based on tests, but rather to determine whether the entire
a part provides students a fair and equal opportunity to learn...."
The Resource Guide ignores
even if they contribute
more to disparate outcomes. In fact, high school courses, judgments a
http://www.siop.org/tip/backissues/tipocto99/22Camara.aspx[1/20/2016 4:16:54 PM]
OCR Issues Draft Guide on Disparate Impact in Educational Testing TIP October 1999
high school curriculum," grades, and rank may
also contribute more to disparate outcomes, th
testing, if an institution places substantially greater weight on these factors.
For example, if te
one of several factors in
admissions, then there is no guarantee those disparate outcomes wi
eliminated. In requiring tests to meet an exceptionally higher
standard than other measures (G
student work, high school
rank, past experience, and opportunities), the Resource Guide will
of valid and objective standardized tests used by educational
institutions, states, and school d
educational
institutions may opt to employ less valid and less objective methods for
high-stak
are not addressed in this Resource Guide.
Eighth, the Resource Guide also sanctions the use of the Uniform
Guidelines on Employment
resource in educational
testing. As the Resource Guide acknowledges in a footnote, there are
differences between educational and employment testing that we
believe undermines any atte
in educational settings.
The Uniform Guidelines were never developed with application to
edu
organizations did not have an opportunity to
comment on extensions of the principles to educ
Guidelines
are over 25 years old and do not reflect current scientific principles of
measureme
practice. The Uniform Guidelines are
outdated and do not conform to the Testing Standards (
their consideration of validity (as accomplished by adopting one of three
distinct types of valid
(this is virtually ignored
in the Guidelines, but is accepted professional practice), differential
p
as well as several other areas (APA, 1985). The Uniform
Guidelines may provide a framewor
guidelines
addressing test use, but they should not be viewed as a substantive resource in
ed
Ninth, statistical analyses should be based on the pool of qualified
applicants, not a general p
not addressed in the
Resource Guide.
Tenth, this Resource Guide implies that once disparate impact is
established that the burden
educational institution to
demonstrate both the educational necessity of the test and then to d
alternative exists throughout the process. This legal interpretation is
incorrect.
Other sections of the Resource Guide viewed as problematic include wording
implying that se
studies are required for each school;
that tests can only be used for purposes they were origi
than for uses where sufficient validation evidence exist); and that
there is a unique methodolo
when they are to be used as
the sole criteria.
On June 18th, the House held a hearing on the OCR Guide and
department officials noted tha
recirculate the current
draft to groups who have already submitted comments on the current d
submit a revised Guide to the National Academy of Sciences Board
of Testing and Assessme
Thereafter, they anticipate
making a final draft available for public review this fall. They will pu
and will have the revised Guide posted on their web
(Coleman, June 21, 1999, personal corre
References:
American Educational Research Association, American Psychological
Association, and Nation
in Education (1985). Standards
for educational and psychological testing. Washington, DC: A
Association.
American Psychological Association (1985). Report of the Ad Hoc Committee
to Develop a Un
Guidelines on Employee Selection
Procedures. Washington, DC: Author.
American Psychological Association (1987). Amicus curiae brief in support
of the petitioner. W
and Trust. Washington, DC:
Author.
Camara, W. (1996). Fairness and public policy in employment testing:
Influences from a profe
Barrett (Ed.), Fair
employment strategies in human resource management. Westport, CT: Qu
Camara, W. and Schmidt, A. (Under Review). Social stratification and group
differences in sta
educational indicators.
Shavelson, R. (June 10, 1996). Correspondence to Norma Cantu.
Watson v. Fort Worth Bank and Trust, 487 U.S. 977 (U.S. Supreme Court,
June 29, 1988).
http://www.siop.org/tip/backissues/tipocto99/22Camara.aspx[1/20/2016 4:16:54 PM]
OCR Issues Draft Guide on Disparate Impact in Educational Testing TIP October 1999
October 1999 Table of Contents | TIP Home | SIOP Home
© 2016 Society for Industrial and Organizational Psychology, Inc. All Rights Reserved
Hit Counter 3687
http://www.siop.org/tip/backissues/tipocto99/22Camara.aspx[1/20/2016 4:16:54 PM]
Home | Privacy Policy | Sitem