AMERICAN EDUCATIONAL RESEARCH ASSOCIATION, INC. et al v. PUBLIC.RESOURCE.ORG, INC.

Filing 134

MOTION for Summary Judgment , MOTION for Permanent Injunction by AMERICAN EDUCATIONAL RESEARCH ASSOCIATION, INC., AMERICAN PSYCHOLOGICAL ASSOCIATION, INC., NATIONAL COUNCIL ON MEASUREMENT IN EDUCATION, INC. (Attachments: # 1 Memorandum in Support, # 2 Statement of Facts, # 3 Declaration Hutter Decl., # 4 Exhibit 1, # 5 Exhibit 2, # 6 Exhibit 3, # 7 Exhibit 4, # 8 Declaration Hudis Decl., # 9 Exhibit A, # 10 Exhibit B, # 11 Exhibit C, # 12 Exhibit T, # 13 Exhibit U, # 14 Exhibit Z, # 15 Exhibit BB, # 16 Exhibit CC, # 17 Exhibit EE, # 18 Exhibit GG, # 19 Exhibit HH, # 20 Exhibit II, # 21 Exhibit JJ, # 22 Exhibit KK, # 23 Exhibit LL, # 24 Exhibit MM, # 25 Declaration Ernesto Decl., # 26 Exhibit NN, # 27 Exhibit OO, # 28 Exhibit PP, # 29 Exhibit QQ, # 30 Exhibit RR, # 31 Exhibit SS, # 32 Exhibit TT, # 33 Exhibit UU, # 34 Declaration Wise Decl., # 35 Exhibit KKK, # 36 Exhibit LLL, # 37 Declaration Camara Decl., # 38 Exhibit MMM, # 39 Declaration Levine Decl., # 40 Exhibit NNN, # 41 Exhibit PPP, # 42 Exhibit QQQ, # 43 Exhibit UUU, # 44 Declaration Geisinger Decl., # 45 Declaration Schneider Decl., # 46 Exhibit Levine Depo Tr., # 47 Exhibit No. 1207 to Levine Depo Tr., # 48 Exhibit No. 1308 to Levine Depo Tr., # 49 No. 1308 to Levine Depo Tr., # 50 Text of Proposed Order)(Elgarten, Clifton)

Case 1:14-cv-00857-TSC Document 60-50 Filed 12/21/15 Page 1 of 13

EXHIBIT NN
Case No. 1:14-cv-00857-TSC-DAR

Case 1:14-cv-00857-TSC Document 60-50 Filed 12/21/15 Page 2 of 13
EXHIBIT

Ì
&ET-

AMERICAN
PSYCHOLOGICAL
ASSOCIATION
April 7, 2000

Ms. Jill Morningstar, Legislative Assistant
U. S. Senator Paul Wellstone
136 Hart Senate Office Building
Washington, DC 20510
Dear Jill:

We very much appreciate your seeking the endorsement of the American Psychological Association (APA)
for the bill introduced by Senator Wellstone this week on Fairness and Accuracy in Student Testing. We
wanted to get back to you quickly with an interim response about the Association's views.
As we mentioned on the phone earlier this week, APA is not in a position to endorse the bill at this time.
APA policy on this issue is guided by The Standards.* Educational and Psychological Testing, the 1999
version of which was developed over a six-year period with the American Educational Research
Association and the National Council on Measuretnent in Education. While we are drafting a more
comprehensive response for the Senator, we wanted to let you know the general areas of our concerns.
First, we believe that the most fair and appropriate way to approach the problems Senator Wellstone seeks
to address is to investigate high stakes decision-making in educational settinus, how such decisions are
informed, and the impact on educational outcomes. The critical issue is not tests, per se (provided they are
valid, reliable instruments) but rather the instances wherein they and other measures of accountability (i.e.
grades), are used inappropriately in making these decisions. Second, the bill appears to misstate the intent
of both the Standards and the National Academy of Sciences (NAS) report by holding tests for tracking,
retention and graduation to the same standard. The NAS study and the Standards treat those uses
differently. Both documents treat graduation tests as certification tests, and do not prohibit assigning them
determinative weight, but offer reconunendations for implementing them fairly.

We understand the Senator wants to offer this amendment to the Elementary and Secondary Education Act
soon and would apprcciate APA's assistance in doing so. Regrettably, APA has not yet adopted a policy
that goes beyond the Standards to address the issues of how high stakes decision-making in schools affects
education and educational outcomes more broadly, particularly for certain groups of children.
Accordingly, it would be impossible for us to offer you language that the Association could endorse within
your timefrarne. However, ve would welcome the opportunity to work with you on your Plan B: an
amendment to authorize specific evaluation research that would provide additional needed infomiation
about the impact of high stakes decision-making on the educational opportunities of different populations
of children, and on the educational system more generally. The NAS report and other sources point to
significant gaps in the data. We hope to provide specific draft language to you late next week.
Again, we appreciate your willingness to work with us. Senator Wellstone has been a champion of many
of the issues most iniportant to psychologists, and we are grateful to him.
Sincerely,

Kob.,
Ellen G. Garrison, Ph.D.
Director of Public Interest Policy

Patricia C. Kobor
Director of Science Policy

750 First Sireel, NE

Washington, DC 20302-4242
202( 33&550D
[phone] TDD

Web. www.oFx,.oro

Please Recycle

AERA_APA_NCME_0031816

Case 1:14-cv-00857-TSC Document 60-50 Filed 12/21/15 Page 3 of 13
0: \ KOS \ KOS00.280

S.L.C.

106T CONGRESS
2D SEssioNr

s

IN THE SENATE OF THE UNITED ST.ATES
Mr. WELLSTON-E introduced the foBowing bill; which was read twice and referred to the Committee on.

A BILL
To provide for fairness and accuracy in student testing.
Be it enacted by the Senate and House of Representa2 Oyes of the United States of America in Congress assembled,
3 SECTION 1. FAIRNESS AND ACCURACY IN STUDENT TEST-

4
5

ING.

(a) FINDLNTGS.Congress finds the following:

6

(1) The use of large-scale achievement tests in

7

education has grown significantly in recent years.

8

States and local school districts have increasingly

9

used these tests in such contexts as raisinz student

10

academic standards to make high-stakes decisions

11

with important consequences for individual students.

YV,1 29T MIL

00/0C/C0

AERA_APA_NOME_0031817

Case 1:14-cv-00857-TSC Document 60-50 Filed 12/21/15 Page 4 of 13
0: \ KOS \KOS00.280

S.L.C.

2
1

such as tracking (assigming students to schools, pro-

2

grams, or classes based on achievement level), pro-

3

motion of students to the next grade, and gradua-

4

tion of students from secondary school.

5

(2) The serious and often adverse co.nsequences

6

resulting from the sole or determinative reliance on

7

large-scale tests h_ave increasingly resulted in clues-

tions and significant concerns by students, parents,
9

teachers, and school administrators about hoi,v to en-

10

sure that such tests are used appropriately and in

11

a manner that is fair.

12

(3) In 1997, Congress directed the National

13

Academy of Sciences to "conduct a study and make

14

-written recommendations on appropriate methods,

15

practices, and safeguards to ensure that, arnonz

16

other things,

17

to assess student performance are not used in a dis-

18

criminatory manner or inappropriately for student

19

promotion, trackin.g, or graduation.".

esting and new tests that are used

20
21

through its National Research Council, completed its

22

study and issued a report entitled 'High Stakes:

23

Testing for Tracking, Promotion and Graduation'.

24

Guided by principles of m.easurement validity, attri-

25

sooZi

(4) In 1999, the National Academy of Sciences,

bation of cause, and effectiveness of trea:tment, the

IVA LVST

00/0C/C0

AERA_APA_NCME_0031818

Case 1:14-cv-00857-TSC Document 60-50 Filed 12/21/15 Page 5 of 13
0: \ KOS \KOS00.280

3
1

National Research Council made key findings for ap-

2

propriate test use in educational settings, in.cluding

3

the following:

4

(A) When tests are used in ways that meet

5

relevant psychometric, legal, and educational

6

standards, students' scores provide important

7

information, that combined with information

8

from other sources, can lead to decisions that

9

promote student learning and equality of oppor-

10

tunity.

11

(B) Tests are not perfect. Test questions

12

are a sample of possible questions that could be

13

as.ked in a given area. Moreover, a test score is

14

not an exact measure of a student's knowledge

15

or skills.

16

(C) To the extent that all students are ex-

17

pected to meet world-class standards, there is a

18

need to provide world-class curricula and in-

19

struction to all students. However, in most of

20

the Nation, much needs to be done before a

21

world-class curriculum and world-class instruc-

22

tion will be in place. At present, curriculum

23

does not usually place sufficient emphasis on

24

student understanding and application of con-

25

900 PI

L.7

cepts, as opposed to memorization and skill

YVA LZ:RT

IJLL

00/0C/C0

AERA_APA_NOME_0031819

Case 1:14-cv-00857-TSC Document 60-50 Filed 12/21/15 Page 6 of 13
0: \KOS \KOS00.280

S.L.C.

4

mastery. In addition, instruction in core sub2

jects typically has been and remains highly

3

stratified. What teachers teach and vliat stu-

4

dents learn vary widely by track, with those in

5

loi,ver tracks receiving far less than a workl-

6

class curriculum.

7

(D) Problems of test validity are greatest

8

among young children, and there is a greater

9

risk of error when such tests are employed to

10

make significant decisions about children who

11

are less than 8 years old or below grade 3, or

12

about their schools. However, well-designed as-

13

sessments may be useful in monitoring trends

14

in the educational development of populations

15

of students who have reached. age 5.

16

(5) The National Research Council made the

17

following recommendations:

18

19

a.nd others who share re,sponsibility for edu-

20

cational outcomes are to discharge their respon-

21:

sibility effectively, they should have accegs to

22

information about the nature and interpretation

23

of tests and test scores.

24

should be made available to the public and

25

LOO

(A) If parents, educators, public officials,

should be incorporated into teacher education

Such information

YVI LZi fIHJ.

00/0C/C0

AERA_APA_NCME_0031820

Case 1:14-cv-00857-TSC Document 60-50 Filed 12/21/15 Page 7 of 13
S.LC.

0: \ KOS \ KOS00.280

5
1

and into educational programs for principals,

2

administrators, public officials, and others.

3

A test may appropriately be used to

4

lead curricular reform, but it should not also be

5

used to make high-stakes decisions about incli-

6

-vidual students until test users can show that

7

the test measures what they have been taught.
High-sfaRes decisions such as tracking,

9

promotion, and g-radvation should not automati-

10

cally be made on the basis of a single test score

11

but should be buttressed by other relevant in-

12

forxnation about the student's lmowledge and

13

skill, such as grades, teacher recommendations,

14

and extenuating circumstances.

15

In general, large-scale assessments

16

should not be used to make high-stakes deci-

17

sions about students who are less than 8 years

18

old or enrolled below grade 3.

19

20

routinely include a well-designed evaluation

21

component, Policymakers should monitor both

22

the intended and unintended consequences of

23

high-stake assessments on all students ancl

24

ROO fpi

High-stakes testing programs should

sionificant subgroups of students, including mi-

rid RZ:RT ART.

00/nC/f:0

AERA_APA_NCME_0031821

Case 1:14-cv-00857-TSC Document 60-50 Filed 12/21/15 Page 8 of 13
0: \ KOS \ KOS00.280

S.L. C.

6
1

norities, English-language learners, and stu-

2

dents with disabilities.

3

(6) These principles and findings of the Na-

4

tional Academy of Sciences are supported in signifi-

5

cant measure by the Standards for Educational and

6

Psychological Testing, adopted ancl approved in De-

7

cember of 1999, by the leading experts and profes-

8

sional organizations on testing, including the _Maier-

9

ican Educational Research Association, American.

10

Psychological Association, and the National Council

11

on Measurement in Education.

12

(b)

TEST PERFORMANCE .If performance on a

13 standardized test is considered as part of any decision
14 about the retention, graduation, tracking, or within-class
15

ability grouping of an individual student by a State edu-

16

cational agency or local educational agency that receives

17 funds under the Elementary and Secondary Education Act
18

of 1965, such test performance sha.11 not be th.e sole deter-

19 minant in such decision and may be considered in rnakinz

20 such decision only if21

22

lidity and reliability for the purpose for which the

23

test's results are being used, including the validity

24

and reliability of any

25

600P

(1) the test meets professional standards of va-

standard set or established for use on the test;

Cut

score or performance

TVA QZ:RT nut

nn/nr7/en

AERAAPANCME0031822

Case 1:14-cv-00857-TSC Document 60-50 Filed 12/21/15 Page 9 of 13
0: \ KOS \ KOS00.280

S.L.C.

7
1

(2) the test allows its asers to make score inter-

2

pretations in relation to a functional performance

3

level,

4

that are made in relation to the performance of oth-

5

ers, is based on State or local content and perfonn-

6

ance standards, and is aligned with the curriculum

7

and classroom instruction;

as distinguished from those interpretations

8

(3) multiple measures of student achievement

9

are utilized, including grades and evaluations by

10

teachers, so that scores from larg_e-scale_aasessments

11

are never the only source of information used nor

12

assigned determinative vveight in making a high-

13

stakes decision about an individual student_

14

(4) students tested have been provided multiple

15

opportunities to demon.strate proficiency in the sub-

16

ject matter covered by the test;

17

(5) the test is administered in accordance with

18

the written guidance from the test developer or pub.

19

lisher;

20

(6) the State educational agency or local edu-

21

cational agency has evidence that the test is of ade-

22

(pate technical quality for each purpose for which

23

the test is used

24
25

OTOZ

(7) the State educational agency or local educational agency provides appropriate accommoda-

YVA 9Z:5TIÌHL

00/oc/c0

AERA_APA_NCME_0031823

Case 1:14-cv-00857-TSC Document 60-50 Filed 12/21/15 Page 10 of 13
0: \ KOS \ KOS00.280

&LC.

8
1

tons and alternate assessments for students with

2

disabilities that provide the students with a valid op-

3

portunity to show what they k_now and can do;

4

(8) the State educational agency or local eclu-

5

cational agency provides appropriate accommoda-

6

tions for students with limited English proficiency,

7

includin.g-

8

(A) if such a student is tested in English,

9

the student received academic instruction pri-

10

marily in English for at least 3 years prior to

11

the test, or if the student received instruction in

12

English for more than such 3 years, the local

13

educational agency determines that the student

14

has achieved sufficient English proficiency to

15

ensure that the test will accurately measure the

16

student's subject matter knowledge and sldlls;

17
18

English proficiency who have not been taught

19

primarily in English for 3 years prior to the

20

test, such students are assessed, to the greatest

21

extent practicable, in the language and form

22

most likely to yield accurate and reliable infor-

23

/nation about what those students know and

94

TTOVI

(B) in the case of students with limited

can do; and

IVA

Z:8T aa

00/0C/C0

AERA_APA_NOME_0031824

Case 1:14-cv-00857-TSC Document 60-50 Filed 12/21/15 Page 11 of 13
0: \ KOS \ KOSOO .280

s.L,C
9

1

(C) in the ease of Spanish-speaking stu-

2

cleats with limited English proficiency, such

3

students are assessed using tests developed and

4

written in Spanish, if Spanish language tests

5

are more likely than English language tests to

6

yield accurate and reliable information on what

7

those students know and can do; and

8

(9) the test is not used for a decision a.bout

9

promotion or placement in special education for a

10

child below the age of 8 or grade 3.

11

I-

(C) EVALUATIONS.

12

(1 )

STATE EDUCATIONAL AGENCTES.Ea.ch

13

State educational agency that receives funds under

14

the Elementary and Secondary Education Act of

15

1965 and uses a standardized test as part of a high

16

stakes decision described in subsection (b), shall con-

17

duct a comprehensive e-valuation of the impact of the

18

test's use on students' education and educational

19

outcomes, with particular con_sideration given to the

20

impact on individual students and subgroups of stu-

21

dents clisaggregated by- socioeconomic .status, race,

22

ethnicity, limited English proficiency, disability, and

23

gender. The State educational agency shall rnake the

24

results of the evaluation available to the public and

25

shall provide clear and comprehensible information

YVA 17:RT 11111

nnino/c,

AERA_APA_NOME_0031825

Case 1:14-cv-00857-TSC Document 60-50 Filed 12/21/15 Page 12 of 13
0: \ kOS \ KOS00.280

S.L.C.

l
about the nature, use, and interpretation of the test
2

and the scores the test generate.

3

(2) LOCAL EDUCATIONAL AGENCY.--Each local

4

educational agency that receives funds uncler the El-

5

ementary and. Secondary Education ,.4..ct of 196:5,

6

uses a standardized test as part of a high stakes de-

7

cision described in subsection (b), and is located in

8

a State that does not conduct a.n evaluation under

9

para,graph (1), shall conduct a comprehensive eval-

10

uation of the impact of the test's use on students'

11

education and educational outcomes, with particular

12

consideration given to the impact on individual stu-

dents and subgroups of students disaggregated by
14

socioeconomic status, race, ethnicity, limited English

15

proficiency, disability, and gender. The local edu-

16

cational agency shall make the results of the evalua-

17

tion available to the public and shall provide clear

18

and comprehensible information about the nature,

19

use, and interpretation of the test and the scores the

20

test generate.

21

(3) DEPARTMENT OF EDUCATION.-Tli

See-.

23

retary shall(A) conduct an evaluation similar to the

24

evaluation described in paragraph (1) among a

22

CTOU

XVd 6Z:RI mu.

nninric,

AERA_APA_NCME_0031826

Case 1:14-cv-00857-TSC Document 60-50 Filed 12/21/15 Page 13 of 13
S L C.

0: \KOS \KOS00 280

11
1

representative sample of States and local &ILI-

2

cational agencies;

(B) report the results of such evaluation to

3

4

Congress; and

(C) make the results of the evaluation

5

6

available to the public.

7

(d) DEFINITION OF STANDARDIZES TEST.In this

8

section the term "standardized test" means a test that

9

is administered and scored under conditions uniform to

10

all students so that the test scores are comparable across

11

individuals.

......

fT00

....

YVA 87:RT inT

nri/ner,n

AERAPPANCME_0031827