Fisher, et al v. Tucson Unified, et al

Filing 1519

REPORT AND RECOMMENDATION of Special Master - RE:UHS Admissions Process Revisions. (Attachments: # 1 Admissions Process Attachment A, # 2 UHS Admissions Process Attachment B, # 3 UHS Admissions Process Fisher Attachment C, # 4 UHS Admissions Process TUSD Attachment D)(SMBE)

November 22, 2013
To: The Honorable David C. Bury
From: Willis Hawley, Special Master
Re: Report and Recommendations Related to Plaintiffs Objections to
TUSD’s Criteria for Admission to University High School
Overview
The Fisher and Mendoza Plaintiffs have both objected to the District’s plan
for changing the criteria for admission to UHS (See Attachment A). The
USP provides that:
a. By April 1, 2013, (since changed by common agreement) the District
shall review and revise the process and procedures that it uses to
select students for admission to UHS to ensure that multiple measures
for admission are used and that all students have an equitable
opportunity to enroll at University High School. In conducting this
review, the District shall consult with an expert regarding the use of
multiple measures (e.g., essays; characteristics of the student’s
school; student’s background, including race, ethnicity and
socioeconomic status) for admission to similar programs and shall
review best practices used by other school districts in admitting
students to similar programs. The District shall consult with the
Plaintiffs and the Special Master during the drafting and prior to
implementation of the revised admissions procedures. The District
shall pilot these admissions procedures for transfer students seeking
to enter UHS during the 2013-2014 school year and shall implement
the amended procedures for all incoming students in the 2014-2015
school year.
The Fisher and Mendoza Plaintiffs object to the following aspects of the
District’s proposed criteria and processes for admission to UHS (See
Attachments B and C):
1. The untested use of the proposed student test of motivation.
2. The failure to test the effects of altering the weights assigned to the
current (2013-14) criteria for admission to UHS.
1

3. The delayed response to the provisions of the USP regarding UHS
admissions.
I concur with these objections and further object to the fact that the
District has not taken actions that would be responsive to the USP that are
being implemented in many schools with missions similar to that of UHS.
In addition, the Mendoza Plaintiffs object to the District’s use of the Illinois
Mathematics and Science School as a argument for the proposed UHS
admission criteria. Fisher Plaintiffs and Mendoza argue that the UHS
admissions plan should include provisions for recruitment and retention of
African American and Latino students. The Fisher Plaintiffs want
admissions criteria that are more inclusive coupled with support for
students who may need extra help to succeed at UHS. The Fisher Plaintiffs
object to the use of the test used by the District in determining admission to
because they believe it to be culturally biased.
I address these concerns below but focus on three conclusions:
The District did not act to implement the provision of the USP related
to UHS admissions criteria in a timely manner. The consequence of
this is that the proposed response to the USP does not adequately
address the provisions of the USP and has led the District to act in the
face of known opposition from the Plaintiffs and the Special Master
and thereby undermine the feasibility of changes in its current
criteria.
The District did not consult with the Plaintiffs and the Special Master
during the drafting of the revised admissions procedures as
provided for in the USP and instead consulted only after it had
reached its initial conclusions. The basic conclusions of the initial
report changed little as a result of subsequent interactions with the
Plaintiffs and the Special Master.
The plan proposed by the District lacks a firm grounding in
research or the practice of other “exam schools” that would allow one
to believe that the action the District proposes to take will accomplish
the goals of the USP. Indeed, the District rejected much of the
2

evidence it gathered from other exam schools, saying that it will
consider what it learned for future years. At the same time, it
proposes to implement the recommendation of a single consultant
with no apparent experience working with schools like UHS or
students like those who attend UHS.
I elaborate on each of these findings later in the report.
This report recommends that the Court immediately direct the District to
implement an alternative course of action that is doable and that will better
achieve the goals of the USP than the District’s proposal. Much of this
course of action, which is described below, was proposed to the District and
was rejected.
I recommend that the Court act on this recommendation as quickly as
possible because the admissions process for UHS is already underway. To
facilitate expedited action, I invited the District to critique a recent version
of the recommendation I am making to the Court. These comments are
enclosed as Attachment D.
These comments and recommendations apply to the proposal by the
District that affects entering freshman because few students are admitted at
other levels so the urgency of action on the criteria for upper class
admission (which involves few students) is not as great as action related to
the freshman class.
The Purpose of the UHS Admissions Revision Proposal of the USP
UHS is, by all accounts, an excellent school. The Plaintiffs have long urged
that the District use admissions procedures that would increase the
enrollment of African American and Latino students at UHS so as to more
nearly represent the demographics of the District’s students. The goal here
is not to alter the rigor of the UHS curriculum or to enroll students who
cannot succeed academically at UHS.
The differences between the racial and ethnic enrollment in UHS and
current enrollment in TUSD high schools (including UHS) in 2012-13 are as
follows:
3

District
UHS

African American
6.2%
1.6%

Latino
56.2
30.9

The freshman enrollment of Latino students in 2009-10 was 29 percent of
the freshman class and 31 percent in 2012-13. From 2010-11 to 2012-13,
the percentage of Latino students enrolling in UHS who do not live in the
District went from one percent of Latino enrollees to almost 10 percent.
African American enrollment in UHS changed from 1 to 4 students. Half of
all current African American students enrolled in UHS do not live in the
District.*
The Failure of the District to Act in a Timely Manner
The District claims that given the fact that the USP was not approved by the
Court until February 2013, it did not have time to conduct the pilot
required by the USP nor to undertake a comprehensive study of alternative
admission measures.
In July 2012, the Court ordered the parties to work on aspects of the USP
about which there was agreement. By July, 2012, increasing the number of
African American and Latino students who attended UHS was identified as
a high priority. Further, enhancing admission opportunities for the subject
classes had been identified as a priority in the Post-Unitary Status Plan
(PUSP), which remained in effect until approval of the USP by the Court.
The District did not mobilize to work on UHS admissions until after the
USP was approved by the Court and even then, its effort was limited as
evidenced by the initial plan for UHS admissions submitted to the Plaintiffs
and the Special Master on July 22. Only after substantial criticisms of the
*Data used here were taken from the District’s Annual Report (November 2013) and from
emails from the District’s Office of Desegregation. They are different from data in the UHS
Admission proposal which show higher Latino enrollment in UHS in 2010-11 and 2011-12.

4

Initial Plan by the Plaintiffs and the Special Master did more intensive work
communications among the parties that resulted in little change in the
District’s proposal. In recognition that the District had not been able to
prepare a viable proposal that addressed the provisions of the USP, the
Plaintiffs and the Special Master agreed, at least three times, to alter the
date for final approval to October 23, but did not agree that this would
mean that changes in the admission criteria the District proposed could not
be altered.
The UHS admissions plan was approved by the Governing Board on
October 22, 2013. Bottom line: the District had more than a year t0work on
the UHS admissions process and the fact that it began this work when it did
is not a viable excuse for the inadequacy of its current proposal which
makes only a minor change in the 2013-14 UHS admission process and the
effect of that change could be counterproductive.
The District argues that it is too late to make changes in the plan that it
knew was opposed by the Plaintiffs and the Special Master in July. The
reason that the issue remained unresolved is because the District did not
act when it could have. Moreover, the District sets a very early deadline for
applying for admission to UHS, months earlier than the deadline set by
most selective colleges and universities. It could have postponed the
application date when it knew that important matters remained contested.
The Failure to Collaborate
The provision of the USP related to UHS admissions is different from many
other parts of the USP that require the District to submit its proposals to
the Plaintiffs and the Special Master for “review and comment” (cf. Section
I.6.1). In the provision relating to UHS, the USP says that, “The District
shall consult with the Plaintiffs and the Special Master during the drafting
(emphasis added) and prior to the implementation of the revised admission
procedures”. As it drafted its plan, the District did not consult. The District
made its plans, the Plaintiffs and the Special Master responded in accord
with the process provided for in Section I.6.1. Had the District involved the
Plaintiffs and the Special Master as it drafted its plan, the District would
have been aware of possible areas of disagreement and, in any event, the
5

Plaintiffs and the Special Master would have known that the District was
not contemplating significant changes in the UHS admission policies and
practices.
The Inadequacy of the District’s Proposed Plan for Admission to UHS
As the District begins the process of recruiting and selecting students for
admission to UHS, the process and criteria are the same as they were in
previous years except that students will take the Children’s Academic
Intrinsic Motivation Inventory (CAIMI) or some other “relevant measure”.
I comment on the CAIMI test below. Students will be given up to five points
for their performance on CAIMI, which will be added to the points students
received from differences in pre-UHS grade point averages and
performance on the Cognitive Abilities Test (CogAT). To be admitted to
UHS a student must have a combined score of 50 made up of weighted
scores based on pre-UHS GPA in core subjects and scores on the CogAT.
UHS reserves the right to increase the minimum eligibility score above 50.
The Validity of Current Admission Criteria
Admission criteria have as their primary purpose the development of
estimates of whether students can be successful in doing and benefitting
from the academic work required by UHS (or any other selective admission
school or college). The first step in evaluating the consequences of using the
existing admission criteria should be to ask how well those criteria predict
success of students in UHS. The District argues that since all admitted
students prosper at UHS--in the sense that students of all races and
ethnicities graduate at high rates--its admission criteria are appropriate
and the weights assigned to GPA and CogAT are valid. But this does not
answer the question of whether different weights for GPA or CogAT scores
would produce similar results and open up admission to a different
demographic mix of students.
One assumes that grades within USP reflect differences in performance at
the school. The District reports that differences in pre-UHS grade point
averages do not predict (in this case, are not correlated with) students’ UHS
grade point averages. This means that there is no basis for assigning
6

different weights to pre-UHS grade point averages. We do not know if there
are differences in GPA among UHS applicants by race because that
analysis, if done, was not shared. But it is easy to see that differences in
GPA have a potentially large affect on admissibility to UHS. Moreover, we
have no way of knowing whether students with a pre-UHS GPA of 2.9,
rather than 3.0 (the current minimum GPA for admission) would not
succeed at UHS.
The UHS admissions committee argues that no student with a CogAT score
of less than 7 should be admitted to UHS. But its analysis of the
relationship between a student’s CogAT score and his or her GPA at UHS is
negligible, except for scores of 9. The District says that the CogAT scores
correlate with tests such as ACT and SAT. But this is irrelevant to the
question of whether they predict performance in UHS. Moreover, the weak
predictability of these tests for success in post-secondary education is why
all selective colleges use measures in addition to the SAT or ACT tests (and
GPAs) to determine who will be admitted.
The District provides no rationale for the weights it assigns to different GPA
and CogAT scores either within or across each time of measure. The
Mendoza Plaintiffs and I have repeatedly asked District to try different
scenarios with past year application information to see what the results
might be. So far as I know they have not done this.
In summary, the criteria in place for admission for UHS in 2013-14, which
remains the foundation for admission for in 2014-15, have little
relationship to the only measure the District has of validity of those criteria.
Nonetheless, the District remains wed to those criteria.
The Children’s Academic Intrinsic Motivation Inventory
When the UHS admissions study group indicated that it was considering
the use of a test of motivation and/or resiliency as an addition to the
current admission criteria, the Plaintiffs and the Special Master indicated
that such an assessment, in principle (emphasis added) was worth
considering. The District consulted with Dr. Lannie Kavesky, whose
expertise is the study of giftedness among elementary students, who
7

identified the CAIMI measure. No other measure was reported to have been
considered despite the fact that no other exam school uses this test and it
has not been validated (so far as one can tell) as a good predictor of success
in an academically selective high school. There is no evidence that the
CAIMI will provide greater diversity in the acceptance pool. In the analysis
presented in Appendix J of its proposal, the District estimates that this test
will likely have little effect on the eligibility of African Americans and will
result in a significant percentage increase in the enrollment of Latino
students. However, this analysis is seriously flawed and overstates the likely
effect. See Attachment E. Indeed, given that the way the test results are to
be used—to make up a deficiency in the qualifying score—a simple spelling
test would have the same effect.
The District says that CAIMI was selected from among other possible
measures (unnamed) because there are studies of its reliability and validity.
As noted, there do not appear to be any studies of the reliability and validity
of the CAIMI for use as a predictive selection tool for admission to a high
school exam school, at least as far as a search of the literature indicates or
that are cited by the District. This is not surprising since no other exam
school the District contacted or researched used the CAIMI or anything like
it.
Other Measures to be Used in Admissions
After the initial criticisms of its plan for UHS admissions, the District
sought to identify what other “exam schools” do in admission. None of the
information reported by the District indicates that a test of motivation
should be used and many exam schools used essays by students; “noncognitive measures” (such as exceptional activities, evidence of extra effort,
leadership roles, personal qualities, etc.); and teacher recommendations.
The District says that it will look into these other measures and will use
student essays for admission in 2015-16 but that it is too late to use them in
the coming year. There is, however, nothing mysterious about the types of
measures suggested above, they are certainly less mysterious than the
CAIMI test. Student essays and non-cognitive measures are used by almost
8

all selective colleges and universities. It would be easy to get examples of
these measurement instruments for other exam schools.
The District says that it will compare the results of the CAIMI to that of
other tests but provides no plan for doing this.
Recommendation
I conclude that the District’s proposal for admissions to UHS in the coming
year is inadequate response to the USP, is inconsistent with the evidence
the District gathered about other exam schools, and that the addition of the
CAIMI is pretty much rolling the dice.
The District has already announced its plan for admission to UHS and
informed parents accordingly. While one could challenge the need to start
the enrollment process in October, that is what the District does. There is
clearly not time to identify an alternative measure of motivation. A
recommendation that the process for admission to UHS be postponed for
several weeks might be justified on procedural and substantive grounds but
the cost to public confidence in the District and the inevitable attacks on the
USP suggest that this option is not desirable.
I recommend that the Court direct the District to:
1. Expedite the review of applicants for admission to UHS using the
criteria used in 2013-14.
2. Develop student essay questions and non-cognitive measures (the
District already has examples of these from other exam schools and
can easily get more) no later than January 15, 2014.
3. Identify applicants who are potentially eligible for admission to UHS
by changing the initial cut score on the aggregated GPA and CogAT
weights from 50 to some number that increases the pool of eligible
candidates by at least 33 percent or a number agreed to by the
District and the Special Master. This will create a preliminary
eligibility pool.
4. As soon as possible, the students in the preliminary eligibility pool
will be invited to write a qualifying essay and complete the
9

5.

6.

7.

8.

questionnaire that identifies non-cognitive student characteristics
typically used in selective school and college admissions.
As an alternative to step 4, the District could ask all applicants to
prepare the essay and to fill out the form identifying particular
experiences and strengths of those who are applying as soon as the
essay topics and questionnaire are prepared.
An additional number of points based on the essays and evidence of
student characteristics related to achievement would be added to the
aggregate GPA and CogAT scores. This number should be
consequential and determined based on the quality of the response to
the alternative measures.
During the next year, applicants to UHS for the 2014-15 school year
(or a sample thereof) will be tested on at least two tests of motivation
and the results evaluated with respect to their impact on the racial
composition of the UHS student body. Alternatively, the District’s
research on motivational assessments may lead to a decision not to
use such an assessment.
The District shall, during the next several months, provide a
justification for the weights it assigns to GPA and the CogAT scores in
determining eligibility for admission to UHS. This analysis shall
inform possible revisions of the admission criteria for 2015-16.

Nothing in this recommendation is meant to restrict the District from any
additional inquiry it chooses to pursue in an effort to increase the numbers
of African American and Latino students who will benefit from the rigorous
academic environment that characterizes UHS. Moreover, nothing in this
proposal is meant to suggest that the academic demands on UHS students
be reduced

10

Comments on Objections not Addressed in the Report Above
Request of Fisher Plaintiffs for Inclusion of Support in the UHS
Admissions Policy
All of the parties agree that it is important to ensure that students who are
admitted to UHS have the support they need to succeed and to graduate.
The District argues that such a provision does not belong in the admissions
criteria but should be dealt with in the Recruitment and Retention plan to
be completed in December and has committed to doing so. I agree with the
District in this case. It is worth noting that: (1) among students declared
eligible for admission, African American and Latino students enroll in
much higher percentages than their white peers, especially in the last two
years for which data were provided and (2) once admitted African
American and Latino students are as likely to graduate as their white peers.
Of course, this could change if different criteria are used in admission
though the goal of changing the admission criteria is to find more valid
measures of capability and motivation, not to admit students unlikely to
succeed in UHS.
Fisher and Mendoza Objection: Need to Address Recruitment
Both Fisher and Mendoza want the District to acknowledge its obligation to
address recruitment, as well as retention, in accord with the relevant
sections of the USP (V.A.5). The District has done so. However, the fact that
there is no plan in place before the application deadline for admission to
UHS (October 4, 2013) suggests that recruiting more African American and
Latino students to UHS than in the past was not a high priority for the
District.
Fisher Objection: The CogAT Test is Culturally Biased
Almost all tests like the CogAT advantage students with strong vocabularies
that are defined by the dominant culture. However, I do not have evidence
that the CogAT is more biased than other options. But the possibility that
the CogAT is less accurate in measuring the cognitive abilities of African
American or Latino students is another reason for using non-cognitive
measures recommended above.
11

Fisher and Mendoza Objection: Using the Illinois Math and Science
Academy to Justify UHS Admissions is Inappropriate
I agree but the District does not use this example in its final proposal.

Attachment E
Possible Effects of the CAIMI Test
The UHS admissions proposal argues that by adding up to five points to the
scores of students as a result of them taking the CAIMI test, the three-year
average of students gaining admission through bonus points from the test is
as follows: Whites-35%, African Americans-5% and Latinos-53%.
Accepting the unlikely TUSD assumption that students would receive five
out of five bonus points and the assumption that all eligible students enroll,
the numbers don’t add up. Taking the two years for which the district
provides admissions data and scores below 5o points by race (they say that
all students over 50 points are admitted) here is the story:
2010-11
Race #Enrolled #Eligible by Bonus Points % Enrollment Increase
White

57

12

21

Af-Am

2

3

150

Latino

60

21

35

White

71

14

20

Af-Am

4

1

25

Latino

67

16

24

2011-12

While the percentage increases for African Americans are high the number
of students is very low. The increase for Latinos is high but nowhere near
12

the 53% increase TUSD calculated (I use a different base but the aggregate
enrollment over time comes from yearly numbers provided by the District).
Moreover, if on average students of all races received three rather than five
points on the CAIMI, the number of qualified Latino students would drop
significantly.
This said, the CAIMI could significantly increase the numbers and, to a
lesser extent, the proportion of Latino students attending UHS although we
have no way to know how different racial/ethnic groups will do on the
CAIMI or if the CAIMI is the best way to assess motivation and resiliency.
And, as noted earlier, since the scores on the test are added on, any test
would have the same effect.

13