Students for Fair Admissions, Inc. v. President and Fellows of Harvard College et al
Filing
419
DECLARATION re 417 MOTION for Summary Judgment by President and Fellows of Harvard College. (Attachments: # 1 Exhibit 1, # 2 Exhibit 2, # 3 Exhibit 3, # 4 Exhibit 4, # 5 Exhibit 5, # 6 Exhibit 6, # 7 Exhibit 7, # 8 Exhibit 8, # 9 Exhibit 9, # 10 Exhibit 10, # 11 Exhibit 11, # 12 Exhibit 12, # 13 Exhibit 13, # 14 Exhibit 14, # 15 Exhibit 15, # 16 Exhibit 16, # 17 Exhibit 17, # 18 Exhibit 18, # 19 Exhibit 19, # 20 Exhibit 20, # 21 Exhibit 21, # 22 Exhibit 22, # 23 Exhibit 23, # 24 Exhibit 24, # 25 Exhibit 25, # 26 Exhibit 26, # 27 Exhibit 27, # 28 Exhibit 28, # 29 Exhibit 29, # 30 Exhibit 30, # 31 Exhibit 31, # 32 Exhibit 32, # 33 Exhibit 33, # 34 Exhibit 34, # 35 Exhibit 35, # 36 Exhibit 36, # 37 Exhibit 37, # 38 Exhibit 38, # 39 Exhibit 39, # 40 Exhibit 40, # 41 Exhibit 41, # 42 Exhibit 42, # 43 Exhibit 43, # 44 Exhibit 44, # 45 Exhibit 45, # 46 Exhibit 46, # 47 Exhibit 47, # 48 Exhibit 48, # 49 Exhibit 49, # 50 Exhibit 50, # 51 Exhibit 51, # 52 Exhibit 52, # 53 Exhibit 53, # 54 Exhibit 54, # 55 Exhibit 55, # 56 Exhibit 56, # 57 Exhibit 57, # 58 Exhibit 58, # 59 Exhibit 59, # 60 Exhibit 60, # 61 Exhibit 61, # 62 Exhibit 62, # 63 Exhibit 63, # 64 Exhibit 64, # 65 Exhibit 65, # 66 Exhibit 66, # 67 Exhibit 67, # 68 Exhibit 68, # 69 Exhibit 69, # 70 Exhibit 70, # 71 Exhibit 71, # 72 Exhibit 72, # 73 Exhibit 73, # 74 Exhibit 74, # 75 Exhibit 75, # 76 Exhibit 76, # 77 Exhibit 77, # 78 Exhibit 78, # 79 Exhibit 79, # 80 Exhibit 80, # 81 Exhibit 81, # 82 Exhibit 82, # 83 Exhibit 83, # 84 Exhibit 84, # 85 Exhibit 85, # 86 Exhibit 86, # 87 Exhibit 87, # 88 Exhibit 88, # 89 Exhibit 89, # 90 Exhibit 90, # 91 Exhibit 91, # 92 Exhibit 92, # 93 Exhibit 93, # 94 Exhibit 94, # 95 Exhibit 95, # 96 Exhibit 96, # 97 Exhibit 97)(Ellsworth, Felicia)
EXHIBIT 35
REBUTTAL EXPERT REPORT OF PETER S. ARCIDIACONO
Students for Fair Admissions, Inc. v. Harvard
No. 14-cv-14176-ADB (D. Mass)
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
TABLE OF CONTENTS
1 Executive Summary ....................................................................................................1
2 Professor Card and I agree on many aspects of my methodology, analysis, and
conclusions.................................................................................................................11
2.1
Professor Card and I largely agree on the relevant dataset..........................11
2.2 Professor Card does not challenge my descriptive analysis about the relative
qualifications of students by race/ethnicity. .....................................................12
2.3 Professor Card generally agrees with my methodological approach to
modeling Harvard’s admissions decisions ........................................................16
3 There are several key flaws in Professor Card’s modeling choices that drive his
conclusions about the size of the Asian-American penalty. ....................................17
3.1 Professor Card’s results are skewed by his decision to include in the analysis
many applicants who are unaffected by racial penalties and preferences......17
3.1.1
Professor Card misleadingly includes non-competitive applicants in his
models, which tends to obscure the racial penalties and preferences
Harvard employs in its admissions process.............................................17
3.1.2
Professor Card errs by including in all of his models those applicants
who are members of Harvard’s special recruiting categories .................19
3.2
Professor Card errs in failing to include interaction terms. ..........................19
3.3 Professor Card’s models ignore the fact that Asian-American applicants face
a penalty in the personal rating........................................................................22
3.3.1
According to standard statistical practice, my model is considered an
excellent fit of the personal rating ...........................................................23
3.3.2
Because there is compelling evidence that racial preferences and
ii
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
penalties affect the personal rating, this rating should not be included
in the analysis. ..........................................................................................25
3.4 Professor Card’s argument that Asian-American applicants are worse on
nonacademic measures is misleading. ..............................................................27
3.5 Parental occupation varies in highly unusual and unexplained ways over
time, undermining its reliability as a variable and its usefulness as a
control.................................................................................................................31
4 Professor Card’s models, once corrected of their key flaws, show that Harvard
imposes a penalty against Asian-American applicants. ..........................................33
4.1 Professor Card’s preferred yearly model is less sound than a pooled
approach. ............................................................................................................34
4.2 Professor Card’s pooled results, with small corrections, show that Harvard
imposes a penalty against Asian-American applicants. ..................................36
4.3 Professor Card’s yearly models, with small corrections, confirm that Harvard
imposes a penalty against Asian-American applicants. ..................................39
4.4 Professor Card’s analysis of applicants whose race is missing further confirms
the existence of an Asian-American penalty ....................................................44
5 Professor Card’s Analysis Actually Demonstrates That Race Is a Determinative
Factor in Harvard’s Admissions Decisions. .............................................................45
5.1 A model with race as the only control would be expected to perform poorly
relative to other factors. ....................................................................................48
5.2 Professor Card’s argument that racial preferences are not relevant for most
African-American and Hispanic applicants misleadingly focuses on
uncompetitive applicants. .................................................................................49
5.3 Professor Card’s method of calculating the importance of unobserved factors
is incorrect and substantially overstates their importance. ............................51
iii
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
6 Professor Card Fails to Refute the Overwhelming Statistical Evidence of a Floor
for African-American Admissions. ...........................................................................54
6.1 Professor Card’s speculation that Harvard would not want to use a floor
based on a non-public admissions rate misses the point. ................................56
6.2 Contrary to Professor Card’s arguments, there is additional evidence that
Harvard began implementing the floor in 2017. ..............................................57
6.3 Professor Card’s analysis of other data does nothing to undermine my claim
that Harvard maintained a floor on the admission rate for single-race
African-American applicants. ...........................................................................58
6.4 Differences in the characteristics of admitted single-race African Americans
after 2016 further support evidence of a floor. .................................................59
7 A Number of the Other Variables Added by Professor Card Are of Questionable
Reliability and Undermine the Confidence of His Conclusions. .............................61
7.1 Intended career varies in highly unusual and unexplained ways over time,
undermining its reliability as a variable and its usefulness as a control. ......62
7.2 Professor Card’s approach to using the rating variables suffers from a smallpopulation problem and masks racial preferences, which undermine its
reliability............................................................................................................63
7.3 Staff interviews are selectively given and thus should not be used as a
control.................................................................................................................66
8 Incorporating Most of Professor Card’s Variables Into My Preferred Model
Confirms My Findings Regarding the Effect of Harvard’s Racial Penalties and
Preferences. ...............................................................................................................67
8.1 Changes advocated by Professor Card that I incorporate in my updated
model ..................................................................................................................68
8.2 The results of the updated preferred model confirm my previous findings and
iv
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
conclusions .........................................................................................................70
8.3 Even incorporating many of Professor Card’s manifestly unsound modeling
choices does not alter the result of my model. ..................................................71
9 My Updated Preferred Model Yields Additional Reasons to Doubt Professor Card’s
Approach....................................................................................................................73
9.1
The penalties Asian-American applicants face are substantial ....................74
9.2 Estimates of my admissions and personal ratings models show that AsianAmerican applicants are strong on non-academic measures. ..........................75
9.3
Dockets with high shares of Asian-American applicants are penalized .......77
v
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
1
Executive Summary
In my opening report, I explained my professional and academic background in
econometrics and my prior scholarly work about the use of race/ethnicity in the
admissions processes of colleges and universities. My report explained how, using
my experience and expertise, I reviewed and analyzed six-years of admissions data
obtained from Harvard College and built a model to test the effect that
race/ethnicity has in the admissions process. The model included and controlled for
more than 200 variables from Harvard’s admissions data, and was constructed
using standard techniques used in my field for statistical modeling. It produced a
number of reliable conclusions about the way an applicant’s race/ethnicity affects
his or her admissions prospects at Harvard. Most importantly, I found:
•
Asian-American applicants as a whole are stronger on many objective
measures than any other racial/ethnic group, including test scores, academic
achievement, and extracurricular activities.
•
Harvard penalizes Asian-American applicants (relative to white applicants)
in the scoring of applicants for admission, particularly in the personal and
overall ratings assigned by Harvard’s admissions officers.
•
Harvard also penalizes Asian-American applicants (again, relative to white
applicants) in the selection of applicants for admission.
•
Race/ethnicity plays a significant role in admissions decisions. In addition to
the racial penalties that Harvard imposes on Asian-American applicants,
Harvard affords substantial racial preferences to Hispanic and AfricanAmerican applicants. The combined effect of the penalties and preferences is
of such great magnitude that, for example, a male non-disadvantaged AsianAmerican student with characteristics that would suggest a 25% probability
of admission would see those chances rise to 95% if he were treated as an
African American.
•
Since the admissions cycle for the class of 2017, the admit rate for those
applicants who identify as African American using the federal IPEDS
(Integrated Postsecondary Education Data System) methodology, i.e., single-
1
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
race African Americans, is almost identical to the admit rate of all other
domestic applicants. The probability of this occurring without direct
manipulation is less than 0.2%.
•
Many of my conclusions are consistent with analyses performed by Harvard’s
own Office of Institutional Research (OIR), including my conclusions about
(1) the relative strength of Asian-American applicants, (2) Harvard’s
discrimination against Asian Americans in the personal ratings, (3) the
penalty Harvard imposes on Asian-American applicants, and (4) the
disproportionate role race plays with respect to Hispanic and AfricanAmerican applicants.
In his report on behalf of Harvard, Professor David Card generally agrees that the
logit model I used is an appropriate way to analyze the effect of race/ethnicity in
Harvard’s admissions process. He does not dispute the objective indicia of quality
regarding the strong qualifications of Asian-American applicants, particularly in
academic achievements. Nor does he dispute that my findings are consistent with
those of OIR.
Professor Card, however, makes a number of modeling choices that lead him to
reach different conclusions than mine: in particular, he contends that the evidence
of a penalty against Asian-American applicants is not compelling. Professor Card
argues that the effect of race on admissions is smaller than I report—although he
still concedes that the use of race substantially increases the admissions prospects
of Hispanic and African-American applicants. Notably, he never challenges the
overwhelming statistical evidence that Harvard has imposed a minimum floor for
the admission of African-American applicants. Instead, Professor Card simply
speculates that there is no reason for Harvard to do so.
None of Professor Card’s arguments are persuasive. His modeling choices are
inconsistent with standard econometric practices and appear designed to understate
the effect of race in the admissions process generally, and on Asian-American
applicants specifically. Moreover, his modeling is not robust—with small
adjustments to his models to correct his methodological flaws, his models actually
2
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
confirm my findings and bolster the conclusion that Harvard imposes penalties on
Asian-American applicants.
Among the key flaws in Professor Card’s approach:
Professor Card’s models are distorted by his inclusion of applicants for
whom there is no reason to believe race plays any role.
As my opening report noted, there are several categories of applicants to whom
Harvard extends preferences for reasons other than race: recruited athletes,
children of faculty and staff, those who are on the Dean’s List or Director’s List
Redacted
Redacted
, legacies, and those
who apply for early admission.1 Because of the significant advantage that each of
these categories confers on applicants, my report analyzed the effect of race on an
applicant pool without these special categories of applicants (the baseline dataset),
which allowed me to test for the effect of race on the bulk of the applicant pool that
did not fall into one of these categories.2
Professor Card, however, includes all of these applicants in his model, taking the
remarkable position that there is no penalty against Asian-American applicants
unless Harvard imposes a penalty on every Asian-American applicant. But this is an
untenable position. I do not assert that Harvard uses race to penalize AsianAmerican applicants who are recruited athletes, children of donors (or others
identified on the Dean’s List), legacies, or other preferred categories. By including
these special recruiting categories in his models, Professor Card obscures the extent
to which race is affecting admissions decisions for all other applicants.
Giving preferences for early action is consistent with the yield rate being higher for early
action applicants. However, unlike the other special recruiting categories, the penalty
against Asian-American applicants who apply early action is similar to the penalty for
those who apply regular action.
1
I also analyzed a dataset that included the special categories of applicants (the expanded
dataset). I included in this dataset interactions for some of the special categories and race,
allowing for the possibility that racial preferences may operate differently for these special
recruiting categories.
2
3
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
Professor Card further exacerbates this problem by including in his calculations the
large majority of applicants whose characteristics guarantee rejection regardless of
their race. Harvard admits a tiny fraction of applicants – only five or six percent in
recent years. This means that a huge proportion of applicants have no realistic
chance of admission. If an applicant has no chance of admission, regardless of his
race, then Harvard obviously does not “discriminate” based on race in rejecting that
applicant. Professor Card uses this obvious fact to assert that Harvard does not
consider race at all in most of its admissions decisions. Further, he constructs his
models in ways that give great weight to these applicants, again watering down the
effect of race in Harvard’s decisions where it clearly does matter. (To put it in
simple terms, it is akin to reducing the value of a fraction by substantially
increasing the size of its denominator.)
Professor Card removes interaction terms, which has the effect of
understating the penalty Harvard imposes on Asian-American applicants.
As Professor Card notes, his model differs from mine in that he removes the
interaction terms. An interaction term allows the effects of a particular factor to
vary with another distinct factor. In the context of racial discrimination, interaction
terms are especially helpful (and often necessary) in revealing where certain factors
operate differently for subgroups within a particular racial or ethnic group. For
example, if a law firm singled out African-American women for discriminatory
treatment but treated African-American males and other women fairly, a regression
model would probably not pick up the discrimination unless it included an
interaction between African-American and female.
Professor Card rightly recognizes that interaction terms should be included in a
model when there is evidence that racial preferences operate differently for
particular groups of applicants; yet he nonetheless removes interaction terms for
variables that satisfy this condition. The most egregious instance of this is Professor
Card’s decision not to interact race with disadvantaged status—even though the
data clearly indicate that Harvard treats disadvantaged students differently by
race.
4
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
Harvard gives a preference to disadvantaged applicants. But as I demonstrated in
my opening report, the preference Harvard gives African-American and Hispanic
applicants for disadvantaged status is much smaller than that given to AsianAmerican and white applicants (Hispanic applicants receive a modest preference for
disadvantaged status, and African-American applicants receive no preference for
disadvantaged status in the selection of applicants for admission). Arcidiacono
Report 8, 64. The interaction term for race and disadvantage allows one to capture
those distinctions. Without it, the size of the preference Harvard gives to
disadvantaged Asian-American and white applicants is muted by the inclusion of
African-American and Hispanic applicants. Since Asian-American applicants are
more likely to be disadvantaged than white applicants, the practical implication of
this is an understatement of the Asian-American penalty.
Professor Card includes the personal rating in many of his analyses,
despite clear evidence that this rating is affected by racial preferences.
Professor Card includes Harvard’s personal rating in his models—notwithstanding
the clear finding yielded by my analysis (and that of OIR) that this rating shows
strong evidence of racial bias. Professor Card contends that my model showing
racial bias in the personal rating is a poor statistical fit, but that is demonstrably
wrong. According to academic works discussing this measure of fit, my model
achieves an “excellent” fit. And Professor Card ignores other indicators of racial
penalties and preferences in the personal rating (such as the substantial
preferences
given
to
African-American
and
Hispanic
applicants),
instead
assuming—against the evidence and the uniform testimony of Harvard’s admissions
officers—that Asian-American applicants as a group are weaker on unobserved
personal qualities.
Professor Card commits other analytical errors that raise doubts about the
reliability of his results.
•
Professor Card claims that Asian-American applicants are weaker on nonacademic measures. In an attempt to support this claim, however, he distorts
the data in two ways. First, he includes legacies, recruited athletes, children
of faculty and staff, and those on the Dean’s/Director’s List in his analysis,
essentially crediting these applicants as having non-academic achievements.
5
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
But their higher admission rates are because they are members of these
specially recruited groups, not because of their non-academic qualifications.
Of course, the reason Professor Card includes them in his analysis is because
Asian-American applicants are underrepresented in these categories, as
compared to the overall applicant pool. This distorts the analysis in a way
that allows Professor Card to make the non-academic qualifications of AsianAmerican applicants appear lower than they actually are. His inclusion of the
personal rating further distorts his results; the racial bias in this rating
artificially holds down the non-academic qualifications of Asian-American
applicants and, at the same time, artificially boosts the non-academic
qualifications of African-American and Hispanic applicants. Removing the
personal rating from Professor Card’s model shows (as does my model) that
Asian-American applicants are at least as strong as white applicants on nonacademic measures, and much stronger on academic measures.
•
Professor Card’s results are heavily influenced by his inclusion of “parental
occupation” (i.e., the occupations of an applicant’s parents) as a control
variable. First, the data produced by Harvard for this field oscillates wildly
from year-to-year, rendering the data unreliable and any results using it
suspect. Second, there is no evidence in the records that Harvard’s
admissions office considers parental occupation important aside from its
value as a measure of SES, which I already control for in my models.
•
Professor Card also uses intended career as a control, even though this
variable suffers from the same kind of inaccuracies as parental occupation.
•
Professor Card also includes the staff interview rating variable. But staff
interviews are offered only to a very small portion of the pool (2.2% of
Professor Card’s dataset); they disproportionately include applicants who fall
within the special recruiting categories (recruited athlete, legacy, etc.); and
those who receive an interview are admitted at a very high rate (roughly
50%). Moreover, the probability of getting a staff interview is much lower for
Asian-American applicants than others, in part because these interviews are
disproportionately given to recruited athletes and legacies. Because staff
6
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
interviews appear to be given on the basis of these other preferences, it is
inappropriate as a control variable.
•
In his yearly analysis, Professor Card also adds controls for reported
extracurricular activities in a way seemingly designed to distort the
extracurricular variable and thereby disfavor Asian-American applicants. He
combines 29 reported categories of extracurricular activities into 12, in a
somewhat arbitrary fashion. He then adds a control for the number of hours
an applicant spends on “work” (i.e., a job). This choice, which ignores the bulk
of the data provided by applicants on the hours they spend on non-academic
activities, seems calculated to disfavor Asian-American applicants. Although
“work” is only the eighth-most popular non-academic activity listed by white
applicants, it is one of the few activities for which they report higher average
hours than Asian-American applicants.
Making small corrections to Professor Card’s own models results in the
finding of a penalty against Asian-American applicants.
Professor Card’s models show significant penalties against Asian-American
applicants once corrective adjustments are made to remedy his various errors. As
stated above, there is no evidence of a penalty against Asian-American applicants
who are in one of the special recruiting categories so I remove applicants in those
categories from Professor Card’s model. For Professor Card’s pooled analysis,
making this one correction plus implementing any one of the following changes
results in a statistically significant penalty against Asian-American applicants: (i)
recognizing the fact that preferences for disadvantaged status vary with race and
therefore interacting race with disadvantaged status; (ii) recognizing that the
personal quality measure includes racial preference and therefore should not be
included in the model; or (iii) recognizing that the parental occupation variables are
unreliable and removing them from the analysis.
Professor Card’s yearly models also show significant Asian-American penalties
when small corrections are made. Once special recruiting categories are removed
from his models, either removing the personal rating or the parental occupation
variables yields evidence of an Asian-American penalty. When the extracurricular
7
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
measures are also corrected, then interacting race with disadvantaged status is
enough to reveal a statistically significant penalty.3
Even Professor Card’s flawed models suggest substantial racial preferences
for Hispanics and African-Americans—preferences that increase once
corrective adjustments are made.
Even Professor Card’s analysis—with all of its flaws—confirms my opening report’s
finding that race plays a “significant role in admissions decisions at Harvard.”
Arcidiacono Report 7-8. Indeed, without making any adjustments to his approach,
his models show that racial preferences are responsible for tripling the number of
African-American admits and doubling the number of Hispanic admits. Professor
Card attempts to explain away these effects, but these efforts can be easily shown to
be both incorrect and very misleading; indeed, his arguments often prove the exact
opposite of his conclusions.
Adopting many of Professor Card’s variables into my models further
confirms my initial findings.
As I have explained, making small corrective adjustments to Professor Card’s
methodology yields results that actually confirm my findings and bolster the
conclusion that Harvard applies racial penalties against Asian-American applicants
and affords large racial preferences to Hispanic and African-American applicants.
On top of supporting my case, this proves the fragility of Professor Card’s models.
For the reasons I’ve described, Professor Card’s approach is flawed among many
dimensions, and appears designed, in many ways, to conceal the effect of Harvard’s
admissions process on Asian-American applicants.
My models, on the other hand, are robust. Indeed, adding many of the new variables
suggested by Professor Card does not materially change my results. My updated
models find that the size of the penalty on Asian-American applicants, and the size
When I refer to statistical significance, I am referring to whether we can be 95% certain
that the measured effect is different from zero. Even without the corrections to the
extracurricular activities, the estimated penalty is statistically significant at the 90% level
in Professor Card’s yearly models when special recruiting categories are removed and race
is interacted with disadvantaged status.
3
8
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
of the preferences for African-American and Hispanic applicants, are just as high if
not higher.
Professor Card offers no analysis to contradict my finding that Harvard
has imposed a floor for admissions of those identifying as African-American
via IPEDS.
In my opening report, I showed that Harvard maintained a floor on the admission
rate for single-race African Americans (as identified by IPEDS) in the classes of
2017, 2018, and 2019. In each of these years, the admit rate of single-race African
Americans was virtually identical to the admit rate of all other domestic applicants.
The chance of this match occurring in three consecutive years (without direct
manipulation) is less than two-tenths of one percent—making it a near certainty
that Harvard was purposely setting a floor on the admission rate of those
applicants.
Professor Card does not challenge that finding. Instead, he speculates that Harvard
had no reason to use a non-public admission rate as a floor, no reason to institute
the floor beginning with the class of 2017, and that Harvard has not set a floor
under other metrics.
None of these responses is persuasive. Why Harvard chose to set a floor and why it
did so in 2017 are not at all relevant to my analysis. But even if they were, there are
a number of reasons in the record that would explain why Harvard would want to
use the IPEDS metric as a floor, and why it did so beginning in 2017. Indeed,
numerous pieces of evidence confirm that Harvard was very concerned about
criticisms tied to its IPEDS data at the precise time the first evidence of the floor
appears in the data. And the fact that Harvard chose to implement this floor, and
not a floor based on another metric, does not change anything. What is certain—and
undisputed—is that Harvard was purposely taking steps to ensure that the
admission rate of single-race African-American applicants approximated or
exceeded the overall admission rate of all other domestic applicants.
*
*
9
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
*
Professor Card’s report changes none of my conclusions; to the contrary, given how
easy it is to alter the results of his models and that my own models report the same
results even incorporating a number of his controls, my opinions in this case have
only been strengthened: Harvard penalizes Asian-American applicants; Harvard
imposes heavy racial preferences in favor of Hispanic and African-American
applicants; and Harvard has been manipulating its admission of single-race
African-American applicants to ensure their admission rate approximates or
exceeds the overall admission rate. Professor Card has demonstrated that it is
possible to mask the true effects of race in Harvard’s admission process by changing
the scope of the analysis in incorrect ways and choosing inappropriate combinations
of control variables. But Professor Card cannot reach these results by applying
accepted statistical methods and treating the data fairly.
10
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
2
Professor Card and I agree on many aspects of my methodology,
analysis, and conclusions
Although the bulk of this report will respond to and rebut criticisms of my work
that Professor Card sets forth in his report, it is useful to note that there are
substantial areas of agreement between the two of us.4
2.1 Professor Card and I largely agree on the relevant dataset
As discussed in my opening report, I reached my conclusions using a dataset
containing Harvard admissions data for the 2014 through 2019 admission cycles. I
then performed two general categories of analysis: (1) descriptive analysis, in which
I drew conclusions based on simple calculations from my dataset; and (2) regression
analysis, in which I used statistical models to estimate how various factors
influence Harvard’s admissions decisions and rating of the applicants.
Professor Card’s analysis modifies my dataset to create a dataset he calls
“Augmented Arcidiacono Data.” Specifically, he creates his dataset by adding
additional control variables to my dataset and then performing what he describes as
“technical corrections” and fixing what he describes as “technical errors.” Card
Report 47-51.
Several of the additional variables that Professor Card adds are problematic in
terms of relevance and reliability, as I explain infra, at 3.5, 7. Beyond that, except
for one “technical error” with which I agree,5 the rest of Card’s modifications are not
“errors” or “corrections,” but merely judgment calls. Because his modifications are
so minor, I have accepted the majority of them in the interest of avoiding
unnecessary disputes.
In formulating my rebuttal report, I have not relied upon any data or material other than
the material produced with Professor Card’s report, the material cited in this report, and
the data and materials identified in my opening report.
4
When the SAT score is not present but an ACT score is present, I now use the ACT science
section in my conversions in the same manner as Professor Card.
5
11
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
2.2 Professor Card does not challenge my descriptive analysis about the
relative qualifications of students by race/ethnicity.
My descriptive analysis is contained primarily in Section 3.1.1 through Section 3.5.4
of my opening report. See Arcidiacono Report 24-53. Professor Card does not
challenge the accuracy of any of this analysis. This is not surprising, because he has
no substantive concerns with my dataset and the descriptive analysis involves
straightforward assessments of the relevant data. Among the specific findings from
the descriptive analysis that Professor Card does not dispute:
2.2.1 Asian-American applicants are, on average, significantly stronger
academically than all other racial groups.
In terms of academic performance, Asian-American applicants are significantly
stronger than all other racial groups. Asian-American applicants have (1) the
highest test scores; (2) the highest high school GPAs; (3) taken more AP exams; and
(4) scored higher on those AP exams than any other racial group. Arcidiacono
Report 33.
Asian-American applicants also are rated higher on Harvard’s metrics for assessing
academic performance than all other racial groups. In particular, Asian-American
applicants’ academic indexes and academic ratings are higher than all other racial
groups.6 For example, in the baseline dataset,7 58.6% of Asian-American applicants
have academic ratings of 3+ or higher, compared with 44.7% of whites, 14.7% of
Hispanics, and 7.3% of African Americans. Arcidiacono Report 33, 36-37.
The “academic index” is a score derived from a formula combining standardized testing
and high-school performance. The “academic rating” is a rating assigned by Harvard
readers.
6
The “baseline” dataset includes all domestic applicants minus certain applicants whose
characteristics were associated with a preference (e.g., legacy, athlete), and the “expanded”
dataset include all domestic applicants. For both datasets, I removed a small number of
applicants who were missing certain information from their application (e.g., test scores).
Arcidiacono Report 2.
7
12
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
2.2.2 Asian-American applicants are so strong academically that their
admission rates would more than double in the baseline dataset if
based on academics alone.
If a random lottery were conducted conditional on being in the top N academic index
deciles, the share of Asian-American admits would rise significantly. For example,
randomly drawing from all those in the top nine academic index deciles would
increase the share of Asian-American admits from 24.9% to 30.4% in the baseline
dataset, a jump of more than 22%. More dramatically, randomly drawing from the
top academic index decile (in the baseline dataset) would cause Asian-American
admits to more than double—resulting in more than 51% of the admitted class
being Asian American. Arcidiacono Report 41-42, 44-45.
But even if the number of admits from all other groups besides whites and Asian
Americans were held fixed and admits for whites and Asian Americans were
randomly drawn from the top decile, the share of the class that was Asian American
would still substantially increase, resulting in an Asian-American admitted share of
36.5%, a 47% increase. Arcidiacono Report 45. This occurs because Asian-American
applicants dominate white applicants in their respective shares of the top academic
decile.
2.2.3 Asian-American applicants are strong in non-academic categories.
Asian-American applicants excel in more than academics. They also have higher
extracurricular ratings and overall alumni ratings than any other racial group.
Asian-American applicants likewise are stronger than African-American and
Hispanic applicants on counselor ratings, teacher 1 ratings, teacher 2 ratings, and
alumni personal ratings, and have similar or slightly lower ratings than whites in
these categories. Arcidiacono Report 37.
2.2.4 Despite their high academic and non-academic ratings, AsianAmerican applicants have lower scores in the subjective personal
rating than all other racial groups.
Despite their superiority on more objective factors, Asian-American applicants have
the lowest scores of the four major racial groups on Harvard’s personal rating—the
13
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
most subjective of all the ratings.8 These low scores on the personal rating are
outliers in several respects.
First, they differ significantly from the scores Asian-American applicants receive
from other individuals, including the ratings from alumni interviewers, teachers,
and counselors. For example, alumni interviewers score Asian-American applicants
higher on the personal rating than African-American and Hispanic applicants and
only slightly lower than white applicants. Arcidiacono Report 37-38.
Second, the low scores Asian-American applicants receive on Harvard’s personal
rating do not square with the higher scores Asian-American applicants receive on
other ratings. As I have shown, higher academic indexes are associated with higher
academic ratings, higher extracurricular scores, and higher personal scores. Yet
even
though
Asian-American
applicants
have
the
highest
academic
and
extracurricular scores, they are ranked substantially lower in the personal category
than the other groups in the same academic index decile. For example, AsianAmerican applicants receive a 2 or better on the personal score more than 20% of
the time only in the top academic index decile. By contrast, white applicants receive
a 2 or better on the personal score more than 20% of the time in the top six deciles,
Hispanics receive such personal scores more than 20% of the time in the top seven
deciles, and African Americans receive such scores more than 20% of the time in the
top eight deciles. Arcidiacono Report 48-50 & Table 5.6.
See, e.g., Chen Depo. at 72 (“Personal quality is one of the categories admissions readers
are asked to assess. It is a subjective determination of a combination of many, many
factors.”); Walsh Depo. at 60-61 (The personal rating involves “[w]hether that student
would contribute to the class, classroom, roommate group, to the class as a whole, their
human qualities…. It is a little hard to talk about in general but sort of add it all up and
get a feeling”); McGrath I Depo at 171 (The reading guidelines for rating the personal
category are “not terribly helpful” and “readers will construe [it] in different ways”);
McGrath II Depo at 360 (The personal rating “includes perhaps likability, also character
traits, such as integrity, helpfulness, courage, kindness”).
8
14
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
2.2.5 Despite their high scores on academic and non-academic ratings,
Asian-American applicants are admitted at lower rates than all
other racial groups.
The Asian-American admit rate was below the total admit rate every year from the
Class of 2000 through the Class of 2019. Asian-American applicants had this low
admit rate despite the fact that during this 20-year span they had higher test scores
than all other racial groups in every year. Indeed, Asian-American applicants as a
whole had higher test scores than both African-American and Hispanic admits.
Arcidiacono Report 24-27 & Figure 1.2.
2.2.6 Among applicants with the same overall rating, Asian-American
applicants are less likely to be admitted than all other racial
groups.
Among those applicants with the same overall rating, Asian-American applicants
are less likely to be admitted than any other racial group. For example, in the
baseline dataset, 81.4% of African-American applicants with an overall rating of 2+,
2, or 2- were admitted; 76.0% of Hispanic applicants with this overall rating were
admitted; 61.0% of white applicants with this overall rating were admitted; and
only 59.4% of Asian-American applicants with this overall rating were admitted.
The gap between white and Asian-American applicants is even larger in the
expanded dataset. Arcidiacono Report 39.
Similarly, higher academic index deciles are associated with higher overall ratings
by both Harvard readers and alumni interviewers. Asian-American applicants
receive overall ratings similar to whites who are one decile lower in terms of their
academic indexes. In the top three deciles, Hispanic applicants are between 2.5 and
4.5 times more likely to receive a 2 or better on the overall rating than AsianAmerican applicants, and African-American applicants are between 4.4 and 9.9
times more likely to receive such a score. Arcidiacono Report 50-52.
2.2.7 Higher academic indexes are associated with higher admit rates
and higher reader ratings. Yet regular-decision Asian-American
admit rates lag behind all other racial groups.
Higher academic index deciles are associated with higher admit rates and AsianAmerican applicants have the highest academic indexes. Yet regular-decision
15
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
Asian-American admit rates are lower than all other racial groups. Asian-American
admit rates in any academic-index decile are roughly equivalent to white admit
rates one academic index decile lower, Hispanic admit rates three deciles lower, and
African-American admit rates five deciles lower. Arcidiacono Report 42-44.
2.2.8 My results are consistent with Harvard’s own findings, as
performed by the Office of Institutional Research (OIR).
Using data over ten years, OIR found that Harvard’s admissions officers assigned
substantially lower personal ratings to Asian-American applicants versus white
applicants, especially when compared to the ratings assigned by teachers,
counselors, and alumni interviewers. Arcidiacono Report 38. OIR also found that
had the academic index and academic rating been used to evaluate the applicants,
Asian Americans would have been 43% of the admitted class. These findings are
consistent with my findings. In both my analysis and OIR’s analysis, the number of
Asian-American admits would more than double if admissions were based on these
two criteria. Arcidiacono Report 45-46.
2.3 Professor Card generally agrees with my methodological approach to
modeling Harvard’s admissions decisions
In my opening expert report, I used regression analysis, and in particular logit
models, to draw various conclusions about Harvard’s admissions process and the
way in which admissions decisions are affected by an applicant’s race. My report
describes the basic methodology and approach, as well as the supporting statistical
equations. See Arcidiacono Report 17-23, Appendix A.
Professor Card “agrees with [my] general approach” because “[m]ultivariate
regression analysis is a widely accepted and common statistical technique in both
academia and litigation.” In particular, Card concludes that a logit model like mine
“is appropriate where, as here, the outcome of interest—in this case admission to
Harvard—is binary, taking values of either zero (not admitted) or one (admitted).”
Card Report 47.
Card instead disagrees with my specific “modeling decisions.” Card Report 47. The
nature of that disagreement, and why my analysis remains more appropriate and
reliable, are described in the rest of this report.
16
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
3
There are several key flaws in Professor Card’s modeling choices that
drive his conclusions about the size of the Asian-American penalty.
Professor Card makes several fundamental errors in his approach that bear directly
on his claims that (1) there is no statistically significant penalty against AsianAmericans, and (2) race plays a lesser role in Harvard’s admissions decisions than I
demonstrated in my opening report. These errors explain the difference in our
conclusions.
3.1 Professor Card’s results are skewed by his decision to include in the
analysis many applicants who are unaffected by racial penalties and
preferences.
In my opening report, I employed accepted statistical methods to demonstrate that
Harvard applies racial penalties and preferences to various racial/ethnic groups.
More particularly, I demonstrated that Harvard applies these penalties and
preferences where they matter—within the band of applicants who are competitive
for admission.
Professor Card’s models operate to conceal these racial penalties and preferences by
diminishing their magnitude. One of the principal ways Professor Card’s models do
so is by his inclusion of applicants who are not impacted by Harvard’s racial
penalties and preferences. The inclusion of such applicants has the practical effect
of making these penalties and preferences appear to be of smaller magnitude than
they actually are. (To put it in simple terms, it is akin to reducing the value of a
fraction by substantially increasing the size of its denominator.)
3.1.1 Professor Card misleadingly includes non-competitive applicants in
his models, which tends to obscure the racial penalties and
preferences Harvard employs in its admissions process.
Harvard is a highly selective school. More than 90% of all domestic applicants were
rejected over this period, and a substantial number of them are not at all
competitive for admission. Those that are affected by racial preferences are
competitive applicants. In my report, I showed that Asian-American applicants who
had particular characteristics would see substantially higher probabilities of
admission were it not for their race. For example, I showed that a male AsianAmerican applicant who was not disadvantaged with observed characteristics that
17
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
would dictate a 25% probability of admission would see his probability of admission
rise to over 36% if treated as a white applicant and to over 95% if treated as an
African-American applicant.
Professor Card argues that I am distorting the picture by examining the effects for
competitive applicants. Professor Card’s approach, however, seeks to dilute the
estimates of preferences by including many applicants whose characteristics are
such that rejection is assured.
Both Professor Card’s and my models show that there is a set of observed
characteristics that guarantee rejection; the models perfectly predict rejection
without the use of race. While arguments can be made regarding the scope over the
set of applicants where one should test for racial penalties and preference, it should
be quite clear that this set should not include those who are sufficiently below the
bar that race could not possibly enter into consideration. By including applicants
who are perfect rejects in his models, Professor Card is able to artificially hold down
the average marginal effect of race with respect to any particular racial group.
Professor Card’s insistence on including perfect predictions in his model implies
that he believes Harvard’s discrimination against certain racial groups and in favor
of others is of no consequence unless Harvard actually discriminates against or in
favor of every applicant within the affected racial/ethnic groups. This is an absurd
proposition. It is a given that Harvard’s low admittance rate means a large number
of applicants will be denied without their race ever becoming a factor. But that does
not exonerate Harvard for its use of race among the competitive pool. “We don’t
always engage in racial discrimination” is not a defense.
A conservative position would be to focus the testing for racial preferences or
penalties on all of those applicants who are not immediately ruled out—which
would mean removing perfect predictions. Under this approach, there will be many
applicants who will be included in the analysis even though their admission chances
are miniscule, and for whom any effect of racial preferences and penalties will
necessarily be small. Throughout my response to Professor Card’s points, I take this
conservative approach, showing the average marginal effects of race for all those
who are not perfectly predicted to be rejected.
18
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
3.1.2 Professor Card errs by including in all of his models those
applicants who are members of Harvard’s special recruiting
categories
Professor Card makes a similar modeling error by always including recruited
athletes, children of faculty and staff, applicants who are on the Dean’s List or
Director’s List, and legacies in his models. Harvard acknowledges that it affords
significant preferences to applicants in these special recruiting categories. By
including these special recruiting categories in his models, Professor Card is able to
obscure the extent to which race is affecting admissions decisions for those not
fortunate enough to belong to one of these groups.
The inclusion of applicants in these special categories specifically tends to obscure
the penalty Harvard imposes on Asian-American applicants. Professor Card’s
inclusion of these applicants reflects his position that there is no penalty against
Asian-American applicants unless Harvard imposes a penalty on every AsianAmerican applicant. But I am not claiming, for example, that Harvard penalizes
recruited athletes who are Asian-American because of their race. My claim is that
the effects of Harvard’s use of race occur outside these special categories. There is
no reason for their inclusion in his models (at least without interactions with race)
other than to conceal the extent to which Harvard penalizes Asian-American
applicants in the admissions process.
3.2 Professor Card errs in failing to include interaction terms.
In Section 5.1.1 of his report, Professor Card argues against the inclusion of
interactions in my models. In discussing the interaction terms between race and
disadvantaged status, Professor Card writes:
The typical approach in a model trying to isolate the effect of AsianAmerican ethnicity on admissions outcomes would be to include an
interaction between race and disadvantaged status only if the effect of
being disadvantaged is different for Asian-American and white
applicants (or, equivalently, if the effect of race is different for
disadvantaged and non-disadvantaged applicants). Prof. Arcidiacono’s
results, however, show that is not the case.
19
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
Card Report 49.
As I discuss in sections 8.1 and 8.3, I believe the various interaction terms I include
are all appropriate. But the one that deserves special attention here—because it is
key to Professor Card’s finding of no Asian-American penalty—is his removal of the
interaction terms between race and disadvantaged status. Here, I show why it is
inappropriate to exclude these interaction terms; in later sections, I show how their
inclusion undermines the reliability of Professor Card’s findings.
First, Professor Card is correct that an interaction between race and disadvantaged
status makes sense when disadvantage has a different effect for different races. But
his analysis becomes misleading when he suggests that the relevant races are only
whites and Asian Americans. Understanding Harvard’s use of race in evaluating
domestic applicants involves distinctions drawn across all four major racial groups
in the applicant pool: Asian Americans, whites, African Americans, and Hispanics.
Indeed, Professor Card does not exclude these groups from his models. One of the
major findings in my report is that although Harvard gives African-American
applicants a large preference, it does not give disadvantaged African-American
students any preference for being disadvantaged. Thus, the effect of being
disadvantaged is different across racial lines—precisely the condition that Professor
Card acknowledges would warrant inclusion of the race/disadvantage interaction
terms. So long as African Americans are used in the estimation of the model, the
model requires these interaction terms. Yet Professor Card does not include them in
his model.
This is a relatively basic point; it is odd that Professor Card misses it. But perhaps
the effect of his excluding these interaction terms from his models explains this. By
excluding the interaction terms between race and disadvantaged status but keeping
African-American applicants in the model, Professor Card significantly weakens the
effect of disadvantage as an explanatory term. His regression model is essentially
finding that disadvantaged status is a fuzzier phenomenon than it actually is and
thus downgrades its role in the admissions process. And, because more AsianAmerican applicants than white applicants are disadvantaged, the weaker effect of
disadvantaged status in his model in turn weakens the distinctions between white
20
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
and Asian-American applicants, thus tending to conceal the magnitude of
discrimination against Asian-American applicants.
It follows that if one compares only Asian-American and white applicants—and
excludes
the
other
races
from
the
analysis
entirely—then
whether
the
disadvantage/race interaction is included is less important. As I illustrate in section
4.2, estimating Professor Card’s models using only white and Asian-American
applicants yields similar penalties against Asian Americans as a model that
includes all races, but interacts race with disadvantaged status. And in both cases,
the penalties are substantially larger than when the same model is estimated using
all races but the interaction terms are removed.
Relatedly, this is why I include interactions of race and disadvantaged status in my
models of Harvard’s ratings. If interactions are important for one racial group, then
they need to be included any time that racial group is included in the analysis.
These interaction terms are helpful in diagnosing the extent to which racial
preferences affect Harvard’s ratings. Professor Card concedes that the overall rating
(and the ultimate admissions decision itself) are affected by race/ethnicity for
African-American and Hispanic applicants. Card Report 51, 81. Both the overall
rating and the admissions decision show substantial preferences for AfricanAmerican applicants, and smaller preferences for disadvantaged status. But both of
these measures also show that African-American applicants either receive a
diminished preference for being disadvantaged (in the ratings) or no preference at
all (in the admissions outcome).9
Similarly, including interaction terms between race and disadvantaged status in
the model more accurately captures the extent to which Harvard’s personal ratings
are affected by racial bias. As I described in my opening report, African-American
applicants receive a larger preference through the personal rating and a smaller
preference for disadvantaged status than other racial groups. That the same
Statistically, this is demonstrated by the fact that the coefficient on the interaction term
between African American and disadvantaged is negative, and either of the same
magnitude or slightly smaller than the positive coefficient on disadvantage itself.
9
21
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
pattern occurs for the overall rating—and is not present in the other ratings
models—provides additional evidence that racial preferences impact the personal
rating. Because racial preferences impact the personal rating, that rating should
not be used in the analysis—a point that Professor Card must concede, given his
own exclusion of the overall rating because it is impacted by racial preferences.
3.3 Professor Card’s models ignore the fact that Asian-American
applicants face a penalty in the personal rating.
My opening report described how the personal rating assigned to applicants by
Harvard’s admissions officers showed clear evidence of racial preferences. Despite
their general strength overall, Asian-American applicants have the lowest share of
1s or 2s (the best ratings) on the personal scores. And while academic qualifications
are generally correlated with higher personal ratings, Asian-American applicants
received lower personal ratings than white applicants despite having better
academic and extracurricular ratings. And similarly situated African-American
applicants receive much higher personal ratings than their Asian-American
counterparts. African-American applicants in the third-worst decile receive higher
personal ratings than Asian-American applicants in the top decile. See Arcidiacono
Report 5-6, 53-61, Table 5.6.
In his report, Professor Card objects to my model that demonstrates a penalty
against Asian-American applicants (compared to whites) and a preference in favor
of African-American and Hispanic applicants. Specifically, he claims that my model
of the personal rating fits the data poorly.
Professor Card’s criticisms are misplaced; the personal rating model I rely upon fits
the data quite well and I show it is within the range of what is considered to be an
“excellent fit.” And the model’s conclusion of a penalty against Asian-American
applicants is unmistakable. Moreover, this conclusion is consistent with similar
findings by Harvard’s own Office of Institutional Research (which Professor Card
does not address). See HARV00065745. And Professor Card estimates no ratings
models of his own to counter my findings and conclusions on this point. Finally, as I
show in section 8.3, adding his additional variables has no effect on my findings of
racial preferences and penalties in the personal rating.
22
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
3.3.1 According to standard statistical practice, my model is considered
an excellent fit of the personal rating
The classic citation for what is considered an “excellent fit” based on the Pseudo Rsquare is McFadden (1979) page 307:
Those unfamiliar with the ρ2 index should be forewarned that its
values tend to be considerably lower than those of the R2 index and
should not be judged by the standards for a ‘good fit’ in ordinary
regression analysis. For example, values of 0.2 to 0.4 for ρ2 represent
an excellent fit.
D. McFadden, “Quantitative Methods for Analysing Travel Behavior: Some Recent
Developments,” Chapter 13 in Behavioral Travel Modeling, D.A. Hensher and P.R.
Stopher, editors, Croom Helm Ltd., 1979.
The ρ2 referred to above later became known as McFadden’s R-Square, or the
Pseudo R-square that I use in my analysis. Note that the value Professor Card
criticizes as “unreliable”—0.28—is within the range characterizing an “excellent”
fit.10
Professor Card further attempts to characterize my model as having a “poor” fit by
using the probability in my model to predict applicant’s personal ratings. He assigns
each applicant the rating that has the highest probability for that applicant, and
then assesses the percent correctly predicted for those who actually received a 1.
Applying this “percent correctly predicted” method to my model assigns zero
applicants a rating of 1. Professor Card criticizes my model for failing to predict a 1
for any of the 47 applicants out of 150,643 (or 0.03%) who actually received a 1 on
the personal rating.
This attack is nonsensical. It is absurd for Professor Card to claim that a failure to
predict the correct personal rating for 0.03% of applicants is evidence of a “poor” fit.
Professor Card makes a similar criticism of my model of the overall rating—where the fit
is even better and thus (again) well within the range understood to be an “excellent fit.”
Card Report 154; Arcidiacono Report Table B.6.8.
10
23
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
My models assign higher probabilities to ratings that occur more often in the data.
When only 0.03% of applicants receive a 1 on the personal rating, the chances of
that rating having the highest probability for any applicant is miniscule.11 Professor
Card’s use of the percentage correctly predicted method just naturally assigns zero
applicants a personal rating of 1.
To further illustrate the absurdity of Card’s standard for considering whether a
model is a poor fit, consider my model of the academic rating, which he refers to as
“more reliable” because of its higher Pseudo R-square. Card Report 70. There are
674 (out of 150,643) applicants who received a 1 on the academic rating, over 13
times the number of applicants who received a 1 on the personal rating. Yet
Professor Card’s method of using my model to assign ratings to individual
applicants would result in zero applicants being assigned a 1.
Indeed, Professor Card’s focus on the model’s ability to correctly predict individual
outcomes is a common error. The classic textbook on discrete choice model is by
Professor Kenneth Train of the University of California, Berkeley. In discussing the
inappropriateness of using the percent correctly predicted, Professor Train writes:
Another goodness-of-fit statistic that is sometimes used, but should actually
be avoided, is the ‘percent correctly predicted’….
Suppose an estimated model predicts choice probabilities of .75 and .25 in a
two-alternative situation. Those probabilities mean that if 100 people faced
the representative utilities that gave these probabilities (or one person faced
these representative utilities 100 times), the researcher’s best prediction of
how many people would choose each alternative are 75 and 25. However, the
‘percent correctly predicted’ statistic is based on the notion that the best
prediction for each person is the alternative with the highest probability.
This notion would predict that one alternative would be chosen by all 100
As another example, suppose data were available on the height of males in one of five
bins, and the last bin was six feet nine inches and higher, something true for about 0.03% of
men. Given virtually any set of observed characteristics outside of height itself, the
probability associated with this bin will never be the highest.
11
24
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
people while the other alternative would never be chosen. The procedure
misses the point of probabilities, gives obviously inaccurate market shares,
and seems to imply that the researcher has perfect information.
Kenneth E. Train, Discrete Choice Methods with Simulation, 69 (2d ed. 2009).
This issue is of course compounded when the events are extremely rare.
3.3.2 Because there is compelling evidence that racial preferences and
penalties affect the personal rating, this rating should not be
included in the analysis.
As explained in my original report, there is strong evidence that race affects the
personal rating, penalizing applicants who are Asian-American and favoring
African Americans and Hispanics. Professor Card does not dispute that Asian
Americans receive disproportionately lower personal ratings. But he argues that
Asian-American applicants have lower scores on the personal ratings because they
have weaker average unobserved characteristics than white applicants.12
Crucially, Professor Card ignores the clear evidence of bias in the personal ratings
in favor of African Americans and Hispanics. For example, Table 6.1 in my previous
report showed that if African-American applicants were treated as whites their
average probability of receiving a 2 or better would fall by 22%, and would fall by
35% if they were treated as Asian-American applicants. And here, I can readily
It is worth noting that no one in Harvard’s admissions office has advanced Professor
Card’s arguments that Asian-American applicants, as a general matter, have some
unobserved qualities that explain lower personal ratings. Indeed, numerous admissions
officers—including Dean Fitzsimmons himself, who has worked in the admissions office for
more than 30 years and reads files to this day—denied that there was any reason to believe
that Asian-American applicants were less qualified on the “personal” metric than any other
applicant. See, e.g., Fitzsimmons Depo. at 347-348; Ray Depo. at 22; Yong Depo at 234-235;
Hansen Depo. at 110-111. Harvard’s own materials likewise leave it to the subjective
judgment of the reader as to how the score should be assigned. See HARV00021322
(instructing readers to assign the personal rating on the following scale: “1. Outstanding. 2.
Very strong. 3. Generally positive. 4. Bland or somewhat negative or immature. 5.
Questionable personal qualities. 6. Worrisome personal qualities”). If Professor Card has
any support for why Asian-American applicants have weaker “personal qualities” than
other racial groups, he does not provide it.
12
25
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
show that the observed characteristics of African-American and Hispanic applicants
predict much weaker—not stronger—ratings vis-à-vis Asian-American and white
applicants.
By Professor Card’s reasoning, this should demonstrate that the personal rating
incorporates racial preferences. If racial preferences are operating through the
personal rating for these groups, then the personal rating is suspect in the same
way that the overall rating is suspect—indeed, Professor Card concedes that the
overall rating is suspect, and thus excludes from his own analysis. Because the data
make clear that racial preferences do, in fact, affect the personal rating, it is
unreasonable for Professor Card to conclude that the estimated negative effect for
Asian-American applicants is not the result of racial penalties against AsianAmerican applicants.13
Finally, Professor Card makes a number of misleading arguments about what my
ratings models show. First, Professor Card states that my finding of a positive and
significant relationship between Asian-American applicants and academic and
extracurricular activities, even after adding controls, somehow suggests that
Harvard cannot be discriminating against Asian-American applicants on the
personal and overall ratings. Card Report 71. But Asian-American applicants are
stronger than any other racial group on the observed characteristics associated with
higher scores on both these ratings. We would therefore also expect them to be
stronger on unobservable characteristics, providing an explanation for why there is
a statistically significant effect of being Asian American on both these activities.
As explained in my opening report, the case for discrimination is very strong when a
group of applicants is strong on the observed characteristics associated with a
particular rating, yet faces a penalty. And if a group of applicants is weak on the
observed characteristics associated with a particular rating, yet receive a higher
than expected rating, it further supports the conclusion that racial preferences
And here too the interaction terms make the case that racial preferences are affecting the
personal rating. That is, there is a substantial preference for being disadvantaged in the
personal rating that is significantly diminished for African Americans, mirroring the
pattern seen for both the overall rating and for admissions itself.
13
26
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
affect this rating. That is what the data show: Asian-American applicants have
observed characteristics associated with higher personal ratings yet receive a
penalty in their personal ratings, and African-American and Hispanic applicants
have observed characteristics associated with lower personal ratings yet receive a
preference in their personal ratings.
Second, Professor Card argues that as more controls are added, the penalty Asian
Americans face on the personal rating is diminished and, therefore, if even more
controls were added, the effect may go away. But it is not universally true that
adding controls leads to lower estimated penalties for Asian-American applicants.
Indeed, in my previous report, adding all the controls basically resulted in the same
penalty for Asian-American applicants as in the model with no controls, implying
that the order in which the controls are added matters. So the inclusion of over 200
controls as a whole does nothing to reduce the Asian-American penalty. Why would
we expect that the next set of controls would lead to different results?
3.4 Professor Card’s argument that Asian-American applicants are worse
on nonacademic measures is misleading.
In my opening report, I noted that my findings of penalties against Asian-American
applicants were particularly striking because these applicants are the strongest on
observable measures. In particular, they have the highest academic ratings, their
ratings on extracurricular activities were better than white applicants, and they
generally received higher ratings on other dimensions with the exception of the
athletic rating and personal rating. See supra, Section 2.
In response, Professor Card contends that Asian-American applicants are weaker
than white applicants on nonacademic measures. Card Report 39, Exhibit 10. To
arrive at this conclusion, Professor Card uses my estimated model of admissions to
form an admissions index for how strong each applicant is based on observed
characteristics. Professor Card then removes from this admissions index the
variables associated with academics, forming a “non-academic” index. Professor
Card finds that Asian-American applicants are generally worse than white
applicants on this metric; more specifically, he finds that Asian-American
applicants have the lowest share of the four major racial/ethnic groups in the top
27
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
decile of non-academic achievements.
The problem is that Professor Card’s methodology is seriously flawed in two
respects. First, Professor Card errs in using the expanded dataset. That model
includes athletes, legacies, the children of faculty and staff, and the applicants on
the Dean’s and Director’s Lists. This means that every student who receives a
preference in one of these special recruiting categories is given a boost in Professor
Card’s measure of his “non-academic” achievements. But this makes no sense.
White applicants are not stronger than Asian-American applicants on “nonacademic” characteristics because they are more likely to be legacies and therefore
treated preferentially in the admissions process. Since Asian-American applicants
are substantially less likely to be in these special recruiting categories, Professor
Card’s classification works to their detriment. See Arcidiacono Report, Table B.3.2.
As I will show, focusing on those who are not in one of these special categories
(which again is where the Asian-American penalty is implemented) paints a
markedly different picture.
In similar fashion, Professor Card includes the personal rating in his measure of
“non-academic achievement.” But as I have shown in Section 3.3.2, the personal
rating incorporates preferences for African-American and Hispanic applicants and
penalties against Asian-American applicants. 14 Using the personal rating as a
marker for non-academic achievement is thus highly misleading.
In Table 3.1N, I show how each of these features results in a distorted picture of the
strength of Asian-American applicants on non-academic measures.
14
See also Arcidiacono Report 37-38. 48-50.
28
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
Table 3.1N: Asian-American applicants are strong on non-academic
measures besides the personal rating
Panels 2-4 remove special recruiting categories. The non-academic ratings in panel 4 are:
extracurricular, athletic, counselor, teacher1, teacher2, alumni personal, and alumni overall.
The first panel of Table 3.1N shows Professor Card’s results from Exhibit 10. The
second panel reflects Professor Card’s Exhibit 10, while also removing those who
are in one of the special recruiting categories. The third panel does the same and
also removes the personal rating. The fourth panel looks only at the non-academic
ratings assigned by Harvard’s admissions officers or alumni interviewers.
29
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
The third panel shows that Asian-American applicants are just as strong as white
applicants on non-academic measures once the personal rating and special
recruiting categories are removed, and substantially stronger than AfricanAmerican applicants.15 Yet when the personal rating is in the model (second panel),
the share of Asian-American applicants in the top decile of the admissions index is
similar to that of African-American applicants and much lower than white
applicants. This further illustrates that racial preferences influence the personal
rating. Including it significantly improves the relative position of African-American
applicants in Professor Card’s non-academic index. And including the personal
rating substantially weakens the relative position of Asian-American applicants in
Professor Card’s non-academic index, despite the fact that they have higher
academic ratings and similar non-academic qualifications to whites when the
personal rating is not included.
But even this third panel incorporates various forms of preferences. Some of these
will favor Asian-American applicants relative to whites (such as disadvantaged
status) and some will not (such as geography). In the fourth panel, I use the portion
of the admissions index that comes from Harvard’s ratings that are not inherently
academic in nature with the exception of the personal rating (having already shown
that this rating is biased).16 These ratings include the following: extracurricular,
athletic, teacher1, teacher2, counselor, alumni personal, and alumni overall. As
panel 4 shows, Asian-American applicants are just as strong as white applicants on
these non-academic ratings.
Note that the findings in the fourth panel also speak directly to Professor Card’s
selective comparisons of white and Asian-American ratings in section 4.2 of his
report. Here he gives equal weight to Harvard’s four profile ratings and ignores the
other ratings measures (school support and the alumni ratings). First, this
Note that Hispanics do well on this measure, at least in the top admissions decile, as the
remaining “non-academic” factors are also affected by preferences, for example, for
disadvantaged status and geography.
15
These ratings include the following: extracurricular, athletic, teacher1, teacher2,
counselor, alumni personal, and alumni overall.
16
30
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
overweights the athletic rating; in practice, the athletic rating is not as important to
the admissions decision as the other ratings once recruited athletes are removed.
Second, it includes the personal rating, which is affected by racial preferences. In
the fourth panel, the weights associated with each rating measure are determined
by how Harvard values them in the admissions process. Here Asian-American
applicants are just as strong as white applicants on the non-academic measures
and, as shown in the previous report, substantially stronger on the academic
measures.
3.5 Parental occupation varies in highly unusual and unexplained ways
over time, undermining its reliability as a variable and its usefulness
as a control.
There also are substantive issues with Professor Card’s additional variables. His
finding of no statistically significant discrimination against Asian Americans hinges
in part on adding these as controls. See Card Report 62-75. But the unusual
variations among some of these variables raise serious doubts about their
reliability.
Parental occupation is one example. Professor Card aggregates mother’s and
father’s occupations into 24 categories. Table 3.2N shows the number of mothers
and fathers in each of these categories for five of the occupations conditional on
being in Professor Card’s pooled dataset; the rest of the occupations are shown in
Appendix Tables B.3.1N, B.3.2N.
31
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
Table 3.2N: Mother’s and father’s occupations vary in non-credible ways
over time
The yearly variations among these five occupations point toward these variables
either being the result of incorrect mappings across years, or being recorded
incorrectly. No mother or father is listed as self-employed in 2014, yet over 900
mothers and over 2,100 fathers are listed as self-employed in each of the other
years. Over 1,000 of mothers and fathers each are listed as low-skilled in 2014, but
in every other year, no more than 50 mothers and fathers were recorded as lowskilled.17
The problems with this variable are not confined to inconsistencies between 2014
and the other years of data. Consider the unemployed category. In 2018 and 2019
there are 10 or fewer unemployed mothers and fathers in each of the years. From
2015 to 2017, however, the number of unemployed mothers was always above 2,200
and the number of unemployed fathers was at least 1,300.
These inconsistencies raise doubts about the reliability of the field and its
A review of Professor Card’s analysis shows an incorrect translation between some of the
coded occupations and how Professor Card aggregates occupations. For example, on the
summary sheet of applicant 1759088, both the applicant’s mother and father were listed as
“Laborer, unskilled.” Handwritten notes show the occupations as “caregiver” and
“newspaper deliveryman”. Yet Professor Card’s classification scheme results in this
applicant being coded as “Skilled Trades Incl. Construction.” The error for this occupation
appears to be in the mapping Professor Card provides in ca_occupation_to_bls_minorg.xlsx.
17
32
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
usefulness as a control. If there is little reason to trust the accuracy of a factor,
incorporating it into a model will not inform the resulting estimates. Professor Card
nowhere offers an explanation for why these data would vary so wildly across these
years. Nor does he provide a particularly compelling explanation for how parental
occupation categories influence admissions decisions. To the extent there is
testimony about this topic in the record, it suggests that parental occupation is
useful mainly to help identify disadvantaged students 18 —and the model I use
already accounts for any applicants that Harvard identified as disadvantaged. As a
result, I see no reason why the parental occupation Professor Card uses would
increase our understanding about the admissions process at Harvard—let alone
serve as a firm basis for opining that there is no significant discrimination against
Asian-American applicants in Harvard’s admissions process
4
Professor Card’s models, once corrected of their key flaws, show that
Harvard imposes a penalty against Asian-American applicants.
In Sections 3.6 and 3.7 of my opening report, I employed statistical methods to
demonstrate that Harvard imposes a penalty against Asian-American applicants in
both the scoring and selection of applicants for admission. In Section 5 of his report,
Professor Card contends that his analysis reveals no significant penalty against
Asian-American applicants. Card Report 46-80.
In analyzing this question, Professor Card employs two versions of his models—one
that pools the six-years of data and analyzes it as a whole (my preferred approach)
and the other separately analyzes each year (Professor Card’s preferred approach).
18
Redacted
33
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
This is one of the most important modeling differences between us, so I will begin
by explaining why the pooled model is the better approach, before demonstrating
that under either approach, making some basic corrections to the models confirms
that there is a penalty against Asian-American applicants.
4.1 Professor Card’s preferred yearly model is less sound than a pooled
approach.
Professor Card contends that regression models of discrimination in Harvard
admissions should focus on individual years, rather than pooling the six years of
data Harvard has disclosed. This is his key rationale:
First, the admissions process at Harvard is, by its nature, an annual
process. Each applicant is compared to other applicants who applied in
that year. A pooled analysis does not reflect how the process actually
works, because it effectively compares applicants from different years to
each other.
Card Report 51.
Professor Card is wrong for two principal reasons. First, he is wrong that all
applicants each year are compared to all other applicants. That is certainly not true
with respect to recruited athletes, who are compared only to other athletes, and it is
largely untrue for legacies. Many of the early action applicants are not compared to
those in the regular admission pool. And, as I have noted earlier, a large proportion
of applicants do not meet minimal Harvard admissions criteria and are thus
eliminated from consideration at an early stage (e.g., their applications do not
receive a second read).
Second, the main effect of using a yearly instead of pooled analysis is that it reduces
the statistical power of the sample. Statistics is largely driven by the law of large
numbers; in any quantitative analysis, the ease of distinguishing random variations
from systematic factors rises in proportion to the square root of the sample size. In
any analysis of discrimination, it is logical and important to use the largest sample
that is relevant to the comparisons involved.
34
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
To see this, consider the example of a large firm that discriminates against women
in making promotions to partnership. Suppose that data disclosed by the firm show
that over a six-year period, women with similar evaluation scores to men, with
similar billings, and so on, are promoted at a substantially lower rate. Suppose
there is substantial corroborating evidence of discrimination in evaluations and
work assignments. Suppose then that the firm’s response is that it is inappropriate
to attack it for a pattern manifest over a six-year period. “Decisions to promote
associates to partnership are made on an annual basis,” the firm’s expert says. “One
must analyze one year at a time.” Doing so would, of course, reduce the statistical
significance of findings of discrimination, but it would not make any sense.
Finally, Professor Card contends that a yearly model is more appropriate because
admit rates for the same rating profile are different across years. Professor Card
gives as an example that those who receive a 2 on all four profile ratings (academic,
personal, extracurricular, and athletic) have admission rates that vary between 61%
and 77% across admissions cycles. Card Report 54. This is misleading: just because
admission rates vary across years for this rating combination does not mean a
pooled model should be ruled out. In fact, my pooled model actually does an
excellent job in predicting these exact fluctuations. Specifically, for those who
receive this rating combination, the correlation between my model’s prediction of
the yearly admission rates and the actual yearly admission rates is extremely high
(0.91). My model is able to explain the differences across years through a variety of
channels, including that admissions are becoming more competitive over time (as
captured by year effects) and in how the other characteristics vary across years for
those who received this particular rating combination (e.g., racial composition,
disadvantaged status).
By using a yearly model, Professor Card achieves results that weaken the effect of
race in Harvard’s admissions process by adding noise to the estimated racial
preferences and penalties. But making even a few adjustments to both his pooled
and yearly models result in significant findings of an Asian-American penalty, and
even more substantial racial preferences than even Professor Card finds to exist for
Hispanic and African-American applicants.
35
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
4.2 Professor Card’s pooled results, with small corrections, show that
Harvard imposes a penalty against Asian-American applicants.
Professor Card first pools the six years of admissions data and runs logit models
similar to mine. But he chooses a different set of controls and, importantly,
substantially restricts how race affects the admissions process. Professor Card’s
pooled results have a number of flaws, many of which have been discussed above,
and all of which have the effect of concealing the extent of Harvard’s discrimination
against Asian-American applicants.
First, Professor Card improperly relies upon the assumption that Harvard’s
discrimination against regular Asian-American applicants is irrelevant unless the
same level of discrimination is present with respect to Asian-American applicants
who are athletes, legacies, and/or members of other special (preferred) recruiting
categories. Including these variables in the analysis—and also ignoring how race
interacts with these variables—serves only to conceal the impact of the penalty
Harvard imposes on Asian-American applicants. It is thus essential to either (1)
remove these applicants from the analysis; or (2) allow for the possibility that the
effect of race is different for these applicants (i.e., interacting these variables with
race).19 By failing to do either, Professor Card makes it impossible to fairly consider
the effects of the racial preferences and penalties Harvard employs in its admissions
process.
Second, Professor Card errs in assuming that racial preferences operate the same
way for disadvantaged students as they do for advantaged students. As noted in
section 3.7 of my opening report, African-American and Hispanic applicants receive
a smaller preference for disadvantaged status than Asian-American and white
applicants, and including them in the analysis without interacting race weakens the
distinctions between white and Asian-American applicants.
Third, Professor Card errs in including Harvard’s personal rating in his models. See
Card Report 69-74. As discussed above, it is clear that racial preferences affect the
Note that removing these observations is equivalent to allowing the coefficients of the
model to be fully interacted with special recruiting status.
19
36
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
personal rating. By ignoring this evidence, Professor Card’s regressions simply
spread Harvard’s discrimination across multiple variables, making the main effect
(Asian-American discrimination) smaller and thus harder to measure as
statistically significant.
Fourth, Professor Card errs by including data for parental occupation that, as
shown in section 3.5, is unreliable, given the wide variation in yearly patterns. See
Card Report 43-45.
Table 4.1N shows that, after removing those in the special recruiting categories,
corrective adjustments to Professor Card’s models that account for any of these
issues results in significant estimates of discrimination against Asian-American
applicants.
Table 4.1N: Small corrective adjustments to Professor Card’s model show
penalties against Asian-American applicants
*=statistically different from zero at the 95% level. Marginal effects are calculated without perfect
predictions.
The first row of Table 4.1N reports the marginal effect for Professor Card’s pooled
model. This is the average marginal effect of Asian-American status itself for all
Asian-American applicants who have a non-zero probability of admission.20 The
Throughout Professor Card’s report when he calculates marginal effects he includes those
whose characteristics are so bad that rejection is guaranteed regardless of their race. This
serves to lessen the actual penalty or preference by averaging in zeros from those who are
clearly not competitive. I have removed those applicants in this analysis to get a more
accurate measure of the marginal effects.
20
37
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
average effect in this first row is small and statistically insignificant. The second
row removes those in the special recruiting categories. The average marginal effect
increases by 61%, but the coefficient remains insignificant—meaning that we
cannot rule out that the effect is different from zero at the 95% level.
The next three rows illustrate how Professor Card’s decision to drop the interaction
of disadvantaged status with race affects the magnitude and statistical significance
of the penalty Asian-American applicants face. Row 3 adds interactions between
race and disadvantaged status, allowing the preference for disadvantaged
applicants to vary by race. Once that is done, it reveals a statistically significant
penalty for Asian-American applicants. In other words, Professor Card’s conclusion
that there is no Asian-American penalty hinges on his error in failing to include
interactions between race and disadvantaged status.
Rows 4 and 5 in Table 4.1N underscore how Professor Card’s model ignores the way
that Harvard’s racial preferences operate in practice. If I estimate the model only on
white and Asian-American applicants (row 4), it shows a statistically significant
penalty against Asian-American applicants. Note how close the estimated marginal
effect is to the one where race is interacted with disadvantaged status. This
confirms that Professor Card’s model downplays the effect of race by ignoring
Harvard’s differential treatment of disadvantaged students. Row 5 further
illustrates this point: by estimating only on students who are not disadvantaged—
including African Americans and Hispanics—Professor Card’s model once again
shows a statistically significant penalty against Asian-American applicants. All of
this demonstrates the fragility of Professor Card’s models, exposing the
unreliability of his finding that there is no Asian-American penalty.
The same effect occurs if we discard Professor Card’s use of the personal rating,
which shows compelling evidence of bias on the part of Harvard’s admission officers.
See Section 3.3.2, supra. Row 6 of Table 4.1N retains Professor Card’s flawed
application of disadvantaged status, but removes the personal rating. Once again,
there is a statistically significant penalty against Asian Americans.
Professor Card’s use of unreliable data on parental occupation likewise skews his
results. In row 7 of Table 4.1N, I keep Professor Card’s flawed application of
38
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
disadvantaged status and the personal rating, but instead remove the parental
occupation variables. Making this change alone once again reveals a statistically
significant penalty against Asian-American applicants.
This exercise reveals the extent to which small adjustments to Professor Card’s
pooled model expose his flawed conclusions. It underscores that his model is not
robust, and that his analysis appears carefully constructed to downplay the extent
to which Harvard’s process penalizes Asian Americans. The next section reveals the
same shortcomings in Professor Card’s yearly model.
4.3 Professor Card’s yearly models, with small corrections, confirm that
Harvard imposes a penalty against Asian-American applicants.
In my report, I analyzed Harvard’s admissions process over the full six-year period.
I chose to pool the data because Harvard’s admissions process (and its use of race)
underwent no material changes during this time, and the six-year period ensures a
larger overall sample size, increasing the confidence in the results.
Professor Card claims that it is inappropriate to analyze the results of Harvard’s
application process by pooling six years of data, and that instead every year should
be treated independently. See Card Report 51-54. I disagree, in part because
treating each year separately decreases the sample size and thus makes it more
difficult to measure the effects of race in Harvard’s admissions decisions. But even
using Professor Card’s yearly approach confirms that Harvard imposes a penalty
against Asian-American applicants, once the key flaws in his model are corrected.
To demonstrate this, I estimated slightly modified versions of Professor Card’s
yearly models, correcting his models for the key flaws I have identified and
explained elsewhere. As before, I removed those applicants in special recruiting
categories, and otherwise adopted the alternative versions of Professor Card’s
models described in the pooled analysis (interacting race with disadvantaged status;
discarding the biased personal rating; 21 and discarding the unreliable parental
Professor Card does do some limited analysis with the personal rating included in his
yearly models. But he presents his results in a misleading way. Namely, his results with
the personal rating in the model) show a statistically significant penalty against Asian
21
39
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
occupation variables).
The first column of Table 4.2N shows the results of these slight corrective
adjustments to Professor Card’s yearly model. Any one of these adjustments reveals
that the estimated penalty against Asian-American applicants is substantially
higher than Professor Card suggests. The models in rows 2 through 7 show a
statistically significant penalty against Asian-American applicants at the 90% level,
with the last three also statistically significant at the 95% level.
Table 4.2N: Small corrective adjustments to Professor Card’s yearly model
show penalties against Asian-American applicants
*=statistically different from zero at the 95% level. Marginal effects are calculated without perfect
predictions.
Again, the models I report take the additional variables that Professor Card has
added at face value, ignoring that these additional variables actually distort the
analysis in ways that tend to conceal the discrimination against Asian-American
applicants. In particular, these models include Professor Card’s questionable use of
the data on extracurricular activities.
Americans—even with all their other flaws. Professor Card claims that this evidence is
nonetheless weak because in only one of the individual years is it statistically significant.
But because Professor Card’s model is estimated at the yearly level, the imprecision of the
estimates becomes much larger. This makes it impossible to rule out the possibility of even
very large penalties in any given year. Professor Card also argues that once 2018—the year
that preceded the SFFA lawsuit—is removed, the average over the remaining five years is
no longer significant. But it is arbitrary to remove 2018 in this manner. One could just as
easily remove 2019, and Card’s model shows significant estimates across the remaining
years. The upshot is that Professor Card’s models that include the personal rating show a
penalty against Asian-American applicants even with no other adjustments to the model.
40
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
Data on extracurricular activities come from applicants listing (1) each activity they
participated in, (2) the years in which they participated in this activity, (3) the
hours per week and weeks per year they participated in the activity, and (4)
whether their participation was during the school year or outside the school year.
Each of the activities is assigned to one of 29 categories (e.g., work, academics,
musical instruments).
In his analysis of these activities, Professor Card considers the first two activities
listed, aggregating the listed activities into one of twelve groups in a somewhat
arbitrary manner. For example, Card aggregates some large categories like
religious and volunteer activities, groups some categories like “school spirit” and
“LGBT” into an “other” category, and leaves “Junior ROTC”—one of the smallest
categories—by itself.
More importantly, the level of participation of the activity is done only for the work
category, where Professor Card calculates the total hours in work activities over the
course of the applicant’s high school career. This distorts the analysis in two ways.
First, it overemphasizes the weight that work is given in the process, as work
activities are only the eighth most popular activity listed for whites. See Card
Report 180, Exhibit 66. Second, white applicants work significantly more hours
than Asian-American applicants. Yet there are many activities where AsianAmerican applicants invest substantially more hours than white applicants. 22
Professor Card provides no explanation in his report for this idiosyncratic approach
to extracurricular activities.
I make the following adjustments to Professor Card’s extracurricular activities in
order to more accurately account for their effect on admissions decisions:
•
Rather than use Professor Card’s groupings when constructing indicators
for each of the first two listed activities, I use the original 29 activity
categories.
This is presumably among the reasons why Asian-American applicants tend to have
higher scores than white applicants on Harvard’s extracurricular rating.
22
41
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
•
Rather than use the total hours of work over the course of the applicant’s
high school career, I consider broader groupings of categories and measure
participation both by (1) counting the number of grades in which the
applicant participated in each activity and (2) indicating whether the
applicant’s total accumulated hours in a category was above the median
for those who had any positive hours in the category.
Making these adjustments more precisely accounts for the impact of extracurricular
activities on admissions decisions. This more accurate picture of extracurricular
activities reveals that the penalty against Asian-American applicants is higher than
Professor Card suggests. The results of the models using additional measures of
extracurricular involvement are given in the second column of Table 4.2N. For each
model, the Asian-American penalty is larger in column 2 than in column 1. Further,
all of the deviations that are in rows 2-7 are statistically significant at the 95%
level. All Professor Card demonstrates is that one can selectively choose and count
extracurricular activities in a way that disadvantages Asian-American applicants—
and thereby conceal the discriminatory nature of Harvard’s admissions process. But
accounting for the full distribution of activities shows that my finding of an AsianAmerican penalty is robust.
Finally, column 3 of Table 4.2N shows the results using the various corrective
adjustments to Professor Card’s models to the six-year pool as a whole, rather than
year-by-year. Examining the marginal effects shows that, if anything, the effects on
Asian-American applicants are more often larger in the year-by-year model.
I suspect that Professor Card prefers the yearly model for the same reason he
prefers to add other irrelevant or unreliable information to his model: it introduces
more noise into the estimates by adding many more variables, all of which tend to
conceal the degree to which Harvard discriminates against Asian-American
applicants. Consider the estimated penalties (marginal effects) in column 1 for the
model using only white and Asian-American applicants. The estimated penalty is
bigger than the corresponding penalty from the pooled model in column 3 (-0.37%
versus -0.34%), yet is not statistically significant at the 95% level.
Professor Card actually shows many results of the marginal effect of being Asian
42
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
American for his yearly models where the effects are rarely significant at the 95%
level (see Exhibits 19, 21, 22, 23, and 25). But because Professor Card has
effectively introduced a lot of noise into his models, it is also not possible to rule out
large penalties against Asian-American applicants. In all the years except for 2019,
the 95% confidence interval in Professor Card’s yearly model with just the special
recruiting categories removed contains an Asian-American penalty of 0.9 percentage
points. This is a large change given the admit rate is 5.1% for Asian-American
applicants who are not within the special recruiting categories.
In fact, 2019 consistently shows the smallest Asian-American penalty of all the
yearly models (or, in some cases, no penalty at all). It also is the first (and only)
admissions cycle after the SFFA lawsuit. In the first column of Table 4.3N, I show
the marginal effects of being Asian American (the Asian-American penalty) by year
for each of the models.
Table 4.3N: The year-over-year evidence of an Asian-American penalty
*=statistically different from zero at the 95% level. Marginal effects are calculated without perfect
predictions.
Note that this is the specification that uses Professor Card’s extracurricular
measures. In every specification that does not include the special recruiting
categories, the marginal effect is negative in all years but 2019. In 2019, the
estimate is positive for all specifications, except in the last column that implements
all three corrections: interacting disadvantage with race, removing parental
occupations, and removing the personal rating.
The last row of Table 4.3N shows the average marginal effect excluding 2019. Now
the marginal effects are significant at the 95% level for specifications (2) through
43
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
(4)—even when Professor Card’s extracurricular controls are used.
Professor Card also argues that the evidence is even weaker for discrimination
against Asian-American applicants when considering particular subgroups, namely
those on California dockets and females. 23 But there are problems with his
subgroup analyses. First, the same criticisms of Professor Card’s yearly and pooled
analyses apply here. It is incorrect to include special recruiting categories; it is
incorrect to ignore racial interactions with disadvantaged status; the parental
occupation variable is unreliable; and the personal rating is biased. Second,
Professor Card uses his yearly model to generate his findings. These yearly models
have very large standard errors that increase when significantly less data are used
in the analysis. What Professor Card has failed to show is whether any of his
subgroup analyses yield results that are statistically different from his other
findings. Third, Professor Card controls for each unique rating combination,
aggregating combinations with less than 100 applicants by how similar they are in
their admission rates. This too serves to hide racial preferences and penalties, an
issue that becomes more salient for smaller estimation samples (which is the case in
his subgroup analysis). As I show in section 7.2, these aggregations in the yearly
models are inappropriate but are surely worse when these aggregations are done at
the yearly subgroup level.24
4.4 Professor Card’s analysis of applicants whose race is missing further
confirms the existence of an Asian-American penalty
Professor Card makes another argument that inadvertently shows a penalty
against Asian-American applicants. In my opening report, I noted that the impact of
racial preferences on Asian-American applicants is likely understated due to some
Asian Americans choosing not to report their race. Without racial preferences, some
of those applicants would see their probabilities of admission rise, as would be the
case for all applicants who are not underrepresented minorities.
As I show in section 9.3, there is actually evidence of discrimination against dockets that
have a higher share of Asian-American applicants.
23
The impact of these aggregations on the magnitude of racial preferences in my pooled
model is substantial. See Table 8.2N.
24
44
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
In an attempt to undermine my argument, Professor Card shows that, using other
sources, it is possible to identify the race for some of the applicants who choose not
to report. Card Report 55. Professor Card notes that when he uses this information
to classify many of those who do not report a race to particular racial groups and
estimates his model, the estimated penalty for Asian-American applicants goes
down.
But Card’s analysis does not undermine my argument at all—it actually shows that
Harvard does not impose a racial penalty on those Asian-American applicants who
do not identify their race.25 That the penalty against Asian-American applicants
falls when some of this group is included as Asian American in the analysis
necessarily means that this group is actually treated better than those who report
their race as Asian-American.
5
Professor Card’s Analysis Actually Demonstrates That Race Is a
Determinative Factor in Harvard’s Admissions Decisions.
I previously demonstrated that race plays a “significant role in admissions decisions
at Harvard.” Arcidiacono Report 7-8. Professor Card does not disagree. See Card
Report 10, 81, 93. Professor Card instead claims that race is not a “determinative
factor” in admissions decisions. He attempts to support this claim by showing the
average marginal effect of race by year for Asian Americans, African Americans,
Hispanics, and applicants who do not identify their race using his preferred yearly
model. See Card Report 81 & Exhibit 26. But in doing so, Professor Card actually
demonstrates that race is in fact a determinative factor in admissions decisions.
Average marginal effects of race show how, on average, admission probabilities
change as a result of the applicant’s race/ethnicity. This is what Professor Card
shows in his Exhibit 26. But what Professor Card leaves out is that average
marginal effects must be interpreted relative to the baseline probability of
Indeed, this coincides with anecdotal evidence that some Asian-American applicants hide
their race on college applications to avoid discrimination. See, e.g., Fearing discrimination,
Asian college applicants don’t always declare ethnicity, Associated Press (Dec. 3, 2011),
www.nydailynews.com/ news/national/fearing-discrimination-asian-college-applicants-dondeclare-ethnicity-article-1.986416.
25
45
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
admission. To illustrate, consider the case of a relatively moderately selective
college, where the average admit rate for a particular racial group is fifty percent in
the absence of racial preferences. If the average marginal effect of race for that
group were six percentage points, then the average admission probability with
racial preferences would be fifty-six percent—i.e., the effect of racial preferences
would amount to a 12% increase in the number of admitted students in this racial
group.
But the impact of racial preferences resulting in a six-percentage-point effect is
much greater at a highly selective school—where the baseline probability of
admission is much lower. For example, if the average admit rate for a specific racial
group were three percent in the absence of racial preferences and the average
marginal effect of race were (again) six percent, that would mean that the average
admission probability with racial preferences would be nine percent. In this
scenario, the effect of racial preferences would be massive—a tripling of the admit
rate and thus the predicted number of individuals admitted for that specific racial
group.
Professor Card’s analysis of his own preferred yearly model shows this very scenario
at Harvard. Table 5.1N below replicates Professor Card’s results for African
Americans and Hispanics in Exhibit 26, but includes the average probability of
admission for these groups—both with and without racial preferences—in order to
illustrate the effect of racial preferences as compared against the baseline. Professor
Card’s own models show that racial preferences are responsible for tripling the
number of African-American admits and doubling the number of Hispanic admits.26
For purposes of this analysis, and consistent with Professor Card’s approach, I am
including within the Hispanic category those applicants whose race/ethnicity is identified
as Native American, Hawaiian/Pacific, or “Other.”
26
46
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
Table 5.1N: Admission probabilities and marginal effects for African
Americans and Hispanics in Professor Card’s yearly models
*indicates statistically significant at the 95% level.
To illustrate, consider domestic applicants for the class of 2014 (in the first row of
Table 5.1N). Absent racial preferences, African-American applicants would be
treated as white applicants. Professor Card’s models predict that if racial
preferences were removed, the average admit rate for African-American applicants
would be 3.38%. This is the baseline (i.e. the starting point absent racial
preferences). Compared against this baseline, Professor Card’s average marginal
effect of race for African-American applicants (7.43%) would increase the admit rate
for African-American applicants to 10.81%, more than tripling the admit rate for
African-American applicants in 2014. And this is not an outlier. Professor Card’s
overall average marginal admit rate for the entire six-year period is 6.12%.
Compared with a baseline of 2.79% (the average admit rate absent racial
preferences for the six-year period), the tripling effect exists for the entire period.
Professor Card’s own preferred model and analysis thus reveal that racial
preferences more than triple the admit rate for African-American applicants.
Professor Card is undoubtedly aware of the multiplying effect that racial
preferences have on African-American and Hispanic admit rates. The fact that he
fails to address them is revealing. His analysis—which demonstrates that racial
preferences alone are responsible for doubling and tripling the admit rates for
African-American and Hispanic applicants—quite obviously demonstrates that race
is a determinative factor in admissions decisions at Harvard.
47
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
So how does Professor Card come to the conclusion that race is not a determinative
factor in admissions decisions at Harvard? Professor Card resorts to three
misleading and/or plainly incorrect arguments, claiming that:
•
A model with race as the only control does a poorer job of explaining
admissions decisions than other sets of controls (e.g. profile ratings,
dockets) See Card Report 83, Exhibit 27.
•
For most African-American and Hispanic applicants, the average
marginal effect of racial preferences is small. See Card Report 84, Exhibit
28.
•
Unmeasured factors are more important than racial preferences. See Card
Report 86-87, Exhibits 29 and 30.
Below I show that each of these arguments is incorrect or misleading. In doing so, I
rely only on Professor Card’s models. As I show in Section 8.2, my preferred model
shows even larger estimates of racial preferences.
5.1
A model with race as the only control would be expected to perform
poorly relative to other factors.
Professor Card’s first argument can be ruled out almost immediately. As I showed
in my opening report, there are vast differences in academic preparation across
racial/ethnic groups. For example, in the expanded data set, over 37% of AfricanAmerican applicants are in the bottom decile of the academic index compared to
fewer than 4% of Asian-American applicants. And less than 1% of African-American
applicants are in the top decile of the academic index compared to almost 18% of
Asian-American applicants See Arcidiacono Report, at Table B.5.1. Because race is
generally correlated with academic preparation, one would expect that race would
have at least some explanatory power with respect to the admissions results.
Professor Card’s findings in Exhibit 27—that race alone explains little variation in
admissions—actually suggests, as a statistical matter, that racial preferences are
quite large.
To more clearly see this, suppose Harvard had a strict quota system, accepting the
48
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
best 10% of each racial group. Estimating a model of admission where the only
control was race would have zero explanatory power even though an explicit quota
was in place. In order to properly evaluate the role of race in the admissions
process, it is paramount that one controls for the relevant factors in the admissions
decision. Professor Card’s analysis in Exhibit 27 does nothing to support his claims.
5.2 Professor Card’s argument that racial preferences are not relevant
for most African-American and Hispanic applicants misleadingly
focuses on uncompetitive applicants.
Professor Card’s second argument is that, for the majority of Harvard applicants,
race is not relevant to the admissions decision. This argument is a dodge. Of course
race is not relevant for a large number of applicants. No one would claim otherwise,
given that Harvard is a highly selective school where more than 90% of all
applicants are rejected. See Section 3.1, supra.
A further example may help illustrate the point. Suppose Harvard sent automatic
rejection letters to the 80 percent of its applicant pool with the lowest standardized
test scores. Further assume that, among the remaining twenty percent, half (ten
percent of the pool) were admitted. Of those who were admitted, suppose Harvard
did so using a specific quota for each racial group, and admitted the other half based
on purely non-racial factors, such as academics, extra-curricular achievement, and
so on. It would still be true that race did not affect most application decisions;
indeed, race would only affect 5% of Harvard’s decisions. But this would be no
defense to the way race was used in admitting the competitive applicants. The fact
that the majority of all applicants are rejected regardless of their race tells us
nothing about the effect race has among those applicants who are seriously
considered for admission to Harvard.
This fallacy can be seen in Exhibit 28 of Professor Card’s report (which is
reproduced in the first two columns of Table 5.2N, below). Here, Professor Card
ranks applicants according to their admissions index which, given the estimates of
his yearly models, describes the strength of applicants based on how the applicant’s
observed characteristics translate into admissions. Professor Card does this ranking
49
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
separately by race, implying that 10% of each racial group is in each decile.27
Table 5.2N: Average marginal effects of race by within race and across
race admission index deciles using Professor Card’s yearly models
The first five rows of column 1 show no effect of race for African-American
applicants in the bottom five deciles. The reason there is no effect in the bottom five
deciles is that Professor Card’s yearly models predict that more than 50% of
African-American applicants have other observed characteristics (combinations of
test scores, Harvard ratings, etc.) where everyone who has these characteristics is
rejected. And I agree that racial preferences are not relevant for uncompetitive
applicants.
I further agree with Professor Card that racial preferences are most salient for the
competitive applicants. But Professor Card makes a mistake when he describes who
is affected by racial preferences:
[T]he applicants with the largest estimated positive effect of race on
their likelihood of admission are the strongest applicants—i.e., those
Part of the note to Exhibit 30 reads “Deciles are constructed by race based on the predicted
probabilities of admission when the race factor is turned off.” Note that whether the race
factor is turned off or not has zero relevance as to who is assigned to what decile, when the
deciles themselves are constructed by race.
27
50
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
whose estimated likelihood of admission is in the top 10% of the
applicant pool absent consideration of race.
Card Report 84.
But Professor Card’s calculations are not for those who are in the top 10% of the
applicant pool, but instead for the top 10% of African-American applicants. The
third and fourth columns of Table 5.2N show the average marginal effects by
admissions index decile where the deciles are constructed across all racial groups
and where racial preferences are turned off.28 Racial preferences are relevant for
the top 10% of African-American applicants (column 1), who are distributed across
the top 30% of the applicant pool (column 3). Given that the admission rate across
all racial groups over this period is slightly over 7%, it is not surprising to find
smaller effects of racial preferences for those in the bottom 70% of applicants.
To further illustrate this point, the last four columns show the share of AfricanAmerican and Hispanic admits in each of the academic index deciles. Using the
within-race deciles, over 82% of African-American admits are in the top decile (the
top 10% of African-American applicants). But this is exactly where the marginal
effects of race are enormous: Professor Card estimates the marginal effect for this
group to be over 47%, as shown in column 1.
5.3 Professor Card’s method of calculating the importance of unobserved
factors is incorrect and substantially overstates their importance.
Professor Card next claims that unobserved characteristics are more important
than race, again suggesting that race is not a determinative factor. Card Report 8586. Professor Card reaches this conclusion using erroneous methods. Properly
accounting for the role of unobserved characteristics shows that Professor Card
vastly overstates the importance of unobserved characteristics relative to race for
Professor Card defines his admissions indexes for this table without accounting for
differences in admission rates by year. Hence, the same applicant would have a higher
index in 2014 than in 2019 as admission rates as a whole were higher in 2014. To form the
across-race deciles, I remove the effects of year by creating the deciles at the year level;
each decile has 10% of each year’s applications. While my method is the correct one, this
has little effect on the patterns shown in Table 5.2N.
28
51
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
African-American and Hispanic applicants. Indeed, I will show that unobserved
characteristics are decidedly less important than race for these two groups.
Professor Card attributes the share unobserved characteristics play in the
admissions decision as the absolute value of the difference between the predicted
probability of admission and the actual admit decision. But this does not equate to
the share of the admissions decisions explained by unobserved characteristics. What
it does give—at least for admits—is whether the applicant has unobservable
characteristics above a particular percentile. Knowing that the unobservable is
above a particular percentile is useful, but not in the way Professor Card uses it.
The predicted probability of admission indicates how often we would expect an
applicant to be admitted given a random draw from the distribution of unobserved
characteristics. Some of those random draws would result in rejection, others in
acceptance. When an applicant is rejected, this tells us that the set of possible
unobserved characteristics had to lie in some range, but not the exact value of the
unobserved characteristic. For example, suppose an applicant has characteristics
associated with a 90% chance of admission and that applicant was admitted. This
means that the applicant’s unobserved characteristics were above the 10th
percentile. Professor Card’s method, however, would imply that unobserved
characteristics explains 10% of the admissions decision, which is simply false.
Knowing that an applicant was admitted and his or her predicted probability of
admission tells us the range of possible values for the applicant’s unobserved
characteristic. Because both Professor Card and I use logit models to estimate the
admissions decisions, our models assume that the unobserved characteristic comes
from a particular distribution. I can use these three pieces of information—the
distribution of the unobserved characteristic, the predicted probability of admission,
and the actual admission decisions—to show:29
•
How often the expected value of the unobserved characteristic is larger
than the preferences for a particular racial/ethnic group; and
29
The derivations of the formulas are given in Appendix A.1.
52
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
•
The probability of the unobserved characteristic being larger than the
preferences for a particular racial/ethnic group.
Table 5.3N shows both Professor Card’s incorrect method, as well as my
calculations.
Table 5.3N: Racial preferences are more important than unobserved
factors for African-American and Hispanic admits using Professor Card’s
models
The first column replicates Professor Card’s result in the last column of Exhibit 29
for Hispanics and African Americans. Here it is important to note that Professor
Card does two misleading things in reporting his results. First, he includes perfect
predictions
in
his
estimates
as
though
(tautologically)
their
unobserved
characteristics were at least as important as race to their admissions chances. But
these observations provide no information on whether race is more or less important
than unobserved characteristics. With over 50% of African Americans having
observed characteristics that result in a 100% chance of rejection, Professor Card’s
inclusion of these applicants in his calculations substantially overstates his actual
findings, even aside from his incorrect method.
Second, Professor Card does not break out the results by admitted and rejected
applicants. Column 2 uses Professor Card’s method but reports the results only for
admits. The differences are striking: Professor Card claims that for 94% of AfricanAmerican applicants, unobserved characteristics are more important than race.
But, even under Professor Card’s own model, unobserved characteristics are more
important than race for only 30% of African-American admits.
Moreover, even this 30% figure is a gross overstatement. In column 3, I show how
53
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
often the expected value of the unobserved characteristic for each admit is bigger
than the estimated racial preference. Using this measure, the unobserved
characteristic is bigger than the racial preference only 4% of the time for AfricanAmerican admits. In column 4, I instead show how often admitted applicants could
expect to draw an unobserved characteristic that was bigger than their racial
preference. The average probability of an African-American admit drawing an
unobserved characteristic that was bigger than their racial preference is 10%. The
corresponding shares are larger for Hispanic admits as racial preferences for
Hispanics are weaker than those for African Americans. Nonetheless, both
measures show that racial preferences are more important than unobserved
characteristics more than 65% of the time. Clearly, then, unobserved characteristics
are substantially less important than racial preferences for these two groups.
6
Professor Card Fails to Refute the Overwhelming Statistical Evidence
of a Floor for African-American Admissions.
In my report, I showed that Harvard maintained a floor on the admission rate for
single-race African-Americans in the classes of 2017, 2018, and 2019. In each of
these years, the admit rate for single-race African Americans (as identified under
the federal Integrated Postsecondary Education Data System (IPEDS)) 30 was
virtually identical to the admit rate for all other domestic applicants, as reflected in
Table 6.1N below:
Table 6.1N: The admit rate for single-race African Americans is
implausibly close to the admit rate for other domestic applicants
IPEDS counts an individual as African American only if the individual marks “Not
Hispanic; Black or African American only.” If the individual marks, for example, Hispanic
and African American, the individual is counted as Hispanic. And if the individual marks
White and African American, the individual is reported as “two or more races.” See
Collecting Race and Ethnicity Data from Students and Staff Using the New Categories,
National Center for Educational Statistics, https://nces.ed.gov/ipeds/Section/collecting_re.
30
54
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
The difference in these two sets of rates is never larger than .00064—about as small
a divergence as is possible, especially given the size of the applicant pool. I found
that the chance of this match occurring in three consecutive years (without direct
manipulation) is less than 0.2%, and arguably much smaller. See Arcidiacono
Report 29.
In response to my analysis, Professor Card has three responses, arguing that:
1) It makes no sense that Harvard would impose a floor on the AfricanAmerican admit rate based on IPEDS metrics, because the admission rate by
race is never publicly reported. See Card Report 88-89.
2) Because Harvard began reporting its results using the federal IPEDS method
before 2017, there is no reason why it would impose a floor during that year.
See Card Report 88-89.
3) Under a variety of alternative measures of race and alternative places where
a floor could be implemented, there is no evidence of a floor. See Card Report
89-93.
None of these responses is persuasive. Indeed, none of them even address the
compelling statistical evidence I present. To begin, Professor Card makes no
attempt to contest the near mathematical certainty that Harvard is, in fact,
manipulating the admissions rate for single-race African Americans to match its
overall rate. That is my primary claim based on the statistical evidence.
Instead, Professor Card provides irrelevant responses, focusing on other admissions
statistics and racial categories that say nothing about what Harvard was doing with
IPEDS admissions rates and single-race African-American applicants beginning
with the Class of 2017. Moreover, further examination of the characteristics of
single-race, African-American admitted applicants confirms that a change occurred
with the 2017 cycle, and further bolsters my conclusion that Harvard was in fact
taking steps to ensure its admission rate for these applicants was at least as high as
the overall admission rate.
55
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
6.1 Professor Card’s speculation that Harvard would not want to use a
floor based on a non-public admissions rate misses the point.
Professor Card notes that under the IPEDS reporting process, admissions rates by
racial group are not publicly reported by the federal government, and Harvard has
declined to make the data public on its own. He assumes that Harvard would have
no reason for imposing a floor that the public would never see.
To begin, there are undoubtedly many ways Harvard could impose racial floors.
They could impose a floor based on the expected number of admits, the share of
admits of a particular race, or the relative acceptance rates of particular races.
Alternatively, Harvard could impose a floor based on the expected number of
enrollees of a particular race. Furthermore, Harvard could do this using a variety of
different measures of race. Invariably, each of these different ways and different
measures would result in different patterns in the data.
However, my claim is that the data show that Harvard implemented a particular
kind of floor using a particular definition of race. Why Harvard chose this particular
way of imposing a floor is irrelevant as a statistical matter.
That said, there are several reasons why Harvard might use a floor that is tied to a
metric not publicly reported:
•
Because rigid floors and other racial quotas are plainly illegal, it seems
logical that if Harvard were attempting to ensure a minimum level of
admissions for a particular race, it would want to use a metric that was not
publicly available, lest its unlawful conduct be detected.
•
Although the IPEDS admissions rates are not publicly reported, they are
tracked within the admissions office and could be used internally—for
example, to rebut any allegations that the admissions office was not
admitting African Americans at a sufficient rate. Indeed, there is evidence
that Harvard was very concerned about the way its IPEDS enrollment
numbers were being perceived by the public in early 2013 (during the
consideration of applications for the class of 2017).31
See HARV00023588 (Feb. 6, 2013 email to Dean Michael Smith) (“[This] is a piece that
explains how we collect and report demographic data, as per federal guidelines.”);
31
56
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
•
Harvard shares IPEDS admissions data by race with other institutions. For
example, Harvard shares annual admissions data by racial group—including
admissions rates—on an annual basis through the Consortium on Financing
Higher Education (COFHE), a voluntary association of 35 selective, private
liberal arts colleges and universities. See, e.g., HARV00004736-38 (setting
forth Harvard admissions rates by racial group and overall admission rates
under IPEDS method); HARV00009158-59 (describing COFHE’s use of
IPEDS data).
•
Likewise, admissions officers from Harvard attend semi-annual meetings of
the Association of Black Admissions and Financial Aid Officers of the Ivy
League and Sister Schools (ABAFAOILSS), at which Harvard officers bring
data on admissions rates—including IPEDS data—and other institutions
appear to share IPEDS admission rates by racial group during the
admissions cycle. See HARV00014684-868; HARV00067679.
Ultimately, Harvard’s reason (or combination of reasons) for establishing a floor for
single-race African-American admissions based on IPEDS metrics is outside both
my (and Professor Card’s) expertise. The data demonstrate that this racial floor
exists. Professor Card does not and cannot dispute that Harvard maintained a floor
on the admission rate for single-race African-Americans in the classes of 2017, 2018,
and 2019.
6.2 Contrary to Professor Card’s arguments, there is additional evidence
that Harvard began implementing the floor in 2017.
Professor Card notes that Harvard changed their reporting of race to the federal
government, using the IPEDS method, before the 2017 cycle. Again, this is
irrelevant to the fact that Harvard maintained a floor on the admission rate for
single-race African Americans in the classes of 2017, 2018, and 2019.
There is evidence that the IPEDS numbers became salient to the admissions office
during the 2017 cycle. For example, Harvard has produced numerous examples of
“one-pagers”—statistical summaries of the applicant pool and admitted class that
are provided on a regular basis to the leadership of the admissions office—from the
HARV00023594 (“[T]he IPEDS reporting system leads to significantly underreported
percentages for all ethnicities except Hispanic Americans. The method used by Harvard
and many peer institutions gives a more complete report of the way many students,
especially those of mixed heritage, actually view their racial and ethnic identities.”).
57
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
2017 and 2018 admissions cycles. Early versions of the one-pagers during the 2017
cycle lack any IPEDS data. See, e.g., HARV00014628 (one-pager from January 7,
2013). It appears that IPEDS numbers are reported on a one-pager for the first time
on or about January 12, 2013. See HARV00019910 (one-pager dated January 12,
2013, comparing early admits for 2016 with early admits for 2017).32 And again,
this change coincides precisely with evidence reflecting increased concern within
the admissions office about IPEDS reporting and the admission of students by
race.33
6.3 Professor Card’s analysis of other data does nothing to undermine
my claim that Harvard maintained a floor on the admission rate for
single-race African-American applicants.
In Exhibits 31 through 34, Professor Card shows changes in the fraction of admitted
students by race/ethnicity over time. He concludes that because these numbers vary
over time, there must not be a floor. But all of Professor Card’s exhibits use a
measure of race that is not the one I claim that Harvard used in imposing a floor on
the admission rate of single-race African-American applicants. Further, it uses an
outcome measure—the fraction of admitted students of a particular race/ethnicity—
that is unrelated to my claim. None of these exhibits have anything to do with my
claim. To repeat, my claim is that there was a floor on the admit rate of single-race
African-American applicants for the classes of 2017 to 2019.
Professor Card also argues that a post-2016 floor cannot be occurring because the
estimated marginal effect of race on African-American admissions (including both
single-race and multi-race African Americans) is smaller in the period between
2017-2019 than in the period 2014-2016. Professor Card’s argument on this point is
misleading, for several reasons:
Further evidence of the then-emerging salience of the IPEDS measure of race is that the
variables used to construct the IPEDS measure were not included in the main data file
Harvard produced for years prior to 2017. After reviewing Professor Card’s report, I
discovered that IPEDS numbers for the pre-2017 years were located in other spreadsheets
provided by Harvard. Its absence from the main data file further indicates that the
admissions office changed its tracking of these data in 2017.
32
33
See HARV00026562; HARV00030511; HARV00023613.
58
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
•
First, and most importantly, the race measure Professor Card uses is (again)
not the one upon which my observation of a floor is based.
•
Second, Professor Card overstates the difference in the marginal effects. The
marginal effects he estimates across years are similar in magnitude and not
statistically different from one another. A similar pattern emerges if the
marginal effects are averaged over the two periods. The difference in the
average marginal effects is small and statistically indistinguishable from
zero.
•
Third, because overall admit rates have been falling over time, it is no
surprise that the marginal effects would be slightly smaller in later years
(though, as noted above, not significantly different).
•
Finally, as explained above, see supra Section 5, how meaningful marginal
effects are necessarily depends on how competitive the pool is: a difference of
six percentage points is much more meaningful when the baseline admit rate
is 5% than when it is 10%.
6.4 Differences in the characteristics of admitted single-race African
Americans after 2016 further support evidence of a floor.
While Professor Card’s response does not address my key claims, there is another
way to test whether Harvard changed its admissions practices with respect to
single-race African Americans in 2017: compare the difference in characteristics
between single-race African-American admits and multi-race African-American
admits in the admitted classes of 2014-16 and the 2017-19 cycles. Because my claim
is that there was a shift in focus towards the admit rate of single-race African
Americans, I would expect to see a change in the strength of admitted single-race
African Americans relative to their multi-race counterparts. I focus on the academic
index as a measure of applicant strength because it is a continuous measure with a
well-defined formula.
59
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
Table 6.2N: Mean academic index for admitted single-race and multi-race
African Americans by class
*=statistically significant at 95% level. Academic index is in standard deviation units.
Difference refers to the single-race academic index minus the multi-race academic index.
Table 6.2N shows the average academic index, in standard deviation units, for
admitted single-race and multi-race African Americans by year. The difference
between the average academic index for single-race African-American admits and
multi-race African-American admits is presented in the third column. There is no
significant difference between the academic index of single-race and multi-race
African American admits in any of the pre-2017 cycles and, as shown in the last row
of the first panel, aggregating across the three pre-2017 cycles shows no significant
differences.
But the results for the post-2016 cycles, shown in the bottom panel, indicate a
markedly different pattern. In each case the difference is negative, and more
negative than any of the differences in the pre-2017 cycles. This gap is significant
for 2019 as well as for the period as a whole. These differences show that Harvard
was admitting single-race African Americans with significantly lower academic
indexes than their multi-race counterparts beginning in the post-2016 period. This
is striking because it is precisely what would be expected if Harvard began
imposing a floor on single-race African-American admit rates after 2016.
Examining the admit rates of single-race and multi-race African Americans in the
different admission cycles further confirms that Harvard changed its practices in
2017. These admit rates are shown in Table 6.3N.
60
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
Table 6.3N: Admit rates for single-race and multi-race African Americans
by class
*=statistically significant at 95%level. Difference refers to the single-race admit rate minus the
multi-race admit rate. Ratio refers to the multi-race admit rate divided by the single-race admit rate.
Single-race African-American admit rates are 3.2 percentage points lower than
multi-race African-American admit rates in the pre-2017 period. After 2016,
however, the difference narrows substantially to 1.6 percentage points. Put another
way, the average admit rate for multi-race African Americans is 50% higher than
the single-race African American admit rate in the pre-2017 period, but only 26%
higher in the post-2016 period.
This additional statistical evidence further confirms that Harvard changed its
admissions practices in 2017 in a manner consistent with the existence of a floor on
admission rates of single-race African Americans such that it was equivalent to the
admission rates for all other domestic applicants. Professor Card has not challenged
the statistical evidence I used in my opening report, instead choosing to focus on
data and racial categories that are irrelevant to the question at hand. The evidence
on this point is thus both statistically compelling and unrebutted.
7
A Number of the Other Variables Added by Professor Card Are of
Questionable Reliability and Undermine the Confidence of His
Conclusions.
Thus far, I have highlighted the numerous errors and questionable modeling choices
that undermine Professor Card’s analysis of the racial penalty Harvard imposes on
61
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
Asian-American applicants and the racial preferences Harvard affords to AfricanAmerican and Hispanic applicants. Professor Card’s findings and conclusions are
further undermined by his inclusion of several variables of questionable reliability.
In his report, Professor Card argues that increasing the number of variables
analyzed in a model necessarily yields more complete results. See Card Report 4050. But that is true only if the variables are (1) relevant to the analysis, (2) correctly
specified (i.e., accurate), and (3) not themselves influenced by racial preferences.
Some of the variables that Professor Card uses violate at least one of these criteria.
One of them is parental occupation, as explained above in Section 3.5. Other
variables that are less important to Professor Card’s result, but still too
questionable to rely on, include intended career and staff interviews. The
weaknesses in these variables are what led me to exclude them from my original
analysis. Further, I disagree with Professor Card’s approach to the ratings data,
believing it introduces unnecessary noise into the model and disguises racial
preferences. In this section, I describe those variables that (in addition to parental
occupation) I choose not to incorporate in my preferred analysis—though in section
8, I show that even including these faulty measures do not affect my findings.
7.1 Intended career varies in highly unusual and unexplained ways over
time, undermining its reliability as a variable and its usefulness as a
control.
Like parental occupation, the applicant’s intended career also varies in ways that
are inconsistent over time, casting doubts upon the reliability of this metric and
further undermining Professor Card’s models. There are fourteen intended career
categories in the Harvard database for these admissions cycles. Table 7.1N shows
the number in each of these categories for five of the intended careers; the full set of
intended careers are shown in the Appendix, see Table B.4.1N.
62
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
Table 7.1N: Intended career varies in non-credible ways over time
The differences across years are enormous for the same intended career. For
academics, the number of applicants who are listed ranges from 13 to 2,247; for law
it ranges from 708 to 2,093. Medicine varies from a low of 3 in 2018 to a high of
6,254 in 2014. Health varies from a low of 85 in 2016 to 4,944 in 2018. Again,
Professor Card provides no explanation as to why he would be confident about the
accuracy of this information or why it varies so widely.
7.2 Professor Card’s approach to using the rating variables suffers from
a small-population problem and masks racial preferences, which
undermine its reliability.
In my original report, I included indicators for each of Harvard’s profile ratings.
Professor Card argues instead that all combinations of the profile ratings should be
included. In Professor Card’s pooled dataset, there are 287 combinations of athletic,
personal, extracurricular, and academic ratings. Of the 287 combinations, 26 of
these combinations yield a perfect prediction of admission—meaning every
applicant who receives these combinations of scores is admitted. Another 153
combinations yield a perfect prediction of rejection—all of the applicants with these
combinations are rejected. This is in part mechanical: of the 179 combinations that
perfectly predict rejection, 53 of the combinations contain only one applicant, and
the median number of applicants in a combination is 13. Professor Card then pools
rating combinations based on their admission rates when the number of applicants
in that category is less than 100.
There are a number of problems with this approach. The first problem is that
aggregating combinations with very small populations leads to admissions patterns
that are inconsistent with Harvard’s ratings. For example, consider the admit
63
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
profile 4341, where the first number is the athletic rating, the second the personal
rating, the third the extracurricular rating, and the fourth the academic rating. Two
individuals were assigned this rating over this period; both were white, and both
were admitted. Yet there are ratings that are objectively higher on Harvard’s scale
that nonetheless have substantially lower admit rates:
•
4321 is a rating profile that is two points higher on the extracurricular rating
and identical on the other three ratings. The 65 applicants that received this
rating profile had an average admit rate of 73.0%;34
•
4331 has a rating profile that is one point higher on the extracurricular and
identical on the other three ratings. The 126 applicants that received this
rating profile had an admit rate of 42.1%;
•
3331 has a rating profile that is one point higher on both the athletic and
extracurricular ratings. The 96 applicants that received this ranking had an
admit rate of 37.5%.
The second problem with this approach is that racial preferences are embedded in
the ratings aggregation. To see this, suppose a particular ratings combination had
more African-American applicants than another ratings combination, but the admit
rates for the two combinations were the same. The average admit rates for the two
groups are in part due to the strength of the rating profiles, but also in part due to
the share of African-American applicants in the two groups. In this example, the
rating profile associated with the second group is actually the better profile as the
admit rate for the first rating profile was more affected by racial preferences.
Professor Card’s aggregation method, then, works to conceal the true effect of racial
preferences.35
These issues are compounded in the yearly analysis, where there are even fewer
numbers in each of the ratings combinations. Across the six admission cycles, 244 of
34
Throughout this section, when I refer to the average admit rate, I am referring to the
average admit rate for the category to which this rating combination was assigned.
I show how Professor Card’s rating scheme conceals racial preferences in Table 8.2N and
Section 8.3.
35
64
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
the 287 categories show up as perfect predictions in at least one of the years.
Further underscoring the small-sample problem, 15 of the categories perfectly
predict admission in one of the years and perfectly predict rejection in at least one of
the other years.
Constructing ratings groupings at the yearly level results in dramatic fluctuations
in the year-by-year admit rates for the same rating, and again in ways that are
inconsistent with higher ratings being associated with higher admit rates. This is
illustrated in Table 7.2N, which shows by year the average admit rates for those
applicants who received each of the following four ratings combinations: 4311, 3321,
4312, and 3312. It also shows the number of observations in each year for that
category.
Table 7.2N: Using Professor Card’s rating combinations for his yearly
regressions leads to inconsistent patterns
The admission rates for the same rating combination fluctuate substantially across
years. The admit rates for 4321 range from 29.7% to 100%; the admit rates for 4312
range from 17.2% to 63.6%. These large fluctuations result because of sampling
variability: using such few observations leads to large sampling error.
Comparing the top two rows to one another as well as the bottom two rows to one
another shows the inconsistent patterns in how the ratings profiles translate into
admission rates. We would expect those who receive a 3321 to be admitted at a
higher rate than those who receive a 4321; it is by all accounts a better score. Yet in
half of the years, this is not the case. And in two of the four years, admission rates
are higher for 4312 than for 3312. The inconsistency itself raises red flags about
65
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
using the ratings data in this way; the fact that it also reflects Harvard’s racial
preferences (as described above) further shows the inappropriateness of Professor
Card’s approach to using the ratings data.
7.3 Staff interviews are selectively given and thus should not be used as
a control.
Redacted
36
Who are these fortunate few who receive staff interviews? Table 7.3N shows the
number and fraction of each of the four main racial/ethnic groups who receive a
staff interview by whether or not they were in one of Harvard’s special recruiting
categories.37
Table 7.3N: Staff interviews vary substantially by race and special
recruiting status
There is an error in how Professor Card codes the scoring of the staff interviews in his
pooled analysis. Namely, Professor Card creates a flag for whether someone received some
combination of 1’s and 2’s on the staff interviewers overall rating and personal rating,
another flag for a combination of 2 on one and a 3 on the other, and finally a flag for two 3’s.
Those who receive one 1 and one 3 are then effectively treated as though they had no staff
interview. This error, however, has virtually no effect on the results due to the small
number of applicants in this category.
36
Special categories are athletes, legacies, faculty or staff children, and Dean’s/Director’s
List selections.
37
66
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
Twenty percent of those who fall into any one of Harvard’s special recruiting
categories receive staff interviews. Those who are in these special recruiting
categories are disproportionately white. 38 For applicants not in one of these
categories, the probability of receiving a staff interview is less than 1.3%. AsianAmerican applicants are least likely to receive a staff interview, both overall and
conditional on the special recruitment status.
Because these interviews clearly depend on preferences, I do not include them in my
analysis.
8
Incorporating Most of Professor Card’s Variables Into My Preferred
Model Confirms My Findings Regarding the Effect of Harvard’s Racial
Penalties and Preferences.
To recap, we can divide Professor Card’s analysis of my report into two broad parts.
In one part, he constructs a model to show that Harvard does not discriminate
against Asian-American applicants vis-à-vis white applicants. As I have shown, this
model is dependent upon many inaccurate assumptions and poor modeling choices.
Moreover, it is not robust: if I change just one or two of these assumptions and
choices, Professor Card’s model no longer supports his findings and conclusions; in
particular, his model confirms the penalty against Asian-American applicants.
In the other part, Professor Card tries to show that Harvard’s racial preferences in
favor of African-American and Hispanic applicants are not substantial or pervasive.
But as I have shown, this analysis is exceedingly weak; even when we use Professor
Card’s own model results, they show that Harvard gives African-American and
Hispanic applicants heavy racial preferences.
For some of the arguments and model specifications Professor Card uses, there is
simply no sound justification for the choices, and it is hard to imagine any reason
for their use other than to intentionally conceal the effects of race in Harvard
admissions. Other adjustments suggested by Professor Card are reasonable, and for
still others there is at least a weak case for inclusion. The question is, are my
8.0% of white applicants are in one of these categories, compared to 2.7% of African
Americans, 2.2% of Hispanics, and 2.0% of Asian Americans.
38
67
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
results as fragile as Professor Card’s? Or are they robust and consistent when we
incorporate specific changes suggested by him? In this section, I examine the
robustness of my model in the context of Professor Card’s analysis.
8.1 Changes advocated by Professor Card that I incorporate in my
updated model
In my updated model, I adopted six general types of revisions that reflect
unobjectionable choices made by Professor Card.
1.
Modifying variables. Professor Card codes several variables in a
different way than my original model. My update incorporates these changes:
•
I treat profile ratings of 7, 8, and 9 as missing values;
•
I include blank teacher ratings as a missing category;
•
When the SAT score is not present but an ACT score is present, I use the
ACT science section in my conversions the same way Professor Card does;
•
I no longer remove from the analysis those who are missing the overall
rating.
2.
Adding variables. I incorporate dozens of additional variables that
Professor Card uses in his analysis, so long as they meet three conditions:
•
They must not be themselves measures contaminated by apparent racial or
other preferences (e.g., I exclude the personal rating and staff interviews);
•
They must display consistent patterns over time, thus demonstrating
reliability;
•
They must be present in each year of the data, so that they can be included in
the pooled analysis.
Many of the additional variables used by Professor Card meet all these restrictions,
and I thus incorporate them in my updated model. These include Professor Card’s
College Board variables on the characteristics of applicant high schools and home
neighborhoods; whether the mother or father is deceased; whether a parent
attended an Ivy League university (other than Harvard); whether a parent attended
graduate school at Harvard; and the type of high school the applicant attended.
68
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
As an alternative to Professor Card’s flawed use of the various combinations of
ratings data to capture the multidimensionality of the applicant, I create indicators
for whether the applicant had each possible combination of a two or better on
Harvard’s four profile ratings, indicators for whether the applicant had two or three
2’s or better on their school support measures, and an indicator for whether the
applicant had 2’s or better on both of the alumni ratings.39
3. Including early admission applicants in my baseline model. In my
original report, my baseline model excluded recruited athletes, legacies, faculty and
staff children, those on the Dean’s/Director’s list, and applicants for early
admission, in order to focus on the part of the admissions process where anti-Asian
discrimination was concentrated, and not on applicants who were subject to special
admissions procedures. Professor Card’s model includes all applicants in a single
model. As I have pointed out, this produces misleading results because Harvard
does not discriminate against Asian-American applicants who are in the special
recruiting categories. But I do not have similar objections about including early
applicants. Although the early admissions process necessarily involves different
considerations than the bulk of the application process, Harvard’s racial penalties
and preferences largely apply in this process the same way they do in the regular
admissions process. I thus include early admission applicants in my updated
baseline model, which is intended to include all applicants whom I believe are at
risk of discrimination.
4.
Racial definitions. Professor Card collapses racial categories in a
different manner than I did in my original report. In my updated model, I use his
definitions, which place Native Americans and Hawaiian/Pacific Islanders into the
“Hispanic” category.
5. Interactions with year. Professor Card claims that a yearly model is
appropriate in part because the composition of the pool changes from year-to-year,
and Harvard may pay attention to this. Indeed, we know exactly how Harvard pays
Note that these are in addition to indicators for each possible value of the individual
ratings (e.g. 2 on the academic rating) that were in my original model.
39
69
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
attention to it through their “one-pagers,” which provides admissions officials with a
snapshot of the current admissions process compared to the prior year. Although
Professor Card errs in employing a yearly model, I can account for year-to-year
changes in my pooled model by including in my model interactions with year and
the characteristics listed on these one-pagers: female, disadvantaged status,
intended major, dockets, and, in some specifications, race.
6.
Reporting results. Professor Card emphasizes the marginal effects of
race in discussing results—in other words, how many percentage points does
membership in a particular race increase or decrease one’s admissions rate? The
numbers below adopt this approach, reporting these marginal effects, but only for
those whose characteristics are such that rejection is not guaranteed (i.e., the
perfect predictions are removed).
8.2 The results of the updated preferred model confirm my previous
findings and conclusions
Table 8.1N, below, shows the marginal effects of race in my original model and my
updated model for my baseline dataset that includes early action applicants.40
Table 8.1N: Basic racial penalties and preferences under my original and
revised model
*=statistically significant at the 95% level. Marginal effects calculated without perfect predictions.
As Table 8.1N shows, the numbers in the updated model are slightly different than
in the original model, but the story is unchanged. African-American applicants
receive extremely large preferences, on average 7.29% off a base of 2.25%; more
40In
order to calculate the marginal effects from my original model, I use the results from
the original report that included both the special recruiting categories and early action
applicants. I then remove the special recruiting categories to calculate the marginal effects.
70
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
than quadrupling their chances of admission. Hispanic applicants experience large
preferences, 4.17% on average off a base of 2.97%, increasing their chances of
admission by 2.4 times. Asian-American applicants experience a substantial
admissions penalty that lowers their chances of admission by a full percentage
point; Asian-American admission rates would be 19% higher if they were treated as
white applicants.
8.3 Even incorporating many of Professor Card’s manifestly unsound
modeling choices does not alter the result of my model.
I now address the modeling choices that Professor Card made and which I find to be
unsound or indefensible. I have dissected many of these choices earlier in the
report. My goal in this section is to explain the degree to which I think Professor
Card’s choices would substantively change the results of my basic analyses, and
thereby make clear which assumptions really matter.
In the first column of the first panel of Table 8.2N, I show marginal effects for each
racial group in the baseline dataset. The rest of the entries show that my findings
are robust along a number of dimensions.
•
Including the personal rating. As noted in earlier sections, Harvard’s
personal rating of applicants is severely contaminated with racial bias;
ratings are inflated for preferred racial groups, and penalized for Asian
Americans. They therefore cannot be included in any sound model of Harvard
admissions that is trying to separate out discriminatory effects. Nevertheless,
as I show in the bottom panel of Table 8.2N, including the personal rating
makes the discriminatory effects in my model smaller (as one would expect),
but it does not make them statistically insignificant, or change their basic
pattern.
•
Including questionable variables. I now show that my model is robust to
the inclusion of the parental occupation and intended career variables,
despite their flaws. As the results in the second column of Table 8.2N show,
including these questionable variables does not materially alter my key
results.
71
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
•
Interaction terms. As noted earlier, Professor Card excludes several
interaction terms used in my model. The most important of these is the
interaction of race and disadvantage status. As I have explained previously,
see supra Section 3.2, this interaction requires inclusion because Harvard
takes a student’s disadvantaged status into account differently for applicants
of different races. Throughout, I keep these interactions in my model.41
My original and updated models also include interactions for gender and
race, and gender and intended major. Including these interactions matters
less for my basic results (as shown in column 3 of Table 8.2N), but they are
an important part of the evidence along other dimensions. They show, for
example, that Harvard significantly penalizes African-American women
relative to African-American men in the personal rating, perhaps because
Harvard wishes to balance out the gender disparity among African-American
applicants (female African-American applicants substantially outnumber
male African-American applicants).
•
Interacting ratings variables. My updated model also declines to follow
Professor Card’s methodology for interacting various ratings combinations.
As shown in section 7.2, the groupings Professor Card uses are too fine and
are based on the false premise that small sets of ratings that have similar
admit rates should be pooled together. They should not be pooled, because the
small sample sizes produce a phenomenon known as “over-fitting”—with
many combinations guaranteed to either be rejected or admitted—and
because their associated admit rates depend on other characteristics of the
applicants. For example, if a particular rating group has a disproportionate
number of African-American applicants, and African Americans receive large
41
I also continue to include interactions between missing SAT2 and race and missing
alumni interview and race. I do this because the missing indicators effectively assign the
same score or rating for all those who are missing. The interactions allow the data to assign
different values based on the race of the applicant.
72
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
preferences, then pooling that rating group will both distort the effect of that
rating and will disguise the effect of race.
As shown in column 4 of Table 8.2N, significant penalties and preferences are
still present when I use Professor Card’s ratings variables from his pooled
analysis. But, consistent with my criticism, the effects of race are attenuated,
and this is especially true for African-American and Hispanic applicants.
Given that adding controls virtually always leads to an increase in the
estimated preferences for African-American applicants, this suggests that
Professor Card’s use of the ratings masks racial preferences.
Finally, the last column of Table 8.2N shows that even if all four changes are
implemented in my preferred model—including the personal rating, controlling for
Professor Card’s suspect variables, removing interactions between gender and race
and gender and major, and using Professor Card’s rating controls—it still results in
substantial racial preferences for African-American and Hispanic applicants and
significant penalties for Asian-American applicants.
Table 8.2N: The racial penalties and preferences I estimate for admissions
are robust to Professor Card’s key changes
*=statistically significant at the 95% level. Marginal effects calculated without perfect predictions.
9
My Updated Preferred Model Yields Additional Reasons to Doubt
Professor Card’s Approach
In this final section, I show three additional results from my updated preferred
model that underscore the weaknesses of Professor Card’s approach and
demonstrate that his findings and conclusions are untenable. First, I show how the
penalties against Asian-American applicants vary with how competitive the
73
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
applicant is. Second, I show that, contrary to Professor Card’s claims, AsianAmerican applicants are as strong as white applicants on non-academic measures.
Third, there is evidence that Asian-American applicants are hurt by other
preferences that Harvard employs. Namely, I show that dockets that have a higher
share of Asian-American applicants have lower admit rates.42
9.1 The penalties Asian-American applicants face are substantial
The estimated effects of the Asian-American penalty depend on the strength of the
applicant. As I have already noted, some applicants are rated in such a way that no
matter their race or their unobserved characteristics, they will be rejected. Hence
for certain applicants, there is no penalty or—in the case of African-American and
Hispanic applicants—no preference. Similarly, Asian-American applicants that only
have very small probabilities of being admitted will see their admissions chances
only slightly improve if Asian-American penalties are removed.
Table 9.1 shows how the Asian-American penalty differs depending on the strength
of the observed characteristics of the applicants. In particular, I use the baseline
dataset to calculate deciles of the Asian-American admissions index for those who
have positive predicted probabilities of admission. These are the applicants affected
by the Asian-American penalty. The deciles are calculated such that 10% of these
Asian-American applicants are in each decile. The first set of columns shows the
results for my preferred model; the second set shows the results for my preferred
model with the personal rating also included.
A more detailed discussion of my updated preferred model is included in Section 3 of
Appendix A
42
74
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
Table 9.1: The effects of the Asian-American penalty at different
admissions deciles
Columns 4 and 6 show the percentage increase in admissions chances if the penalty
were removed. The average penalty faced across all deciles in my preferred model is
almost one percentage point. Because the overall Asian-American admit rate is
5.2%, removing the penalty would increase the Asian-American admit rate by
19.2%.
As would be expected, the effect varies substantially across the admissions index
deciles. The biggest percentage point increases are for the most competitive
applicants; these applicants see a 6.2 percentage point increase in their admissions
probabilities, a 14.8% increase. The percentage point increases are smaller in the
lower deciles, but as their base probability of admission is smaller, the percentage
increases are higher: those in the bottom deciles only see a 0.02 percentage point
penalty, but removing this penalty would increase their admission rate by 40%.
9.2 Estimates of my admissions and personal ratings models show that
Asian-American applicants are strong on non-academic measures.
Throughout his report, Professor Card claims that the Asian-American penalties
found in my models of both admissions and the personal rating can be explained by
Asian-American applicants being weaker on non-academic dimensions. As I showed
using a corrected version of Professor Card’s Exhibit 10, this is not supported by the
data in my original model. See Table 3.1N. It is also not supported in my updated
model.
As before, I construct an admissions index which assesses applicants’ strengths
based on how their observed characteristics translate into a probability of
75
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
admission, after removing race and year effects. I then construct deciles of the
admissions index, with higher deciles associated
with
stronger observed
characteristics. Based on the admissions deciles for my updated baseline dataset
and my preferred model, Asian-American applicants are the strongest group overall,
with 13.1% of Asian-American applicants in the top decile. As shown in the first
panel of Appendix C, Table 7.3R, this is higher than the respective numbers of
white (10.5%), Hispanic/Other (5.7%), and African-American applicants (4.1%). The
second panel shows that even with the personal rating included, Asian-American
applicants are the strongest group.
But to further test Professor Card’s claim that Asian-American applicants are
actually weak on non-academic characteristics (which he claims are more likely to
be in the unobservables), I create a non-academic index following Professor Card’s
approach in Exhibit 10, removing those variables that are explicitly academic in
nature (e.g., test scores, grades, academic ratings).43 Results from my preferred
model are shown in the first panel of Appendix C, Table 7.4R. Asian-American and
white applicants have the same share in the top decile (11.3%); Asian-American
applicants have a greater share of the following decile, and have smaller shares in
the bottom deciles. It is thus clear that on non-academic measures other than the
personal rating, Asian-American applicants are at least as strong as white
applicants. The second panel illustrates the bias when the personal rating is
included—only then do Asian-American applicants fall behind white applicants on
non-academic measures.
The same point can be illustrated through the effects of Harvard’s other ratings
besides personal and academic. These include the following ratings: extracurricular,
athletic, teacher1, teacher2, counselor, and both of the alumni ratings. Creating the
admissions index using these variables alone shows the same pattern as seen in
Appendix C, Table 7.5R. Asian Americans have greater representation in the top
deciles than white applicants as long as the personal rating is not included; when it
Since I am using the baseline dataset, I am not vulnerable to the mistake Professor Card
makes by including preferences for special recruiting categories as part of his non-academic
index.
43
76
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
is included, they have the lowest share in the top decile. This is yet another
indication that the personal rating exhibits bias.
The same point is demonstrated when I examine the non-race variables that affect
the personal rating. I create personal indexes in a similar manner to the admission
indexes described above; higher indexes are associated with higher probabilities of
receiving a high rating on the personal quality measure. In Appendix D, Table
B.6.13R, I show the representation of each racial group in the resulting deciles
(using the baseline dataset without academic factors.) 44 Under this approach,
Asian-American applicants are actually 0.07 standard deviations stronger than
white applicants—even without considering any academic factors. Looking instead
at deciles of the non-academic rating components, Asian-American applicants are
slightly worse than whites, but the margin is less than -0.02 standard deviations.
These results make clear that any differences between white and Asian-American
applicants on non-academic ratings is quite small, and—contrary to Professor
Card—could not possibly explain the substantial differences in their personal
ratings.
9.3 Dockets with
penalized.
high
shares
of
Asian-American
applicants
are
Harvard could also impose racial preferences or penalties through indirect
channels, such as geographic preferences based on the demographics of the targeted
areas. If certain dockets have high shares of Asian American applicants, and
Harvard want to disfavor them, it could simply penalize these dockets.
To investigate this possibility, I examined the relationship between the estimated
docket by year fixed effects and the Asian-American share of domestic applicants
from each docket-year combination. Using the docket-by-year effects from my
preferred model and expanded dataset, I find that the larger the share of AsianAmerican applicants in that docket-year combination, the more negative the
Results with the academic factors included are shown in Appendix D, Table B.6.11R.
Here, too, Asian-American applicants are at least as strong as white applicants on the
observed characteristics associated with higher personal ratings.
44
77
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
estimated docket-year fixed effect will be. More precisely, a one-standard deviation
in the share of Asian-American applicants in a docket-year leads to a reduction in
the admissions index of 0.14.45 This reflects the penalty Harvard imposes on dockets
with a high share of Asian-American applicants, and it is more than one-third the
magnitude of the Asian-American penalty Harvard already imposes on AsianAmerican applicants.46 Put differently, Harvard imposes a penalty on applicants
from any docket with a high share of Asian-American applicants, and that penalty
is more than a third of the direct penalty Harvard imposes on Asian-American
applicants generally.
To be clear, this penalty is not imposed solely on Asian-American applicants; its
effects extend to any applicant from that docket regardless of race. But because this
penalty is imposed only on those applicants in dockets with a high share of AsianAmerican applicants, this strongly indicates that its real target is Asian-American
applicants themselves. There are two important points here: first, it appears that
Harvard penalizes Asian-American applicants in indirect ways on top of the already
substantial penalties it imposes on Asian-American applicants; second, my findings
thus tend to understate the true magnitude of the penalties Harvard imposes on
Asian-American applicants.
This is the penalty using the expanded dataset; the penalty is larger in the baseline
dataset.
45
46The
coefficient on Asian American is -0.39 using my preferred model with the expanded
dataset. Note that this is the coefficient for male Asian-American applicants who are not
disadvantaged.
78
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
Dated: January 29, 2018
s/ Peter S. Arcidiacono
Peter S. Arcidiacono
79
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
1
Expected errors conditional on choices
In Appendix A of my original report, I defined a latent index πi where i indexes individuals and where
πi = Xi γ + εi
(1)
The university accepts individual i if πi > 0. In the above equation, Xi represents attributes about candidate
i that I observe in the data. The εi represents the unobserved characteristic of the individual. To characterize
the role of unobservables, I need to be able to calculate the expected value of εi conditional on the admission
decision.
A mathematically equivalent model–one that leads to same the estimation procedure and model predictions–
would be instead to define the payoff the university receives from accepting the applicant and rejecting the
applicant respectively as u1i and u0i and where:
u1i = Xi γ +
u0i =
1i
0i
The university admits the applicants when u1i − u0i > 0. Note that ε in equation (1) is then identical
to
1i
E(
1
−
−
0i .
0 |y
I want to recover E(ε|y = 1) where y indicates admission. This is the same as recovering
= 1) but this second way is mathematically easier to derive the expectation.
Under this second way of expressing the logit model,
1
and
0
are distributed Type 1 extreme value. This
error distribution has the following property:
E( 0 ) = γ = P r(y = 0)E( 0 |y = 0) + P r(y = 1)E( 0 |y = 1)
(2)
where γ is Euler’s constant.
Rearranging terms yields:
E( 0 |y = 1) =
γ − P r(y = 0)E( 0 |y = 0)
P r(y = 1)
(3)
The previous literature has shown that E( 0 |y = 0) can be expressed as:1
E( 0 |y = 0) = γ − ln(P r(y = 0))
(4)
Substituting (4) into (3) yields:
γ − P r(y = 0)[γ − ln(P r(y = 0))]
P r(y = 1)
P r(y = 0) ln(P r(y = 0))
= γ+
P r(y = 1)
E( 0 |y = 1) =
(5)
(6)
Recognizing that:
E( 1 |y = 1) = γ − ln(P r(y = 1))
1
(7)
See, for example, V.J. Hotz and R.A. Miller “Conditional Choice Probabilities and the Estimation of Dynamic Models”, Review
of Economic Studies, Vol. 60, No.3, July 1993., page 504.
1
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
we can form E(
1
−
0 |y
E(
1
= 1) using:
−
0 |y
= 1) = − ln(P r(y = 1)) −
P r(y = 0) ln(P r(y = 0))
P r(y = 1)
(8)
The individual probabilities of admission (P r(y = 1)) and rejection (P r(y = 0)) then translate directly into
how strong we expect the applicant to be on unobserved characteristics conditional on being admitted. This
can then be compared to the estimated admissions preference which I label μ (this is the coefficient on race in
the logit model) to see how often the expected unobserved characteristic is bigger than the racial preference.
2
Probability of unobserved draws
The previous section showed how to calculate the expected value of the unobservable characteristic conditional on three pieces of information: (i) the distribution of the unobserved characteristic, (ii) the probability
the individual was admitted, and (iii) whether the individual was actually admitted. These three pieces of information can also be used to calculate the probability the unobserved characteristic is bigger than the racial
preference for each applicant.
The probability of the unobserved factor being greater than μ is given by one minus the logistic cumulative distribution function.
P r( > μ) = 1 −
1
1 + exp(−μ)
(9)
The probability of the unobserved factor being greater than μ conditional on being admitted given observed
characteristics x, where these observed characteristics x translate into an admit probability of P r(y = 1), is
given by:
P r( > μ|y = 1, x) = min
1−
1
1+exp(−μ)
P r(y = 1)
,1
(10)
The reason for the min operator is that some individuals would have x’s such that is assured that their unobservable characteristics had to be bigger than μ.
2
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
3.
In this Appendix, I document the results of my analysis from moving to my
updated models. While my results are generally robust to the changes I have
made, there are three important points to keep in mind regarding the
samples under consideration.
First, because I have adopted Professor Card’s controls from the College
Board data, information for the U and V dockets are dropped from the
analysis. U and V dockets contain data on applicants who are living abroad,
and the College Board data only contain information on those attending high
school domestically. This works to lower the sample sizes (as well as the
number of seats in the counterfactuals).
Second, I now include early action applicants in my baseline dataset. This
increases the number of observations in the baseline dataset, and thus
correspondingly increases the number of seats in the counterfactuals. It also
changes some of the descriptive statistics, for two reasons. First, early action
applicants tend to be stronger than regular decision applicants, which is one
reason they have higher admit rates. Second, Asian-American applicants are
more likely to apply early action (once special recruiting categories are
removed). This changes the baseline descriptive tables, showing that when
early action applicants are included in the baseline dataset, admit rates for
Asian-American applicants are sometimes higher than the admit rates for
white applicants.
Third, I no longer include athletes in my expanded model. Athletes by far
have the highest admit rates and it is clear that the admission process for
this group is very different.1
Additional time and analysis has underscored the extent to which recruited
athletes are truly outliers, even within the special recruiting categories. For
example, the probability of getting admitted with an academic rating of 4 is
minuscule for non-athletes (.076%) and nearly a thousand times greater for athletes
(70.46%). One in seven admitted athletes have an academic rating of 4 or worse; the
rate for non-athletes is one in every 600. Recruited athletes also make up a much
1
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
In the next three sections I summarize the findings of my updated models,
and how they relate to my original conclusions.
3.1 The role of race in the scoring of applicants for admission
In my opening report, I showed that there is a significant penalty against
Asian-American applicants in the scoring of applicants for admission despite
the fact that Asian-American applicants are stronger on the observed
characteristics than all the other races/ethnicities. I also showed that there is
a significant preference given to African-American and Hispanic applicants in
both the personal and overall ratings.
These findings are unaltered when I use the updated sample and employ
additional control variables. Tables B.6.1R through B.6.8R in Appendix D
present a series of ordered logit estimates of the probability of receiving a
particular rating on one of Harvard’s components. 2 For ease of tracking
multiple variables, the ratings have been recoded so that higher values are
associated with better ratings.
Consistent with my original report, my revised regressions indicate that the
personal and overall ratings are biased against Asian-American applicants,
and not the product of having better unobserved characteristics (as Professor
Card contends). They further show that preferences are given to AfricanAmerican and Hispanic applicants in these ratings:
•
As objective controls are added to the models for the academic and
extracurricular ratings, the race coefficients tend to become smaller in
magnitude. This suggests that if more observables were added, the
effect of race would continue to diminish.
•
A very different pattern emerges for more subjective ratings, such as
the personal and overall ratings. As additional controls are added to
smaller portion of the applicant pool than legacies or early action applicants.
2Moving
across the columns within a particular Harvard component rating (academic, for
example) shows how the results change as more controls are added.
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
the model, the race coefficients tend to increase in magnitude, with the
racial preference for African-American and Hispanic applicants getting
larger and the racial penalty for Asian-American applicants becoming
stronger. This pattern is inconsistent with the notion that racial
preferences simply reflect the impact of unobserved characteristics.
•
Appendix C Table 6.1R shows how the probability of receiving a
personal rating of two or better would change for each race/ethnicity if
they were treated like each of the other races/ethnicities. Had Asian
American applicants been treated as white applicants, the probability
of receiving a two or better personal rating would increase by 4
percentage points, reflecting a 21% increase. If treated like Hispanic
applicants, their probability of receiving a two or better would rise by
38%, and if treated like African-Americans, it would rise by 58%.
Similar patterns exist for the overall rating.
While the racial penalty Harvard imposes on Asian-American applicants is
especially stark for the personal and overall rating, there is some evidence
that Harvard penalizes Asian-American applicants in Harvard’s scoring of
the teacher and counselor reports. In each of these models, Asian-American
applicants are subjected to a penalty despite being stronger than all other
racial groups on the observed characteristics associated with high ratings.
See Appendix D, Tables B.6.11R and B.6.12R.
My updated models thus confirm that racial preferences work throughout the
admissions process, not simply at the final decision point. Professor Card
concedes that the overall rating contains racial preferences; Tables B.6.3R
and B.6.4R shows that the pattern of racial preferences/penalties is
extremely similar for the overall and personal ratings models. This is why it
is improper for Professor Card to control for the personal rating in the
admissions model.
2.2 The role of race in the selection of applicants for admission
My opening report showed that Harvard imposes a penalty on Asian-
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
American applicants in the selection of applicants for admission—a penalty
that is separate and apart from the penalty that Harvard imposes against
them in the scoring of applicants for admission. This Asian-American penalty
in the selection of applicants is unaltered by the changes in the sample and
control variables discussed previously.
Appendix D Table B.7.1R and Table B.7.2R display estimates of a series of
logit models of admission for the updated baseline and expanded dataset,
respectively. Model 5 is the preferred specification, as it includes all controls
other than the personal rating. The changes in the race coefficients as
additional controls are included mimic the patterns seen in the personal
rating. The race coefficients for African-American and Hispanic applicants
become larger and positive as additional applicant characteristics are
included. This occurs because African-American and Hispanic applicants are
weaker on the observed characteristics that predict admission, meaning that
the racial preference has to grow to explain the admissions decisions.
Table 7.2R in Appendix C puts the admissions penalty against AsianAmerican applicants in context. It shows that Asian-American admit rates
would increase by 19% if Asian Americans were treated as whites in the
preferred model. The preferences for African-American and Hispanic
applicants are even larger in magnitude than the Asian-American penalty. In
the preferred model, admit rates for Asian American applicants in the
baseline dataset would increase almost three-fold if they were treated like
Hispanic applicants, and over five-fold if they were treated like AfricanAmerican applicants.
Similar to the ratings models, my updated models assess whether the
penalties Asian Americans suffer could reasonably be attributed to
unobservable characteristics. Indexes can be constructed net of year and race
that give the strength of the applicant based on the controls, effectively
aggregating all the measures Harvard uses and weighting them the same
way the data indicates that Harvard weighs them in their admissions
decisions. These indexes are not well defined for those who have
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
characteristics that perfectly predict rejection and admission, so I focus on
deciles of the admissions indexes where those who have characteristics that
guaranteed rejection were assigned to the bottom decile, and characteristics
that guaranteed admission to the top decile.3 These deciles then give the
strength of the application based on how the characteristics of the applicant
translate into admissions probabilities net of race/ethnicity.
Appendix C Table 7.3R shows the share of each racial/ethnic group that is in
each of the deciles for the preferred model, as well as a variation that
includes the overall and personal ratings for the baseline and expanded
models, respectively. These deciles show that, based on observables, AsianAmerican applicants are substantially less likely to be in the bottom five
deciles and are substantially more likely to be in the top deciles. For the
preferred model, the share of Asian-American applicants rises steadily with
every decile; the opposite trend occurs for African-American applicants. And
even when the personal rating is added, Asian Americans are still
overrepresented at the top of the distribution. Selection on unobservables
would have to be working in the opposite direction of selection on observables
to explain the negative Asian-American coefficient.
2.3 How the removal of preferences would impact the admitted
class
As in my opening report, I evaluate how the removal of penalties and
preferences for particular racial groups would affect admissions rates, fixing
the overall admissions rate in a particular year for a particular dataset
(baseline or expanded) to match with the data. For example, turning off the
penalty against Asian-American applicants would increase the number of
Asian Americans admitted. If no other adjustments were made, then
Harvard’s admitted class would be larger than Harvard intended. The
constant term in the logit admissions models is thus lowered for all groups
until the model-predicted overall probability of admission is the same as the
Note that I include perfect predictions here to show the total strength of the
applicant pool by race. Including the perfect predictions in this instance is
appropriate because we are looking at the full distribution of the effects.
3
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
probability of admission in the data.4 Results for these models are given in
Appendix C, Tables 8.1R and 8.2R.
Similar to my previous report, I find that the removal of racial penalties and
preferences has a profound impact on the admitted class:
•
Using the updated baseline dataset and my preferred model, removing
the Asian-American penalty in admissions results in increased AsianAmerican admits in all years. The model predicts 261 more AsianAmerican admits over this six-year period, a 13% increase.
•
Removing preferences for African-American and Hispanic applicants
(but keeping the penalty against Asian American applicants) results in
even larger gains, with 537 more Asian-American admits over the
period, an increase of more than 26%. And removing all racial
preferences and penalties—treating everyone as though they were
white—raises the number of Asian Americans by 799, a 40% increase.
The second panel of Table 8.1R in Appendix C looks at the share of the
admitted class by race/ethnicity under the different policies. Again, the
results are striking:
•
In the preferred model, removing the penalty against Asian Americans
increases their share of the admitted class by at least 2.3 percentage
points in all years, with the largest change in 2018 of 4.7 percentage
points.
•
Treating all applicants in a manner similar to whites has dramatic
effects: the share of admits who are Asian American increases by more
than 10 percentage points (a 40% increase in share), while the share of
admits who are African American falls by over 11 percentage points (a
72% decrease in share).
•
The fact that the racial composition of Harvard’s admitted class
depends so strongly on racial preferences indicates that race is a
determinative factor in admissions decisions.
My updated model again finds that the effect of removing racial preferences
To perform this exercise, I re-estimate the preferred model (Model 5) and the model
that includes the personal rating (Model 6) but now allowing for race times year
effects. Including these interactions ensures that, in each year, the admissions rate
for each racial/ethnic group matches the actual admit rate for that group.
4
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
on African Americans and Hispanics admit rates depends on disadvantaged
status. The estimates show that Harvard has a preference for disadvantaged
applicants, but that preference is smaller for Hispanics, who already receive
a large preference, and nonexistent for African Americans. With the removal
of racial preferences, disadvantaged African Americans and Hispanics receive
the same preference as other disadvantaged applicants. As shown in
Appendix C Table 8.3R (using Models 5 and 6), this preference is smaller
than the preference with racial preferences, but nonetheless substantial:
•
Disadvantaged African-American applicants see a 52% fall in the
number of admitted students in the preferred model.
•
For non-disadvantaged African-American applicants, the decline is
much larger at 80%. This occurs because the added boost that nondisadvantaged African-American applicants receive because of their
race is significantly smaller than the added boost disadvantaged
African-American applicants receive because of their race. As a result,
the share of African-American admits who are disadvantaged shifts
from 29% to 50%.
•
Similar patterns, though not quite as stark, occur for Hispanic
applicants: the drop in admits is 60% for non-disadvantaged students
and below 36% for disadvantaged students.
Using the expanded dataset and incorporating athletes5 brings additional
insight into how all of the preferences Harvard employs (race, legacy, athlete,
etc.) work against Asian-American applicants. Appendix C Table 8.2R shows
how the admitted class would change as racial and other preferences are
eliminated. Focusing on the preferred model and the scenario where race,
legacy, and athlete preferences are eliminated:
5 Recall that my updated expanded model no longer includes athletes. Hence all
counterfactuals that do not involve athletes treat the admissions decisions for athletes as
unchanged. When I do counterfactuals with athletes included, I replace their athletic rating
and extracurricular rating with 2s and then use the model to predict their admissions
probabilities.
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
•
The total number of Asian-American admits would rise by over 1,200
over the six-year period, or more than 50%.
•
In contrast, African-American admits would fall by 939 over the same
six-year window, a decline of over 68%.6
6 The number of white admits would increase by only 3% in this scenario. This occurs
because while the removal of racial preferences tends to favor white applicants, the removal
of legacy and athlete preferences harms white applicants.
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY