Students for Fair Admissions, Inc. v. President and Fellows of Harvard College et al

Filing 419

DECLARATION re 417 MOTION for Summary Judgment by President and Fellows of Harvard College. (Attachments: # 1 Exhibit 1, # 2 Exhibit 2, # 3 Exhibit 3, # 4 Exhibit 4, # 5 Exhibit 5, # 6 Exhibit 6, # 7 Exhibit 7, # 8 Exhibit 8, # 9 Exhibit 9, # 10 Exhibit 10, # 11 Exhibit 11, # 12 Exhibit 12, # 13 Exhibit 13, # 14 Exhibit 14, # 15 Exhibit 15, # 16 Exhibit 16, # 17 Exhibit 17, # 18 Exhibit 18, # 19 Exhibit 19, # 20 Exhibit 20, # 21 Exhibit 21, # 22 Exhibit 22, # 23 Exhibit 23, # 24 Exhibit 24, # 25 Exhibit 25, # 26 Exhibit 26, # 27 Exhibit 27, # 28 Exhibit 28, # 29 Exhibit 29, # 30 Exhibit 30, # 31 Exhibit 31, # 32 Exhibit 32, # 33 Exhibit 33, # 34 Exhibit 34, # 35 Exhibit 35, # 36 Exhibit 36, # 37 Exhibit 37, # 38 Exhibit 38, # 39 Exhibit 39, # 40 Exhibit 40, # 41 Exhibit 41, # 42 Exhibit 42, # 43 Exhibit 43, # 44 Exhibit 44, # 45 Exhibit 45, # 46 Exhibit 46, # 47 Exhibit 47, # 48 Exhibit 48, # 49 Exhibit 49, # 50 Exhibit 50, # 51 Exhibit 51, # 52 Exhibit 52, # 53 Exhibit 53, # 54 Exhibit 54, # 55 Exhibit 55, # 56 Exhibit 56, # 57 Exhibit 57, # 58 Exhibit 58, # 59 Exhibit 59, # 60 Exhibit 60, # 61 Exhibit 61, # 62 Exhibit 62, # 63 Exhibit 63, # 64 Exhibit 64, # 65 Exhibit 65, # 66 Exhibit 66, # 67 Exhibit 67, # 68 Exhibit 68, # 69 Exhibit 69, # 70 Exhibit 70, # 71 Exhibit 71, # 72 Exhibit 72, # 73 Exhibit 73, # 74 Exhibit 74, # 75 Exhibit 75, # 76 Exhibit 76, # 77 Exhibit 77, # 78 Exhibit 78, # 79 Exhibit 79, # 80 Exhibit 80, # 81 Exhibit 81, # 82 Exhibit 82, # 83 Exhibit 83, # 84 Exhibit 84, # 85 Exhibit 85, # 86 Exhibit 86, # 87 Exhibit 87, # 88 Exhibit 88, # 89 Exhibit 89, # 90 Exhibit 90, # 91 Exhibit 91, # 92 Exhibit 92, # 93 Exhibit 93, # 94 Exhibit 94, # 95 Exhibit 95, # 96 Exhibit 96, # 97 Exhibit 97)(Ellsworth, Felicia)

EXHIBIT 37

UNITED STATES DISTRICT COURT
FOR THE DISTRICT OF MASSACHUSETTS

STUDENTS FOR FAIR ADMISSIONS,
INC.,

Civil Action No. 1:14-cv-14176

Plaintiff,
v.
PRESIDENT AND FELLOWS OF
HARVARD COLLEGE (HARVARD
CORPORATION),
Defendant.

REBUTTAL REPORT OF DAVID CARD, Ph.D.
March 15, 2018

CONFIDENTIAL

Table of Contents
1. ASSIGNMENT AND SUMMARY OF FINDINGS ............................................................................................ 3 
1.1. Assignment ....................................................................................................................................3 
1.2. Summary of opinions ....................................................................................................................3 
2. A RELIABLE MODEL OF HARVARD’S WHOLE-PERSON EVALUATION PROCESS REQUIRES
DETAILED INFORMATION ON THE MANY NON-ACADEMIC AND CONTEXTUAL
FACTORS THAT HARVARD CONSIDERS ................................................................................................10 
2.1. Harvard’s admissions process seeks to find candidates with “distinguishing
excellences” across a variety of dimensions, not just academic achievement ......................10 
2.2. Harvard’s admissions process collects a lot of information on non-academic
performance ............................................................................................................................12 
2.3. Asian-American and White applicants possess different qualifications and
backgrounds, on average, across a variety of dimensions .....................................................15 
3. PROF. ARCIDIACONO’S KEY CRITICISMS OF MY ADMISSIONS MODEL REFLECT A
MISUNDERSTANDING OF HOW HARVARD’S ADMISSIONS PROCESS WORKS ....................18 
3.1. The personal rating is an important factor in admissions decisions, and excluding it
from the admissions model is not justified .............................................................................19 
3.2. There is no basis for Prof. Arcidiacono’s decision to exclude parental occupation,
intended career, or staff interviews ........................................................................................34 
3.3. Prof. Arcidiacono’s use of a pooled model is inconsistent with an essential feature of
Harvard’s admissions process and thus has no methodological basis ..................................43 
3.4. Prof. Arcidiacono’s decision to exclude certain types of applicants from his model is
inconsistent with how Harvard’s admissions process works, and is methodologically
unsound ...................................................................................................................................46 
4. AN ADMISSIONS MODEL THAT INCLUDES RELEVANT INFORMATION FINDS NO
EVIDENCE OF BIAS AGAINST ASIAN-AMERICAN APPLICANTS .................................................53 
4.1. My preferred regression model shows no evidence of bias against Asian-American
applicants ................................................................................................................................53 
4.2. Analysis of key subgroups of the data provides further evidence that there is no bias in
Harvard’s admissions process ................................................................................................60 
4.3. Prof. Arcidiacono’s new allegation of bias against dockets with larger shares of
Asian-American applicants lacks any causal credibility .......................................................64 
4.4. Other technical criticisms of my model do not change my findings..........................................67 
5. THE EVIDENCE IS NOT CONSISTENT WITH ADMISSIONS DECISIONS BEING
DETERMINED BY RACE ALONE................................................................................................................69 

CONFIDENTIAL

Page 1

5.1. Race alone is uninformative in Harvard’s decision process .....................................................69 
5.2. The fact that race has a relatively large effect on the probability of admissions for
some candidates cannot be taken as evidence that race is “determinative”..........................71 
6. DOCUMENTS AND HARVARD’S DATA UNDERMINE PROF. ARCIDIACONO’S CLAIM THAT
HARVARD IMPOSED A FLOOR ON THE ADMISSION RATE FOR SINGLE-RACE AFRICANAMERICAN APPLICANTS STARTING WITH THE CLASS OF 2017 ................................................75 
6.1. The record does not support Prof. Arcidiacono’s claim of a floor on single-race
African-American admissions starting with the class of 2017 ..............................................76 
6.2. The pattern that Prof. Arcidiacono claims as evidence of manipulation is not as
unlikely as he suggests ............................................................................................................79 
6.3. The relative quality of single-race African-American admitted students did not fall
starting with the class of 2017, further undermining the idea of a floor on their
admission rate .........................................................................................................................80 
7. MR. KAHLENBERG DOES NOT SHOW THAT HARVARD COULD ACHIEVE A COMPARABLY
DIVERSE AND HIGH-QUALITY CLASS WITHOUT CONSIDERING RACE ................................85 
7.1. The academic literature establishes that race-neutral alternatives diminish selective
universities’ ability to select on quality ..................................................................................86 
7.2. Mr. Kahlenberg’s new simulations confirm that the substitution of race-neutral
alternatives for Harvard’s race-conscious admissions process would change the
characteristics of the class and compromise its quality .........................................................90 
7.3. Other race-neutral alternatives are unlikely to generate diversity without changing
class characteristics and compromising class quality ............................................................97 
7.4. Conclusion ................................................................................................................................101 
8. APPENDIX A..........................................................................................................................................................103 
8.1. Documents Relied Upon ...........................................................................................................103 
9. APPENDIX B: OTHER TECHNICAL CRITIQUES OF PROF. ARCIDIACONO’S REPORT.........108 
9.1. Appendix B.1 Constructing categories for parental occupations ............................................108 
9.2. Appendix B.2: Error in Prof. Arcidiacono’s difference-in-difference estimates ....................112 
9.3. Appendix B.3: Using absolute deviation to measure the importance of unobserved
characteristics is appropriate ................................................................................................113 
10. APPENDIX C ....................................................................................................................................................... 116 
10.1. List of variables included in model of admission ...................................................................116

CONFIDENTIAL

Page 2

1. ASSIGNMENT AND SUMMARY OF FINDINGS
1.1. Assignment
1. I have been asked to review the Rebuttal Expert Report of Peter S. Arcidiacono
(“Arcidiacono Rebuttal”) and the Reply Declaration of Richard Kahlenberg (“Kahlenberg Rebuttal”)
in support of the claims of the Plaintiff, Students for Fair Admissions, Inc. (“SFFA”); assess the
reliability of the analyses therein; and comment on how those analyses affect (if at all) my opinions,
which I previously outlined in my first report (“Card Report”).
2. As with my first report, in conducting my review of the two rebuttals, I have relied on a
variety of sources of information, including the data and documents I relied on in my first expert
report (summarized in Appendix A of that report). Additionally, I have reviewed all of the relevant
supporting materials submitted by Prof. Arcidiacono and Mr. Kahlenberg for their rebuttals.
Appendix A to this report includes an updated list of the documents and data on which I relied in
forming the opinions expressed in this report.
1.2. Summary of opinions
3. In his rebuttal, Prof. Arcidiacono offers a variety of critiques of, and responses to, the
analysis in my first report. While his rebuttal covers much ground, his disagreements with my
analysis can be traced to a relatively simple methodological difference: My analysis is grounded in,
and motivated by, the actual process that Harvard employs in making admissions decisions, based on
my careful review of the available testimony and documentary evidence in the record. As I will detail
in this report, Prof. Arcidiacono’s analysis, on the other hand, is grounded in a variety of
misunderstandings about how Harvard’s process works, what factors Harvard values in the
admissions process, and how candidates are admitted. These fundamental misunderstandings explain
why Prof. Arcidiacono and I reach different conclusions, and why Prof. Arcidiacono’s admissions
model yields unreliable results.
4. There should be no dispute that a statistical model that reliably assesses SFFA’s claims of
bias against Asian-American applicants in this case must include as much information as possible
about the underlying process Harvard employs in making admissions decisions, given the available
data. In my first report I noted that “[a] basic tenet of econometric research is that the selection of
control variables should be informed by the research question at hand and the specific outcome that is
being modeled,” and that, as a result, “the first step in my analysis is to add to Prof. Arcidiacono’s
fullest models (Models 5 and 6) any variables missing from his models that Harvard considers in the

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 3

admissions process.”1 Indeed, in Sections 3 and 4 of my first report I spent more than 30 pages
summarizing in detail, based on the extensive record evidence, the factors Harvard values and the
process Harvard uses to collect as much information as possible regarding the many factors that drive
its admissions decisions. Additionally, for each piece of information in my model, I carefully
explained the reason for its inclusion and how it related to the actual decision process I was
modeling.
5. In both his original report and his rebuttal, Prof. Arcidiacono takes a different approach. He
models what he apparently believes Harvard’s process should value. In his 78-page rebuttal, Prof.
Arcidiacono does not refute the testimony or documents from Harvard College’s Office of
Admissions and Financial Aid (“Admissions Office”) or from individual admissions officers
regarding the core aspects of Harvard’s whole-person admissions process that I rely on to develop my
model. Instead, he repeatedly mischaracterizes the admissions process in a manner that misses critical
aspects of how Harvard makes decisions, and then relies on those mischaracterizations as a basis for
excluding critical information from the model that undermines his key findings. Most notably, he
continues to focus on the relative strength of Asian-American applicants on academic dimensions
while downplaying the fact—detailed repeatedly in Harvard’s documents and extensively
summarized in my first report—that academic excellence is the most abundant trait in Harvard’s
applicant pool, and that, as a result, it is not a particularly effective way for applicants to distinguish
themselves. In Section 2, I summarize the key aspects of Harvard’s admissions process that Prof.
Arcidiacono has mischaracterized and/or omitted.
6. In Section 3, I turn to a more detailed discussion of my key methodological and factual
disagreements with Prof. Arcidiacono, all of which derive from his apparent misunderstanding of
how Harvard’s admissions process works. Specifically, in Section 3, I explain how Prof.
Arcidiacono’s finding of alleged “bias” against Asian-American applicants depends entirely on his
decision to exclude important pieces of information from his admissions model—including
information about candidates’ life experiences, interests, and family backgrounds—that the record
indicates are essential to the admissions process. By excluding this critical information, he creates a
problem of “missing data” or “omitted variable bias” in his models, which leads to a misleading
appearance of discrimination against Asian-American applicants. As I show below, once all relevant
information is included, there is no evidence of discrimination.
7. The most critical example of this problem is Prof. Arcidiacono’s decision to exclude the
personal rating from his model. As I detail in Section 3.1, Prof. Arcidiacono continues to assert that
the personal rating is “biased” against Asian-American applicants and should therefore be completely
1

Expert Report of David Card, Students for Fair Admissions, Inc. v. President and Fellows of Harvard College (Harvard
Corporation), December 15, 2017 (“Card Report”), p. 47.
HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 4

excluded from his admissions model. Prof. Arcidiacono’s argument for excluding the personal rating
rests entirely on a set of empirical analyses that, when objectively considered, do not support his
claim of bias. Specifically, Prof. Arcidiacono has constructed a model of the personal rating that finds
that Asian-American applicants have lower personal ratings than White applicants, controlling for
other available factors. However, as I detailed in my first report (and again in Section 3.1 below), a
critical limitation of this analysis is that Prof. Arcidiacono’s personal ratings model cannot control for
all information that Harvard relies on when assigning personal ratings (including but not limited to
the personal essay, the full text of teacher and guidance counselor recommendation letters, any
supplemental recommendation letters, and any comments from alumni interviewers that have arrived
before the personal rating is assigned). As he does elsewhere, Prof. Arcidiacono simply ignores this
clear fact about Harvard’s admissions process, and asserts—contrary to the facts—that his model is
sufficiently robust to reliably measure racial “bias” in the personal rating. A more objective
interpretation of Prof. Arcidiacono’s personal ratings model is that it is not capable of reliably
determining whether the personal rating is in fact “biased” or whether his model is simply missing
critical aspects of the admissions process (e.g., the personal essay and other related information) that
could explain the differences in personal ratings if available. As I explained in my first report, this is
a very standard “omitted variable bias” problem that arises in statistical modeling where key pieces of
information are not included in the model.
8. What is particularly striking about Prof. Arcidiacono’s claim that his personal rating model
provides evidence of racial “bias” is that he reaches the opposite conclusion with regard to the
differences he finds in favor of Asian-American applicants in his academic and extracurricular ratings
models. Specifically, in his rebuttal, Prof. Arcidiacono argues that the positive bias in favor of AsianAmerican applicants that he finds in his academic and extracurricular ratings models reflects nothing
more than Asian-American applicants being “stronger on unobservable characteristics” that are
missing from his models.2 This, of course, is the exact explanation he rejects when interpreting the
negative unexplained gap between Asian-American and White applicants in the personal ratings as
racial “bias,” even though (as I show in my first report) Asian-American applicants are less strong, on
average, on the non-academic dimensions Harvard evaluates. For example, as I show below, White
applicants are more likely than Asian-American applicants to have strong scores across the three
ratings that Harvard’s interviewer handbook identifies as among the important inputs to the personal
rating—the teacher rating, the guidance counselor rating, and the alumni rating. White applicants are
also more likely to have strong scores across the three non-academic profile ratings (extracurricular,
athletic, and personal).
9. Prof. Arcidiacono cannot selectively apply the same reasoning in opposite ways. Either
2

Rebuttal Expert Report of Peter S. Arcidiacono, Students for Fair Admissions, Inc. v. President and Fellows of Harvard
College (Harvard Corporation), January 30, 2018 (“Arcidiacono Rebuttal”), p. 26.
HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 5

Harvard is biased against Asian-American applicants on the personal rating and biased in favor of
Asian-American applicants on the academic and extracurricular ratings, or his ratings regressions
reflect missing factors that are hard to measure – as he asserts about the academic and extracurricular
ratings. The fact that the single rating that Prof. Arcidiacono concludes is “biased” is the rating
where, on average, Asian-American applicants fare slightly less well than other groups makes clear
that Prof. Arcidiacono is selectively interpreting the evidence from his models. His decision to
exclude the personal rating from his overall admissions model is even more problematic when we
consider that documents from Harvard’s Admissions Office identify “unusually appealing personal
qualities” and “outstanding capacity for leadership” as two types of distinguishing excellence
Harvard seeks, and that the personal rating is the most relevant piece of available data that captures
such factors. As summarized in my first report, Harvard’s training materials for admissions officers
provide numerous tangible examples of the importance placed on personal qualities in evaluations of
individual candidates. Yet Prof. Arcidiacono excludes the personal rating without providing any
objective evidence of bias, and asserts that his model without the personal rating is more reliable than
one that includes it. Neither choice is defensible.
10. In a similar argument, Prof. Arcidiacono asserts that my inclusion of parental occupation
in the admissions model is flawed because there is “no evidence in the records that Harvard’s
admissions office considers parental occupation important aside from its value as a measure of SES”
and because it allegedly “oscillates wildly from year-to-year.”3 As I explain in Section 3.2 below,
deposition testimony and other evidence in the record indicate that parental occupation is an
important piece of information that Harvard admissions officers consider when reading applications
and discussing candidates at admissions meetings. Further, of the available variables that reflect
socioeconomic status in Harvard’s data, it contains the most detailed information. There is simply no
basis for excluding an important piece of information that Harvard considers in its process. Prof.
Arcidiacono also overstates the year-to-year fluctuations in the occupation data I rely on and fails to
recognize that year-to-year fluctuations are not of concern so long as the model allows the effect of
parental occupation to vary from one year to the next (as mine does). There is no objective
justification for excluding this important information from the model. Prof. Arcidiacono’s
methodological motivation appears to be that excluding parental occupation substantially increases
the alleged disparity between Asian-American applicants and White applicants. As I explain in
Section 3.2, Prof. Arcidiacono applies similarly flawed reasoning in arguing that staff interview
ratings and the applicant’s intended career—two variables that Harvard uses in its process—should
also be excluded from the model.
11. In Section 3.3 and Section 3.4, I address two additional methodological flaws in Prof.
Arcidiacono’s analysis, which also stem from his apparent misunderstanding of how Harvard’s
3

Arcidiacono Rebuttal, p. 6.
HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 6

admissions process works. First, Prof. Arcidiacono continues to pool all the applicants who apply in
different years into a single regression model, even though Harvard’s admissions process is distinct in
each year. As I explain in Section 3.3, Prof. Arcidiacono’s approach imposes the clearly incorrect
assumption that applicants from the class of, for example, 2019 are competing for slots with
applicants from the class of 2014. There is simply no evidentiary or logical defense for such an
assumption. The only credible methodological reason Prof. Arcidiacono offers for pooling together
applicants from six different classes into a single model is that estimating a separate model for each
separate year of applicants “reduces the statistical power of the sample.” As I explained in my first
report and again below, by estimating a separate model for each class of applicants and then taking
the average of the relevant results across all years of data, I resolve this problem with no reduction in
the statistical power of the sample. In fact, the statistical power of my model—as summarized by the
precision of the estimated average marginal effect of Asian-American ethnicity—is actually slightly
higher than that of Prof. Arcidiacono’s pooled model.
12. Second, Prof. Arcidiacono argues that a particular set of purportedly “special” candidates
(e.g., recruited athletes, lineage applicants) should be excluded from the admissions model because
Harvard allegedly conducts a “special” admissions process for such candidates.4 As I explain in
Section 3.4, I have seen no evidence that Harvard conducts a separate admissions process for such
candidates, nor has Prof. Arcidiacono presented any such evidence. While it is true that Harvard gives
a “tip” in its admissions process to candidates who have certain characteristics, that “tip” in no way
guarantees their admission nor does it remove the need for such candidates to possess other
characteristics valued by Harvard. For example, as I show below, even among candidates who
possess the “special” characteristics, the subset who are ultimately admitted are the ones who possess
many other characteristics Harvard values. Further, my admissions model shows that such candidates
have stronger overall profiles and more valued characteristics than other applicants, even when their
“tip” is removed. In other words, no applicant is guaranteed admission simply based on one trait—
every applicant still has to compete with the larger pool on other dimensions. As with the issues
discussed above, Prof. Arcidiacono’s decision to exclude these candidates from his model is not
consistent with how the admissions process works, and removes important information from his
model that helps quantify how Harvard makes decisions across the many characteristics it values.
13. In Section 4, I shift the discussion away from Prof. Arcidiacono’s critiques of my
analysis, and show that when all relevant variables Harvard considers in its admissions process are
included in my admissions model, I continue to find no evidence of bias against Asian-American
applicants. Indeed, my model shows no evidence of bias even when I incorporate the various
technical concerns raised by Prof. Arcidiacono. As part of the discussion in Section 4, I also address
Prof. Arcidiacono’s new claim that the alleged bias against Asian-American applicants is not applied
4

Arcidiacono Rebuttal, p. 69.
HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 7

to all applicants, but is instead somehow concentrated on a subset of applicants. As a conceptual
matter, I discuss the implausibility of such a theory of discrimination. What is the basis of Prof.
Arcidiacono’s new claim? What evidence in the record supports it? Why would Harvard pursue such
a complex form of discrimination? Prof. Arcidiacono does not provide answers to these questions.
14. Moreover, I show that the results I presented in my initial report demonstrating that there
is evidence of a positive (though statistically insignificant) estimated effect of Asian-American
ethnicity for two large subgroups, Asian-American women and Asian-American applicants from
California, continue to hold even after I make modifications to address Prof. Arcidiacono’s
criticisms. These large subgroups account for nearly two-thirds of Asian-American applicants. As I
discussed in my first report, this finding strongly supports my broader point that the unexplained gaps
in admission rates between Asian-American and White applicants are best interpreted as reflecting
differences in characteristics that are not perfectly measured by the admissions data, rather than racial
bias against Asian-American applicants.
15. In Section 5, I turn to the question of whether race is a determinative factor in the Harvard
admissions process. In his rebuttal, Prof. Arcidiacono opines that it is. His opinion is predominantly
driven by the relatively large effect of race for the subset of competitive African-American and
Hispanic candidates. This effect is not a new finding; I discussed it in my initial report. What I show
below, however, is that the magnitude of this effect is not unique to race and, thus, cannot support the
inference that race is a determinative factor in the admissions process. More specifically, what we see
in the data is that, for any candidate who has a relatively strong profile, the incremental marginal
effect of adding any additional valued characteristic to his or her profile—e.g., a strong academic or
extracurricular or personal profile rating—can be relatively large, and in some cases larger than the
effect of race. This pattern is exactly what would be expected given Harvard’s whole-person
admissions process. In order to be admitted from the highly competitive applicant pool at Harvard,
any candidate must have multiple areas of strength in his or her profile, which means that changing
any single characteristic of a candidate who is otherwise already competitive can substantially raise
his or her chance of being admitted. Importantly, this feature of the process does not imply that any
single characteristic on its own determines admissions decisions. In fact, the situation is just the
opposite. Without being strong on multiple dimensions valued by Harvard, a candidate has little
chance of admission. This is clearly evident from the large fraction of African-American and
Hispanic applicants who are rejected in the admissions process. Only when a candidate reaches a
certain level of overall strength can any additional characteristic help his or her candidacy
significantly.
16. In Section 6, I turn to Prof. Arcidiacono’s allegation that Harvard is imposing a floor on
the African-American admission rate. As I discuss in that section, three considerations undermine

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 8

Prof. Arcidiacono’s claims. First, Prof. Arcidiacono does not present persuasive facts in support of
his claim that Harvard was so concerned about the single-race African-American admission rate that
it chose to manipulate that rate. Indeed, his explanation for the claim has shifted between his two
reports, reflecting the fact that the documentary record does not support his story. Second, the pattern
in the data that Prof. Arcidiacono claims as evidence of manipulation could be explained by chance.
As I discuss in Section 6 below, given that Harvard has used at least three different classifications of
race (New Methodology, Old Methodology, IPEDS) during the period in question, and that each
classification includes multiple racial groups, a finding that the admission rate for a racial subgroup is
close to the overall admission rate is not as unusual as Prof. Arcidiacono would have one believe.
Finally, as I noted in my first report, if Harvard had in fact imposed a floor on African-American
admission rates, we would expect to see the relative quality of African-American students fall. Prof.
Arcidiacono presents an analysis claiming that this happened. As I discuss in Section 6, his analysis
contains a calculation error that, when corrected, reverses his finding.
17. In Section 7, I respond to several arguments from the Kahlenberg Rebuttal about whether
race-neutral alternatives are a viable way for Harvard to achieve its diversity goals, while also
maintaining the overall quality of its student body. As I show below, the two additional race-neutral
alternatives Mr. Kahlenberg presents in his rebuttal do not allow Harvard either to achieve its
diversity goals or to maintain the overall quality of the student body. Both alternatives generate a
reduction in the share of the student body that is African-American and a decrease in the overall
quality of the class. As I discuss in Section 7, these results are not surprising—the broad academic
literature has established that race-neutral alternatives can achieve racial diversity at selective
institutions only at a cost to the quality of the admitted class. None of the analyses Mr. Kahlenberg
presents in his rebuttal changes the central finding of my first report: the overall profile of the student
body would change significantly if Harvard ceased considering race as one factor among many.

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 9

2. A RELIABLE MODEL OF HARVARD’S WHOLE-PERSON EVALUATION PROCESS
REQUIRES DETAILED INFORMATION ON THE MANY NON-ACADEMIC AND
CONTEXTUAL FACTORS THAT HARVARD CONSIDERS
18. As detailed in Sections 3 and 4 of my original report, the starting point for my analysis of
Harvard’s admissions process is a detailed assessment of the key factors considered in that process.
This assessment is necessary to ensure that my model captures as much information as possible that
Harvard actually relies on when evaluating candidates.
19. Understanding the decision process is a standard methodological approach to any
empirical project, and, as noted above, it is especially critical for understanding why Prof.
Arcidiacono and I reach different conclusions about the key issues in this case. As I detail in this
report, Prof. Arcidiacono appears to misunderstand certain aspects of Harvard’s process, which leads
him to exclude important pieces of information from his model.
20. Given the importance of an accurate understanding of Harvard’s admissions process, in
this section I provide a brief summary of the key features that I detailed in my first report. I focus on
the features that will be relevant to the analysis in the remainder of this report.
2.1. Harvard’s admissions process seeks to find candidates with “distinguishing excellences” across
a variety of dimensions, not just academic achievement
21. The guiding principle of Harvard’s admissions process is a full evaluation of each
candidate’s high school achievements (on many dimensions), life experiences, and personal
background. As noted in my first report, this principle is succinctly described in the Admissions
Office’s 2014 – 2015 Interviewer Handbook (“Interviewer Handbook”), in a passage entitled “The
Search for Distinguishing Excellences”:
Redacted

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 10

Redacted

.5
22. The Interviewer Handbook then identifies some specific qualities that are common ways
for a candidate to distinguish herself from the applicant pool. For example, the Interview Handbook
states Redacted

6

23. One important fact to note about Harvard’s whole-person evaluation process, as evidenced
by the list above, is that academic qualifications are just one of many factors Harvard values. In fact,
traditional academic achievements like high test scores and GPAs are some of the most abundant
traits in the applicant pool. For example, as I explained in my first report, there were only 1,756
domestic applicants admitted in the class of 2019, yet 2,741 applicants had a perfect SAT Verbal
score, 3,450 applicants had a perfect SAT Math score, and over 8,000 applicants had a perfect GPA
(Exhibit 1). In fact, the “Interviewer Handbook” estimates that Redacted
7

24. Because strong test scores and GPA alone are so abundant, the evaluation of academic
quality extends beyond such quantitative measures. As I noted in my first report, academic evaluation
also accounts for the admissions officer’s knowledge of the applicant’s high school; the high school's
curriculum; appraisals of the candidate’s academic work by Harvard faculty––referred to as faculty
reads; academic honors or awards; and writing skills.8

5

Interviewer Handbook, 2014-2015, HARV00001392 – 1438 (“Interviewer Handbook”) at HARV00001400 – 01.
Interviewer Handbook at HARV00001401 – 02.
7
Interviewer Handbook at HARV00001401. Other documents from Harvard support this account of the admissions
process. For example, in a presentation given to guidance counselors at schools in the Sarasota, Florida area, Harvard
admissions officer Kanoe Williams explained that test scores are just a “small piece” of Harvard’s whole-person
evaluation; that, “in general, we can tell pretty quickly if a student will be an academic fit for our school”; and that “the
lengthier part of the conversation typically focuses on intangibles, the qualitative pieces” (Sarasota Presentation, “KLW Sarasota Presentation,” HARV00013561 – 65 at HARV00013563 – 64).
8
Card Report, p. 27
6

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 11

Academic excellence is abundant in Harvard’s applicant pool

Source: Arcidiacono Data
Note: Data are from applicants to the class of 2019 in Prof. Arcidiacono’s original expanded sample including athletes. Harvard converts
applicant GPAs to a 35-80 scale.

2.2. Harvard’s admissions process collects a lot of information on non-academic performance
25. As noted in my first report, in order to collect reliable information on the numerous
“distinguishing excellences”—many of which are inherently difficult to quantify—Harvard employs
an admissions process that emphasizes the evaluations and perspectives of multiple admissions
officers, interviewers, and faculty members. Harvard describes this as “a rigorous comparative”
process.9 It involves the collection of detailed information about each candidate’s life experience,
achievements, academic potential, personality, and family/community background through a variety
of interviews, essays, and relevant data (including evaluations from Harvard faculty). I described the
details of the process in my first report as follows:
Once all applications from a particular docket have been reviewed, the
subcommittee for that docket meets to discuss the applications. My
understanding is that during this process, the first reader summarizes the
strength of the applications he or she has read. Subcommittee members

9

Interviewer Handbook at HARV00001408.
HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 12

discuss applications, and then vote on each application to recommend an
action to the full Committee. The degree of support expressed for
applicants is noted to allow for comparisons with applicants from other
subcommittees. The full Admissions Committee then meets to discuss
the candidates recommended by each subcommittee. For Regular
Decision applicants, full committee meetings take place over the course
of approximately two weeks during March. My understanding is that
during the full committee process, the first reader, or area person, for an
application generally presents the applicant’s file to the full Committee,
and may choose to project portions of the application on a screen during
the discussion so that the Committee can review important components
of the application. For example, deposition testimony indicates that the
admissions officer presenting the case might use excerpts of visual art or
music submissions or academic papers to highlight an applicant’s skills,
and that discussions in subcommittee or in full Committee on a single
applicant may range in length up to a half hour or more. The full
Committee compares all candidates across all subcommittees (footnotes
omitted).10
26. Central to Harvard’s evaluation process are the four profile ratings, which are designed to
capture the detailed data and information collected during the evaluation process on four key
dimensions of quality Harvard values: academic, extracurricular, athletic, and personal. As detailed in
my first report, Harvard’s admission data on profile ratings bear out the importance of non-academic
strength in Harvard’s process. Below are several important patterns in the admission data discussed in
my first report that demonstrate the value Harvard places on non-academic qualities:
• Non-academic skills are scarce: “[A]pplicants who are highly rated on
non-academic dimensions are much scarcer than applicants with a high
academic rating. Exhibit 5 shows that about 42% of applicants have an
academic rating of 1 or 2, while fewer than 25% of applicants receive a
1 or 2 on each of the other three profile ratings. Applicants with a rating
of 2 or better on at least three dimensions are even rarer—just 7% of the
applicant pool. These data indicate that high ratings on non-academic
dimensions (and particularly on multiple non-academic dimensions)
10

Card Report, pp. 24–25.

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 13

distinguish applicants in the pool much more effectively than a high
academic rating”11
• Non-academic skills explain admissions decisions better than
academic skills: “Another way to see the importance of non-academic
dimensions relative to academic dimensions of excellence is to examine
how important each element is in explaining which applicants are
admitted…. In Prof. Arcidiacono’s expanded sample, the Pseudo RSquared of a model that includes only the academic rating as a control
variable is 0.09, while the Pseudo R-Squared of models that include
each of the three non-academic ratings as the sole control variables are
0.20 (personal), 0.09 (extracurricular), and 0.08 (athletic), and the
Pseudo R-Squared for a model that includes all three non-academic
ratings as control variables is 0.32. In non-technical terms, this means
that non-academic factors (taken together) explain more than three times
as much of the variation in admissions decisions as the academic rating
does. That should not be surprising, since exceptional non-academic
qualities are less common in the applicant pool than exceptional
academic qualities and are thus more likely to distinguish applicants
from one another” (footnote omitted).12
• Being multi-dimensional is important: “Exhibit 6 shows that only
12% of admitted students are “one-dimensional stars” with a rating of 1
on one dimension but fewer than three ratings of 2 or better, while 46%
are multi-dimensional applicants with three or four ratings of 2 or better,
and 31% have two ratings of 2 and two ratings of 3. These statistics are
yet another way to show the value that Harvard places on applicants who
distinguish themselves on multiple dimensions.”13
• Athletic rating is important: “Harvard’s admissions data confirm the
importance of the athletic rating. For example, applicants with an
athletic rating of 2 have an admission rate of 12%. That is substantially
higher than the overall admission rate of approximately 7%, [for
domestic applicants], and is the same as the admission rate of applicants
with an academic rating of 2. Further, as shown above, receiving a rating
11

Card Report, pp. 28–29.
Card Report, pp. 29–30.
13
Card Report, p. 30.
12

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 14

of 2 on all four profile ratings is associated with an admission rate of
68%, while receiving a rating of 2 on the three non-athletic ratings and a
rating of 3 or worse on the athletic rating is associated with an admission
rate of only 48%. This contrast provides further evidence of the
incremental importance of an athletic rating of 2” (footnote omitted).14
27. Despite these facts about Harvard’s admissions process, Prof. Arcidiacono has repeatedly
focused on academic achievement as the most important characteristic in the admissions process. For
example, in both of his reports Prof. Arcidiacono employs Harvard’s Academic Index as a general
measure of quality for applicants to Harvard. (Harvard’s Academic Index is a value which
summarizes an applicant’s strength across SAT scores, ACT scores, and grades (GPA).)15 In fact, in
his first report and again in his rebuttal, he offered an analysis that predicted how Harvard’s class
would look if it only considered Academic Index.16 This type of analysis does not reflect Harvard’s
process, and, as I will detail below, is an important reason why Prof. Arcidiacono’s claim of “bias”
against Asian-American applicants is flawed. The fact that Prof. Arcidiacono chose to present such
an analysis underscores his fundamental misunderstanding of Harvard’s process.
2.3. Asian-American and White applicants possess different qualifications and backgrounds, on
average, across a variety of dimensions
28. SFFA’s and Prof. Arcidiacono’s core theory of racial bias against Asian-American
applicants centers on Prof. Arcidiacono’s conclusion that, on average, Asian-American applicants are
academically stronger than applicants of other races, but they are admitted at a lower rate than White
applicants. This theory ignores two critical facts: First, strong academic performance is just one
factor in the process, and is also one of the most abundant characteristics in the applicant pool—that
is, it does little to distinguish some applicants from others. Second, Asian-American applicants are,
on average, not as strong as White applicants on several important non-academic measures. As
detailed in my first report, the data show that, while Asian-American applicants have stronger
average academic measures, they are weaker on average on athletic and personal ratings, less likely
to be strong on multiple ratings, and less likely to have high ratings across all three non-academic
ratings taken together. Exhibit 2 and Exhibit 3 below report the same data as in my first report.

14

Card Report, p. 31. It is worth noting that, in his rebuttal, Prof. Arcidiacono continues to make the incorrect assertion
that athletic rating is less important than other ratings. Prof. Arcidiacono states, “the athletic rating is not as important to
the admissions decision as the other ratings once recruited athletes are removed” (Arcidiacono Rebuttal, p. 31). As shown
above, Harvard’s admissions data directly contradict this statement.
15
Ivy League AI Calculator Information 20051.xlsx, HARV00001895, Tab “IVY LEAGUE AI 2005-2006.”
16
Expert Report of Peter S. Arcidiacono, Students for Fair Admissions, Inc. v. President and Fellows of Harvard College
(Harvard Corporation), October 16, 2017 (“Arcidiacono Report”), pp. 44–45; Arcidiacono Rebuttal, p. 13.
HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 15

White and Asian-American applicants excel in different dimensions

Source: Arcidiacono Data
Note: Data are from applicants to the classes of 2014 – 2019 in Prof. Arcidiacono’s original expanded sample including athletes. Ratings of
2- and above are considered “2 or Better” in this analysis. +/- rating designations were introduced beginning with the class of 2019.

For a given academic rating, White applicants tend to have better non-academic ratings than
Asian-American applicants

Source: Arcidiacono Data
Note: Data are from applicants to the classes of 2014 – 2019 in Prof. Arcidiacono’s original expanded sample including athletes.

29. In addition to differences in non-academic strength, there are also average differences

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 16

between White and Asian-American applicants (i.e., differences between Asian-American and White
applicants (on average)) on numerous other factors Harvard weighs in its admissions decisions,
including proxies for life experience and interests like parental occupation, intended major, intended
career, geography, and high school.17
30. In Section 3, I analyze the differences between Asian-American and White applicants in
more detail, when I address Prof. Arcidiacono’s claim that the differences in personal ratings
observed in Harvard’s sample are a result of racial bias, rather than differences in the underlying
characteristics and backgrounds, on average, between the two groups. As I explain in that section, an
objective analysis of the many average differences between Asian-American and White applicants
does not support Prof. Arcidiacono’s conclusion of bias. Rather, it supports a much simpler
conclusion—while the set of Asian-American high-school students who apply to Harvard tend on
average to be slightly stronger than the set of White applicants in academic respects, they tend to be
weaker in non-academic dimensions.

17

Throughout my report, I use the term “average differences” to refer to differences in the average level of a given
attribute across groups.
HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 17

3. PROF. ARCIDIACONO’S KEY CRITICISMS OF MY ADMISSIONS MODEL REFLECT A
MISUNDERSTANDING OF HOW HARVARD’S ADMISSIONS PROCESS WORKS
31. The different conclusions that Prof. Arcidiacono and I reach with regard to SFFA’s claim
of alleged bias against Asian-American applicants can be explained by three critical methodological
errors in Prof. Arcidiacono’s analyses. First, Prof. Arcidiacono excludes several variables key to
Harvard’s admissions process—in particular, the personal rating, parental occupation, intended
career, and staff interview ratings—claiming that that they are biased and/or unreliable. Second, Prof.
Arcidiacono pools together applicants from different admissions cycles into a single admissions
model. Third, Prof. Arcidiacono excludes data from several categories of applicants—lineage
applicants, recruited athletes, children of Harvard faculty and staff, and applicants on the Dean’s or
Director’s interest lists—under the erroneous assumption that they are subject to a separate
admissions process that is free of the alleged bias Prof. Arcidiacono’s flawed model purports to show.
32. As I explain below, each of these key methodological choices by Prof. Arcidiacono stems
from an incorrect (or incomplete) view of how Harvard’s admissions process works, and all of them
lead to a model that focuses too much on academic achievement and ignores important non-academic
factors. The unexplained gap in admissions in Prof. Arcidiacono’s models reflects a standard
“omitted variable bias” that arises when important variables are excluded from a statistical model.18
This type of “omitted variable bias” is such a common methodological problem in econometric
analysis that a popular textbook describes it as the first issue that a researcher should think about.19 It
occurs whenever a regression model omits a control variable that is correlated with the independent
variable of interest (in this case, race) and influences the outcome variable (in this case, admissions).
It is problematic because it causes the model to misattribute to other independent variables the effect
on the outcome actually caused by the omitted variable.
33. For example, suppose that younger employees in a firm are concerned that they are paid
less than older employees due to some form of bias by their employer. And suppose that the
18

James H. Stock and Mark W. Watson, Introduction to Econometrics (Pearson, 2015), pp. 183–184 (“If the regressor is
correlated with a variable that has been omitted from the analysis and that determines, in part, the dependent variable,
then the OLS estimator will have omitted variable bias.”); Sharmila Choudhury, “Reassessing the Male-Female Wage
Differential: A Fixed Effects Approach,” Southern Economic Journal 60(2), 1993, pp. 327–340 at p. 327 (“The
conventional approach of economists has been to estimate earnings as a function of various socio-economic
characteristics. The observed wage gap is decomposed into a part explained by productivity related factors and an
unexplained residual, traditionally labelled as discrimination. While it is possible that the unexplained variation in
earnings is the result of discrimination, it is also possibly the result of model misspecification ... we address the
misspecification that could possibility arise from omitted variables…”).
19
James H. Stock and Mark W. Watson, Introduction to Econometrics (Pearson, 2015), pp. 232–234 (“The starting point
for choosing a regression specification is thinking through the possible sources of omitted variable bias… A control
variable is not the object of interest in the study; rather it is a regressor included to hold constant factors that, if neglected,
could lead the estimated causal effect of interest to suffer from omitted variable bias.”).
HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 18

employer’s pay guidelines identify two attributes of each employee that affect pay: the employee’s
level of education, and his or her number of years of experience in the occupation. In this example,
the outcome variable is the employee’s salary, and the independent variable of interest is his or her
age. If a regression controls only for the employee’s age and education—but not his or her number of
years of experience—then the regression may misattribute to the employee’s age an effect on salary
that is actually due to the employee’s number of years of experience. That is because the number of
years of experience is correlated with both the employee’s age and his or her salary.
34. The record makes clear that Prof. Arcidiacono’s model is missing key variables and is
therefore flawed. Further, as I showed in my first report (and as I show again in Section 4 of this
report), when all of the information Harvard relies on in making its admissions decisions is included
in the admissions model, there is no statistical evidence that Asian-American applicants are being
admitted at lower rates than White applicants.
3.1. The personal rating is an important factor in admissions decisions, and excluding it from the
admissions model is not justified
35. As detailed in my first report, Prof. Arcidiacono’s claim that the personal rating is biased
against Asian-American applicants is not supported by his own analysis. In his rebuttal, Prof.
Arcidiacono once again asserts that the personal rating is biased against Asian-American applicants,
pointing (once again) to the unexplained gap between White and Asian-American applicants in his
personal rating regression, i.e., the gap that remains after controlling for the other candidate
characteristics that are included in his model.
36. Prof. Arcidiacono provides two arguments for why the unexplained gap in his personal
ratings regression is evidence of racial bias. First, he argues that his personal ratings regression is
reliable because it has a high explanatory power by academic standards. Second, he argues that there
is no alternative explanation for the unexplained gap because there is no evidence that AsianAmerican applicants are weaker, on average, on the observable factors that affect the personal
ratings.20 In what follows, I address both arguments and explain why, despite his claims to the
contrary, (a) his personal ratings regression is missing key factors that affect the rating; (b) it is
reasonable to conclude that Asian-American applicants are weaker, on average, on those missing
factors; and (c) as a result, the unexplained gap is not evidence of bias.
3.1.1. Prof. Arcidiacono’s personal ratings regression is missing critical information
37. I start with Prof. Arcidiacono’s claim that his personal ratings regression reliably captures
20

Arcidiacono Rebuttal, pp. 22–23, 26–27.
HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 19

relevant factors that drive the personal rating, and thus reliably measures the alleged “bias” against
Asian-American applicants.
38. In my first report, I argued that Prof. Arcidiacono’s personal ratings regression “cannot
reliably explain the assignment of personal ratings” because (a) it had a relatively low Pseudo RSquared,21 and (b) it suffered from omitted variable bias due to the omission of the personal essay
and other difficult-to-measure factors that affect the personal rating.22 In response to my criticisms,
Prof. Arcidiacono cites an academic paper from 1979 by Daniel McFadden that indicates that the
Pseudo R-Squared of Prof. Arcidiacono’s personal ratings regression (0.28) is sufficiently high to be
considered an “excellent fit.”23 He also offers some discussion and analysis of the predictive power of
his personal ratings regression.24
39. While I have no major disagreement with the paper he cites, or the calculations he
presents,25 I believe that Prof. Arcidiacono’s response entirely misses the broader point of my
original critique. Most importantly, even a model that has a relatively high Pseudo R-Squared still
may suffer from omitted variable bias in estimating the effect of Asian-American ethnicity. In fact,
Prof. Arcidiacono himself recognizes this principle. For example, when discussing his own academic
rating regression—which has a Pseudo R-Squared (0.56) that is nearly double that of his personal
rating regression (0.28)—Prof. Arcidiacono explains that the differences across racial groups in
academic ratings capture “unobservable characteristics” outside of his model, not racial bias.26 In
other words, he believes his academic rating model, with a Pseudo R-Squared of 0.56, suffers from
omitted variable bias. A second factor, however, is that the risk of omitted variable bias is larger
when the model has lower explanatory power. This is widely understood in the academic literature,
and implies that Prof. Arcidiacono’s personal rating model is even more vulnerable to omitted

21

As noted in my first report, Pseudo R-Squared captures how well a variable or set of variables can explain outcomes—
in this case, admissions decisions. The statistic takes on values from zero to one; the closer it is to zero for a given model,
the less information the variables in that model provide about admissions decisions, while a value closer to one means the
model explains a higher proportion of the variability in the actual decisions.
22
Card Report, pp. 69–70.
23
Arcidiacono Rebuttal, p. 23.
24
Arcidiacono Rebuttal, pp. 23–25.
25
I note, however, that the McFadden paper that Prof. Arcidiacono relies on was written in 1979, when access to, and
computing power sufficient to analyze, the type of detailed microdata analyzed in this case did not exist. In modern
empirical analyses—particularly where, as here, voluminous data are available—the Pseudo R-Squared of Prof.
Arcidiacono’s personal ratings regression of 0.28 would not be considered strong.
26
Arcidiacono Rebuttal, p. 26.

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 20

variable bias than his academic rating model.27
40. The key issue with Prof. Arcidiacono’s personal ratings regression, therefore, is not that
its Pseudo R-Squared value is lower than might be optimal, but that the model is missing various
factors that inform the personal rating that are not quantifiable and thus are not observed in the data.
For example, the model is missing an assessment of the applicant’s personal essay. As I explained in
my first report, Harvard’s Interviewer Handbook explicitly identifies information in the personal
essay as a consideration central to the personal rating,28 as does testimony from numerous admissions
officers.29 Prof. Arcidiacono’s regression also includes only limited and incomplete summary
information from the teacher and guidance counselor reports. For example, available application files
produced in this case indicate that many applicants have recommendation letters from more than two
teachers and/or supplementary letters of reference from figures like research supervisors or
extracurricular instructors, even though such information is not reflected in the database.30 The
available applications also indicate that teacher and guidance counselor letters often review both
academic and personal qualities, but are summarized by Harvard admissions officers using a single
27

Emily Oster, “Unobservable Selection and Coefficient Stability,” Brown University and NBER Working Paper #19054,
August 9, 2016, p. 3 (“The key observation is that the quality of the control is diagnosed by how much of the variance in
the outcome is explained by its inclusion or, equivalently, how much the R-squared moves when the controls are
introduced. Omitted variable bias is proportional to coefficient movements, but only if such movements are scaled by the
change in R-squared when controls are included.”).
28
Interviewer Handbook at HARV00001401 Redacted

29

Deposition testimony indicates that the personal essay is also a key factor in evaluating personal qualities. See, for
example, Deposition of Roger Banks, May 4, 2017 (“Banks Deposition”), p. 80 (“Q. Okay. So for the last category, the
[personal qualities]—the main inputs you would look at were recommendations, interview, and anything else? A. The
candidate’s essay.”); Deposition of Brock Walsh, June 28, 2017 (“Walsh Deposition”), p. 60 (“Q. How would you
calculate that score?…[A.] I would like to take into consideration whatever relevant information I had were that his essay,
her essay, her interview, and the opinions about that applicant as expressed by others.”); Deposition of Tia Ray, June 7,
2017 (“Ray Deposition”), pp. 21–22 (“Q. What are the materials that you use—materials or considerations that go into
determining this person’s score?…A. For example, content in recommendation letters, personal essays.”).
30
For example, application HARV00079421 – 75 contains three teacher recommendations. As evidenced by the reader’s
notes at HARV00079422, all three letters were taken into consideration when reviewing the application: “Support prose is
very nice; SSR says she is the best in the class, T3 says best in 13 years, and T2 says one of the best in 20 years. There are
noticeable checks down for concern for others on both T1 and T2, though, and many references to how driven she is.”
Application HARV00079519 – 63 contains three teacher letters (at HARV00079543 – 9), as well as a supplementary
letter of support from the school's Club Faculty Advisor (at HARV00079554 – 5). Application HARV00079476 – 518
also contains three teacher recommendations (at HARV00079500 – 6), as well as a recommendation from the applicant’s
Tae Kwon Do instructor (at HARV00079510 – 1). As another example, application HARV00079325 – HARV00079420
contains two recommendations from teachers (at HARV00079349 – 55), as well as a letter of recommendation from the
supervisor of a lab at Redacted
where the applicant conducted research (at HARV00079381 – 2).

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 21

rating from which it is impossible to separate the academic qualities from the personal qualities.31
41. The fact that key factors central to the determination of the personal rating are missing
from Prof. Arcidiacono’s personal rating regression, and the fact that it has a relatively weak Pseudo
R-Squared (as compared to his academic rating regression), mean that Prof. Arcidiacono’s personal
ratings regression cannot establish that differences across ethnic groups in the personal rating are
caused by racial bias (as he claims), rather than by average differences in other personal
characteristics not included in the database that Prof. Arcidiacono and I analyze.
42. What is particularly striking about Prof. Arcidiacono’s claim of bias in the personal rating
is how he reaches the opposite conclusion for the academic and extracurricular ratings, even when the
pattern of empirical evidence is the same. For example, Prof. Arcidiacono interprets the unexplained
racial gaps in his academic and extracurricular ratings that favor Asian-American applicants as
evidence that his model is missing important information, rather than that Harvard is biased in favor
of Asian-American applicants. However, when faced with an unexplained gap in in the personal
rating regression (that favors White applicants rather than Asian-American applicants), he rules out
this same “unobserved characteristics” explanation and instead jumps to the conclusion that there is
bias against Asian-American applicants. In light of the known factors that inform the personal rating
but are not captured in Prof. Arcidiacono’s personal rating model, and the relatively low explanatory
power of his personal rating model, it is indefensible for Prof. Arcidiacono to claim that the disparity
in personal ratings between Asian-American and White applicants is evidence of bias, while
simultaneously arguing that the (converse) disparity in his academic ratings model is due to omitted
variable bias.
43. Prof. Arcidiacono cannot selectively apply the same reasoning in different ways. Either
his ratings regressions reflect missing factors that are hard to measure—as he asserts about the
academic and extracurricular ratings—or they reflect a complex (and unusual) form of racial
discrimination whereby Harvard favors Asian-American applicants on academic and extracurricular

31

For example, see the teacher and guidance counselor recommendations in application HARV00079325 –
HARV00079420. A letter from the applicant’s math teacher describes both the student’s mathematical aptitude, as well as
the student’s personal qualities, such as his willingness to help other students and his “politeness and respect for his
fellow man, as well as his sense of humor” (at HARV00079351). Note that the reader’s summary of the guidance
counselor’s letter mentions both academic qualities like intelligence and curiosity, as well as more personal descriptors:
“GC describes his ‘insatiable curiosity,’ and calls him the most well-mannered, pleasant, and intelligent student ever” (at
HARV00079386). As another example, see application HARV00079812 – 52. The applicant’s letters showcase both his
academic achievements and personal qualities, something both readers comment on. One reader writes: “Teachers praise
his natural intelligence but also speak to his habit of helping his peers and his sense of humor and relaxed personality
outside of academics. There’s a great deal of raw talent and interest here, and I imagine he would use this place well,” and
the other writes “[Redacted] is highly respected by his teachers for his academic strength and good PQ’s (They say he has
a great sense of humor, though he is very focused on academics.)” (at HARV00079813).
HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 22

dimensions and disfavors them on personal dimensions.
44. Prof. Arcidiacono’s inconsistent interpretation of his own ratings models is a key issue
that I identified in my first report as evidence against his conclusion of bias in the personal rating
process. As I noted in that report:
[S]uch a pattern calls into question whether the effects [Prof.
Arcidiacono’s] models attribute to race are more properly explained by
factors that are missing from his models (either because he does not
include them or because they are unobservable). If Harvard were in fact
biased against Asian-American applicants, it would make little sense for
Harvard to give an unexplained advantage to Asian-American applicants
in the academic and extracurricular ratings. On the other hand, if
Harvard were not biased, but the ratings models were simply missing
relevant variables that explain the differences across race in ratings
assignments, it would not be surprising to see an inconsistent pattern of
“bias” across the profile ratings.32
3.1.2. The data show that, on average, Asian-American applicants are weaker on non-academic
factors that affect the personal rating
45. The explanation Prof. Arcidiacono offers for his selective interpretation of the alleged bias
across the different ratings regressions is that “the case for discrimination is very strong when a group
of applicants is strong on the observed characteristics associated with a particular rating, yet faces a
penalty.”33 Prof. Arcidiacono then argues that there is no evidence that Asian-American applicants
are weaker, on average, on factors associated with the personal rating, and thus that it is proper to
infer discrimination. Yet a key driver of this conclusion is Prof. Arcidiacono’s focus on AsianAmerican applicants academic qualifications, rather than on the non-academic factors that affect the
personal rating. This logic does not make sense because, as discussed above, the unobservable factors
that are missing from Prof. Arcidiacono’s personal ratings regression are not academic factors. They
are non-academic factors like the personal essay, other recommendation letters, and any other
discussion that informs a candidate’s personal quality. Thus, the fact that Asian-American applicants
are stronger on academic factors is not sufficient evidence to conclude that they are also stronger on
unobservable factors that affect the personal rating.
46. As I detailed in my first report, there are three key patterns in the data that indicate that,
32
33

Card Report, p. 71.
Arcidiacono Rebuttal, p. 26.
HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 23

on average, Asian-American applicants are weaker on non-academic dimensions.
• First, if we look at the four profile ratings Harvard relies on, we see that,
while many Asian-American applicants are stronger on academic
qualifications, they are, on average, weaker across all non-academic
measures. See Exhibit 3 above, which I have reproduced from my first
report.
• Second, the patterns in Prof. Arcidiacono’s own personal ratings
regression strongly suggest that Asian-American applicants are on
average weaker across non-academic measures. In those regressions, the
estimated negative effect of Asian-American ethnicity shrinks as he adds
non-academic factors. Specifically, the logit coefficient falls by nearly
30 percent, from -0.547 to -0.391, when he adds controls for
neighborhood and school background and for the relevant ratings that
feed into the personal rating, like the teacher, guidance counselor, and
alumni ratings.34 This is a critically important finding because, as noted
above, the types of information missing from the personal rating
regression are non-academic in nature. Thus, the fact that the measured
disparity in ratings between Asian-American and White applicants
shrinks as additional non-academic factors are added to the ratings
model suggests that the disparity would shrink further if other nonacademic factors (such as information from the personal essay) could be
added. In fact, it is well understood in the academic literature that this
pattern of a declining effect when additional controls are added to the
model is a red flag that other unobserved factors are potentially
correlated with the variable of interest.35 In this case, it suggests that
unobserved factors are correlated with Asian-American ethnicity in a
way that leads the model to overstate the negative effect of AsianAmerican ethnicity.

34

Arcidiacono Rebuttal, Appendix D, Table B.6.7R.
Emily Oster, “Unobservable Selection and Coefficient Stability,” Brown University and NBER Working Paper #19054,
August 9, 2016, p. 3.

35

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 24

• Third, the non-academic factors in Prof. Arcidiacono’s own admissions
model contradict his claim that there is no evidence that Asian-American
applicants are weaker, on average, on non-academic dimensions. I
created a non-academic admissions index that summarizes the collective
admissions strength of each candidate across all non-academic factors in
Prof. Arcidiacono’s model, using the same methodology Prof.
Arcidiacono used to create what he called an “admissions index.”36
Using this non-academic admissions index, I found that the AsianAmerican applicants were less likely than White applicants to be in the
top half, as well as the top decile, of this index.37
47. In his rebuttal, Prof. Arcidiacono attempts to counter these analyses with new arguments
that are misleading or factually incorrect. First, Prof. Arcidiacono asserts that the scores AsianAmerican applicants receive on the key ratings that inform the personal rating (alumni, teacher, and
counselor ratings) “differ significantly” from the overall personal rating scores assigned to AsianAmerican applicants.38 In response to this claim I have analyzed the sum of the ratings from alumni
interviewers, teachers, and guidance counselors, and compared them across Asian-American and
White applicants. The best possible ratings sum is 5 because there are five ratings (two teacher
ratings, one guidance counselor rating, and personal and overall ratings from an alumni interviewer)
and ratings vary from 1 to 5 (the lower the rating, the stronger it is, with a rating of 1 being the best).
For example, an applicant who received teacher, guidance counselor, and alumni ratings of 1, 2, 2, 2,
and 1 would have a ratings sum of 8. The sum can be viewed as an overall summary measure of the
strength of each candidate as reflected in the interviews and materials submitted by external
reviewers. If Prof. Arcidiacono’s claim that Asian-American and White applicants are of similar
quality on these ratings is true, then we should not see any major differences between these two
groups.
48. As shown in Exhibit 4, this is simply not the case. For a given level of academic ratings,
Asian-American applicants are less likely than White applicants to receive strong ratings collectively
across these five ratings. For example, Exhibit 4 shows that among applicants with an academic
rating of 2, White applicants are more likely to have strong scores (i.e., a lower sum) than AsianAmerican applicants, indicating that they are stronger on average than Asian-American applicants on
these dimensions, which inform the personal rating.39 This is demonstrated by the fact that the

36
SAT, GPA, Academic Index, academic rating, and academic rating interaction variables are considered academic
factors for purposes of constructing this index.
37
Card Report, p. 39, Exhibit 10.
38
Arcidiacono Rebuttal, p. 14.
39
This same pattern is observed for other academic ratings as well. See workpaper.

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 25

distribution for Asian-American applicants (represented by red bars) is shifted right (i.e. toward
higher sums) relative to the distribution for White applicants (represented by blue bars). Exhibit 5
shows that the same pattern is true for other levels of the academic rating. Specifically, it shows the
share of White applicants and the share of Asian-American applicants who have very strong school
support and alumni interview ratings (measured as having a ratings sum of 11 or less) for each
category of academic rating. Among applicants with an academic rating of 2, 42% of White
applicants have a ratings sum of 11 or less, compared to 38% of Asian-American applicants, with
similar gaps among applicants with other competitive academic rating levels.40

Among applicants with an academic rating of 2, White applicants tend to have stronger school
support and alumni ratings than Asian-American applicants

Source: Augmented Arcidiacono Data
Note: Data are from applicants to the classes of 2014 – 2019 in Prof. Arcidiacono’s expanded sample including athletes.

40

Prof. Arcidiacono states in his report that “African-American and Hispanic applicants have observed characteristics
associated with lower [worse] personal ratings yet receive a preference in their personal ratings” (Arcidiacono Rebuttal, p.
27). Again, Prof. Arcidiacono is basing this claim primarily upon an analysis of the Academic Index, not the nonacademic characteristics (such as the school support and alumni interview ratings) that actually determine the personal
rating. An examination of the sum of school support and alumni interview ratings for African-American applicants shows
that, contrary to Prof. Arcidiacono’s assertion, they actually have observed characteristics associated with stronger
personal ratings. See workpaper.
HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 26

For a given academic rating, White applicants tend to have stronger school support and alumni
ratings than Asian-American applicants

Source: Augmented Arcidiacono Data
Note: Data are from applicants to the classes of 2014 – 2019 in Prof. Arcidiacono’s expanded sample including athletes.

49. It is important to note that Harvard’s Interviewer Handbook specifically states that Redacted
Redacted

The Handbook
identifies Redacted
Redacted

as important sources in identifying applicants with
41
Redacted
In other words, when looking at the
observable ratings that both Harvard and Prof. Arcidiacono identify as relevant inputs into the
personal rating, White applicants are collectively stronger than Asian-American applicants. Under
Prof. Arcidiacono’s own logic, this fact provides an alternative explanation for the unexplained gap
in personal ratings Prof. Arcidiacono finds. Because Asian-American applicants are weaker on
average on key observable factors that affect the personal rating, it is entirely plausible that the
unexplained gap in the personal rating reflects differences in unobservable factors that are missing
from the personal ratings regression, rather than racial bias against Asian-American applicants.
50. If I measure non-academic qualities more broadly, summing not only school support and
alumni interview ratings but also the extracurricular, personal, and athletic ratings, Asian-American
applicants are on average even weaker relative to White applicants. In this case, the ratings sum
consists of a sum over eight ratings, where in each case a rating of 1 is the best possible rating, so the
best possible ratings sum is 8. An applicant who received a rating of 2 on all eight ratings would have
a sum of 16. Exhibit 6 shows the distribution of ratings sums over these eight ratings for Asian41

Interviewer Handbook at HARV00001401.

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 27

American and White applicants who received an academic rating of 2, and Exhibit 7 shows the share
of Asian-American and White applicants who have a ratings sum of 18 or less by academic rating.
Again we see that Asian-American applicants are weaker on non-academic qualities, i.e., have higher
ratings sums than White applicants.42 Again this is demonstrated by the fact that the red distribution
(that of Asian-American applicants) is shifted right, i.e. toward larger ratings sums, relative to the
blue distribution (that of White applicants). And, again, because the factors that are missing from the
personal rating regression are non-academic in nature, these differences in non-academic factors
provide an alternative explanation for the unexplained gap Prof. Arcidiacono finds in his personal
rating regression.

Among applicants with an academic rating of 2, White applicants have stronger non-academic
ratings (school support, alumni, and profile other than academic)

Source: Augmented Arcidiacono Data
Note: Data are from applicants to the classes of 2014 – 2019 in Prof. Arcidiacono’s expanded sample including athletes.

42

The pattern observed in Exhibit 6 also holds for other academic ratings. The patterns observed in Exhibit Exhibit 6 and
Exhibit Exhibit 7 also hold if I exclude the personal rating. See workpaper.
HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 28

For a given academic rating, White applicants have stronger non-academic ratings (school
support, alumni, and profile other than academic)

Source: Augmented Arcidiacono Data
Note: Data are from applicants to the classes of 2014 – 2019 in Prof. Arcidiacono’s expanded sample including athletes.

51. In his rebuttal, Prof. Arcidiacono also challenges my argument that adding more nonacademic variables to his personal ratings regression reduces the unexplained gap in personal ratings
between Asian-American and White applicants. Specifically, he claims that it is not “universally
true” that adding controls to his personal ratings regression leads to a lower estimated marginal effect
of Asian-American ethnicity, and notes that “adding all the controls basically resulted in the same
penalty for Asian-American applicants as in the model with no controls.”43 The critical flaw in this
logic is that Prof. Arcidiacono includes academic variables in his analysis.44 What matters is not
whether it is “universally true” that all controls shrink the unexplained gap. What matters is whether
the crucial information omitted from the model that is known to inform the personal rating (e.g. the
personal essay, additional recommendation letters, etc.) would shrink the unexplained gap. The most
reliable way to test how such missing information affects the unexplained gap in personal ratings is to
test how similar non-academic variables that we can observe in the data impact the unexplained gap.
As noted, above, Prof. Arcidiacono’s own analysis shows that the non-academic variables that are
important to the personal rating (like the teacher, guidance counselor, and alumni interview ratings)

43

Arcidiacono Rebuttal, p. 27.
There is another flaw in this analysis, which is that Prof. Arcidiacono adds interaction terms to his model as he moves
from his model with no controls (model 1) to his model with all controls (model 5). By adding these interaction terms, he
changes the group of applicants the Asian coefficient applies to. For example, his model 1 Asian-American coefficient
represents the effect of Asian-American ethnicity for all Asian-American applicants but his model 5 coefficient represents
the effect of Asian-American ethnicity for only Asian-American applicants who are male, not flagged as disadvantaged,
and not recruited athlete, lineage, Dean’s or Director’s list, or children of Harvard faculty and staff applicants.

44

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 29

shift the unexplained gap towards zero.45 Thus, it is likely that more non-academic variables would
have the same effect. Prof. Arcidiacono’s argument misses the point.
52. Additionally, in his rebuttal, Prof. Arcidiacono responds to the analysis from my first
report that compares Asian-American and White applicants on the “non-academic” index from Prof.
Arcidiacono’s admissions model. Specifically, in Table 3.1.N he presents a “corrected” version of my
analysis, which he claims shows that Asian-American applicants are just as strong as White
applicants on non-academic dimensions.46 However, his results are driven by his exclusion from the
sample of many of the strongest White applicants: lineage applicants, recruited athletes, applicants on
the Dean’s or Director’s interest lists, and children of Harvard faculty and staff. He refers to these
excluded applicants as “specially recruited” applicants, but I will refer to them as ALDC (Athlete,
Lineage, Dean/director list, Children of faculty/staff) applicants. Comparing the full sample of AsianAmerican and White applicants provides further evidence that White applicants are stronger, on
average, on non-academic dimensions.47
53. As demonstrated in the first panel of Exhibit 8, once ALDC applicants are included in the
sample, White applicants are more likely than Asian-American applicants to fall in the top deciles of
the non-academic admissions index. For example, if we look at Row 6 of the first panel, we see that
12.03% of White applicants are in the highest decile of the non-academic index, whereas only 7.75%
of Asian-American applicants are. The highest decile of the non-academic index represents the group
of applicants with the strongest chance of admissions based on all non-academic variables in the
admissions model. The fact that White applicants are more likely to be in that group indicates that
White applicants are stronger than Asian-American applicants on non-academic dimensions. This is
true even if I accept Prof. Arcidiacono’s other modifications, such as removing the effect of the “tips”
associated with being an ALDC applicant (see second panel of Exhibit 8), removing the effect of the
personal rating (see third panel of Exhibit 8), or doing both (see fourth panel of Exhibit 8). In each
case, White applicants are more likely to be strong on non-academic dimensions, i.e., fall in the top
deciles of the non-academic admissions index.

45

Another way to see that additional non-academic variables help shrink the unexplained gap is to take Prof.
Arcidiacono’s personal ratings regression and estimate it year-by-year. When I do this, I can add additional variables to
the model in later years that Prof. Arcidiacono omits from his model because they do not exist for all years (as well as add
additional non-academic factors controlled for in my model but not in his). Doing this, I find that the unexplained gap in
ratings between Asian-American and White applicants shrinks even further. Using Prof. Arcidiacono’s preferred activities
measures instead of mine in the model described above also causes the unexplained gap to shrink even more. See
workpaper.
46
Arcidiacono Rebuttal, pp. 29–30.
47
This conclusion is supported by Prof. Arcidiacono’s own analysis. In the calculations performed by Prof. Arcidiacono
for appendix tables 7.5R and B.6.13R of his rebuttal, he also created versions of these tables in the same manner as I
describe above. In each case, his analysis shows White applicants are stronger than Asian-American applicants on nonacademic dimensions (SFFA-HARVARD 0002359_admissionsLogitsIndices.do).
HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 30

White applicants rank higher than Asian-American applicants on non-academic admissions index
Non-Academic
Admissions
Index Decile

White

AsianAmerican

AfricanAmerican

Hispanic

Without Removing Additional Effects
1. 5 or lower

46.46%

51.80%

55.27%

54.33%

2. 6

10.10%

10.35%

9.04%

9.48%

3. 7

10.12%

10.53%

9.00%

9.26%

4. 8

10.47%

10.10%

8.85%

8.90%

5. 9

10.81%

9.47%

9.06%

9.18%

6. 10

12.03%

7.75%

8.79%

8.85%

Remove Effect of ALDC “Tips”
7. 5 or lower

46.98%

51.28%

55.04%

53.81%

8. 6

10.20%

10.24%

8.95%

9.42%

9. 7

10.31%

10.25%

8.82%

9.39%

10. 8

10.69%

9.98%

8.80%

8.58%

11. 9

10.80%

9.56%

9.02%

9.02%

12. 10

11.03%

8.69%

9.37%

9.77%

Remove Effect of Personal Rating
13. 5 or lower

46.45%

51.59%

55.70%

54.63%

14. 6

10.36%

10.14%

9.02%

9.16%

15. 7

10.30%

10.19%

8.90%

9.27%

16. 8

10.34%

10.28%

9.13%

8.96%

17. 9

10.61%

9.84%

8.76%

9.02%

18. 10

11.94%

7.96%

8.50%

8.96%

Remove Effect of Personal Rating and ALDC “Tips”
19. 5 or lower

47.02%

50.92%

55.42%

54.20%

20. 6

10.46%

10.02%

8.89%

9.14%

21. 7

10.59%

9.91%

8.79%

9.06%

22. 8

10.53%

10.02%

9.12%

8.77%

23. 9

10.84%

9.69%

8.64%

8.96%

24. 10

10.56%

9.42%

9.14%

9.88%

Source: Augmented Arcidiacono Data; College Board Cluster Data; U.S. Census Data
Note: Data are from applicants to the classes of 2014 – 2019 in Prof. Arcidiacono’s expanded sample including athletes. The non-academic
admissions index is constructed using the updated approach put forth by Prof. Arcidiacono in Tables 7.4R and 7.5R in Appendix C of his
rebuttal report. The shares within each panel for a given race sum to 100%.

54. Prof. Arcidiacono also attempts to rebut my analyses of the personal rating by claiming

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 31

that an internal analysis by OIR finds evidence consistent with his analysis. Specifically, he asserts
that, “[u]sing data over ten years, OIR found that Harvard’s admissions officers assigned
substantially lower personal ratings to Asian-American applicants versus white applicants, especially
when compared to the ratings assigned by teachers, counselors, and alumni interviewers.”48 As I
described in my first report, Prof. Arcidiacono’s characterization of OIR’s findings is wrong. OIR did
not in fact reach any conclusions about bias in Harvard’s process. OIR described its analysis of
differences between White and Asian-American applicants as a “basic” analysis, and specifically
noted that it could not account for many factors that it could not measure, like the personal essay,
context variables, and socioeconomic status. Indeed, one important advantage of my model is that it
controls for these types of differences better than either the preliminary and incomplete model used
by OIR, or the incomplete model advocated by Prof. Arcidiacono. As I explained in my first report:
Prof. Arcidiacono points to documents produced in this litigation from
Harvard’s Office of Institutional Research (OIR), summarizing
statistical analyses performed by that office, as supposedly corroborating
his findings and his methodology. A careful review of the relevant
analyses, however, indicates that OIR’s research methodology actually
supports my methodological approach over Prof. Arcidiacono’s.
Specifically, the documents indicate that OIR understood that its models
were “basic” and “preliminary” and that, like Prof. Arcidiacono’s, they
were missing important factors in the admissions process—particularly
non-academic factors. For example, one of the documents states that
“[t]here are a variety of factors that quantitative data is likely to miss or
ratings not capture,” and then lists as examples “[e]xceptional talent,”
“[t]he role of context cases,” “[t]he role of the personal
statement/essay,” and “[m]easures of socioeconomic status” (footnote
omitted).49

48

Arcidiacono Rebuttal, p. 16.
Card Report, p. 66. Deposition testimony corroborates the idea that OIR researchers viewed their research as
incomplete/preliminary. For example, see Deposition of Erica Bever, July 13, 2017 (“Bever Deposition”), pp. 135–136
(“Q. Okay. And do you have any basis to believe that data would have been incomplete? A. Yes. Q. And why is that? A.
Because since I have moved to admissions and financial aid, I have a better understanding of admissions data. Q. And
what in particular do you think you have an understanding of now that you didn’t know then? A. The process. Q. And
what in particular—how in particular does that affect the reliability of the data that OIR would have used in 2013? A. We
oversimplified the process. Q. And what do you mean by ‘oversimplified the process’? A. So in that analysis we just
reviewed only four ratings were included. Q. And—and why does that oversimplify the process? A. There are many other
49

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 32

55. It is also worth noting that, outside of the Harvard data, there is evidence that the
populations of Asian-American and White students applying to colleges differ from each other on
non-academic factors. For example, as I noted in my first report, academic literature indicates that, on
average, Asian-American high school students apply to selective universities at higher rates than
students from other ethnic groups, even after controlling for whether or not a student is qualified on
key academic dimensions.50
56. This is important because an implicit assumption in Prof. Arcidiacono’s argument that the
personal rating is biased is that, absent bias by Harvard, there should be no average differences in
personal ratings between White and Asian-American applicants to Harvard. However, given the
underlying differences in application behavior to select universities between the two groups found in
the academic literature, there is no reason to expect that Asian-American and White applicants to
Harvard would possess the same qualifications and/or life experiences, on average, across the many
dimensions Harvard evaluates—even if we assume that the two groups have the same average
qualifications and/or life experiences in the population at large. Indeed, as detailed throughout this
section, the differences we see across the key profile ratings are consistent with differences we see in
the inputs into those ratings. Further, the fact that Prof. Arcidiacono selectively concludes that racial
bias is the cause of only one of those average differences (the difference in personal ratings)
establishes the unreliable and selective nature of his methodology.
57. Finally, even if we assumed—counter to the evidence—that the unexplained racial gaps in
Prof. Arcidiacono’s ratings regression reflect racial bias, the proper test for whether such alleged bias
in the ratings led to bias in admissions decisions would be to estimate an admissions model where
factors we review in admissions.”), and p. 156 (“A. So again this does not reflect the process by which we do admissions.
Q. And why doesn’t it reflect that? A. Because we review many factors, some of which can be data and some of which
are not.”). See also Deposition of Mark Hansen, July 19, 2017 (“Hansen Deposition”), pp. 195–196 (“Q. Are there other
factors that you may not have thought of earlier today, that also might explain the apparent difference in likelihood of
admission, based on one racial identification? […] THE WITNESS: Yes. (BY MR. DULBERG): Q. The modelling that
you undertook in Exhibit 4, does not take social economic status into account; correct? […] THE WITNESS: That
appears correct, yes. Q. There are other factors and data that are not reflected in these models; correct? […] THE
WITNESS: Certainly, yes.”).
50
Card Report, p. 37, footnote 64 (“Sandra Black, Kalena Cortes, and Jane Lincove, “Apply Yourself: Racial and Ethnic
Differences in College Application,” NBER Working Paper #21368, 2015; Sandra Black, Kalena Cortes, and Jane
Lincove, “Academic Undermatching of High-Achieving Minority Students: Evidence from Race-Neutral and Holistic
Admissions Policies,” American Economic Review: Papers & Proceedings, 105(5), 2015, pp. 604–610; Amanda Griffith
and Donna Rothstein, “Can’t Get There from Here: The Decision to Apply to a Selective College,” Economics of
Education Review, 28(5), 2009, pp. 620–628; David Card and Alan Krueger, “Would the Elimination of Affirmative
Action Affect Highly Qualified Minority Applicants? Evidence from California and Texas,” Industrial and Labor
Relations Review, 58(3), 2005, pp. 416–434.”).

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 33

Harvard’s actual (allegedly biased) ratings are replaced with the predicted ratings from Prof.
Arcidiacono’s ratings regression, with the alleged “biases” that Prof. Arcidiacono measures netted
out. In Exhibit 9, I present the results of such a model. Specifically, I use his ratings models (for the
personal, academic, and extracurricular ratings) to predict “bias-free” personal, academic, and
extracurricular ratings for each applicant and then include these ratings in my preferred model instead
of the actual ratings assigned by Harvard admissions officers.51 As Exhibit 9 shows, using these
ratings I continue to find no evidence of bias against Asian-American applicants. Although the
overall estimated effect becomes slightly more negative, it is still far from statistically significant,
and there is still a mix of positive and negative effects over the six classes.

There is no consistent or statistically significant evidence of bias against Asian-American
applicants even adjusting for what Prof. Arcidiacono alleges as bias in the personal rating

Source: Augmented Arcidiacono Data; College Board Cluster Data; U.S. Census Data
Note: Table shows the average marginal effect of race on admission for Asian-American applicants relative to White applicants from
updated Card model using adjusted academic, extracurricular, and personal ratings. Data are from applicants to the classes of 2014 – 2019
in Prof. Arcidiacono’s expanded sample including athletes. An applicant’s adjusted rating is the rating with the highest predicted
probability according to Prof. Arcidiacono’s rating model excluding other profile ratings from the controls and turning off the effect of
race. * indicates significance at the 5% level. Marginal effects are reported as percentage point values.

3.2. There is no basis for Prof. Arcidiacono’s decision to exclude parental occupation, intended
career, or staff interviews
58. Beyond his decision to exclude the personal rating from his admissions models, Prof.
Arcidiacono identifies several other variables that I included in my admissions model that he claims
51

Prof. Arcidiacono includes profile ratings as independent variables in his ratings regressions. For example, in his
personal rating model, he includes the academic and extracurricular ratings as independent variables. For this analysis, I
re-estimate his ratings models removing the profile ratings from the set of control variables since the purpose of this
exercise is to ensure that the allegedly biased ratings are not impacting my estimate of the effect of Asian-American
ethnicity.
HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 34

should not be included: parental occupation, intended career, and staff interviews. As with the
personal rating, including these variables reduces the alleged “bias” Prof. Arcidiacono finds in his
regression model. This fact helps to explain why Prof. Arcidiacono reaches a different conclusion
about the alleged bias against Asian-American applicants than I do.
59. In this sub-section, I summarize the evidence that definitively shows that all of these
variables are important to Harvard’s admissions process, and that Harvard relies on the information
they contain in deciding whom to admit. For these reasons, I conclude that there is no objective basis
for their exclusion from the admissions model. As with the personal rating, Prof. Arcidiacono’s
failure to consider inputs that are critical to Harvard’s decision-making process, without sufficient
factual basis for doing so, results in a model that fails to accurately reflect the process being modeled.
3.2.1. Parental Occupation
60. Prof. Arcidiacono asserts that parental occupation should be excluded from a model of
admissions for two reasons: it is allegedly not important to Harvard’s process, and it behaves in an
allegedly unreliable manner over time.
61. In support of his first critique, Prof. Arcidiacono states that “there is no evidence in the
records that Harvard’s admissions office considers parental occupation important aside from its value
as a measure of [socioeconomic status].”52 This argument does not justify excluding important
information from the model for at least two reasons.
62. First, as explained in detail in my first report, and summarized above in Section 3, the full
context of each candidate’s life is essential to Harvard’s review process. Harvard seeks to admit
candidates with a wide range of experiences and skills who can engage with and help educate
classmates and faculty. Parental occupation is an important fact from which Harvard gleans
information about family background and socioeconomic status. The importance of parental
occupation in the admissions process is supported by numerous pieces of evidence in the record. For
example:

52

Arcidiacono Rebuttal, p. 6.

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 35

• Parental occupation is one of the pieces of information reported on
docket sheets used by admissions officers during committee meetings.53
• Parental occupation is also reported on an applicant’s Summary Sheet
(used by admissions officers to synthesize application information).54
The summary sheet is viewed by admissions officers as containing key
pieces of information that guide the discussion about a candidate.55
• Deposition testimony from admissions officers confirms that they
consider parental occupation a relevant piece of information in the
admissions process.56
• Additionally, Harvard’s Discussion Guide to its Casebook (which is
used to train admissions officers) contains examples in which Redacted
Redacted

53

Docket sheets, “P, R & S Dockets- Official #1 Class of 2018,” HARV00056250 – 57311 at HARV00057289;
Deposition of William Fitzsimmons, August 3, 2017 (“Fitzsimmons Deposition”), pp. 254–255 (“…and then the entire
application would go up on the screen. So all the parental information…everything we've discussed…”); Deposition of Marlyn
Elizabeth McGrath, Volume I, June 18, 2015 (“McGrath Deposition 2015”), pp. 180–181 (“Q. And you said at the point that

it goes to committee, a physical docket is prepared? A. A physical docket is prepared. Q. And what does that look like? A.
It has...a list of the candidates from that school with certain salient information like test scores, grades, ... , parental
occupation...”).
54
See, for example, Summary sheet, HARV00076219 – 20.
55
Deposition of Chris Looby, June 30, 2017 (“Looby Deposition”), pp. 30–31 (“Q. And what is a ‘summary sheet’? A.
It’s a sheet that provides a summary of the application. Q. And how is a summary sheet used? …A. Many ways. Q. Can
you please list them? […] A. Provides an overview for anyone who might view that application. Q. Any others? A. Could
be used during the presentation of an application. Q. Any other ways? A. I believe they can be used for training
purposes.”); McGrath Deposition 2015, pp. 96–97 (“There’s—there is a space on the electronic thing, and there’s a piece
of paper in the paper version, where people, as they review, once the folder’s complete that I just described, once that’s
complete and begun to be read, the people who read the folder, the readers make their comments on a sheet of paper,
which then is available to the subsequent readers and to the full committee. Q. Is that sometimes referred to as a summary
sheet? A. Yes. Q. And are comments from admissions officers or other people in your office always placed on the
summary sheet either in the electronic or paper format, depending on timing? A. That’s the idea, yes.”)
56
Fitzsimmons Deposition, p. 201 (“Q. How does Harvard determine whether or not an applicant is socioeconomically
disadvantaged? A. …We also have information at the outset about the parents’ educational and professional
backgrounds.”); McGrath Deposition 2015, pp. 180–181 (“Q. And you said at the point that it goes to committee, a
physical docket is prepared? A. A physical docket is prepared. Q. And what does that look like? A. It has...a list of the
candidates from that school with certain salient information like test scores, grades, ... parental occupation...”); Looby
Deposition, p. 59 (“Q. What types of information would you assess in trying to determine whether you should code an
applicant as disadvantaged? … A. Could be parent jobs.”).

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 36

Redacted

57

• Finally, beyond the record in this case, occupation is also one of the
most informative measures of socioeconomic status relied upon in the
economics and social sciences literature.58
63. Second, even if one were convinced that parental occupation is used by Harvard only as
an indicator of socioeconomic status (as Prof. Arcidiacono seems to be), it would still be
inappropriate to exclude it from the model. Prof. Arcidiacono and I agree that Harvard considers
socioeconomic status in its admissions process, and we both include multiple measures of
socioeconomic status besides parental occupation (e.g., whether the applicant has been identified as
“disadvantaged,” whether the applicant received a fee waiver, and whether the applicant applied for
financial aid).59 Prof. Arcidiacono’s decision to exclude just one of the available measures of
socioeconomic status—a measure that is clearly considered in Harvard’s admissions process—is
unwarranted and inconsistent with his own methodology of including multiple socioeconomic
factors.
64. Moreover, relative to the other socioeconomic factors Prof. Arcidiacono includes in his
model, parental occupation is a more detailed and more informative measure of socioeconomic status.

57

Discussion Guide to the 2012 Casebook, HARV00018164 – 176 (“Casebook Discussion Guide”) at HARV00018167 –

8.
58

See, for example, David Zimmerman, “Regression Towards Mediocrity in Economic Stature,” The American Economic
Review 82(3), 1992, pp. 409–429; Otis Dudley Duncan, “A Socioeconomic Index for All Occupations,” in Occupations
and Social Status, ed. Albert J. Reiss, Jr. (Free Press, 1961), pp. 109–138; Greg J. Duncan and Katherine A. Magnuson,
“Off With Hollingshead: Socioeconomic Resources, Parenting, and Child Development,” in Socioeconomic Status,
Parenting, and Child Development, ed. Marc H. Bornstein and Robert H. Bradley (Lawrence Erlbaum Associates, Inc.,
2003), pp. 83–106.
59
Other measures such as the College Board neighborhood median income variable are only proxies which may or may
not accurately reflect an individual applicant’s circumstances.

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 37

For example, whether the applicant applied for financial aid provides only very limited information
on an applicant’s socioeconomic status because it is a simple binary indicator of “Yes” or “No.” In
fact, the vast majority of domestic applicants (76%) apply for financial aid.60 The fee waiver variable
and disadvantaged flag are also similarly limited measures of SES; like the financial aid variable both
are simple “dummy” variables that delineate only a binary indicator of socioeconomic status—i.e.,
fee waiver versus no fee waiver. Parental occupation, by contrast, includes 24 categories.
65. One formal way to demonstrate that parental occupation contains more relevant
information than the other three measures of socioeconomic status is to calculate how well each
variable (on its own) explains admissions decisions for domestic applicants to Harvard using a simple
regression. When I do this, I find that parental occupation explains more about admissions decisions
than any of the other three individual-specific measures of socioeconomic status. Specifically, a
model with just parental occupation has a Pseudo R-Squared of 0.011, while models with only fee
waiver, only financial aid, and only disadvantaged have Pseudo R-Squared values of 0.0004, 0.0063,
and 0.0023.61 In other words, without controlling for any other factors, parental occupation explains
more than twenty times as much as fee waiver, nearly twice as much as financial aid, and almost five
times as much as the disadvantaged flag. Finally, outside the record of this case, a substantial body of
social science literature uses parental occupation as an indicator of socioeconomic status.62 Given all
of these facts, there is simply no basis for Prof. Arcidiacono to exclude parental occupation from the
admissions model, even if it were (as he views it) purely a measure of socioeconomic status.
66. Prof. Arcidiacono also claims that the parental occupation field in Harvard’s admissions
database should be excluded because it fluctuates “wildly” from year to year, a pattern that he claims
proves it is unreliable.63 There are two important reasons why this critique is wrong.
67. First, Prof. Arcidiacono overstates the severity of these allegedly “wild” fluctuations by
focusing on a small number of occupation categories that exhibit changes over time. While it is true
that some occupational categories do fluctuate in size over time, the majority of occupations—
including the most common—behave in a stable fashion. Indeed, if one looks at the patterns across
all occupations and all years (as shown in Prof. Arcidiacono’s own tables B.3.1N and B.4.2N), it is
clear that the changes Prof. Arcidiacono complains about are much less pronounced than he suggests.
60

See workpaper.
See workpaper.
62
See, for example, David Zimmerman, “Regression Towards Mediocrity in Economic Stature,” The American Economic
Review 82(3), 1992, pp. 409–429; Otis Dudley Duncan, “A Socioeconomic Index for All Occupations,” in Occupations
and Social Status, ed. Albert J. Reiss, Jr. (Free Press, 1961), pp. 109–138; Greg J. Duncan and Katherine A. Magnuson,
“Off With Hollingshead: Socioeconomic Resources, Parenting, and Child Development,” in Socioeconomic Status,
Parenting, and Child Development, ed. Marc H. Bornstein and Robert H. Bradley (Lawrence Erlbaum Associates, Inc.,
2003), pp. 83–106.
63
Arcidiacono Rebuttal, p. 6.
61

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 38

Furthermore, these fluctuations primarily reflect a change in how data about occupations were
recorded in Harvard’s database starting in 2015. Harvard’s database indicates a switch from using
one set of occupation codes in 2014, to using two sets in 2015, with the majority of applicants
classified under the new system. As a result, certain occupational categories appear to “fluctuate”
between 2014 and later years. For example, Prof. Arcidiacono notes that while about 1,000 parents
per year were coded as self-employed in each year from 2015 to 2019, no parents were coded as selfemployed in 2014; this is because Harvard’s database did not have a code for self-employed parents
in 2014. Similarly, the “low skill” occupation appears to drop sharply in 2015; but this is simply
because “low skill” was not an option under Harvard’s new, more prevalent coding. The data also
suggest that Harvard stopped consistent use of the “unemployed” code starting with the class of 2018.
Such recoding is a common issue in many data sets that are widely used in econometric research,
such as the American Community Survey, Public Use files of the decennial censuses, and the Current
Population Survey.64
68. Most importantly, there is a simple solution to the problem Prof. Arcidiacono identifies—
estimate the admissions model separately for each year of applicants, then pool the estimated racial
effects from each year into a single summary measure. As explained in my original report, a key
methodological advantage of estimating separate admissions models for each year is that it accounts
for the fact that the overall composition of Harvard’s applicant pool changes substantially from year
to year. In fact, I provided several examples of key variables whose distributions changed
substantially over time, including intended concentration, docket, and early action. Such year-to-year
compositional changes are not a problem at all so long as the admissions models are estimated
separately for each year. Such an approach ensures that applicants who apply in a year where, say,
there are an unusually large number of applicants whose parents are engineers are compared only to
64

U.S. Census Bureau, “Industry and Occupation Code Lists & Crosswalks,” available at
https://www.census.gov/topics/employment/industry-occupation/guidance/code-lists.html, accessed March 8, 2018; U.S.
Bureau of Labor Statistics, “Historical comparability of occupation and industry data from the Current Population
Survey,” available at https://www.bls.gov/cps/cpsoccind.htm, accessed March 8, 2018; IPUMS USA, “ACS Occupation
Codes,” available at https://usa.ipums.org/usa/volii/c2ssoccup.shtml, accessed February 19, 2018.
Indeed, Prof. Arcidiacono himself has used some of these datasets, and has also mapped occupational codes from
university-specific data to ACS categorizations; see Peter Arcidiacono et al., “Recovering Ex Ante Returns and
Preferences for Occupations Using Subjective Expectations Data,” NBER Working Paper #20626, October 2014, p. 9
(“We utilize data on wages, college major, and current occupation from the 2009-2011 ACS.... Majors in the ACS were
categorized similarly to the Duke data. Several majors in the ACS are not offered at Duke; to the extent they clearly fell
into one of the six major categories, they were included in that category. Occupations were constructed by matching the
occupations categories in the ACS with the occupation groupings in the Duke data.”) and footnote 17 (“[s]cience,
computing, and engineering occupations were coded as science and technology careers; medicine was coded as a health
career; business and finance were coded as business careers; legal was coded as a law career; nonprofit occupations as
well as local, state or federal occupations were coded as government/nonprofit”).

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 39

the other applicants who applied within that same year (where the pool has that same feature).65
69. In fact, Prof. Arcidiacono acknowledges—and partially implements—this solution in his
own analysis. Specifically, after I pointed out in my first report that variables like intended
concentration, docket, and early action behaved differently across years, Prof. Arcidiacono changed
his admissions model to allow those variables to have different effects in different years. In other
words, for some categorical variables with categories that change over time, Prof. Arcidiacono chose
to employ a standard methodological solution that allowed him to retain those variables (rather than
exclude them from the model, as he advocates for parental occupation).66 Prof. Arcidiacono’s
inconsistent application of this approach, and his insistence on leaving out parental occupation, is
noteworthy because parental occupation is a variable that reduces the alleged “bias” against AsianAmerican applicants found in Prof. Arcidiacono’s model (as I show in Section 4 below).
70. Perhaps the most striking example of Prof. Arcidiacono selectively applying which
variables to include or not based on their volatility over time is the disadvantaged flag. As noted
above, Prof. Arcidiacono views this flag as a critical indicator of socioeconomic status, and relies
heavily on this variable for his opinions. Yet, like parental occupation, the coding of the
disadvantaged flag varies substantially over time. For example, the share of applications flagged as
disadvantaged nearly doubles in 2019 relative to years prior (in 2018 it is 9.9% and in 2019 it is
17.8%).67 Such a large unexplained change is the exact reason Prof. Arcidiacono cites for excluding
parental occupation from his model.68 It is telling that when choosing between two variables that
exhibit similar volatility, Prof. Arcidiacono chose to include the factor that leads to an estimated
effect of Asian-American ethnicity that is more negative (disadvantaged) and to exclude the factor
that leads to a less negative effect (parental occupation).69
71. An alternative approach, commonly used to address concerns that certain categories of a
variable like parental occupation change over time in an unreliable manner, is to include the volatile
categories (e.g., “Unemployed”) in a combined “missing and unstable” category across all years, and
leave the other more stable categories (e.g. “Lawyers, Judges”) in the model. This solution allows the
model to use the parental occupation information that Prof. Arcidiacono believes is reliable, rather
65

In Appendix B.1 I address a more technical criticism from Prof. Arcidiacono about how I aggregated the available
occupational categories to create the indicator variables in my regression. I show that the findings of my main regression
models are robust to a variety of reasonable ways of constructing occupation categories.
66
Arcidiacono Rebuttal, pp. 69–70.
67
See workpaper.
68
Arcidiacono Rebuttal, pp. 31–33. (“These inconsistencies raise doubts about the reliability of the field and its
usefulness as a control. If there is little reason to trust the accuracy of a factor, incorporating it into a model will not
inform the resulting estimates. Prof. Card nowhere offers an explanation for why these data would vary so wildly across
these years.”)
69
See workpaper.
HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 40

than throw out parental occupation information entirely. Prof. Arcidiacono does not explore this
alternative approach. As I will discuss in Section 4, in order to be certain that the issues Prof.
Arcidiacono raises do not affect the final conclusions of my results, I have also estimated my
preferred model making this modification (despite the fact that my year-by-year model already
addresses this issue) and show that it leads to the same conclusions.
3.2.2. Intended Career
72. Prof. Arcidiacono also argues that data on applicants’ intended careers should be excluded
from a model of admissions. As I noted in my first report, intended career is another piece of
information Harvard relies on to better understand an applicant’s life experience and interests. This
should not be surprising. A student body in which all students had the same career interests, or the
same intellectual interests, would have less diversity of thought. I included intended career in my
admissions model both because the record indicates it can meaningfully influence an applicant’s
chance of admission, and because it exhibits differences between ethnic groups. Specifically, I noted:
[A]n applicant’s future plans and fields of interest can be critical to the
assessment of how the applicant will contribute to the Harvard
community both inside and outside the classroom. For example, the
Casebook Discussion Guide notes the following about one candidate:
Redacted

70

73. It is worth noting that a recent study that analyzed how undergraduates at Harvard and
Stanford gain information about different careers, and make different career choices when they leave
college, found that (a) 30% of White students in the sample were interested in “impact careers”
(defined as education, public service, nonprofits, and philanthropy, and “creative-class” careers, such
as academia and journalism), as compared to 15% of Asian-American students,and (b) 56% of AsianAmerican students in the sample were interested in Consulting or Finance, as compared to 41% of
White students. 71

70

Card Report, p. 44.
Amy J. Binder, Daniel B. Davis, and Nick Bloom, “Career Funneling: How Elite Students Learn to Define and Desire
‘Prestigious’ Jobs,” Sociology of Education 89(1), 2016, pp. 20–39 at pp. 24–25.

71

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 41

74. Despite the clear importance of intended career in the admissions process, and the fact
that preferences for intended career differ on average between ethnic groups, Prof. Arcidiacono
excludes the variable from his model. He claims, as he did for parental occupation, that intended
career “varies in highly unusual and unexplained ways over time, undermining its reliability as a
variable and its usefulness as a control.”72 As noted above, however, one of the main purposes of
estimating the admissions model separately for each admissions class is to solve this exact problem.
As I detailed in my original report, the exact composition of each admissions class changes from year
to year on any number of dimensions. These types of year-to-year compositional changes, however,
do not pose any methodological problems for the admissions model because the admissions model is
focused on comparing applicants within the same year.
75. Prof. Arcidiacono also overstates the changes that occur over time in the intended career
variable. Prof. Arcidiacono’s Table 7.1N shows how the intended career variable changes over time
for all categories.73 As is clear, the biggest change occurs in 2018; outside of 2018 the values across
all of the categories are generally stable. As with parental occupation, Prof. Arcidiacono’s omission
of intended career is not defensible. A more reliable solution would be to simply allow the variable to
have different effects in different years, as he does with other variables that change substantially over
time, and as I do below in Section 4 with my year-by-year model.
3.2.3. Staff Interviews
76. Prof. Arcidiacono also excludes staff interviews from the admissions model. As with
intended career, Prof. Arcidiacono does not dispute the importance of staff interviews to the
admissions process. Instead he argues that the variable should be excluded because he understands
that staff interviews are given to only a small portion of the applicant pool and are less likely to be
given to Asian-American applicants, and because people receiving staff interviews have a good
chance to be admitted.74
77. The fact that only a small number of applicants participate in staff interviews is not a
sufficient basis to exclude staff interview ratings from the model. Given the competitive nature of the
process, and the many dimensions over which Harvard tries to distinguish between so many strong
applicants, additional information (like the staff interview) helps improve the model.
78. That said, to address the concerns Prof. Arcidiacono raises, in Section 4, I test whether the
exclusion of the staff interview in my model in any way changes the overall conclusions from my key
72

Arcidiacono Rebuttal, p. 62.
Arcidiacono Rebuttal, p. 63.
74
Arcidiacono Rebuttal, pp. 66–67.
73

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 42

findings. As I show there, it does not. Whether or not we include it in the model does not
significantly change the effect associated with Asian-American ethnicity.
3.3. Prof. Arcidiacono’s use of a pooled model is inconsistent with an essential feature of Harvard’s
admissions process and thus has no methodological basis
79. In Section 5.3 of my original report, I explained why it is critical to analyze Harvard’s
admissions decisions separately by year. In his rebuttal, Prof. Arcidiacono challenges that approach
for two reasons. First, he claims that I am incorrect in asserting “all applicants each year are
compared to all other applicants.”75 Second, he asserts that estimating models separately by year
“reduces the statistical power of the sample.”76 Prof. Arcidiacono’s claims lack any factual support
and are entirely without merit.
80. First and foremost, Harvard’s admissions process is, in fact, a year-by-year process.
Applicants from the class of 2019 are not compared to applicants from the class of 2017, and any
analysis that assumes they are is inherently flawed. Prof. Arcidiacono’s response that it is “wrong that
all applicants each year are compared to all other applicants”77 (which I will turn to in the next
section of this report) completely misses this point. What is relevant is not whether every candidate
within a year is compared to every other candidate in that year, but whether applicants in different
years are compared to each other. Again, it is nonsensical to assume that an applicant for the class of
2019 is competing with an applicant for the class of 2017, yet that is the assumption Prof.
Arcidiacono imposes in his own model. On this critical issue, Prof. Arcidiacono offers no rebuttal.
81. The second reason Prof. Arcidiacono offers for pooling applicants from different years
into a single model is that estimating a separate model for each year “reduces the statistical power of
the sample.”78 Prof. Arcidiacono offers a hypothetical example of discrimination against women in
promotions at a law firm over a six-year period, and asserts that in that hypothetical example
performing the analysis year-by-year would “reduce the statistical significance of findings of
discrimination, but it would not make any sense.” 79 First, it is worth noting that the example of
promotion to partner at a law firm is fundamentally different from admissions to Harvard because,

75

Arcidiacono Rebuttal, p. 34.
Arcidiacono Rebuttal, p. 34.
77
Arcidiacono Rebuttal, p. 34.
78
Arcidiacono Rebuttal, p. 34.
79
Arcidiacono Rebuttal, p. 35.
76

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 43

among many other things, candidates not promoted at a law firm in a given year could be (and
typically are) considered again for promotion in future years. Second, as I explained in my first
report, I resolve the purported problem of reduced statistical significance by taking the average of the
estimated race effects (e.g. the effect of Asian-American ethnicity) from the models for each year of
data, which is a standard statistical approach to this issue.80 In other words, Prof. Arcidiacono’s
hypothetical is thoroughly misleading. He ignores the important fact that after doing the analysis by
year, I average the results across years to ensure statistical power.
82. In fact, it is possible to directly compare how precisely the effect of Asian-American
ethnicity can be estimated (i.e. the standard error) by Prof. Arcidiacono’s pooled model versus my
year-by-year model, in which the six yearly estimates are averaged into a single effect representing
the average effect over the six classes of applicants. As we can see in Exhibit 10, the precision of the
two approaches is nearly identical. Specifically, in the first two panels of this exhibit, I estimate Prof.
Arcidiacono’s model pooled and then also separately year-by-year. The appropriate measure of
precision for each model is the standard error. As a general matter standard errors decrease (and
precision increases) when a model has more data. What we see is that the standard error from the
pooled model and the year-by-year model averaged across years is nearly identical at 0.15.81 The
reason for this is simple: by averaging the estimates across years, I am taking advantage of the same
number of observations as Prof. Arcidiacono does by pooling them.82 Prof. Arcidiacono’s assertion
that there is a reduction in statistical power from fitting year-by-year models is obviously not correct.
Moreover, the results in the second and third panels of Exhibit 10 demonstrate that the standard error
of the average effect from my year-by-year model is actually smaller than that of Prof. Arcidiacono’s
pooled model (0.14 vs. 0.15). This is because my year-by-year model does a better job of explaining
admissions decisions. Thus, contrary to Prof. Arcidiacono’s assertion, my model, fit year-by-year and
then averaged, has greater precision (i.e., greater statistical power) in estimating the effect of Asian-

80

Card Report, p. 67, footnote 116 (“To ensure that my year-by-year estimates are comparable with Prof. Arcidiacono’s
pooled estimate, I average the six year-by-year estimates to obtain an average effect across all six years of data. This
approach allows me to use all the available years of data but estimate models that more accurately reflect Harvard’s
admissions process.”).
81
The standard error for the weighted average of the yearly effects is computed according to the following formula:
2019

. .

2

. .∗
2014

82

Technically, while Prof. Arcidiacono and I both take advantage of the same number of observations, there is an
additional trade-off which affects the precision of the estimates. This relates to which method has more “degrees of
freedom” and which method has a better fit. The degrees of freedom refer to the total number of observations in the
sample minus the total number of parameters being estimated. My method of averaging the estimates from the 6 yearly
models utilizes the same number of observations as Prof. Arcidiacono’s but has more parameters because I estimate a
separate model for each year. This means that my estimate has fewer degrees of freedom relative to Prof. Arcidiacono’s.
But, because my year-by-year models fit the data better, on balance, the precision of my estimates is slightly higher.
HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 44

American ethnicity on admissions than his pooled model.

Estimating a model either pooled or year-by-year will produce extremely similar measures of
statistical precision

Source: Augmented Arcidiacono Data; College Board Cluster Data; U.S. Census Data
Note: Models are estimated on Prof. Arcidiacono’s expanded sample including athletes. The first panel shows standard errors for Prof.
Arcidiacono’s model estimated year-by-year. The overall standard error (0.15) is the standard error for the weighted average of the yearly
effects. The second panel shows the standard error for Prof. Arcidiacono’s pooled model. The third panel shows the overall standard error
for the weighted average of the yearly effects as estimated from the Card model.

83. Before moving on, it is worth noting that, in my first report, I identified another important
reason for estimating the model separately for each year—the pooled model imposes the assumption
that Harvard places the same value on each characteristic across all years. In my first report, I
explained this problem as follows:
Second, a closely related problem with the pooled model is that it
imposes the assumption that every factor in the admissions process has
the same effect from year to year. Given that the applicant pool changes
from year to year, it is quite possible that the relative abundance and

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 45

scarcity of relevant factors can also change, which can cause the value
Harvard places on any given factor to also change from year to year.83
84. As noted above, Prof. Arcidiacono’s rebuttal has partially acknowledged the importance
of this methodological concern. Specifically, for some of the key variables that he chose not to
exclude from his model, Prof. Arcidiacono has added interaction terms between those variables and
the year variables. However, his decision to selectively interact only some of his variables across
years, rather than estimate separate models for each year, is a less neutral approach than my approach
of estimating each model separately by year, because it involves more subjective judgment. Not
surprisingly, perhaps, given the patterns associated with his other modeling choices, Prof.
Arcidiacono’s decision to not estimate his models separately by year has the effect of increasing the
alleged disparity between Asian-American and White applicants.
3.4. Prof. Arcidiacono’s decision to exclude certain types of applicants from his model is inconsistent
with how Harvard’s admissions process works, and is methodologically unsound
85. In both of his reports, Prof. Arcidiacono performs all of his key analyses on a restricted
sample of applicants that he calls his baseline sample. This sample excludes recruited athletes,
lineage applicants, those on the Dean’s or Director’s interest lists, and children of a member of
Harvard’s faculty and staff—applicants I refer to as ALDC applicants (Athlete, Lineage,
Dean/Director list, Children of faculty and staff) for brevity. In addition, in his initial report, he
excluded early admission applicants from his baseline sample.84 My first report criticized his use of
the baseline sample; in his rebuttal, Prof. Arcidiacono continues to defend his use of the baseline
sample.
86. Prof. Arcidiacono offers two main justifications for his use of the baseline sample. First,
he argues that the candidates he excludes from the baseline sample are “subject to special admissions
procedures[,]” i.e., their admission process is distinct from that of other applicants.85 Second, in his
rebuttal, he offers the new argument that the purportedly different admissions process for ALDC
candidates is not affected by the alleged discrimination, and thus inclusion of such applicants in an
empirical analysis will “obscure the penalty Harvard imposes on Asian-American applicants.”86 In
this section, I explain why neither of the reasons offered by Prof. Arcidiacono is based on available
facts in the record or is a methodologically sound reason for excluding these candidates from the

83

Card Report, pp. 51–52.
In his first report Prof. Arcidiacono excluded early action applicants from his baseline sample, but in his rebuttal he
includes them. Arcidiacono Rebuttal, p. 69.
85
Arcidiacono Rebuttal, p. 69.
86
Arcidiacono Rebuttal, pp. 19, 69.
84

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 46

analysis.
3.4.1. Prof. Arcidiacono’s claim that ALDC candidates are part of a “special” admissions process,
and, thus, do not compete with other candidates is not supported by the data, documents, or
depositions
87. In my first report, I argued that Prof. Arcidiacono’s baseline sample was flawed because it
was inconsistent with how Harvard’s actual admissions process worked. I am aware of no evidence in
the record that Harvard conducts a different admissions process for certain types of candidates
whereby those candidates do not compete against candidates from the broader pool. Certainly, Prof.
Arcidiacono has not presented any such evidence. As explained in my first report:
Harvard compares all of its applicants in each year to all other applicants
in the pool for that year; it does not conduct separate admissions
processes for discrete subsets of the pool. Harvard seeks a diverse class
in each year on any number of dimensions—academic, extracurricular,
geographic, racial and ethnic, and so on. Thus, the fact that some
candidates with particular attributes (such as lineage applicants or
recruited athletes) have a higher likelihood of admission does not mean
that they should be completely excluded from the analysis. Such
candidates are still compared to other candidates on all dimensions, and
their candidacy can affect how other decisions are made. By throwing
such information out of the analysis, the model cannot use that
information to explain why other applicants were or were not admitted.87
88. Prof. Arcidiacono responds by continuing to insinuate that ALDC candidates are part of a
different process88—yet his rebuttal still offers no evidence to support that assertion.89 As I explain
87

Card Report, p. 57. In my initial report, I made two additional points. First, I noted that excluding early admission
applicants was particularly problematic because (a) early admission applicants who are not admitted early remain in the
regular pool of applicants, and (b) early admission did not exist for the classes of 2014 and 2015, which means that
excluding early admission applicants has a differential effect on the sample in those two years. Second, I noted that
throwing out these data reduced the precision of his statistical model (Card Report, pp. 57–58).
88
Arcidiacono Rebuttal, pp. 34, 69; Arcidiacono Rebuttal, Appendix A, p. 3.
89
If anything, the evidence suggests the opposite. See, for example, an email chain in which the women’s hockey coach
asks for feedback from admissions officers on a draft email she plans to send to Dean Fitzsimmons in advance of
admissions committee deliberations, in which she advocates for the admission of her recruits. Stone writes: “I am
compelled to reach out about the importance of next week’s admissions meeting for our hockey program. I am fully
aware that there are many qualified applicants for next year’s class; and I feel strongly that these Redacted
are

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 47

below, exclusion of this large and relatively well-qualified group of applicants from the admissions
model removes important information about how Harvard balances the many characteristics it
considers in its decision process, and, thus, makes the model less reliable. Given the lack of any
evidence of a separate process for ALDC candidates, and the importance of including a large and
diverse sample in the model to allow accurate estimation of the tradeoffs between different
characteristics in the review process, excluding ALDC applicants from the model is not
methodologically defensible.
89. To better understand why excluding these ALDC candidates from the admissions model
reduces the reliability of the model, it is helpful to consider the example of a candidate with an
academic rating of 1 who is not an ALDC candidate. As I showed in my first report, applicants with
an academic rating of 1 (and no other profile ratings of 1) have a 68% chance of admission.90 That is
more than ten times the average admission rate. Yet, while an academic rating of 1 certainly elevates
a candidate’s chances of admission, Harvard still evaluates all other dimensions of such a candidate’s
profile, which may entail comparing her to other candidates who perhaps have lower academic
credentials, but who display more well-rounded excellence on multiple dimensions. Prof.
Arcidiacono apparently agrees with this logic because he does not exclude candidates with an
academic rating of 1 from his model.
90. The same logic holds for Prof. Arcidiacono’s ALDC applicants. While it is certainly true
that Harvard gives a “tip” to competitive candidates in certain categories, that “tip” by no means
assures admission; nor does it remove the need for strength on other dimensions. A “tip” is just one
part of an applicant’s candidacy, and her remaining characteristics are considered in light of the many
other highly qualified candidates in the applicant pool. By removing ALDC applicants from the
admissions model, Prof. Arcidiacono’s model is less reliable because it is not able to use the
information from those applicants’ other characteristics to help identify the tradeoffs that Harvard
makes across candidates when deciding whom to admit.
91. The data bear this out. Specifically, if we look at the admissions data for the ALDC
candidates that Prof. Arcidiacono excludes from his baseline sample, it is evident that many ALDC
candidates have a high chance of admission, with or without the “tip” they receive for belonging to
one of the ALDC categories. For example, one way to see the strength of ALDC applicants relative to
the broader applicant pool is to compare the predicted probability of admission (according to my
no exception. I recognize their testing may not be that of others, yet what they will bring to the Harvard classroom,
athletic area and community is immeasurable” (Email from Katey Stone to Grace Cheng and Nathan Fry, “FW: Harvard
Women’s Ice Hockey,” November 30, 2012, HARV00022645). See also Harvard College, “Frequently Asked
Questions,” available at https://college.harvard.edu/frequently-asked-questions, accessed February 2, 2018 (Question: “Is
there a separate admissions process for prospective athletes?” Answer: “No. We encourage students with athletic talent to
contact our Athletic Department for information about any of Harvard’s 42 varsity athletic teams.”).
90
Card Report, p. 28, Exhibit 4.
HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 48

model) for ALDC applicants to that of non-ALDC applicants. Exhibit 11 shows predicted
probabilities for both ALDC and non-ALDC applicants. In this exhibit, the predicted probabilities for
ALDC applicants are calculated after turning off the “tip” associated with their ALDC status, and the
results clearly show that ALDC applicants are stronger applicants even without the “tip” they receive
for their ALDC status. Specifically, there are far more non-ALDC applicants with very low predicted
probabilities of admission (i.e., predicted probabilities between 0% and 5%) and far more ALDC
applicants with more competitive predicted probabilities. 21% of ALDC applicants have predicted
probabilities of admission that are higher than 20%, compared to only 8% of non-ALDC applicants.
92. It is also the case that, among ALDC candidates who apply to Harvard, the candidates
who are admitted are much stronger than the ones who are denied admission. Specifically, ALDC
applicants who are admitted have an average predicted probability of admission that is 61 percentage
points higher than that of ALDC applicants who are denied admission.91 In other words, ALDC
applicants exhibit numerous traits other than their ALDC status that matter in determining whether
they are admitted to Harvard, and it is their strength across multiple dimensions that is central to
whether they are ultimately admitted. These facts imply that the ALDC sample provides important
information to the admissions model that helps the model more reliably estimate the effect of those
traits in the admissions process.

91

See workpaper.
HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 49

ALDC applicants have higher predicted probabilities of admission than non-ALDC applicants, even
without their ALDC “tip”

Source: Augmented Arcidiacono Data; College Board Cluster Data; U.S. Census Data
Note: Data are from applicants to the classes of 2014 – 2019 in Prof. Arcidiacono’s expanded sample including athletes. ALDC applicants’
predicted probability of admission is calculated removing the effect of being an ALDC applicant (i.e. removing the effect of being a
recruited athlete, on the Dean’s or Director’s list, a lineage applicant, or a child of Harvard faculty and staff).

3.4.2. Prof. Arcidiacono’s claim that ALDC candidates should be excluded because there is no
disparity in admissions decisions for such candidates is methodologically unsound
93. As noted, Prof. Arcidiacono’s rebuttal introduces a new and different argument about why
the ALDC applicants must be removed from his baseline sample. Prof. Arcidiacono now claims that
ALDC applicants are part of a separate process that does not discriminate, and thus any analysis of
discrimination that includes them operates to “obscure” discrimination where it allegedly does
occur.92 As I explain below, this argument is factually unsupported, methodologically flawed, and
highly problematic.
94. The first and most important flaw in this approach is that ALDC applicants are not, in
fact, considered in a separate admissions process. All Harvard applicants are reviewed in the same
admissions process.

92

Arcidiacono Rebuttal, pp. 19, 34.
HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 50

95. Second, Prof. Arcidiacono’s claim that only a subgroup of applicants is subject to the
alleged bias appears to be a form of data mining, a process whereby the researcher selectively
chooses a subsample of the data to obtain a desired result. To help understand what this is, and why it
is so problematic, consider a scenario in which it is undisputed that Harvard does not discriminate.
Even in this scenario, it will be true, purely by chance, that some subgroups of Harvard’s data will
exhibit statistically significant unexplained disparities between racial groups due to a baseline level of
unobserved differences in characteristics not in the model. Because there is no discrimination, these
disparities will run in both directions—i.e., some subgroups of the data will show an unexplained gap
in favor Asian-American applicants and some will show an unexplained gap in favor White
applicants.
96. Given this reality, if one employs Prof. Arcidiacono’s approach of excluding subgroups of
the data where there is no evidence of a statistical disparity, then one is stacking the deck in favor of
finding bias in the remaining data. This does not mean that there is discrimination in those remaining
subgroups—it means only that the researcher has selectively analyzed the data to find a favorable
result. That is why, as a general matter, the most reliable methodological approach for testing for a
significant disparity in admission rates across race is to include all data points, and then to test
whether there is a systematic disparity.
97. An exception to this approach can be made if there is clear evidence outside of the data
that whatever alleged discriminatory behavior being analyzed is indeed limited to a subset of the data.
In such a case, it might be appropriate to limit the sample. But Prof. Arcidiacono provides no external
evidence that the alleged discrimination asserted by SFFA is not relevant for ALDC candidates. Nor
is there any logical reason to assume it is not. If Harvard were in fact biased against Asian-American
applicants, why would it not impose its supposed discriminatory preferences against legacies, or
children of faculty, or athletes? What would be its motivation for selectively imposing such racial
preferences? Prof. Arcidiacono’s decision to exclude ALDC candidates appears to be based solely on
the fact that the data show no negative effect of Asian-American ethnicity for this particular set of
applicants. It is, thus, neither an appropriate nor an objective approach to building a model directed at
analyzing the effect of Asian-American ethnicity on admissions decisions.
98. In fact, examining the effect of Asian-American ethnicity for the applicants Prof.
Arcidiacono excluded suggests why he excluded them. Many ALDC applicants have an estimated
effect of Asian-American ethnicity that is positive. For example, the estimated effect of AsianAmerican ethnicity among lineage applicants is 3.12 percentage points. This means that among
lineage applicants, Asian-American applicants are admitted at a rate that is roughly three percentage
points higher than the rate at which the model would expect White applicants with identical
characteristics to be admitted. Similarly, the estimated effect of Asian-American ethnicity for

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 51

applicants who are on the Dean’s or Director’s interest lists or who are children of Harvard faculty
and staff is positive. 93 Of course, these estimated positive effects do not mean that Harvard is biased
in favor of Asian-American applicants in the specified categories. Instead, these positive effects show
that there may be unobserved characteristics that vary both with race and with membership in the
specified categories, and that affect applicants’ likelihood of admission. That is a central reason why
it is inappropriate to exclude the specified categories from the model, as Prof. Arcidiacono does.
99. Before moving on, it is worth noting that Prof. Arcidiacono does acknowledge that an
alternative to excluding the data of ALDC applicants entirely is to include their data, and then add
interaction terms between the race variables and the relevant dummy variables for the ALDC
categories at issue (recruited athletes, lineage applicants, etc.). Prof. Arcidiacono asserts that such an
approach is an alternative way to address his concern that such candidates should not be included in
the model because they are not discriminated against.94 In Section 4, I estimate a sensitivity analysis
that employs this methodology. I continue to find no evidence of bias even with this approach.

93
94

See workpaper.
Arcidiacono Rebuttal, p. 36.
HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 52

4. AN ADMISSIONS MODEL THAT INCLUDES RELEVANT INFORMATION FINDS NO
EVIDENCE OF BIAS AGAINST ASIAN-AMERICAN APPLICANTS
100. In Section 3 above, I offered an explanation of the key methodological differences
between Prof. Arcidiacono’s approach and mine. In general, the differences in our approaches reflect
differences in our modeling of how Harvard’s admissions process works. As I explained above, given
Harvard’s clear philosophy of identifying “distinguishing excellences” across a wide variety of
dimensions and evaluating each application within the context of the applicant’s life experiences and
opportunities, the most reliable model of Harvard’s admissions process should include as much
relevant information as possible about such distinguishing excellences and context factors. This is the
approach I take. Prof. Arcidiacono, on the other hand, seeks to exclude several highly relevant pieces
of information from the model, under the claim that they are either unreliable or biased (or both).
101. In this section, I present results from my admissions model. As I show below, when all
relevant observable information is included, I find no evidence of bias against Asian-American
applicants. Further, even when I perform a variety of sensitivity checks on my model to
accommodate specific points raised by Prof. Arcidiacono, I continue to find no evidence of bias.
Additionally, I show that for large subgroups of applicants (specifically, female applicants and
applicants from California), there is evidence of a positive (though statistically insignificant)
estimated effect of Asian-American ethnicity, suggesting that the negative effect of Asian-American
ethnicity that Prof. Arcidiacono attributes to bias actually reflects unobserved differences between
applicants, not bias. Finally, I show that Prof. Arcidiacono’s argument that Harvard discriminates
against all applicants on dockets where Asian-American applicants are more common is severely
flawed—both conceptually and in its empirical implementation.
4.1. My preferred regression model shows no evidence of bias against Asian-American applicants
102. For the reasons detailed above in Section 3, the year-by-year admissions model I
presented in my initial report remains my preferred model with one small change. For ease of
comparing our results, I have adopted Prof. Arcidiacono’s revised method of modeling the various
ratings variables (i.e. the profile ratings, schools support ratings, and alumni interview ratings).95
Implementing the modifications suggested by Prof. Arcidiacono (such as removing parental
occupation and other key variables that capture information considered by admissions officers,
estimating the model pooled across all years, or excluding ALDC applicants from the estimation
95

In his rebuttal, Prof. Arcidiacono updated his ratings variables to account for the possibility that specific combinations
of ratings can have different effects (by using interaction terms for certain combinations of ratings), as I had done in my
initial report. Thus, for ease of comparison I use his approach in this report. A comparison of my initial report results with
my results in this report demonstrates that the findings of my model are qualitatively the same with his approach versus
my original approach. Card Report, p. 68, Exhibit 19.
HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 53

sample) leads to a model that less accurately reflects the actual process, and is, therefore, less
reliable. Therefore, I do not adopt these suggestions. See Appendix C for a full list of the variables in
my updated model.
103. As shown in Exhibit 12, using my preferred admissions model, I continue to find no
evidence of bias against Asian-American applicants. The average effect of Asian-American ethnicity
is statistically insignificant, both overall and in each of the six years. The effect is slightly positive in
three of the six years and slightly negative in three, with an overall effect (-0.05 percentage points)
that—as with the effects for each individual admission class—is statistically indistinguishable from
zero.

Year-by-year logit models of admission show no consistent or statistically significant evidence of
bias against Asian-American applicants

Source: Augmented Arcidiacono Data; College Board Cluster Data; U.S. Census Data
Note: Table shows the average marginal effect of race on admission for Asian-American applicants relative to White applicants using Prof.
Arcidiacono’s expanded sample including athletes. * indicates significance at the 5% level. Marginal effects are reported as percentage
point values.

104. To illustrate the key differences between Prof. Arcidiacono’s updated preferred model
and my preferred model and how these differences impact the estimated effect of Asian-American
ethnicity, in Exhibit 13 I modify Prof. Arcidiacono’s model step by step until it matches mine. This
approach shows how each incremental methodological change to the model changes the estimated
effect of Asian-American ethnicity. The systematic pattern in Exhibit 13—whereby changes from
Prof. Arcidiacono’s model lead to a smaller (less negative) estimated effect of Asian-American
ethnicity in the admissions process—shows that many of Prof. Arcidiacono’s modeling choices
appear to have been driven by the fact that they increase the alleged bias against Asian-American
applicants (i.e., result in an estimated effect that is more negative), rather than by any objective
evaluation of how Harvard’s admissions process actually works.

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 54

105. For example, a model that makes no changes to his model other than adding ALDC
applicants back into the sample reduces Prof. Arcidiacono’s estimate of the negative effect of AsianAmerican ethnicity by 20%. Similarly, adding the personal rating, parental occupation information,
intended career information, and staff interview ratings also significantly reduces his estimated effect.
A model that includes ALDC applicants in addition to this information (and that is estimated year-byyear) eliminates 92% of Prof. Arcidiacono’s estimated effect, resulting in an effect that is not
statistically distinguishable from zero.

Prof. Arcidiacono’s modeling decisions overstate the effect of Asian-American ethnicity on
admissions

Source: Augmented Arcidiacono Data; College Board Cluster Data; U.S. Census Data
Note: [1] Data are from Prof. Arcidiacono’s sample. Marginal effects are calculated relative to White applicants. * indicates significance at
the 5% level. Marginal effects are reported as percentage point values. [2] ALDC applicants include lineage applicants, children of Harvard
faculty and staff, recruited athletes, and applicants on the Dean or Director’s interest lists. Such applicants are added to the sample and
indicators for ALDC groups are added to the model. [3] Additional controls include measures of participation in extracurricular activities
and indicators for being born in the United States and having lived outside of the United States. [4] Includes interactions of female with
intended concentration and race, interactions of race with indicator for Early Action, and interactions of race with missing SAT 2 average,
missing alumni rating, and indicator for having a converted GPA of 35.

106. An important pattern in Exhibit 13 is that once the key changes I discuss in Section 3 are
made to the model (i.e., the model is estimated by year, ALDC candidates are added back in, and
information on the personal rating, parental occupation, intended career, and staff interview are
included (row 6 onward)), the alleged bias is close to zero and statistically insignificant. While I
believe that all of these changes to the model are necessary for it to be a reliable representation of the
admissions process, I have also considered whether, starting with my updated model, any of my key
findings are sensitive to the remaining methodological changes Prof. Arcidiacono argues I should
implement. I walk through each of these in turn below.
107. First, Prof. Arcidiacono states that it is acceptable to include ALDC applicants in my
HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 55

model as long as I interact the relevant variables (recruited athlete indicator, lineage applicant
indicator, etc.) with the race variables. Specifically, he says: “It is thus essential to either (1) remove
these [ALDC] applicants from the analysis; or (2) allow for the possibility that the effect of race is
different for these applicants (i.e., interacting these variables with race).”96 I have done as he suggests
and estimated my updated model by allowing the effect of an applicant’s ALDC status to vary by
race (i.e. included interactions of ALDC status with race) and I find that my results are not sensitive
to this change. The average effect over the six years is unchanged. (See Exhibit 14.) 97

There is no consistent or statistically significant evidence of bias against Asian-American
applicants even when the effect of ALDC status is allowed to vary by race

Source: Augmented Arcidiacono Data; College Board Cluster Data; U.S. Census Data
Note: Table shows average marginal effect of race on admission for Asian-American applicants relative to White applicants using the
updated Card model with interactions of race and indicators for ALDC groups. * indicates significance at the 5% level. Marginal effects are
reported as percentage point values.

108. Second, Prof. Arcidiacono argues that I “[err] in failing to include interaction terms.”98
Specifically, he is concerned that my model does not allow the effect of disadvantaged status to vary

96

Arcidiacono Rebuttal, p. 36.
When Prof. Arcidiacono estimates his model that employs this approach and includes lineage applicants, children of
Harvard faculty and staff, and applicants on the Dean’s or Director’s interest lists, he continues to exclude recruited
athletes from the sample. As noted above, it is my understanding that recruited athletes are part of the same admissions
process as all other applicants. I therefore include recruited athletes in my preferred model. However, because Prof.
Arcidiacono presents a model that excludes recruited athletes, I also estimate my model excluding these applicants and
confirm that it does not affect my finding that there is no evidence of bias against Asian-American applicants. See
workpaper.
98
Arcidiacono Rebuttal, p. 19.
97

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 56

by race.99 I modify my updated model to allow the effect of disadvantaged status to vary by race and
find that my conclusion (that there is no evidence of bias against Asian-American applicants) is not
sensitive to this change. Although the average marginal effect across the six years becomes slightly
more negative, it is still statistically indistinguishable from zero and there is still a mix of positive and
negative effects across the six years (see Exhibit 15).

There is no consistent or statistically significant evidence of bias against Asian-American
applicants even when the effect of disadvantaged status is allowed to vary by race

Source: Augmented Arcidiacono Data; College Board Cluster Data; U.S. Census Data
Note: Table shows the average marginal effect of race on admission for Asian-American applicants relative to White applicants using the
updated Card model with interaction of race and indicator for disadvantaged. * indicates significance at the 5% level. Marginal effects are
reported as percentage point values.

109. Third, Prof. Arcidiacono disagrees with how I have used the available extracurricular
activity information in the Harvard database. He complains that two of my modeling decisions—
aggregating the 29 activities into 12 groups and omitting hours spent on activities other than work—
cause me to understate the estimated effect of Asian-American ethnicity.100 The variables he uses in
place of mine have their own limitations, however. For example, Prof. Arcidiacono uses a coarse
measure of hours of participation, which fails to identify the students most committed to a particular
activity, and his measure of years of participation in a given activity gives equal weight to all
instances of participation, regardless of the seriousness of the student’s commitment. These features
appear to reward breadth of participation slightly more than depth. While I disagree with a number of
Prof. Arcidiacono’s decisions about how to use the available information about applicants’

99

Prof. Arcidiacono also includes in his preferred model a number of other interaction variables, such as allowing the
effect of race or intended concentration to vary by gender, or allowing the effect of having a missing alumni interview
rating to vary by race. I do not include these interaction terms in my model. As I discussed in my initial report, the choice
to include interactions should be informed by a clear economic theory or methodological goal since there are hundreds of
potential interactions one could add to the admissions model (Card Report, p. 49).
100
Arcidiacono Rebuttal, pp. 40–41.
HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 57

extracurricular activities from the database, I modify my updated model to incorporate his activity
variables and show that I still find no evidence of bias against Asian-American applicants. (See
Exhibit 16.)

There is no consistent or statistically significant evidence of bias against Asian-American
applicants even when Prof. Arcidiacono’s preferred measures of extracurricular activity
participation are used

Source: Augmented Arcidiacono Data; College Board Cluster Data; U.S. Census Data
Note: Table shows the average marginal effect of race on admission for Asian-American applicants relative to White applicants using the
updated Card model with Prof. Arcidiacono’s preferred measures of extracurricular activity participation. * indicates significance at the 5%
level. Marginal effects are reported as percentage point values.

110. Prof. Arcidiacono criticizes my choice to include an applicant’s total hours worked but
exclude hours spent on other activities because work activities “are only the eighth most popular
activity listed for whites,” and elevating work above other extracurricular activities “distorts the
analysis” since White applicants work more hours than Asian-American applicants.101 I included this
particular measure because the number of hours a student works in a job is a straightforward measure
of a student’s socioeconomic status as I have previously discussed, and there are limited individualspecific measures of socioeconomic status available in the database. Crucially, Prof. Arcidiacono’s
preferred measures of hours spent working (whether above or below median hours) cannot identify
fully the degree to which there is variation among applicants in hours worked at a job.
111. Fourth, as discussed in Section 3.2.1 above, Prof. Arcidiacono criticizes my model
because it includes parental occupation variables that he considers “unreliable,” primarily because
they vary from one year to the next.102 Although my year-by-year model addresses his concern, I
have also conducted a sensitivity where I classify parent’s occupation into a combined “missing or
101
102

Arcidiacono Rebuttal, p. 41.
Arcidiacono Rebuttal, pp. 31–33.
HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 58

unstable” category if it falls in one of the five categories Prof. Arcidiacono claims are problematic
due to their fluctuations over time (Other, Homemaker, Unemployed, Low Skill, and SelfEmployed). Again, as demonstrated in Exhibit 17, my results are not at all sensitive to this decision,
and my conclusion that there is no evidence of bias against Asian-American applicants is
unchanged.103

There is no consistent or statistically significant evidence of bias against Asian-American
applicants even when I modify my parental occupation variables to address Prof. Arcidiacono’s
critique

Source: Augmented Arcidiacono Data; College Board Cluster Data; U.S. Census Data
Note: Table shows the average marginal effect of race on admission for Asian-American applicants relative to White applicants using the
updated Card model with modifications to parental occupation controls, grouping ‘Laborer (Unskilled)’, ‘Low Skill’, ‘Self-Employed’,
‘Unemployed’, ‘Homemaker’, and ‘Other’ as one occupation category. * indicates significance at the 5% level. Marginal effects are reported
as percentage point values.

112. Fifth, Prof. Arcidiacono argues that the staff interview rating should not be included in
the admissions model.104 Although I disagree with excluding this information (as discussed in Section
3.2.3 above), doing so does not alter my conclusions—there continues to be no statistically
significant evidence of bias against Asian-American applicants (see Exhibit 18).

103

As mentioned earlier, Appendix B.1 presents results for an additional sensitivity that addresses Prof. Arcidiacono’s
more technical critique about how I aggregated occupational categories.
104
Arcidiacono Rebuttal, p. 66.
HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 59

There is no consistent or statistically significant evidence of bias against Asian-American
applicants even if staff interview ratings are excluded from the model

Source: Augmented Arcidiacono Data; College Board Cluster Data; U.S. Census Data
Note: Table shows average marginal effect of race on admission for Asian-American applicants relative to White applicants using the
updated Card model removing the indicator for receiving a staff interview rating. * indicates significance at the 5% level. Marginal effects
are reported as percentage point values.

4.2. Analysis of key subgroups of the data provides further evidence that there is no bias in
Harvard’s admissions process
113. In my initial report, I showed that there was a positive (though statistically insignificant)
effect of Asian-American ethnicity for two key subgroups of Asian-American applicants: female
applicants and applicants from California dockets. (Recall that the Admissions Committee divides
Harvard applicants into dockets based on the geographic region of each applicant’s high school.) In
that report, I explained that analysis of subgroups of applicants is a well-accepted method for helping
assess whether an average unexplained gap between two groups of applicants is caused by
discrimination or, instead, by unobserved differences between the two groups. In that report, I wrote
the following:
One way to examine whether a racial disparity is attributable to bias is to
assess whether it is robust and consistent across subgroups and time
periods in the data. If discrimination against Asian-American applicants
were the cause of the racial disparity in admission rates, one would
expect to see a systematic and robust racial difference in admission rates
across all relevant subgroups and time periods. By contrast, if the gap
instead reflects differences across race in factors that Harvard considers
when making admissions decisions—but that are missing from the
model—it is much more likely that the gap will vary across subgroups
HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 60

because, simply by chance, some subgroups in the data are likely to be
particularly strong or weak, in aggregate, on factors not accounted for in
the model.105
114. In his rebuttal, Prof. Arcidiacono attempts to counter the subgroup analysis I presented
in my initial report with three main critiques, none of which is sound. First, he reiterates his main
criticisms of my model. Second, he states that I have failed to show that these subgroups are
“statistically different from [my] other findings.” Third, he states that his concerns with the rating
combinations I use in my main model are exacerbated when I estimate my analysis at the subgroup
level.106 In the remainder of this section, I explain why these critiques are flawed, and why my key
results hold.
115. First, in Section 3 above, I have extensively described the shortcomings of Prof.
Arcidiacono’s criticisms of my main model. As explained, my model more accurately reflects the
process Harvard actually uses to select among applicants. My model does not filter out any applicants
in order to obtain a particular result. My model is also less subject to the omitted variable bias
endemic in Prof. Arcidiacono’s analysis, and it properly considers the fullest possible set of criteria
Harvard uses when selecting students.
116. Prof. Arcidiacono’s second complaint—that I have not tested for statistically significant
differences between this subgroup and the overall main model—misses the point of the analysis. The
goal of this analysis is to help distinguish between a hypothesis of bias and a hypothesis of
unmeasured differences. If Harvard were in fact biased in its decisions, I would not expect to see a
small, positive (though statistically insignificant) estimated effect of Asian-American ethnicity in two
of the largest subgroups of Asian-American applicants (accounting for nearly two-thirds of AsianAmerican applicants). I would instead expect to see a robust pattern of negative bias across most
Asian-American subgroups. Thus, the patterns I observe in my subgroup analysis are more consistent
with the fact that different groups of applicants have different unobserved characteristics than with a
theory of discrimination in which the admissions committee targets its animus to an arbitrary (and
relatively small) subgroup of Asian-American applicants.
117. Finally, to respond to Prof. Arcidiacono’s concerns that the ratings combinations I used
in my preferred year-by-year model are not robust to subgroup level analysis because of the small
105
106

Card Report, p. 75.
Arcidiacono Rebuttal, p. 44.

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 61

sample size, I replicate my results from the prior report, using my updated model which uses Prof.
Arcidiacono’s ratings variables instead of ratings combinations. As Exhibit 19 and Exhibit 20 show,
these results are robust to Prof. Arcidiacono’s concerns.107

The estimated effect of Asian-American ethnicity is positive (though statistically insignificant) for
Asian-American women

Source: Augmented Arcidiacono Data; College Board Cluster Data; U.S. Census Data
Note: Table shows the average marginal effect of race on admission for Asian-American applicants relative to White applicants using the
updated Card model on the sample of female applicants. * indicates significance at the 5% level. Marginal effects are reported as
percentage point values.

107

There is also no evidence of bias against Asian-American applicants (and, if anything, evidence of a positive effect of
Asian-American ethnicity) for female applicants and applicants from California within the sample Prof. Arcidiacono
prefers (excluding applicants who are lineage, recruited athletes, children of Harvard faculty staff, or on the Dean’s or
Director’s interest lists). See workpaper.
HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 62

The estimated effect of Asian-American ethnicity is positive (though statistically insignificant) for
Asian-American applicants from California

Source: Augmented Arcidiacono Data; College Board Cluster Data; U.S. Census Data
Note: Table shows the average marginal effect of race on admission for Asian-American applicants relative to White applicants using the
updated Card model on the sample of applicants applying from California dockets. * indicates significance at the 5% level. Marginal efects
are reported as percentage point values.

118. As discussed above in Section 3, Prof. Arcidiacono’s rebuttal report proposes a new
theory that Harvard’s alleged discrimination against Asian-American applicants does not apply to the
ALDC applicants he claims have a different admissions process because there is no negative gap in
admission rates between White and Asian-American applicants in those groups. If one takes seriously
Prof. Arcidiacono’s new claim that Harvard somehow discriminates only against some subgroups of
Asian-American applicants, my analysis above shows that applicants from California and female
applicants are also not discriminated against. Such applicants represent nearly two-thirds (64%) of all
domestic Asian-American applicants to Harvard over the six classes from 2014 through 2019.108 Prof.
Arcidiacono does not explain how or why Harvard would discriminate only against Asian-American
men from states other than California, and it would seem to be nonsensical for Harvard to run a costly
and highly complex admissions process and only discriminate against Asian-American applicants
from some states—excluding those from the state with the largest fraction of Asian-American
applicants. A more sensible interpretation of the differences across subgroups in the effect of AsianAmerican ethnicity is that there are unobserved differences between various subgroups of candidates
that cause the disparity to be positive for some, negative for others, and statistically indistinguishable
from zero as a whole.
119. In sum, the subgroup results discussed above and in my initial report reveal patterns that
are entirely inconsistent with systematic discrimination against Asian-American applicants. SFFA

108

See workpaper.
HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 63

and Prof. Arcidiacono have offered no coherent explanation or theory of bias to explain these
patterns. Without any explanation or theory for why such a pattern of discrimination would exist, the
fact that so many subgroups of Asian-American applicants show no evidence of bias is strong
evidence in support of my broader claim that the small differences in admission between AsianAmerican applicants and White applicants are best interpreted as differences between the two groups
of applicants in characteristics that are not perfectly measured by the admissions data, rather than by
racial bias against Asian-American applicants.
4.3. Prof. Arcidiacono’s new allegation of bias against dockets with larger shares of Asian-American
applicants lacks any causal credibility
120. Section 9.3 of Prof. Arcidiacono’s rebuttal offers yet another new theory of how Harvard
allegedly imposes racial “penalties.” He claims that Harvard “could also impose racial preferences or
penalties through indirect channels such as geographic preferences based on the demographics of the
targeted areas.”109 In making this claim, Prof. Arcidiacono presents no documentary evidence of such
behavior by Harvard. Nonetheless, he presents an analysis that purports to show that Harvard is
biased against dockets with a larger share of Asian-American applicants. Specifically, he alleges that
Harvard penalizes applicants of all races, regardless of other qualifications and/or life experiences,
simply because they come from dockets that happen to have higher shares of Asian-American
applicants.110 There are significant problems with this argument.
121. First, putting this claim in perspective shows just how unlikely and unfounded it is. The
California dockets (discussed in the previous section) are the domestic dockets with the three largest
shares of Asian-American applicants in the sample. These dockets contain over 33,000 applicants,
61% of whom are not Asian-American.111 Prof. Arcidiacono’s claim is that Harvard penalizes each
and every one of the applicants on these dockets (and other dockets with a high share of AsianAmerican applicants) as a way to impose a racially motivated penalty targeted at Asian-American
applicants. The apparent logic here is that, rather than impose a direct penalty on Asian-American
applicants, Harvard is penalizing large swaths of its applicant pool simply because they are from an
area with a high share of Asian-American applicants. This is an ill-founded claim based on an
unusual theory of discrimination. As I will discuss in more detail below, it is particularly unusual
because the estimated effect of ethnicity for Asian-American applicants from the California dockets
is positive (though statistically insignificant). Why would Harvard admit Asian-American applicants
on California dockets at higher rates than White applicants from those dockets with similar
characteristics and then at the same time penalize applicants of all races on these dockets in an
109

Arcidiacono Rebuttal, p. 77.
Arcidiacono Rebuttal, p. 78.
111
See workpaper.
110

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 64

attempt to discriminate against Asian-American applicants? Again, Prof. Arcidiacono offers no
evidence that Harvard is pursuing such a strange policy.
122. His analysis also suffers from at least two empirical flaws. To understand these flaws, it
is helpful to first explain how Prof. Arcidiacono performs this analysis. Prof. Arcidiacono starts by
collecting the coefficients for each of the separate dockets from his admissions model. These
coefficients represent the average effect of being from a given docket on an applicant’s probability of
admission after controlling for other factors in the admissions model (within a given year). The
proper way to interpret these coefficients is that they capture unobserved factors that are not in the
admissions model, but that are specific to that docket, that might increase or decrease the applicant’s
probability of admission. For example, students from different dockets will be coming from different
high schools, and, thus, may have different levels of preparedness. Those differences would be
captured by the docket coefficients because “preparedness” is not a variable directly accounted for in
the admissions model.
123. Prof. Arcidiacono then takes these coefficients from each docket, correlates them with
the share of the docket that is Asian-American, and finds a negative correlation. Based on this
analysis alone, he then asserts that this correlation shows a causal relationship between a docket’s
admission rate and the share of Asian-American applicants on the docket. A causal relationship of
this sort would reflect a discrimination scheme in which Harvard penalizes whole dockets so that it
can indirectly penalize Asian-American applicants.
124. The first fundamental flaw with this analysis is that it controls for no other docketspecific characteristics. The differences between dockets measured by the docket fixed effects could
be due to any number of things: unobserved measures of school composition, socioeconomics, or
even geographic proximity to Harvard. Prof. Arcidiacono’s simple correlation analysis does not allow
him to discern whether the different admission rates across dockets are due to the share of AsianAmerican applicants in that docket or any number of other things that differ across dockets.
125. One way to see the unreliable nature of Prof. Arcidiacono’s finding is in Exhibit 21. In
the first row, we see Prof. Arcidiacono’s claim that an increase in the share of Asian-American
applicants on a docket causes Harvard to discriminate against everyone in that docket. In the second
row, using the exact same approach, I show that Harvard has an even more intense “bias” against
dockets with a high share of applicants who receive a guidance counselor rating of 1 or 2. In other
words, taking Prof. Arcidiacono’s analysis seriously, one would be forced to conclude that Harvard
discriminates against students with strong ratings from guidance counselors by lowering the
admission rate on dockets where they are most common. Of course, this is not true. The only
plausible interpretation of the results in Exhibit 21 is, instead, that there are other features of each

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 65

docket that affect the admission rate that are correlated with receiving a strong guidance counselor
rating and Asian-American ethnicity. This is yet another example of a common pattern in Prof.
Arcidiacono’s empirical analysis—implementing analyses that do not control for relevant factors and
then interpreting the results as evidence of bias against Asian-American applicants.

Simple changes to Prof. Arcidiacono’s analysis of docket-level bias show that his allegations are not
credible

Source: Augmented Arcidiacono Data; College Board Cluster Data; U.S. Census Data
Note: Docket-year fixed effects are obtained from Prof. Arcidiacono’s preferred admissions model estimated using applicants to the classes
of 2014 – 2019 who are in his expanded sample excluding athletes. Regressions of docket-year fixed effects on shares also contain year
fixed effects. * indicates significance at the 5% level.

126. Perhaps more importantly, if Harvard were in fact trying to penalize Asian-American
applicants from specific dockets where they are most highly concentrated, a much more plausible
scenario would be that Harvard would “raise the bar” a bit higher for only the Asian-American
applicants from those dockets—rather than imposing a penalty on the majority of applicants, who are
not Asian-American, in those dockets. In fact, this is a theory that I explored in my first report and
again in Section 4.2 of this report. The proper way to test such a theory is to estimate the full
admissions model (controlling for as much information as possible) on applicants from such dockets,
and then test whether there is any evidence of a larger disparity between White and Asian-American
applicants in the dockets where Asian-American applicants are more concentrated. As I showed in
my first report (and again in Section 4.2 above), that analysis shows that, if anything, AsianAmerican applicants are admitted at slightly higher rates relative to similarly qualified White
applicants in the California dockets where they are most concentrated.
127. It is difficult to imagine a plausible theory of bias against Asian-American applicants
whereby Harvard would penalize all applicants from dockets with high concentrations of AsianAmerican applicants as a way to indirectly penalize Asian-American applicants relative to White
applicants, yet simultaneously treat Asian-American applicants in those same dockets a bit better than
White applicants. Given the patterns in the data, Prof. Arcidiacono’s theory is simply not credible.

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 66

4.4. Other technical criticisms of my model do not change my findings
128. Prof. Arcidiacono also offers a handful of minor technical critiques of my methodology
that do not affect my conclusions. For example, he criticizes my decision to include all applicants
(including those for whom admission or rejection is perfectly predicted) in my average marginal
effect calculation, and my decision to model profile ratings using variables that reflect specific
combinations of the four profile ratings as opposed to modeling the effect of each profile rating
separately. In this section, I address these criticisms.
129. Prof. Arcidiacono disagrees with my decision to calculate an average marginal effect
over all domestic applicants. Specifically, he argues that I should exclude from my calculation
applicants who are not competitive, i.e., those who have a marginal effect of race that is zero due to
the fact that their rejection is perfectly predicted by one of the variables in the model.112 He argues
that including such applicants is misleading and causes me to “dilute the estimates of preferences by
including many applicants whose characteristics are such that rejection is assured.”113 He further
states that “[a] conservative position would be to focus the testing for racial preferences or penalties
on all of those applicants who are not immediately ruled out—which would mean removing perfect
predictions.”114 While this is a minor point, there are a number of problems with Prof. Arcidiacono’s
arguments.
130. First, it is important to note that this decision has no effect on the average marginal effect
of Asian-American ethnicity in my updated model. As shown above in Exhibit 13, the effect remains
the same when I account for perfectly predicted applicants in my marginal effect calculation.
131. Second, Prof. Arcidiacono seems to be confused, because what I did in my first report
(and continue to do in this report) is exactly what he says I should do. Although the average marginal
effects reported in my initial report reflect an average that includes all applicants, the statistical test I
conduct to determine whether the effect is statistically indistinguishable from zero is based only upon
applicants who are not perfectly predicted, i.e., the applicants Prof. Arcidiacono says the test should
be based on. This means that the results of my statistical test are the same whether or not I include
people who are perfectly predicted in the calculation of my average marginal effect. Additionally, as
just noted, the average marginal effect does not substantively change.
132. Third, contrary to Prof. Arcidiacono’s assertion, I include in my average marginal effect
112

For example, in my model, if every single applicant with an academic rating of 5 in a given year were rejected,
applicants with an academic rating of 5 in that year would be “perfectly predicted” and thus the marginal effect of their
ethnicity (whether it be Asian-American or something else) would be zero.
113
Arcidiacono Rebuttal, p. 18.
114
Arcidiacono Rebuttal, p. 18.
HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 67

calculation (but not my statistical test) not only the least competitive applicants (those whose
rejection is perfectly predicted) but also the most competitive applicants (those whose admission is
perfectly predicted). While there are far fewer applicants who are perfectly predicted to be admitted,
contrary to Prof. Arcidiacono’s claim, my approach was not one-sided in the sense that it only
included uncompetitive applicants. I included both the most competitive and the least competitive
applicants in my reported average marginal effect.
133. Prof. Arcidiacono also disagrees with my decision to control for combinations of the four
profile ratings in my year-by-year models. He argues that my approach “works to conceal the true
effect of racial preferences,”115 yet his own analysis shows that using combinations of ratings as
control variables changes the average marginal effect of Asian-American ethnicity by only 0.02
percentage points relative to his method (when using his model with the personal rating, which, as
discussed in Section 3, omits a number of key variables).116 As I explained in my initial report, I use
these ratings combinations variables in my year-by-year models not to “conceal the true effect of
racial preferences” but because there is not enough data to estimate separately the effect of specific
ratings that are very rare in the data due to limited sample size (e.g., personal ratings of 1), and to
allow my model to account for any additional weight Harvard places on specific combinations of
ratings. To ensure that this decision did not impact my findings, I also conducted a sensitivity
analysis in my initial report where I ran a pooled model two ways—using ratings combinations and
using Prof. Arcidiacono’s preferred ratings variables—and showed that the results were the same.117
134. Finally, as discussed in Section 4.1, despite his criticisms of my use of ratings
combinations as control variables, Prof. Arcidiacono changed his own ratings variables in his new
report to allow them to account for specific combinations of ratings, just as I did in my original
model. As noted above, given this change, I have adopted his ratings variables for all of the
regressions in this report to eliminate any further claims that those variables are the cause of the
different conclusions we reach. They are not.

115

Arcidiacono Rebuttal, p. 64.
Arcidiacono Rebuttal, p. 73, Table 8.2N.
117
Card Report, p. 48, footnote 84.
116

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 68

5. THE EVIDENCE IS NOT CONSISTENT WITH ADMISSIONS DECISIONS BEING
DETERMINED BY RACE ALONE
135. In my initial report, I presented a variety of analyses that explored whether race was a
“determinative” factor in admissions. As I discussed in my original report, while race does have a
significant effect on the probability of admission for some applicants, the data also show that—
consistent with Harvard’s whole-person admissions process—each candidate who is admitted to
Harvard has multiple dimensions of quality. These facts about Harvard’s admissions process are not
consistent with race being a “determinative” factor in admissions.
136. In his rebuttal, Prof. Arcidiacono argues that race is “determinative” because race can
have a relatively large effect on admissions for the subset of applicants who are highly
competitive.118 As I explain in this section, Prof. Arcidiacono has interpreted the data incorrectly.
Specifically, I show below that (a) race has limited explanatory power by itself, (b) race has less
explanatory power than other key variables in the full regression model, (c) the marginal effect of
race is very small for almost all applicants, and (d) variables other than race can have a large (or even
larger) effect on admissions for individual applicants. All of these patterns are consistent with the fact
that, to be admitted to Harvard, applicants must have multiple areas of strength, and race is not a
determinative factor.
5.1. Race alone is uninformative in Harvard’s decision process
137. In my initial report, I included an analysis that measured how well different factors in the
admissions process helped explain Harvard’s admissions decisions. Specifically, I showed that a
model that considered only race had almost no explanatory power—a Pseudo R-Squared of just
0.002. By contrast, models that include only information on the student’s socioeconomic background
were much more powerful, and models that include only student profile ratings were even more
powerful (Pseudo R-Squared equal to 0.33).119 What these analyses establish is that race alone does
not determine whether or not an applicant is admitted, and numerous other characteristics, on their
own, are much better predictors of Harvard’s admissions decisions.
138. Prof. Arcidiacono responded to this analysis by asserting that: 1) a model with only race
as an explanatory variable should be expected to perform poorly, and 2) the fact that race alone
118
119

Arcidiacono Rebuttal, pp. 49–51.
Card Report, p.83, Exhibit 27.

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 69

explains very little about admissions decisions actually suggests that racial preferences are, somehow,
quite large.120 Prof. Arcidiacono explicitly states that “[i]n order to properly evaluate the role of race
in the admissions process, it is paramount that one controls for the relevant factors in the admissions
decision.”121 This statement from Prof. Arcidiacono highlights a central difference in how Prof.
Arcidiacono and I quantify the relative importance of race in Harvard’s admissions process. Prof.
Arcidiacono is focused on whether race can have a relatively large effect for some candidates, once
we account for their other qualifications and/or life experiences. However, as I explained in my first
report, if race alone truly determines whether any individual applicant is admitted, then knowing that
applicant’s other characteristics should not matter. The fact that 91% of African-American applicants
and 93% of Hispanic applicants are not admitted indicates that other qualities besides race are highly
relevant in determining who is and is not admitted.122 Prof. Arcidiacono’s acknowledgement that he
needs to know the other characteristics of each applicant to assess the importance of race is an
explicit recognition that no single factor in the admissions process (including race) is determinative.
139. Nonetheless, I modify my analyses from my initial report to address Prof. Arcidiacono’s
concerns. I estimate my updated model that includes all factors (not just one factor at a time as I did
in my previous report) and then calculate how the model’s explanatory power (as measured by
Pseudo R-Squared) would change if I were to remove the effect of factors (such as race) one at a
time. This exercise allows me to respond directly to Prof. Arcidiacono’s critique, by seeing how
much explanatory power the model loses when race is excluded, compared to excluding other
important factors from the model.
140. Exhibit 22 shows the Pseudo R-Squared for my updated model as well as for my updated
model after removing the explanatory power of several different factors (such as race) one at a time.
The results of this exercise are unambiguous. Unsurprisingly, the removal of race has the least effect
on the explanatory power of the model. For example, turning off the effect of the teacher and alumni
ratings reduces the model’s explanatory power by 50%, but turning off the effect of race causes a
drop of only 10%. Turning off other factors, such as the academic, personal, or extracurricular
ratings, also have a larger impact on the model’s explanatory power than race. This pattern is
completely inconsistent with Prof. Arcidiacono’s assertion that race is a central factor in the
admissions model; other factors are clearly more important.

120

Arcidiacono Rebuttal, p. 48.
Arcidiacono Rebuttal, p. 49.
122
See workpaper.
121

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 70

Race explains far less about admissions decisions than other key factors such as ratings

Source: Augmented Arcidiacono Data; College Board Cluster Data; U.S. Census Data
Note: Data are from applicants to the classes of 2014 – 2019 in Prof. Arcidiacono’s expanded sample including athletes. Predicted
probabilities are computed seperately each year, from which the pooled Pseudo R-Squared values are computed.

5.2. The fact that race has a relatively large effect on the probability of admissions for some
candidates cannot be taken as evidence that race is “determinative”
141. Prof. Arcidiacono further argues that my original report “misleadingly” focuses on
uncompetitive applicants, which purportedly obscures the larger effect of race for competitive
applicants.123 He then points to the relatively large effect of race for the subset of African-American
applicants who are most competitive as evidence that race is determinative for those applicants.124 As
I have discussed both in this report and in my original report, in order to reliably test for the alleged
racial bias against Asian-American applicants, it is important to consider the full population of
applicants, rather than strategically select particular cases. In fact, observing the number of students
for whom race does not have an effect is a way to understand the upper limit of how important a
single characteristic can be. Thus, I continue to find that it is highly relevant to the issues in the case
that race has little to no effect on the admission outcomes for the vast majority of applicants to
Harvard.
142. Despite that, in this section, I consider the additional question of whether the relatively
large effect of race that exists for the subset of highly competitive applicants should be taken as
evidence that race is determinative even for them. As I discuss below, it should not. The reason is
that, because Harvard assesses a wide variety of characteristics across all students, it turns out that
many distinguishing characteristics (whether race or another trait) will produce a more powerful

123
124

Arcidiacono Rebuttal, p. 49.
Arcidiacono Rebuttal, p. 51.
HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 71

effect for students on the margin of being admitted or not admitted.
143. To help understand these points, consider Exhibit 23 below. In this exhibit, I compare
the average marginal effect of several different characteristics. In particular, I compare the size of the
marginal effect associated with being African-American or Hispanic to the size of the marginal effect
for several non-racial characteristics, including being a lineage applicant, or receiving a top profile
rating (academic, personal, or extracurricular). I adopt Prof. Arcidiacono’s method of looking at the
marginal effect for each decile of the admissions index, and show the relative effect of each
characteristics on the probability of admission within each decile.

The effect of race follows the same pattern across deciles as other characteristics in Harvard’s
admissions process

Source: Augmented Arcidiacono Data; College Board Cluster Data; U.S. Census Data
Note: Data are from applicants to the classes of 2014 – 2019 in Prof. Arcidiacono’s expanded sample including athletes. Deciles are
constructed by year, across the full sample, based on the predicted probabilities of admission after removing the effect of the given
characteristic. Marginal effects are computed for applicants with the given characteristic relative to the baseline (i.e. White, non-lineage,
academic rating of 3, extracurricular rating of 3, and personal rating of 3). Marginal effects are reported as percentage point values. “-”
indicates that there are no applicants with a given characteristic in a given decile.

144. The pattern here is clear. For most characteristics, the marginal effect of that
characteristic is highest for candidates in the top admissions deciles. This is a natural consequence of
the fact that Harvard places a very high value on multiple dimensions of quality, rather than on any
single attribute. For candidates in the lower deciles (who have relatively few observed characteristics
that Harvard values), the “tip” for African-American race, or for being a lineage applicant, or for
having a profile rating of 1 is very small, because having just one major strength is not enough to
ensure a high probability of admission. Candidates in the upper deciles, on the other hand, are
relatively strong on at least one dimension, and in most cases several dimensions, that Harvard
values. For these candidates, the extra “tip” for any one additional strength can be large. Importantly,
the pattern that Prof. Arcidiacono demonstrates for race—the largest effects concentrated among the
strongest candidates—is present for other characteristics in Exhibit 23, such as being a lineage
HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 72

applicant or a candidate with a top profile rating.
145. If we compare the patterns of the effect of race specifically with the pattern of the effect
of receiving an academic or extracurricular rating of 1, a notable finding is that the effect of a top
academic or extracurricular rating is both larger and more widespread across the full distribution of
applicants than the effect of race. If race were truly “determinative” then we would not expect to see
other factors with a larger effect than race, and we would not expect to see such a small effect for
race among the least competitive applicants. The fact that we do means that the incremental value of
race is smaller than the incremental value of having a top academic or extracurricular rating. Such
patterns are consistent with race being one of many factors that can help distinguish a candidate—not
the only one (or even the most important one).
146. Exhibit 24 presents another way to see this same point for African-American
candidates—the group of candidates Prof. Arcidiacono focuses on most. Exhibit 24 demonstrates two
important facts. First, it shows that the effect of race is small (9 percentage points or less) for the vast
majority of African-American applicants (those in deciles 1 to 7).125 This can be seen in the second
column, which reports the average marginal effect of race for African-Americans. Second, it shows
that for the subset of African-American applicants for whom the marginal effect of race is largest
(applicants in the 9th and 10th admissions index deciles), the marginal effect of ratings is substantially
larger than the marginal effect of race. For the strongest applicants (those in the 10th decile of the
admissions index), the effect of ratings is almost twice the size of the effect of race.

125

Note that I use Prof. Arcidiacono’s preferred approach of constructing the index across all races, not just AfricanAmerican applicants. 86.4% of all African-American applicants are in deciles 1 to 7. See workpaper.
HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 73

The effect of race is smaller than that of ratings for African-American applicants

Source: Augmented Arcidiacono Data; College Board Cluster Data; U.S. Census Data
Note: Data are from applicants to the classes of 2014 – 2019 in Prof. Arcidiacono’s expanded sample including athletes. Deciles are
constructed by year, across the full sample, based on the predicted probabilities of admission after removing the effect of race. All ratings
include the four profile ratings, teacher and guidance counselor ratings, and alumni ratings. Marginal effects are computed for AfricanAmerican applicants relative to the baseline (i.e. White, and ratings of 3 for applicants with ratings of 1 and 2). Marginal effects are
reported as percentage point values.

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 74

6. DOCUMENTS AND HARVARD’S DATA UNDERMINE PROF. ARCIDIACONO’S CLAIM
THAT HARVARD IMPOSED A FLOOR ON THE ADMISSION RATE FOR SINGLE-RACE
AFRICAN-AMERICAN APPLICANTS STARTING WITH THE CLASS OF 2017
147. In his first report and again in his rebuttal, Prof. Arcidiacono claims that Harvard
manipulated its admission rates to create a floor on the admission rate for single-race AfricanAmerican applicants starting with the class of 2017. His only empirical evidence of this claim is the
fact that the difference between the admission rate for single-race African-American applicants and
all other applicants was very small for the classes of 2017, 2018, and 2019 using the IPEDS
definition of race.126 As I explained in my first report, Prof. Arcidiacono’s claim of manipulation is
not supported by any other evidence in the record. In this section, I highlight three critical reasons to
be skeptical of Prof. Arcidiacono’s claim.
148. First, Prof. Arcidiacono has substantially changed his theory for why Harvard would
manipulate the African-American admission rate. For example, in his initial report, Prof. Arcidiacono
claimed that, “[f]or the class of 2017 and going forward, Harvard adopted a new methodology for
coding race and ethnicity that was consistent with federal standards for reporting of race and
ethnicity,” and that because this “new” methodology excluded multi-racial African-American
students from the “African-American” category, this change “prompted concern at Harvard that the
new reporting would understate the number of African-American admits to Harvard.”127 As a result,
he asserted that Harvard imposed a floor on single-race African-American admissions. In response to
my pointing out that the documents did not support this claim (including the fact that Harvard
adopted the methodology at issue three years before the class of 2017),128 in his rebuttal Prof.
Arcidiacono shifts the catalyst from alleged public concern, to potential concern from admissions
officers at peer institutions who participate in particular cross-institutional meetings (discussed
below). As I explain below, his argument continues to rest on a selective reading of the available
documents and appears to be an attempt to justify ex post his alleged finding of “manipulation” in a
particular set of years for a particular racial group.
149. Second, it is important to understand that the pattern Prof. Arcidiacono identifies as
evidence of manipulation can be easily explained by random chance. Given that Harvard has at least
three operative definitions of race (New Methodology, Old Methodology, IPEDS) and several racial
groups under each definition, finding three years in a row in which one racial group’s admission rate
is close to the overall admission rate is not as unlikely as Prof. Arcidiacono suggests. Yet Prof.
Arcidiacono takes this expected pattern in the data, attempts to fit certain facts around it, and then
126

Arcidiacono Report, pp. 27–30.
Arcidiacono Report, pp. 27–28.
128
Card Report, pp. 88–89.
127

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 75

claims it is evidence of manipulation.
150. Finally, Prof. Arcidiacono’s new argument that the relative quality of single-race
African-American admitted students fell starting with the class 2017 is not supported by his own
analysis. As I show below, he makes a statistical error that when corrected undoes his results.
6.1. The record does not support Prof. Arcidiacono’s claim of a floor on single-race AfricanAmerican admissions starting with the class of 2017
151. In my initial report, I identified several reasons why Prof. Arcidiacono’s claim that
Harvard began to manipulate the African-American admission rate with the class of 2017 did not
make sense. Specifically, I explained how Harvard adopted the IPEDS methodology for federal
reporting three years before the alleged floor was implemented, and that the documents Prof.
Arcidiacono cited did not demonstrate particular concern from Harvard about public perceptions of
the single-race African-American admission rate starting in 2013 (the class of 2017 admissions
cycle). Indeed, the record shows that Harvard does not even use IPEDS when reporting statistics on
race to the press and public. Using Harvard’s preferred and most commonly used methodology for
classifying race, I demonstrated that the racial composition of the Harvard class does fluctuate
somewhat year to year. I also showed that, inconsistent with a floor on the rate of admissions, the
relative quality of African-American admitted students did not fall starting with the class of 2017.129
152. In his rebuttal, Prof. Arcidiacono stands by his general claim, stating that it “is certain—
and undisputed—that Harvard was purposely taking steps to ensure that the admission rate of singlerace African-American applicants approximated or exceeded the overall admission rate of other
domestic applicants.” 130 In addition, he modifies his story for why Harvard would choose to
manipulate these admission rates, suggesting that perhaps Harvard’s participation in admissions
industry group meetings for COFHE and ABAFAOILSS precipitated the alleged floor on
admission.131 Prof. Arcidiacono does not offer any evidence on these points, relying instead on
speculation without any basis in fact. Below I highlight several flaws in his theories.
153. First, despite Prof. Arcidiacono’s assertion that “Harvard was purposely taking steps”132

129

Card Report, pp. 87–93.
Arcidiacono Rebuttal, p. 9; Card Report p. 88.
131
Arcidiacono Rebuttal, p. 57.
132
Arcidiacono Rebuttal, p. 9 (emphasis added).
130

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 76

to bring about this statistical phenomenon, his report is notable for its lack of any direct evidence of
motivation or intent on the part of Harvard. The main motivation he posits for why Harvard would
manipulate the admission rates is that Harvard was “very concerned about criticisms tied to its IPEDS
data at the precise time the first evidence of the floor appears in the data.”133 This claim follows the
pattern of the claims in his first report, where he motivated his analysis with the unsubstantiated
assertion that the IPEDS methodology “prompted concern at Harvard that the new reporting would
understate the number of African-American admits to Harvard,” and that this concern led to Harvard
implementing a floor.134 However, Prof. Arcidiacono does not present a single “criticism tied to
[Harvard’s] IPEDS data” that might have precipitated the alleged floor on African-American
admissions. Again, Harvard had been reporting IPEDS figures to the federal government for years
before allegedly implementing the floor, and Harvard released information on the racial composition
of the admitted class to the public using its preferred “New Methodology” for classifying race—not
IPEDS.135
154. In lieu of direct evidence that Harvard was facing criticism for its use of IPEDS data,
Prof. Arcidiacono cites several documents that he claims provide evidence that Harvard was “very
concerned about the way its IPEDS enrollment numbers were being perceived by the public in early
2013.”136 In these documents, Harvard officials discuss how data on race are collected, and express
the view that the IPEDS method for classifying race can be “confusing” and “misleading” since it
does not always align with how students identify their own race. These documents provide no
evidence that Harvard’s concerns about racial statistics led it to manipulate admission rates.137
155. In his rebuttal, Prof. Arcidiacono also introduces and emphasizes the fact that Harvard
133

Arcidiacono Rebuttal, p. 9.
Arcidiacono Report, pp. 28–30.
135
Card Report, pp. 88–89.
136
Arcidiacono Rebuttal, pp. 56–57.
137
Prof. Arcidiacono cites the following material in support of his claim: “Addendum on the collection and reporting of
data on race and ethnicity,” HARV00030509 – 12 at HARV00030511. This memo discusses how demographic data is
reported by IPEDS, and how the IPEDS methodology differs from Harvard’s preferred method for reporting race. It is an
edited version of HARV00023592 – 4, which Prof. Arcidiacono also cites. Both versions of the document explain that
“the IPEDS reporting system leads to significantly underreporting percentages for all ethnicities except Hispanic
Americans. The method used by Harvard and many peer institutions gives a more complete report of the way many
students, especially those of mixed heritage, actually view their racial and ethnic identities” (at HARV00023594 and
HARV00030510 – 11, emphasis added); Email from Jeff A. Neal to William R. Fitzsimmons et al., “FW: Draft Annual
Admissions Applications Gazette Article,” February 6, 2013, HARV00023588 (“It explains the difference between
what’s reported in IPEDS (basically, all students get one ethnicity and they all add up to 100%) and what [Harvard]
report[s] publically (students pick as many ethnicities as they think apply to themselves and all [Harvard’s] students’
ethnicities add up to more than 100%).”).
134

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 77

has long participated in COFHE meetings and ABAFAOILSS “Round Robins” in which admissions
officers from various colleges and universities share, confidentially, statistics about race and
admissions.138 He documents that Harvard officials attended these meetings and shared data on race
and admissions. He then implies that the exchange of information at those meetings somehow created
an incentive for Harvard to introduce a floor on African-American admissions.139 But he offers no
documents or testimony to support this implication.
156. Indeed, nothing in the record supports Prof. Arcidiacono’s arguments that these meetings
encouraged universities to implement racial quotas. Christina Lopez, a representative for
ABAFAOILSS, testified that the point of Round Robins is for universities to share information and
best practices with respect to diversity and recruiting.140 A primary mission of ABAFAOILSS is
“sharing best practices across [its member] institutions to ensure equity and inclusion.”141 The
evidence further shows that Round Robins were a way for universities to disseminate helpful
information on successful recruiting tactics, like which particular cities, schools, and communitybased organizations seem to yield strong under-represented applicants.142 The fact that Harvard
admissions officers attended these meetings is indicative of Harvard’s commitment to recruiting
under-represented applicants—not evidence that it allegedly manipulated admission rates.
157. In sum, none of the documentary evidence presented by Prof. Arcidiacono convincingly
indicates that “Harvard was very concerned about criticisms tied to its IPEDS data at the precise time
138

Arcidiacono Rebuttal, p. 57; COFHE Admissions Statistics, Class Entering 2013, HARV00004683 – 4789; Deposition
of Christina Lopez, May 22, 2017 (“Lopez Deposition”), pp. 57–59 (“As stated in the constitution, information that is
shared is to be used as confidential… Q. Why is this information confidential? ... A. Enrollment information for colleges
is not public information.”).
139
Arcidiacono Rebuttal, pp. 56–57.
140
Lopez Deposition, p. 53 (“Q. I want to ask a few questions about the Round Robin meetings. What are the Round
Robin meetings? A. Round Robin is a separate—it is a part of our meetings where we share enrollment and application
numbers. Q. And what is the purpose of a Round Robin? A. The purpose is to share best practices as well as recruitment
information across institutions.”).
141
Lopez Deposition, p. 21 (“Our mission is to work for access for under-represented students in higher education as
determined by our constitution, providing access and sharing best practices across our institutions to ensure equity and
inclusion of under -- for historically under-represented groups in higher education”), p. 36 (“The mission of
ABAFAOILSS is to maintain, increase, and solidify access in equity for under-represented students in higher education,
as well as provide a space for those who serve within those admissions offices to have a space to share of their
experiences serving in those capacities and within their office.”).
142
Lopez Deposition, p. 80 (“If a school is doing recruitment in a certain state and they have found that their recruitment
strategy in terms of bringing in group travel with other schools or hitting a certain city and community-based
organizations is producing strong applicants, then other schools may want to know where those particular places are and
also include those in their strategies as well. If I found a school that was doing great work, I would want to share that with
my colleagues.”).

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 78

the first evidence of the floor appears in the data.”143
6.2. The pattern that Prof. Arcidiacono claims as evidence of manipulation is not as unlikely as he
suggests
158. Given the lack of a clear incentive or motivation for Harvard to manipulate admission
rates, a basic threshold question for Prof. Arcidiacono’s claim is: are the alleged “manipulated”
patterns in the data sufficiently rare that they cannot be explained by random chance? As I discuss
below, Prof. Arcidiacono’s claim does not surpass this basic threshold.
159. First, it is important to note that the more outcomes and data points that can be examined
in a search for evidence of an alleged floor on admissions, the greater the probability of finding such
a pattern just by chance. Indeed, this fundamental observation is the basis for widespread concerns in
the research community over “data mining.”144 Prof. Arcidiacono himself acknowledges this fact,
noting that the array of outcomes one could search over to find alleged evidence of a quota is vast:
“[T]here are undoubtedly many ways Harvard could impose racial floors. They could impose a floor
based on the expected number of admits, the share of admits of a particular race, or the relative
acceptance rates of particular races. Alternatively, Harvard could impose a floor based on the
expected number of enrollees of a particular race. Furthermore, Harvard could do this using a variety
of different measures of race.”145 That Prof. Arcidiacono found a pattern consistent with one of many
prospective floors is not particularly surprising.
160. Second, the specific pattern Prof. Arcidiacono homes in on is not as unlikely as he
suggests. Prof. Arcidiacono computes a very specific number in his initial report: the probability that
the single-race African-American admission rate matched overall admission rate for the classes of
2017, 2018, and 2019, which he computes to be approximately 0.2%. He argues that this means there
is a 0.2% probability that the correspondence between admission rates between 2017 and 2019
happened by chance.146 It is important to remember that the available admissions data that Prof.
Arcidiacono analyzes includes: six years of data, at least three operative definitions of race (New
Methodology, Old Methodology, IPEDS), and several racial groups under each definition. Given
these six years of data and three definitions of race, there is nothing surprising about finding three
years in a row in which one racial group’s admission rate is close to the overall admission rate.

143

Arcidiacono Rebuttal, p. 9.
For example, see Garret Christensen and Edward Miguel, “Transparency, Reproducibility, and the Credibility of
Economics Research,” NBER Working Paper #22989, December 2016, pp. 15–17.
145
Arcidiacono Rebuttal, p. 56.
146
Arcidiacono Report, p. 29.
144

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 79

161. Given that there are many racial groups and three-year periods to search over to find
evidence of an alleged quota, Prof. Arcidiacono’s estimate of 0.2% is likely to vastly understate the
probability of finding such a coincidence in the data. For example, imagine that Prof. Arcidiacono
were simply looking for a stretch of three years between 2014 and 2019 in which the admission rate
for any particular racial group matched the admission rate for all other groups using either IPEDS, the
New Methodology, or the Old Methodology. There are eight racial categories under the New
Methodology, eight under the IPEDS methodology, and at least seven under the Old Methodology,
for a total of 23 groups.147 With 23 racial groups and four possible three-year stretches to search over,
Prof. Arcidiacono has 92 opportunities (23 multiplied by four) to find the pattern of interest. Assume
for the sake of simplicity that Prof. Arcidiacono’s calculation is correct, and assume that for any
given racial group and three-year stretch there is a 0.2% chance that that group’s average admission
rates match the admission rate for other applicants.148 Because there are 92 combinations to check,
not just one, the chances of seeing evidence of an alleged quota are actually much higher than 0.2%.
Indeed, with 92 options to search over, the probability of seeing an allegedly “suspicious” three-year
stretch simply by chance is about 17% (that is, one minus 99.8% to the 92nd power).149 In other
words: the probability that Prof. Arcidiacono would find evidence of his particular type of floor is
likely much higher than the 5% threshold typically used to reject that an event occurred by chance.
Furthermore, as I noted above in paragraph 159, this is only one of many types of floors that Prof.
Arcidiacono states he could have searched over. Combined with the lack of any credible documentary
evidence discussed above, a more reasonable interpretation of the patterns in the data is that they are
due simply to chance.
6.3. The relative quality of single-race African-American admitted students did not fall starting with
the class of 2017, further undermining the idea of a floor on their admission rate
162. As noted in my first report, if a floor was imposed on African-American admission rates,
it would very likely generate a decline in the relative quality of African-American admitted students
as compared to other admitted students. The data reflect no such decline, as I showed in my first

147

The racial groups are as follows. New Methodology: White, Asian, Black, Hispanic (including Mexican and Puerto
Rican), Native American, Native Hawaiian, multi-racial, and race unknown. IPEDS: White, Asian, Black, Hispanic
(Including Mexicans and Puerto Ricans), Native Americans, Native Hawaiians, multi-racial, and race unknown. Old
Methodology: White, Asian, Black, Hispanic (including Mexican and Puerto Rican), Native American (including
Hawaiian), Other, Unknown. See, for example, “Ethnicity Backgrounds – Classes of 2014 – 2017,” HARV00005106;
“Applicants, Admits, and Matriculants – Old Methodology NLNA,” HARV00001851 – 56 at HARV00001851.
148
The probability could vary, but for illustrative purposes I assume it is fixed at 0.2% for all permutations.
149
The calculation is: 1 - (1-0.002)^(23*4) = 16.8%.

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 80

report.150
163. In his rebuttal, Prof. Arcidiacono responds to this analysis by presenting evidence that he
claims shows a statistically significant decrease in the relative Academic Index of single-race
African-American admitted students as compared to multi-racial African-American admitted
students.151 Prof. Arcidiacono refers to this relative change in the Academic Index as the “double
difference,” which he reports as statistically significant in his Table 6.2N.152 He then interprets this
“double difference” as evidence that the relative quality of single-race African-American admitted
students fell after 2017, consistent with a floor on single-race African-American admissions. He also
notes that there is an allegedly significant increase in the relative admission rate of single-race
African-American applicants versus multi-racial African-American applicants before and after 2017
(which he again refers to as the “double difference” in his table 6.3N).153
164. Prof. Arcidiacono makes a critical calculation error in this analysis that, when corrected,
reverses his key finding. Specifically, his calculations of the statistical significance of the “double
differences” in his tables 6.2N and 6.3N are implemented incorrectly, resulting in standard errors that
are too small. In other words, he overstates the precision of his estimates, which makes his results
look statistically significant when they are not (see Appendix B.2). Once I correct this error, the
“double differences” reported in tables 6.2N and 6.3N are not statistically significant at the 5% level
(Exhibit 25). In other words, when Prof. Arcidiacono’s own analysis is done correctly, there is no
statistically significant change in the relative admission rate of single-race and multi-racial AfricanAmerican admitted students before and after 2017, nor is there a statistically significant change in the
relative average Academic Index of single-race and multi-racial African-American admitted students
before and after 2017.
165. Furthermore, the relative quality of single-race African-American admitted students did
not fall as compared to that of multi-racial African-American admitted students on a more
comprehensive array of metrics. Exhibit 25 mimics Prof. Arcidiacono’s analysis of the Academic
Index in his Table 6.2N, but uses each of the four profile ratings, as well as the admissions index,
which summarizes an applicant’s overall probability of being admitted according to my updated
model. I remove the effect of race when computing the admissions index. For each characteristic, I
report the difference between single-race and multi-race African-American admitted students in the
period spanning 2014 – 2016, the period spanning 2017 – 2019, and the difference-in-difference
between these two numbers (what Prof. Arcidiacono calls the “double difference”). If the difference150

Card Report, p. 89.
Arcidiacono Rebuttal, pp. 59–60.
152
Arcidiacono Rebuttal, p. 60, Table 6.2N.
153
Arcidiacono Rebuttal, p. 61, Table 6.3N.
151

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 81

in-difference is negative and significant (as denoted by a star), this suggests that the relative quality
of single-race African-American admitted students fell.
166. I find that as measured by Harvard’s four profile ratings and the admissions index
(removing the effect of race), the relative strength of single-race African-American admitted students
did not fall relative to the strength of multi-racial African-American admitted students after 2017.
None of the difference-in-difference estimates in Exhibit 25 is statistically significant. This is highly
inconsistent with there being a floor on the admission of single-race African-American students.

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 82

The relative quality of single-race African-American admitted students did not fall in 2017
African-American
Admitted Students
Single-Race
Multi-Race
Admission Rate [2]
1. Average 2014 – 2016
2. Average 2017 – 2019
3. Difference-in-Difference

Difference [1]

6%
6%

10%
8%

-3% *
-2% *
2%

0.20
0.23

0.19
0.34

0.01
-0.12 *
-0.13

Fraction with Academic Rating of 1 or 2 [2]
5. Average 2014 – 2016
53%
6. Average 2017 – 2019
55%
7. Difference-in-Difference

48%
57%

5%
-2%
-7%

Fraction with Extracurricular Rating of 1 or 2 [2]
8. Average 2014 – 2016
47%
9. Average 2017 – 2019
48%
10. Difference-in-Difference

52%
49%

-5%
-1%
4%

Fraction with Personal Rating of 1 or 2 [2]
11. Average 2014 – 2016
74%
12. Average 2017 – 2019
74%
13. Difference-in-Difference

76%
72%

-2%
2%
4%

Fraction with Athletic Rating of 1 or 2 [2]
14. Average 2014 – 2016
20%
15. Average 2017 – 2019
22%
16. Difference-in-Difference

24%
28%

-5%
-6%
-1%

Average Admissions Index [4]
17. Average 2014 – 2016
18. Average 2017 – 2019
19. Difference-in-Difference

0.31
0.32

-0.07 *
-0.06 *
0.01

Average Academic Index [2][3]
4. Average 2014 – 2016
5. Average 2017 – 2019
6. Difference-in-Difference

0.24
0.26

Source: Augmented Arcidiacono Data; College Board Cluster Data; U.S. Census Data
Note: [1] * indicates statistical significance at the 5% level. [2] Consistent with Prof. Arcidiacono's analyses, data are from domestic
admitted applicants, including prior admitted applicants and excluding deferred admitted applicants. [3] Academic Index values are in
standard deviation units. Average Academic Index calculations exclude students with GPA flags. [4] Data are from admitted applicants in
Prof. Arcidiacono’s expanded sample including athletes (my preferred year-by-year regression model sample). The admissions index is
constructed using applicants' predicted probability of admission after removing the effect of race.

167. In sum, Prof. Arcidiacono’s assertions of a floor on African-American admissions are
supported neither by the record nor by Harvard’s data on applicant quality. The pattern Prof.
Arcidiacono takes as evidence of an alleged floor could be due to chance, and he provides no credible
documentary evidence to suggest otherwise. Furthermore, the data show no evidence of a decline in
HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 83

the relative quality of the group that is allegedly receiving preferential admissions treatment—
undermining the idea that such a floor exists.

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 84

7. MR. KAHLENBERG DOES NOT SHOW THAT HARVARD COULD ACHIEVE A
COMPARABLY DIVERSE AND HIGH-QUALITY CLASS WITHOUT CONSIDERING RACE
168. In my initial report, I analyzed a set of race-neutral alternatives proposed by SFFA and
its expert, Richard Kahlenberg. Consistent with the existing economics literature on race-neutral
alternatives—including the papers cited by Mr. Kahlenberg—I found that these policies were
unlikely to increase diversity without diminishing the quality of Harvard’s admitted class and/or
changing its characteristics in other ways that I understand matter to Harvard.154 In his rebuttal, Mr.
Kahlenberg presents three main critiques of my analysis. In this section, I address each in turn and
explain why none affects the main conclusions of my first report.
169. First, Mr. Kahlenberg offers additional arguments from the economic literature on raceneutral alternatives. He asserts that the literature supports the claim that race-neutral alternatives can
achieve diversity at low cost to quality at selective institutions.155 As I explain below, none of the
papers Mr. Kahlenberg cites, and none of the new arguments he advances, supports his claims. In
fact, the papers he cites support the main conclusion from my first report: race-neutral alternatives
reduce the ability of universities to admit students with other characteristics they value, with a
particularly large effect for more selective institutions.
170. Second, Mr. Kahlenberg presents a series of criticisms of my simulations, and offers
several new simulations of his own. As I show below, his criticisms of my simulations are either illfounded or irrelevant. My core findings are robust to his suggested changes. Moreover, the new
simulations presented by Mr. Kahlenberg actually underscore my initial findings: none of the raceneutral alternatives he offers generates a comparably diverse student body, and all result in larger
changes to class quality than the version of my simulation Mr. Kahlenberg chooses as a benchmark,
as measured by Harvard’s academic, personal, and extracurricular ratings, among other indicia.
171. Finally, Mr. Kahlenberg criticizes my analysis of expanding financial aid, recruiting
efforts, and transfer admissions as race-neutral policies for increasing diversity, as well as my
evaluation of eliminating deferred admission and using place-based admissions policies. I address
each of his criticisms below.

154

Card Report, p. 95.
Rebuttal Expert Report of Richard D. Kahlenberg, Students for Fair Admissions, Inc. v. President and Fellows of
Harvard College (Harvard Corporation), January 30, 2018 (“Kahlenberg Rebuttal”), pp. 2–5.

155

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 85

7.1. The academic literature establishes that race-neutral alternatives diminish selective universities’
ability to select on quality
172. My initial report cited the extensive academic literature that finds that the use of raceneutral alternatives by selective universities necessarily comes with a meaningful cost to class
quality.156 Mr. Kahlenberg criticizes several papers that I rely on, but does not introduce any new
academic articles beyond those cited in his and my initial reports. His criticisms do not alter or
undermine the conclusions in my initial report. Below, I discuss each of those papers in light of Mr.
Kahlenberg’s criticisms, and explain how his additional criticisms about those papers are misleading.
173. My report cites a 2004 study by Carnevale, Rose, and Strohl, which finds that it is
difficult for race-neutral alternatives to reproduce the level of racial diversity seen under admissions
policies that directly consider race.157 In his reply, Mr. Kahlenberg states that “Card fails to mention
that ten years later, these same professors found two alternatives that produced greater racial
diversity and higher mean SAT scores[.]”158 However, in the 2014 paper Mr. Kahlenberg refers to,
those authors heavily qualify these findings:
In the end, we find that “race-blind” and “race-conscious” (giving an
added boost to underserved minorities) forms of affirmative action can
substitute for the use of “race alone” in college admissions. But these
alternatives are only available if elite colleges are willing to risk lower
average test scores (in the case of two of our five simulations, one
estimate is higher but not statistically significant) and thereby lower
graduation rates.159
174. In other words, ten years after their initial paper, Carnevale, Rose, and Strohl still
conclude that considering race in the admissions process is the most efficient way to produce
increased racial diversity. Further, they conclude that race-blind admissions policies do, indeed, come
at a cost. Lastly, the researchers also caution that alternative approaches require “substantial

156

Card Report, pp. 97–103.
Card Report, p. 101.
158
Kahlenberg Rebuttal, p. 4
159
Anthony P. Carnevale, Stephen J. Rose, and Jeff Strohl, “Achieving Racial and Economic Diversity with Race-Blind
Admissions Policy,” in The Future of Affirmative Action, ed. Richard Kahlenberg, (Century Foundation Press, 2014),
(“Carnevale, Rose, and Strohl 2014”), pp. 187–202 at p. 188.
157

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 86

disruption in the admissions practices and enrollments of selective colleges.”160
175. Mr. Kahlenberg also suggests that the studies I cited by Thomas Kane and Reardon et al.,
are too limited in scope to be insightful. He writes that “Card cites studies by Thomas Kane and Sean
Reardon finding that using income instead of race in admission will not produce the same level of
racial diversity” and states these are “of little value here because they measure only the use of income
and not, as I propose, a broad set of socioeconomic variables[.]”161 This criticism is incorrect. First,
Kane does discuss targeting “broader” socioeconomic indicators, and argues that given his findings,
such policies were not likely to be much more successful. Second, Reardon et al. do not measure
“only the use of income.” Rather, they look at the Texas Top Ten Percent Plan, which Mr.
Kahlenberg champions, and conclude that it would lead to a 10% reduction in minorities attending
selective schools.162
176. Later, Mr. Kahlenberg discusses a paper by Mathew Gaertner, cited in both his earlier
report and my report, about the costs to race-neutral alternatives.163 Mr. Kahlenberg attempts to
downplay Gaertner’s conclusion that race-neutral alternatives “are complicated to implement and
may lower the academic quality of the admitted class and the likelihood of success for admitted
students.”164 In particular, Mr. Kahlenberg comments that, “Card fails to mention that Gaertner
concludes low-income students do about as well academically as underrepresented minority students
admitted through race-based affirmative action programs. And Gaertner argues that academic support

160

Carnevale, Rose, and Strohl 2014, p. 201.
Kahlenberg Rebuttal, p. 2.
162
Thomas J. Kane, “Racial and Ethnic Preferences in College Admissions,” Ohio St. Law Journal 59, 1998, pp. 971–996
at p. 990 (“There may be other characteristics that are more highly correlated with race than income alone, such as family
wealth or neighborhood poverty rates, that a college might use to construct a ‘race-blind’ measure for promoting racial
diversity. However, since blacks and Hispanics are only 6.8 percent of the highest-scoring youth, it would be difficult to
find a preference that would yield even a majority of black or Hispanic youth…even if high-scoring black or Hispanic
youth were thirteen times more likely to meet some combination of wealth, neighborhood, and family income criteria than
other youth, they would still represent less than half of the high-scoring youth meeting the criteria.”); Sean Reardon,
Rachel Baker, and Daniel Klasik, “Race, income, and enrollment patterns in highly selective colleges, 1982-2004,”
Center for Education Policy Analysis, Stanford University, 2012, pp. 1–25 at p. 14 (“Our simple simulations show that
admissions policies like the Texas Top Ten Percent rule…alone are unlikely to increase the proportion of black and
Hispanic and low-income students enrolled in highly-selective colleges…more sophisticated simulation models suggest
that the Top Ten Percent rules would…lead to a 10% reduction in the proportion of black and Hispanic students attending
highly-selective colleges and universities.”).
163
Kahlenberg Rebuttal, p. 3; Expert Report of Richard D. Kahlenberg, Students for Fair Admissions, Inc. v. President
and Fellows of Harvard College (Harvard Corporation), October 16, 2017 (“Kahlenberg Report”), pp. 12–13; Card
Report, p. 101.
164
Card Report, p. 101.
161

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 87

for low-income students should be the proper response, not ceasing to admit such students.”165 I do
not disagree that supporting low-income students is a worthwhile endeavor. In fact, the results from
my admissions model suggest that Harvard gives an admissions “tip” to students who are flagged as
disadvantaged, and to students whose parents work in lower-paying occupations. I do disagree with
Mr. Kahlenberg’s characterization of Gaertner’s findings. In fact, Gaertner writes the following,
contrary to Mr. Kahlenberg’s characterization:
Across outcomes, strictly overachieving class-based admits can be
expected to perform quite well—better, in fact, than typical
undergraduates. The forecasts for strictly disadvantaged admits,
however, are not as encouraging. Their GPAs, graduation rates, and
earned credit hours lag far behind the baseline. This said, given
additional time in college, disadvantaged admits’ graduation rates
accelerate comparatively quickly […] thereby narrowing the graduation
gap. To sum, analysis of college outcomes for historical surrogates
suggest college success for class-based admits is possible, but it is far
from guaranteed[.]166
177. Mr. Kahlenberg also objects to my characterization of Sigal Alon’s race-neutral
simulations, which Mr. Kahlenberg cited in his initial report.167 As I noted in my initial report, Alon’s
race-neutral simulations do not consistently show that racial diversity would meet or exceed current
levels. In her one simulation where the fraction of African-American and Hispanic admitted students
surpasses that achieved by considering race, Alon finds that this diversity comes at the cost of a
decline in academic selectivity.168
178. Additionally, Mr. Kahlenberg argues that the literature’s focus on “efficiency” is
misleading. Specifically, Mr. Kahlenberg conflates the concepts of “efficiency” and “administrative
convenience,” and then argues that “convenience” is not the measure by which we should judge raceneutral alternatives.169 Contrary to Mr. Kahlenberg’s argument, “efficiency” and “convenience” are
two logically separate concepts that cannot be easily combined. In my report, and in the many papers
we both cite, an “efficient” policy refers to an admissions policy that obtains the desired outcome
(diversity) while minimizing the cost in other dimensions of the admitted class, such as academic
165

Kahlenberg Rebuttal, p. 3.
Matthew N. Gaertner, “Advancing College Access with Class-Based Affirmative Action,” in The Future of Affirmative
Action, ed. Richard Kahlenberg (Century Foundation Press, 2014), pp. 175–186, at pp. 184–186.
167
Kahlenberg Rebuttal, p. 4; Kahlenberg Report, p. 13.
168
Card Report, p. 102; Sigal Alon, Race, Class, and Affirmative Action (New York, NY: Russell Sage Foundation,
2015), pp. 254–256.
169
Kahlenberg Rebuttal, pp. 2–3.
166

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 88

preparedness or extracurricular excellence. The race-neutral alternatives I evaluate are inefficient not
because they are “inconvenient,” but because they reduce Harvard’s ability to select applicants along
other dimensions I understand it may value. For example, putting four times the weight on
socioeconomic characteristics is a dramatic shift in policy that effectively reduces the relative weight
that Harvard places on other characteristics it values, like extracurricular, athletic, and academic
achievement. Similarly, a geographic quota or percent plan would severely constrain the extent to
which Harvard can admit well-qualified candidates from exceptionally competitive areas.
179. Nevertheless, it is important to note that in addition to imposing efficiency costs (as this
term is used in the literature, and in my report), race-neutral alternatives may also impose
administrative and financial costs. Utilizing ZIP code, high school, or other geographic quotas, for
example, could generate a massive increase in applications from less competitive areas or high
schools, increasing the costs of Harvard’s admissions process. Further increases in financial aid and
recruiting would also be costly. Mr. Kahlenberg dismisses these costs, but they may be of legitimate
concern to a university, including Harvard.170
180. Mr. Kahlenberg also argues that the finding in the literature regarding the difficulty of
employing race-neutral alternatives at selective institutions is irrelevant because the schools studied
“could have done more to promote diversity.”171 In other words, Mr. Kahlenberg appears to be
arguing that analyses of prior attempts by selective universities to use race-neutral alternatives are
inherently flawed, because they cannot rule out that those universities could have implemented more,
or different, policies. Speculating that other selective universities “could have done more” shows a
fundamental misunderstanding of the concept of empirical evidence. The existing literature on raceneutral alternatives provides such evidence by examining actual attempts by universities to
implement race-neutral policies. Mr. Kahlenberg offers no factual support for the efficacy of the raceneutral policies he claims universities could have employed. Speculation is not the same as evidence.
181. Additionally, Mr. Kahlenberg suggests that one explanation for why race-neutral
alternatives have been less effective in generating diversity at selective universities like U.C.
Berkeley, UCLA, and Michigan is that these schools faced a “special disadvantage in recruiting
minority students because they were prohibited by state law from using racial preferences, but their
competitors were not.”172 Mr. Kahlenberg fails to note that Harvard would face the exact same
“special disadvantage” if prohibited from using race as a factor in admissions. This fact renders the
experience of these schools all the more relevant to assessing the potential effectiveness of race-

170

Kahlenberg Rebuttal, pp. 24–25.
Kahlenberg Rebuttal, p. 5.
172
Kahlenberg Rebuttal, p. 5.
171

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 89

neutral alternatives at Harvard.
7.2. Mr. Kahlenberg’s new simulations confirm that the substitution of race-neutral alternatives for
Harvard’s race-conscious admissions process would change the characteristics of the class and
compromise its quality
182. Mr. Kahlenberg’s rebuttal simulates Harvard’s admitted class under two new raceneutral alternatives that he claims address several shortcomings of the simulations I modeled in my
first report. In this section, I first summarize the criticisms that Mr. Kahlenberg offers of my
simulations, and then discuss the findings of his new simulations.
183. As I show below, even if I accept Mr. Kahlenberg’s new simulations, they support the
main findings of my first report. Specifically, both of his new simulations show that race-neutral
alternatives substantially alter the characteristics of the admitted class and diminish its quality, as
measured by Harvard profile ratings and other indicia. Moreover, although Mr. Kahlenberg’s new
simulations increase the fraction of Asian-American and Hispanic admitted students, they still result
in a pool of admitted African-American students that is substantially smaller than the current pool.
This pattern is not surprising, and is fully consistent with the conclusions from the broader academic
literature that race-neutral alternatives cannot achieve diversity at selective institutions without a
meaningful cost to quality.
7.2.1. Mr. Kahlenberg’s criticisms of my simulations
184. Mr. Kahlenberg criticizes my simulations in five primary ways. First, Mr. Kahlenberg
criticizes the way in which I boost the probability of admission for low-SES students in my
simulations, and offers his own variation on my methodology.173 In my simulations, I simulate giving
a “low-SES boost” to applicants who exhibit the following characteristics: disadvantaged, requested a
fee waiver, first generation college student, neighborhood median income less than or equal to
$65,000. An applicant who meets all four criteria receives the full low-SES boost, while an applicant
who meets only two criteria receives a boost equal to one-half the full boost. I start by setting the
value of the full boost at two additional points to an applicant’s admissions index, and then scale this
boost up across my various simulations.174 Mr. Kahlenberg suggests that on the one hand, the set of
socioeconomic criteria I target in my simulations is too limited. On the other hand, he argues that
labeling applicants with median neighborhood below $65,000 as low-SES is too generous—he thinks
the threshold ought to be lower.

173
174

Kahlenberg Rebuttal, pp. 31–32.
The admissions index is the input into the logit function that determines an applicant’s probability of admission.
HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 90

185. To address these alleged deficiencies, he proposes a new weighting scheme. He starts by
constructing indices that measure neighborhood and high school SES. The indices give equal weight
to three factors: parental income, parental education, and percentage of families speaking a language
other than English at home.175 Then, just as in my simulations, Mr. Kahlenberg gives applicants with
certain socioeconomic characteristics a low-SES boost by adding a value to their admissions index.
The value of the boost is equal to 1.6 multiplied by the number of low-SES criteria an applicant
meets, where the criteria are: disadvantaged, requested a fee waiver, first generation college student,
applicant obtains a neighborhood SES index score in the bottom third of the distribution, and
applicant obtains a high school SES index score in the bottom third of the distribution.176
186. Second, Mr. Kahlenberg argues that it is important to simulate the effect of eliminating
Early Action by turning off the preference Harvard accords such applicants because he argues that
Early Action “disproportionately benefits white and wealthy students.”177 He criticizes me for not
doing so in my simulations.178 Mr. Kahlenberg’s proposal is problematic, however. The large,
positive effect of applying Early Action on admission is a composite of many unobservable factors
that distinguish those who apply early from those who do not.
187. Early Action applicants may be better qualified in unobservable dimensions. In addition,
as Prof. Arcidiacono explains, “[g]iving preferences for early action is consistent with the yield rate
being higher for early action applicants.”179 The higher yield rate for Early Applicants may reflect
many applicant characteristics, including their stronger interest in Harvard, better fit with a particular
department or area of study, or greater affinity for Harvard’s extracurricular offerings. Thus, Mr.
Kahlenberg’s position that preferences for Early Action applicants primarily reflect a reward for
being “white and wealthy” is at odds with that of Prof. Arcidiacono, who designed Mr. Kahlenberg’s
simulations, and who acknowledges that the preference for Early Action is consistent with traits
Harvard may value. Furthermore, my initial report demonstrated that when Harvard restored Early
Action for the class of 2016 after having eliminated it for several years, this change was not
associated with a decline in the fraction of African-American, Hispanic, and Other (non-Asian)
minority race (“AHO”) applicants, admitted students, or matriculants.180 If anything, matriculation
175

Kahlenberg Rebuttal, pp. 30–31.
Kahlenberg Rebuttal, pp. 30–31; see also SFFA-HARVARD 0002346_simulation6.do and SFFA-HARVARD
0002347_simulation7.do in Mr. Kahlenberg’s backup.
177
Kahlenberg Rebuttal, p. 18.
178
Kahlenberg Rebuttal, pp. 18–19.
179
Arcidiacono Rebuttal, p. 3, footnote 1.
180
Card Report, pp. 147–150.
176

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 91

rates were higher for students of all races under the Early Action regime, particularly for AHO
admitted students.181 This undermines the idea that eliminating Early Action would be a strong lever
for generating diversity.
188. Third, Mr. Kahlenberg repeatedly criticizes me for removing preferences for recruited
athletes in my simulations.182 That is surprising because four of his five simulations do the same.
Indeed, SFFA did not propose this policy in its Complaint—it was introduced by Mr. Kahlenberg
himself in his initial report.183 I removed preferences for athletes in my featured simulations in order
to be conservative, employing more of his proposed race-neutral tools, rather than fewer. While his
preferred simulation (Simulation 4) excludes this policy, Mr. Kahlenberg’s only reason for restoring
the athletic preference is that “removing athletic preferences in connection with race neutral
alternatives is sometimes perceived as radical.”184 It is unclear why Mr. Kahlenberg would introduce
and feature a policy, only to disparage it so severely—but regardless, my initial findings are robust to
restoring the preference for recruited athletes (see the discussion of Exhibit 26 below).
189. Fourth, Mr. Kahlenberg makes factually incorrect statements about my simulations. He
states, for example, that my initial report “does not simulate the racial impact of eliminating …
preferences for the children of alumni, donors, faculty and staff, and those admitted through the Zlist.”185 But the only practice that Mr. Kahlenberg identifies in that statement that I do not simulate is
removing consideration of whether an applicant’s parents could donate to Harvard. The reason I do
not, as I explained, is because I do not have data that identifies these applicants—and neither does
Mr. Kahlenberg, so he cannot simulate this effect, either.
190. Fifth, Mr. Kahlenberg makes the broad critique of my initial analysis that, because any
given race-neutral alternative he proposed in his initial report is unlikely to be effective on its own,

181

Card Report, pp. 149–150.
Kahlenberg Rebuttal, pp. 11–12, 32.
183
Kahlenberg Report, pp. 45–46.
184
Kahlenberg Report, p. 46.
185
Kahlenberg Rebuttal, p. 11. Mr. Kahlenberg also cites a paper published in his book contending that legacy
preferences are not associated with higher alumni giving (Kahlenberg Rebuttal, p. 12; Kahlenberg Report, p. 33). This
paper uses aggregated data at the college level to examine the determinants of mean alumni giving, rather than the
preferred approach of examining the donation decisions of individual alumni with different potential incentives to give.
Moreover, it relies on limited proxies for alumni characteristics, such as mean Pell grants per currently enrolled student as
a measure of alumni wealth. In my opinion the empirical analysis in this paper is very weak and is uninformative about
the reasons for alumni giving.
182

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 92

proper analysis must consider all of them in conjunction.186 On this point, I agree with Mr.
Kahlenberg, which is why I conducted just such a combined analysis in my initial report. It is
important to evaluate the potential effects of multiple race-neutral alternatives used in conjunction. In
fact, I explained this challenge in my first report, and included results for a simulation that combined
the following race-neutral alternatives:
• Eliminated preferences for lineage applicants, recruited athletes,
children of Harvard faculty and staff, and applicants on the Dean and
Director’s interest lists
• Increased the preference given to low-SES applicants
• Admitted students equally across College Board clusters (the placebased policy simulated in Mr. Kahlenberg’s initial report)
• Doubled the number of disadvantaged applicants Harvard was able to
attract (assuming no change to the quality of disadvantaged applicants)
191. In that analysis, I found that, even taken together, these policies were unlikely to
generate both diversity and class quality.187 Tellingly, Mr. Kahlenberg does not comment on the
above simulation, even though it directly addresses his concern.
7.2.2. The results of Mr. Kahlenberg’s new simulations support the conclusions of my first report
192. Mr. Kahlenberg puts forward two additional simulations in his rebuttal. Mr.
Kahlenberg’s Simulation 6 retains the same sample and regression model as my own. As in my
simulations, he eliminates consideration of race, lineage status, whether an applicant’s parents are
Harvard faculty and staff, whether the applicant appears on the Dean’s or Director’s interest list, and
the proportion of the applicant’s high school and neighborhood that is African-American, Hispanic,
and White. As noted above, he makes three main changes to my simulations. He does not remove
186

Kahlenberg Rebuttal, p. 1 (“These strategies, when used in tandem with one another, can produce the educational
benefits of diversity[.]”), p. 10 (“[M]y opening report never suggested that increasing financial aid is a stand-alone
strategy that would automatically increase racial diversity.”), p. 12 (“[T]he elimination of preferences that tend to favor
wealthy and white students was not meant to be a stand-alone race-neutral alternative[.]”), p. 18 (“[M]y contention is not
that community college transfers alone is the answer; it is that increasing the number of community college students at
Harvard is one piece of a larger solution[.]”).
187
Card Report, pp. 137–138, Exhibit 53.

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 93

consideration of recruited-athlete status, he eliminates the preference associated with applying Early
Action, and he gives applicants with certain socioeconomic characteristics a low-SES boost by
adding a value to their admissions index that reflects a slightly different set of low-SES criteria than
my model, as described above.188 Mr. Kahlenberg’s Simulation 7 is the same as Simulation 6, except
that it restores consideration of Early Action status.
193. Exhibit 26 presents the results of Mr. Kahlenberg’s Simulation 6. Starting first with its
effect on diversity, we see that Simulation 6 generates a substantial increase in the fraction of
Hispanic or Other applicants in the admitted class. The fraction of admitted students who are AfricanAmerican, however, remains 30% lower than in the actual class. His Simulation 7 performs similarly.
194. In his initial report, Mr. Kahlenberg reported the average profile ratings associated with
each of his simulated classes.189 Notably, he excludes those outcomes when he reports results for
Simulations 6 and 7.190 Fortunately, Prof. Arcidiacono’s code for Mr. Kahlenberg’s simulations
computes and records not only information on ratings, but also all of the metrics I used in my own
report to evaluate how different polices impact class characteristics. Mr. Kahlenberg chooses not to
report the findings from Prof. Arcidiacono’s output, but I report them in full in Exhibit 26.
195. When discussing his new simulations, Mr. Kahlenberg uses my “4x low-SES boost”
simulation as a benchmark. I follow suit in this section. It is worth noting that in his rebuttal, Mr.
Kahlenberg argues that this simulation represents a “viable” race-neutral alternative for Harvard,
despite the fact that the simulation results in a decline in the fraction of students with top profile
ratings, and a fraction of African-American admitted students that is about 30 percent lower than that
of the current class (among other changes to class characteristics).191 His arguments do not change
my conclusion that this race-neutral alternative produces a class that is different from the current
class in dimensions I understand Harvard cares about; but I use the “4x low-SES boost” simulation as
a benchmark to be consistent with Mr. Kahlenberg.
196. First, although Mr. Kahlenberg’s new simulations generate a larger fraction of AfricanAmerican and Hispanic admitted students, as compared to my 4x low-SES simulation, his
simulations (like mine) produce a class that has a full 30% fewer African-American students than the
actual admitted class—a dramatic drop. Mr. Kahlenberg’s simulations also result in a class with
slightly lower average SAT and ACT scores, as compared to my simulation. Because he retains a
188

Kahlenberg Rebuttal, pp. 29–32.
Kahlenberg Report, Appendix C.
190
Kahlenberg Rebuttal, pp. 33, Appendix A.
191
Kahlenberg Rebuttal, p. 21 (“Instead, in this section, I show that (1) Card incorrectly concludes that Arcidiacono
Simulation 4 and Card’s 4x are not viable race neutral alternatives; and (2) a new simulation from Card’s model
(Simulation 6) demonstrates viable race-neutral alternatives.”).
189

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 94

preference for athletic recruits, his simulated classes do have better athletic ratings than in my 4x
simulation. Importantly, Mr. Kahlenberg’s new simulations reduce the fraction of the admitted class
with academic, personal, and extracurricular ratings of 1 or 2. These reductions are larger than in my
benchmark simulation.
197. In sum, Mr. Kahlenberg’s new simulations are no better at increasing the fraction of
African-American students in the class (relative to a baseline in which Harvard does not consider
race) than my simulations, and come at a higher cost to other factors that Harvard values in the
admissions process, including academic excellence. In other words, Mr. Kahlenberg’s new
simulations simply reinforce the point made repeatedly in my initial report and in the academic
literature: it is extremely difficult to generate diversity using race-neutral alternatives without
inflicting costs in other dimensions a university may value.

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 95

Kahlenberg’s Simulation 6 and 7: Impact on class characteristics
Predicted Class Without Consideration of Race and Factors that
Allegedly Advantage White Applicants
Card’s Simulation
(4x SES Boost)

Kahlenberg’s
Simulation 6

Kahlenberg’s
Simulation 7

Actual
Admitted
Class

Predicted
Value

% Change

Predicted
Value

% Change

Predicted
Value

% Change

Outcome Measures

[A]

[B]

([B]-[A])/[A]

[C]

([C]-[A])/[A]

[D]

([D]-[A])/[A]

1.
2.
3.
4.
5.

Race
White
Asian-American
Hispanic or Other
African-American
Race Missing

676
402
233
234
134

589
508
293
163
127

-13%
+26%
+26%
-30%
-6%

541
523
330
164
121

-20%
+30%
+42%
-30%
-10%

561
521
313
160
123

-17%
+30%
+34%
-32%
-8%

6.
7.
8.
9.

Academic
Average Composite SAT Score
Average Composite ACT Score
Average Converted GPA
Average Academic Index

2244
33.1
77.0
228

2189
32.7
77.1
225

-2%
-1%
+0.1%
-1%

2173
32.5
77.0
225

-3%
-2%
+0.02%
-1%

2180
32.5
77.0
225

-3%
-2%
+0.02%
-1%

Fraction with Profile Rating of 1 or 2
Academic
76%
Extracurricular
62%
Personal
71%
Athletic
27%

66%
57%
64%
18%

-13%
-9%
-11%
-33%

61%
54%
62%
20%

-19%
-13%
-13%
-26%

63%
55%
63%
21%

-17%
-12%
-11%
-22%

259

86

-67%

61

-76%

81

-69%

72

19

-73%

13

-81%

18

-75%

180

88

-51%

144

-20%

159

-11%

44

17

-61%

12

-74%

16

-64%

839

851

+1%

858

+2%

851

+1%

10.
11.
12.
13.

Applicant Characteristics
14. Number of Lineage Students
Number of Double Lineage
15.
Students
16. Number of Recruited Athletes
Number of Children of Harvard
17.
Faculty and Staff
18.

Number of Students on Dean’s
and Director’s Interest Lists

19. Number of Female Students

Redacted

20.
21.
22.
23.
24.
25.
26.
27.

Concentration
Social Sciences
Humanities
Biological Sciences
Physical Science
Engineering
Computer Science
Mathematics
Unspecified

25%
15%
21%
7%
13%
6%
6%
7%

24%
13%
23%
8%
13%
6%
7%
6%

-5%
-9%
+11%
+6%
+5%
-7%
+3%
-9%

24%
12%
24%
7%
14%
6%
6%
6%

-4%
-15%
+12%
-5%
+14%
-4%
+1%
-6%

24%
12%
24%
7%
14%
6%
6%
7%

-2%
-14%
+12%
-5%
+8%
-6%
+0.5%
-3%

28.
29.
30.
31.
32.

Geography
Number Rural
Number in Northeast
Number in Midwest
Number in South
Number in West

59
694
207
379
399

87
604
217
407
451

+48%
-13%
+5%
+7%
+13%

87
615
164
392
509

+47%
-11%
-21%
+3%
+27%

82
630
170
391
488

+39%
-9%
-18%
+3%
+22%

Source: Augmented Arcidiacono Data; College Board Cluster Data; U.S. Census Data; Kahlenberg Production

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 96

Note: My simulation (“Card’s Simulation (4x SES Boost)”) consists of applicants to the class of 2019 in Prof. Arcidiacono’s expanded
sample including athletes, who are in my preferred year-by-year regression model from my affirmative report. The simulation eliminates
consideration of race, lineage status, recruited-athlete status, whether an applicant’s parents are Harvard faculty and staff, whether the
applicant appears on the Dean’s or Director’s interest list, and the proportion of the applicant’s high school and neighborhood that is
African-American, Hispanic, and White. In addition, recruited athletes are reassigned to rating combinations in the regression sample that
contain the next highest athletic rating. Applicants with certain socioeconomic characteristics are given a low-SES boost by adding a value
to their admissions index. The value is equal to 2 multiplied by the number of characteristics an applicant displays out of the following:
disadvantaged, requested a fee waiver, first generation college student, neighborhood median income less than or equal to $65,000.
Kahlenberg’s simulation 6 retains the same sample and regression model from my simulation. Simulation 6 eliminates consideration of
the same characteristics as my simulation except for recruited-athlete status. Simulation 6 also eliminates consideration of Early Action
status. Applicants with certain socioeconomic characteristics are given a low-SES boost by adding a value to their admissions index. The
value is equal to 1.6 multiplied by the number of characteristics an applicant displays out of the following: disadvantaged, requested a fee
waiver, first generation college student, applicant obtains a neighborhood SES index score in the bottom third of the distribution,
applicant obtains a high school SES index score in the bottom third of the distribution. The neighborhood and high school SES indices are
constructed by equally-weighting three standardized factors: parental income, parental education, and percentage of families speaking a
language other than English at home. Kahlenberg’s simulation 7 is the same as simulation 6 except that it retains consideration of Early
Action status.

7.3. Other race-neutral alternatives are unlikely to generate diversity without changing class
characteristics and compromising class quality
198. Mr. Kahlenberg also critiques my analyses of the role that different approaches to
financial aid, recruiting, transfer admissions, and deferred admission could play in increasing
diversity at Harvard. Additionally, he lays out a vague suggestion for how Harvard could allegedly
use place-based admissions policies to increase diversity. I address each of these critiques below.
7.3.1. Increasing financial aid
199. Mr. Kahlenberg argues that Harvard could offer more generous financial aid and that
doing so could increase the diversity of its admitted class.192 I evaluated this claim in my initial report
by looking at how historical expansions in financial aid influenced the composition of applicants,
admitted students, and matriculants. I focused on Harvard’s most recent expansion: starting with the
class of 2016, Harvard expanded its threshold for attendance at zero personal cost from $60,000 to
$65,000. In my initial report, I showed that this expansion was not associated with an increase in
AHO applicants, admitted students, or matriculants.193 My report also explained why this might not
be surprising: even with a threshold of $60,000 for attendance at zero personal cost, Harvard was
already free for the vast majority of Hispanic and African-American households.194 I took this as
evidence that future expansions in financial aid were unlikely to be a powerful lever for increasing

192

Kahlenberg Rebuttal, pp. 9–10.
Card Report, pp. 142–145.
194
Card Report, p. 140.
193

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 97

racial diversity.
200. In his rebuttal, Mr. Kahlenberg complains that I should not treat this expansion as an
increase in financial aid, because it was accompanied by a reduction in aid to families making
between $150,000 and $180,000.195 This complaint is misplaced, in my view. The key policy change
in question—raising the threshold for attendance at zero personal cost—is targeted at applicants at
the lower end of the income distribution, where both Mr. Kahlenberg and I would expect to see a
response from disadvantaged applicants. Thus, I maintain that looking at the most recent expansion in
the threshold for zero-parental contribution is a helpful and informative exercise for understanding
how future increases in aid might affect the pool of applicants, admitted students, and matriculants.
201. It is also worth noting that elsewhere in his report, Mr. Kahlenberg implicitly admits that
financial aid is already exceptionally generous at Harvard for both disadvantaged and middle-class
applicants. In evaluating my simulations, he writes “[t]he problem with Card’s $65,000 threshold [for
identifying low-SES applicants] is that it includes middle-class as well as economically
disadvantaged neighborhoods. (In 2016, the median household income was $57,617.)”196 In other
words, Mr. Kahlenberg considers families making $65,000 middle class. By Mr. Kahlenberg’s
standard, Harvard already requires no financial contribution from applicants from disadvantaged
neighborhoods, as well as many middle-class ones.
7.3.2. Increasing recruiting
202. Mr. Kahlenberg dismisses my concern that doubling the pool of disadvantaged
applicants to Harvard may be challenging, and that doing so would likely have an impact on the
average quality of disadvantaged applicants.197 He suggests that there is a large pool of qualified
candidates who do not apply to Harvard, noting that “82% of American high schools have not a
single applicant to Harvard, one of the world’s best known colleges.”198 As the statement itself
implies, this is probably not because these students are unaware of Harvard. It is more likely that
these students are either academically unprepared for Harvard, have personal reasons for not applying
(e.g., because they prefer to attend college in a different part of the country), lack information on the
application process or the availability of financial aid, or are concerned about their own fit with

195

Kahlenberg Rebuttal, p. 10.
Kahlenberg Rebuttal, pp. 31–32.
197
Kahlenberg Rebuttal, pp. 15–17.
198
Kahlenberg Rebuttal, p. 16.
196

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 98

Harvard in terms of academic and extracurricular interests. My reading of the available materials in
this case is that Harvard is aware of the potential information gaps that face some students and that
Harvard actively tries to provide this information through extensive recruiting visits, social media
campaigns, targeted mailings, and engagement with local public schools.199 As I detailed in my
previous report, Harvard also purchases extensive search lists from testing agencies, and conducts a
variety of forms of direct outreach by staff and students involved in the wide array of recruiting
programs I described in my initial report.200
203. Second, Mr. Kahlenberg cites the work of economists Caroline Hoxby and Christopher
Avery to support his point that there is a pool of talented, low-income applicants who do not apply to
selective colleges, despite being qualified for admission.201 While I agree with those authors that
there are likely talented students in disadvantaged circumstances who currently do not apply to
selective schools like Harvard, as I explained in my report, those same authors indicate that many of
these students are “isolated from other high achievers, both in terms of geography and in terms of the
high schools they attend,” rendering them particularly difficult to reach.202 As detailed at length in my
initial report, Harvard already engages in extensive recruiting, and Harvard already employs the very
interventions suggested by Hoxby and Avery in their paper.203
204. Mr. Kahlenberg also cites an academic study suggesting that there are about 5,160
Hispanic students and 2,580 African-American Pell Grant recipients “who have test scores
comparable to those of students at selective colleges but who do not now attend such institutions.”204
He takes this as evidence that Harvard could easily double its pool of disadvantaged applicants
without any impact on quality. If anything, I think these figures underscore how challenging
expanding the pool of qualified disadvantaged candidates can be. Between 2014 and 2019, Harvard
flagged several thousand students each year as being disadvantaged (over 4,700 in 2019).205 By
comparison, the authors of this study identify fewer than 8,000 potential applicants, an unknown
number of whom may have already applied to Harvard and been rejected. Further, test scores are the

199

Card Report, pp. 120–122.
Card Report, pp. 120–122. These programs include the Undergraduate Minority Recruitment Program, the Harvard
First Generation Program, the Harvard College Connection, Project Teach, and the Cambridge-Harvard Summer
Academy.
201
Kahlenberg Rebuttal, p. 16.
202
Card Report, p. 122, footnote 198.
203
Card Report, pp. 120–122.
204
Kahlenberg Rebuttal, pp. 16–17.
205
See workpaper.
200

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 99

only metric by which the authors measure quality, which is not particularly helpful for understanding
how well these potential applicants would stack up against current candidates for admission to
Harvard.
7.3.3. Increasing transfer admissions
205. Mr. Kahlenberg also argues that Harvard could increase diversity by increasing the
number of transfer applicants Harvard admits from community colleges.206 In my first report, I
showed that current transfer applicants are no more diverse than Harvard’s regular applicant pool,
and pointed out that admitting a large number of transfer students would require restricting the size of
the regularly-admitted freshman class, as so few students drop out of Harvard.207 In his rebuttal, Mr.
Kahlenberg argues that it is irrelevant to analyze the composition of current transfer students, since
Harvard only accepted two such students to the classes of 2014 to 2019.208 Mr. Kahlenberg seems to
have missed the point: I did not analyze the racial composition of current transfer students. I analyzed
the composition of the pool of transfer applicants. To reiterate: the pool of students who apply to
transfer to Harvard is no more diverse than the pool of applicants who apply to the freshman class. As
a result, I would not expect an increase in the number of transfer students Harvard admitted to have
an impact on racial diversity.
7.3.4. Eliminating deferred admission
206. In his rebuttal, Mr. Kahlenberg states that my simulations do not address his suggestion
that Harvard could increase diversity by ending the practice of deferred admission.209 This is factually
inaccurate. As stated in my initial report, I simulate the effect of eliminating deferred admission (the
“Z-list”) by using a model that fills all seats in the entering class with students who apply in a given
year, based on their characteristics.210 All of my simulations eliminate the practice of deferred
admission.

206

Kahlenberg Rebuttal, pp. 17–18.
Card Report, p. 119.
208
Kahlenberg Rebuttal, p. 17.
209
Kahlenberg Rebuttal, p. 11.
210
Card Report, p. 104.
207

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 100

7.3.5. Place-based admissions policies
207. Mr. Kahlenberg’s initial report advocated for placed-based policies, such as admitting
top students by high school (e.g., the Texas Top 10 Percent Plan) or by ZIP code.211 He particularly
emphasizes the latter. Now, acknowledging my point that it is not possible for Harvard to admit top
students from every U.S. high school or ZIP code, Mr. Kahlenberg pivots from that style of placebased policy in his rebuttal. Instead, he suggests that Harvard “could easily seek excellence and
socio-geographic diversity by enrolling top students from all of the College Board’s 33 ‘Educational
Neighborhood Clusters,’ as we model, or some variation of Harvard’s choosing.”212 In a footnote, he
then suggests that Harvard could admit top students across buckets of ZIP codes in lieu of College
Board clusters.
208. First, Harvard already “seek[s] excellence and socio-geographic diversity,” and Harvard
already “enroll[s] top students from all of College Board’s 33 ‘Educational Clusters.’”213 Aside from
Mr. Kahlenberg’s suggestion that Harvard consider only race-neutral criteria in its holistic admissions
process, I fail to see how this vague proposal differs from Harvard’s current practices.
209. Further, as I showed in my initial report, the policy Mr. Kahlenberg initially proposed
(which admitted an equal number of applicants across clusters) results in a class with lower personal,
academic, extracurricular, and athletic ratings, as compared to the status quo.214 One reason for this
change is that imposing geographic quotas would necessarily limit Harvard’s ability to accept
additional students from the most competitive geographic slates. Simply substituting buckets of ZIP
codes for College Board clusters—which are themselves collections of census tracts—seems no more
promising than using clusters themselves.
7.4. Conclusion
210. In this section, I addressed Mr. Kahlenberg’s criticisms of my analysis of race-neutral
alternatives, and demonstrated the robustness of my findings. I maintain the position put forth in my
first report: even considered in combination, the race-neutral alternatives put forward by Mr.
Kahlenberg are blunt and ineffective instruments for generating a diverse class, as they limit

211

Kahlenberg Report, pp. 36–39.
Kahlenberg Rebuttal, pp 14–15.
213
See workpaper.
214
Card Report, pp. 151–153.
212

HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 101

Harvard’s ability to select applicants based on other characteristics that it values. The policies I
analyzed either did little to generate a class comparable in diversity to the current class, or did so only
by significantly changing class characteristics and compromising class quality. Whether these
policies generate a class that meets Harvard’s educational needs is beyond the scope of my
opinion,215 but the empirical evidence is clear: using the race-neutral alternatives proposed by Mr.
Kahlenberg to generate diversity comes at a cost.

___________________________
David Card
March 15, 2018

215

I understand that a committee led by Dean Michael Smith will address the question of whether or not these raceneutral policies would generate an admitted class that meets Harvard’s educational needs.
HIGHLY CONFIDENTIAL: ATTORNEYS’ EYES ONLY

Page 102

8. APPENDIX A
8.1. Documents Relied Upon
Expert Reports
Expert Report of David Card, Ph.D., Students for Fair Admissions, Inc. v. President and Fellows of
Harvard College (Harvard Corporation), December 15, 2017.
Expert Report of Peter S. Arcidiacono and backup materials, Students for Fair Admissions, Inc. v.
President and Fellows of Harvard College (Harvard Corporation), October 16, 2017.
Rebuttal Expert Report of Peter S. Arcidiacono and backup materials, Students for Fair
Admissions, Inc. v. President and Fellows of Harvard College (Harvard Corporation), January 30,
2018.
Expert Report of Richard D. Kahlenberg and backup materials, Students for Fair Admissions, Inc.
v. President and Fellows of Harvard College (Harvard Corporation), October 16, 2017.
Rebuttal Expert Report of Richard D. Kahlenberg and backup materials, Students for Fair
Admissions, Inc. v. President and Fellows of Harvard College (Harvard Corporation), January 30,
2018.
Harvard Admissions Data
HARV00001203 – 53; HARV00001373 – 80; HARV00006413 – 818; NEVO Admissions Data for
the classes of 2014 – 2019.
HARV00001224; HARV00001322; Lists of database fields produced.
HARV00001895; HARV00001985; HARV00002725 – 29; HARV00003489; Documents
pertaining to calculation of Academic Index.
Depositions
Deposition of Brock Walsh, June 28, 2017.
Deposition of Chris Looby, June 30, 2017.
Deposition of Christina Lopez, May 22, 2017.
Deposition of Erica Bever, July 13, 2017.
Deposition of Mark Hansen, July 19, 2017.
Deposition of Marlyn Elizabeth McGrath, Volume I, June 18, 2015.
Deposition of Roger Banks, May 4, 2017.
Deposition of Tia Ray, June 7, 2017.
Deposition of William Fitzsimmons, August 3, 2017.
Academic Articles
Alan S. Blinder and Alan B. Krueger, “Alternative Measures of Offshorability: A Survey
Approach,” Journal of Labor Economics 31(2), 2013, pp. S97–S128.
Amanda Griffith and Donna Rothstein, “Can’t Get There from Here: The Decision to Apply to a
Selective College,” Economics of Education Review 28(5), 2009, pp. 620–628.

CONFIDENTIAL

Page 103

Amy J. Binder, Daniel B. Davis, and Nick Bloom, “Career Funneling: How Elite Students Learn to
Define and Desire “Prestigious” Jobs,” Sociology of Education 89(1), 2016, pp. 20–39.
Christian Gourieroux et al., “Generalized Residuals,” Journal of Econometrics 34, 1987, pp. 5–32.
David Card and Alan Krueger, “Would the Elimination of Affirmative Action Affect Highly
Qualified Minority Applicants? Evidence from California and Texas,” Industrial and Labor
Relations Review 58(3), 2005, pp. 416–434.
David H. Autor and Michael J. Handel, “Putting Tasks to the Test: Human Capital, Job Tasks, and
Wages,” Journal of Labor Economics 31(2), 2013, pp. S59–S96.
David Zimmerman, “Regression Toward Mediocrity in Economic Stature,” The American
Economic Review 82(3), 1992, pp. 409–429.
Emily Oster, “Unobservable Selection and Coefficient Stability,” Brown University and NBER
Working Paper #19054, August 9, 2016.
Garret Christensen and Edward Miguel, “Transparency, Reproducibility, and the Credibility of
Economics Research,” NBER Working Paper #22989, December 2016.
James J. Heckman, “Sample Selection Bias as a Specification Error,” Econometrica 47(1), 1979,
pp. 153–161.
Peter Arcidiacono et al., “Recovering Ex Ante Returns and Preferences for Occupations using
Subjective Expectations Data,” NBER Working Paper #20626, October 2014.
Peter Arcidiacono, Jane Cooley, and Andrew Hussey, “The Economic Returns to an MBA,”
International Economic Review 49(3), 2008, pp. 873–899.
Sandra Black, Kalena Cortes, and Jane Lincove, “Academic Undermatching of High-Achieving
Minority Students: Evidence from Race-Neutral and Holistic Admissions Policies,” American
Economic Review: Papers & Proceedings 105(5), 2015, pp. 604–610.
Sandra Black, Kalena Cortes, and Jane Lincove, “Apply Yourself: Racial and Ethnic Differences in
College Application,” NBER Working Paper #21368, 2015.
Sean Reardon, Rachel Baker, and Daniel Klasik, “Race, Income, and Enrollment Patterns in Highly
Selective Colleges, 1982-2004,” Center for Education Policy Analysis, Stanford University, 2012,
pp. 1–27.
Sharmila Choudhury, “Reassessing the Male-Female Wage Differential: A Fixed Effects
Approach,” Southern Economic Journal 60(2), 1993, pp. 327–340.
Thomas J. Kane, “Racial and Ethnic Preferences in College Admissions,” Ohio St. Law Journal 59,
1998, pp. 971–996.

Books and Book Chapters
Anthony P. Carnevale, Stephen J. Rose, and Jeff Strohl, “Achieving Racial and Economic
Diversity with Race-Blind Admissions Policy,” in The Future of Affirmative Action, ed. Richard
Kahlenberg (Century Foundation Press, 2014), pp. 187–202.
Chad Coffman, Tara O’Neil, and Brian Starr, “An Empirical Analysis of Legacies Preferences on
Alumni Giving at Top Universities,” in Affirmative Action for the Rich, ed. Richard D. Kahlenberg
(New York: Century Press, 2010), pp. 101–121.

CONFIDENTIAL

Page 104

Daron Acemoglu and David Autor, “Skills, Tasks and Technologies: Implications for Employment
and Earnings,” in Handbook of Labor Economics, Volume 4A, ed. Orley Ashenfelter and David
Card (San Diego, CA: North-Holland, 2011), pp. 1043–117.
Greg J. Duncan and Katherine A. Magnuson, “Off With Hollingshead: Socioeconomic Resources,
Parenting, and Child Development,” in Socioeconomic Status, Parenting, and Child Development,
ed. Marc H. Bornstein and Robert H. Bradley (Lawrence Erlbaum Associates, Inc., 2003), pp. 83–
106.
James H. Stock and Mark W. Watson, Introduction to Econometrics (Pearson, 2015).
Matthew N. Gaertner, “Advancing College Access with Class-Based Affirmative Action,” in The
Future of Affirmative Action, ed. Richard Kahlenberg (Century Foundation Press, 2014), pp. 175–
186.
Otis Dudley Duncan, “A Socioeconomic Index for All Occupations,” in Occupations and Social
Status, ed. Albert J. Reiss, Jr. (Free Press, 1961), pp. 109–138.
Sigal Alon, Race, Class, and Affirmative Action (New York, NY: Russell Sage Foundation, 2015).
Public Press/Websites
Harvard College, “Frequently Asked Questions,” available at
https://college.harvard.edu/frequently-asked-questions, accessed February 2, 2018.
IPUMS USA, “ACS Occupation Codes (OCC),” available at
https://usa.ipums.org/usa/volii/c2ssoccup.shtml, accessed February 19, 2018.
U.S. Census Bureau, “Industry and Occupation Code Lists & Crosswalks,” available at
https://www.census.gov/topics/employment/industry-occupation/guidance/code-lists.html,
accessed March 8, 2018.
U.S. Bureau of Labor Statistics, “Historical comparability of occupation and industry data from the
Current Population Survey,” available at https://www.bls.gov/cps/cpsoccind.htm, accessed March
8, 2018.
Produced documents
HARV00001392 – 1438, “Interviewer Handbook, 2014-2015”
HARV00001851 – 56, “Applicants, Admits, and Matriculants – Old Methodology NLNA”
HARV00004683 – 89, COFHE Admissions Statistics, Class Entering 2013
HARV00005106, “Ethnicity Backgrounds – Classes of 2014 – 2017”
HARV00013561 – 65, Sarasota Presentation, “KLW - Sarasota Presentation”
HARV00018164 – 76, “Discussion Guide to the 2012 Casebook”
HARV00022645, Email from Katey Stone to Grace Cheng and Nathan Fry, “FW: Harvard
Women’s Ice Hockey,” November 30, 2012
HARV00023588, Email from Jeff A. Neal to William Fitzsimmons, February 6, 2013
HARV00023592 – 4, "An Addendum on the collection and reporting of data on race and ethnicity"
HARV00030509 – 12, “Addendum on the collection and reporting of data on race and ethnicity”
HARV00056250 – 57311, Docket sheets, “P, R & S Dockets- Official #1 Class of 2018”
HARV00076219 – 20, Summary sheet
HARV00079325 – 420, Application file
HARV00079421 – 75, Application file
CONFIDENTIAL

Page 105

HARV00079476 – 518, Application file
HARV00079519 – 63, Application file
HARV00079812 – 52, Application file
Other Data and Data Documentation
College Board, “2010 College-Bound Seniors,” Total Group Profile Report, p. 4, available at
https://research.collegeboard.org/programs/sat/data/archived/cb-seniors-2010, accessed December
14, 2017.
College Board, “2011 College-Bound Seniors,” Total Group Profile Report, p. 4, available at
https://research.collegeboard.org/programs/sat/data/archived/cb-seniors-2011, accessed December
14, 2017.
College Board, “2012 College-Bound Seniors,” Total Group Profile Report, September 24, 2012,
p. 4, available at https://research.collegeboard.org/programs/sat/data/archived/cb-seniors-2012,
accessed December 14, 2017.
College Board, “2013 College-Bound Seniors,” Total Group Profile Report, September 2013, p. 4,
available at https://research.collegeboard.org/programs/sat/data/archived/cb-seniors-2013, accessed
December 14, 2017.
College Board, “2014 College-Bound Seniors,” Total Group Profile Report, October 7, 2014, p.4,
available at https://research.collegeboard.org/programs/sat/data/archived/cb-seniors-2014, accessed
December 14, 2017.
College Board, “2015 College-Bound Seniors,” Total Group Profile Report, September 30, 2015,
p. 4, available https://research.collegeboard.org/programs/sat/data/archived/cb-seniors-2015,
accessed December 14, 2017.
College Board, “Descriptor PLUS Cluster Description Guide,” available at
http://media.collegeboard.com/digitalServices/pdf/miscellaneous/ClusterDescriptionGuide.pdf,
accessed December 14, 2017.
College Board, “Segment Analysis Service Tagging/Historical Analysis Output File,” available at
http://media.collegeboard.com/mSSS/media/pdf/segment-analysis-tagging-output-file-sep2013.pdf,
accessed December 14, 2017.
College Board, “Segment Analysis Service: An Educationally Relevant Geodemographic Tagging
Service,” available at http://media.collegeboard.com/mSSS/media/pdf/segment-analysis-serviceoverview.pdf, accessed December 14, 2017.
College Board Segment Analysis Service, Applicant-level data (“College Board Cluster Data”).
National Center for Education Statistics, “Number of educational institutions, by level and control
of institution: Selected years, 1980-81 through 2013-14,” Digest of Education Statistics 2015,
December 2016.
United States Bureau of Labor Statistics, “2010 SOC User Guide,” February 2010, available at
https://www.bls.gov/soc/soc_2010_user_guide.pdf, accessed December 13, 2017.
United States Bureau of Labor Statistics, “2010 Standard Occupational Classification,” January
2009, available at https://www.bls.gov/soc/soc_structure_2010.pdf, accessed December 13, 2017.
United States Census Bureau, “2010 ANSI Codes for Places.”
United States Census Bureau, “2010 FIPS Codes for Counties and County Equivalent Entities.”
United States Census Bureau, “Core based statistical areas (CBSAs), metropolitan divisions, and
combined statistical areas (CSAs),” July 2015.

CONFIDENTIAL

Page 106

United States Census Bureau, Current Population Survey 2017 Annual Social and Economic
Supplement, HINC-01: Selected Characteristics of Households by Total Money Income, available
at https://www.census.gov/data/tables/time-series/demo/income-poverty/cps-hinc/hinc-01.html,
accessed December 14, 2017.
United States Census Bureau, "Census regions and Divisions in the United States," available at
https://www2.census.gov/geo/pdfs/maps-data/maps/reference/us_regdiv.pdf, accessed December
14, 2017.
United States Census Bureau, “ZIP Code Tabulation Areas,” 2016 U.S. Gazetteer Files, available
at https://www.census.gov/geo/maps-data/data/gazetteer2016.html, accessed December 14, 2017.
All other documents cited in this report and in my initial report.

CONFIDENTIAL

Page 107

9. APPENDIX B: OTHER TECHNICAL CRITIQUES OF PROF. ARCIDIACONO’S REPORT

9.1. Appendix B.1 Constructing categories for parental occupations
211. Parent occupations are stored in a field in the NEVO database. This field uses two
distinct sets of occupational codes, and the prevalence of either set of codes changes over time, most
notably between 2014 and 2015, when the second set of codes is first introduced. I harmonize the two
sets of codes by mapping them to the Bureau of Labor Statistics’ occupational categories, which are
standard occupational categories used in the labor economics literature.216 Prof. Arcidiacono also
takes issue with the way I aggregate occupational categories to create indicator variables in my
regression. His critique is vague: it is limited to a footnote that states my translation is “incorrect” and
is supported only by one example he disagrees with.
212. To harmonize the two sets of occupational codes Harvard used within each year, I create
a mapping between these codes and the Bureau of Labor Statistics’ occupational codes. This also
allows me to aggregate occupations that the BLS considers similar into “BLS major groups.” I
combine some of these groups in order to create a meaningful but parsimonious set of occupational
variables. I also split some “major groups” to create categories that more accurately reflect parents’
socioeconomic status, e.g., I split doctors from nurses, who would otherwise remain together in the
“healthcare practitioners” category. The complete mapping is shown at the end of this section in
Exhibit 28.
213. Prof. Arcidiacono critiques my mapping by pointing to one example of an applicant
whom he believes was miscoded. He writes that for this applicant: “Handwritten notes [on the
applicant’s summary sheet] show the occupations as ‘caregiver’ and ‘newspaper deliveryman’. Yet
Professor Card’s classification scheme results in this applicant being coded as ‘Skilled Trades Incl.
Construction. ’”217
214. First, it is important to note that handwritten notes on summary sheets are not available
in Harvard’s data, and that Harvard only produced summary sheets for several hundred out of more
than a hundred thousand applicants. My analysis of occupations is based on the most comprehensive
216

Daron Acemoglu and David Autor, “Skills, Tasks and Technologies: Implications for Employment and Earnings,” in
Handbook of Labor Economics, Volume 4A, ed. Orley Ashenfelter and David Card (San Diego, CA: North-Holland,
2011), pp. 1043–1171 at pp. 1048–1049, p. 1164; IPUMS USA, “ACS Occupation Codes (OCC),” available at
https://usa.ipums.org/usa/volii/c2ssoccup.shtml, accessed February 19, 2018; Alan S. Blinder and Alan B. Krueger,
“Alternative Measures of Offshorability: A Survey Approach,” Journal of Labor Economics 31(2), 2013, pp. S97–S128
at pp. S100–S101; David H. Autor and Michael J. Handel, “Putting Tasks to the Test: Human Capital, Job Tasks, and
Wages,” Journal of Labor Economics 31(2), 2013, pp. S59–S96 at p. S71.
217
Arcidiacono Rebuttal, p. 32, footnote 17.
HIGHLY CONFIDENTIAL – ATTORNEYS’ EYES ONLY

Page 108

data at hand—the variables coded in Harvard’s database.
215. In the example at hand, Harvard’s data lists the applicant’s parents as “laborers,
unskilled.” I mapped “laborers, unskilled” into the BLS category that covers construction workers, as
laborers traditionally worked in this field. Because construction is a mix of unskilled laborers and
more skilled tradespeople, I also combined construction with the BLS group for skilled trades. This
yields an occupational category that consists of unskilled laborers, construction workers, and
tradespeople. While this scheme may not capture the granularity of “newspaper deliveryman,” it does
capture something sensible and pertinent about status and class.
216. Because the data are somewhat complex and require harmonization, Prof. Arcidiacono
would have us toss this information out entirely. I disagree. I use the sensible mapping reported
below to create parsimonious but meaningful occupational categories that reflect information salient
to admissions officers. I then include these in my year-by-year model.
217. While my preferred categorization is reliable, my results are robust to alternative
occupational categorizations that are reasonable and that fully address Prof. Arcidiacono’s critiques. I
re-estimate my model using the following changes to occupational categories, separately and at the
same time:
• Creating a “low skill” group that consists of parents coded as “laborer,
unskilled” and “low skill.” In my preferred classification, parents coded
as “laborer, unskilled” were grouped with construction workers and
skilled tradesmen.
• Combining parents coded as homemakers, self-employed, other, and
unemployed into one category.
218. My results are robust to this change. Exhibit 27 reports the marginal effect of being
Asian-American for these models. The effect of being Asian-American remains insignificant (on
average and in each of the six years).

HIGHLY CONFIDENTIAL – ATTORNEYS’ EYES ONLY

Page 109

My results are robust to changes in occupational classifications

Source: Augmented Arcidiacono Data; College Board Cluster Data; U.S. Census Data
Note: Table shows the average marginal effect of race on admission for Asian-American applicants relative to White applicants using Prof.
Arcidiacono’s previously defined expanded sample. * indicates significance at the 5% level. Marginal effects are reporteed as percentage
point values.

Construction of occupational categories

Card Category

BLS Major
or Minor
Group

0

Other

-

Includes 99-0004, Undecided; 99-0002
and 00-0003, Retired; 99-0003 and 000004, Other; or missing

-

1

Homemaker

-

Includes 00-0001, Homemaker

Includes 2010-21, Homemaker
(full-time)

2

Unemployed

-

Includes 99-0001 and 00-0002,
Unemployed; 99-0005, Disabled

-

3

Skilled Trades
Incl.
Construction and
Extraction

47, 49, 51

Includes 47, Construction and Extraction;
49, Installation, Maintenance and Repair;
51, Production

Includes 2010-42, Skilled
Trades; 2010-44, Semi-Skilled
Worker; 2010-43, Laborer
(unskilled)

Includes 35, Food Preparation and
Serving; 53, Transportation and Material
Moving; 37, Building and Grounds
Cleaning and Maintenance; 45, Farming,
Fishing and Forestry; 31, Healthcare
Support; 39, Personal Care and Service

Includes 2010-15,
Conservationist or Forester

4

Low Skill
Occupations

35, 53,
37, 45, 31,
39

5

Self-Employed

-

BLS, "99-XXXX", and "00-XXXX"
Codes

“2010-XX” Codes

Includes 2010-07, Business
Owner or Proprietor

HIGHLY CONFIDENTIAL – ATTORNEYS’ EYES ONLY

Page 110

Card Category

6

7

8

Business
Executive
(management,
administrator)
Other
Management
Occupations
(Excl. Business
Execs)
Business and
Financial
Operations
Occupations

BLS
Major
or
Minor
Group

BLS, "99-XXXX", and "00-XXXX"
Codes

“2010-XX” Codes

11-1, 11-2,
11-3

Includes 11-1, Top Executives; 11-2
Advertising, Marketing, Promotions,
Public Relations, and Sales Managers; 11-3,
Operations Specialties Managers

Includes 2010-06, Business
Executive (management,
administrator); 2010-20, Foreign
Service Worker (including
diplomat); 2010-32,
Policymaker/Government

11-9

Includes 11-9, Other Management
Occupations

Includes 2010-34, School
Principal or Superintendent;
2010-12, College Administrator
or Staff

13

Includes 13, Business and Financial
Operations

Includes 2010-01, Accountant or
Actuary

9

Computer and
Mathematical
Occupations

15

Includes 15, Computer and Mathematical

Includes 2010-14, Computer
Programmer or Analyst

10

Architecture and
Engineering
Occupations

17

Includes 17, Architecture and Engineering

Includes 2010-18, Engineer;
2010-03, Architect or Urban
Planner

19

Includes 19, Life, Physical, and Social
Science

Includes 2010-35, Scientific
Researcher; 2010-11, Clinical
Psychologist

21

Includes 21, Community and Social
Services Occupations

Includes 2010-09, Clergy
(minister, priest); 2010-10,
Clergy (other religious); 2010-33,
School Counselor; 2010-36,
Social, Welfare, or Recreation
Worker

23-1

Includes 23-1, Lawyers, Judges, and
Related Workers

Includes 2010-25, Lawyer
(attorney) or Judge

25-1

Includes 25-1, Postsecondary Teachers

Includes 2010-13, College
Teacher
Includes 2010-38, Teacher or
Administrator (elementary);
2010-39, Teacher or
Administrator (secondary)

11

12

13

Life, Physical,
and Social
Science
Occupations
Counselors,
Social Workers,
and Other
Community and
Social Service
Specialists
Lawyers, Judges,
and Related
Workers

14

Postsecondary
Teachers

15

Pre-K through
Grade 12
Educational
Instruction and
Library
Occupations

25-2, 253, 25-4,
25-9

Includes 25-2, Preschool, Primary,
Secondary, and Special Education School
Teachers; 25-3, Other Teachers and
Instructors; 25-4, Librarians, Curators,
and Archivists; 25-9, Other Education,
Training, and Library Occupations

16

Entertainers and
Performers,
Sports and
Related Workers

27-2

Includes 27-2, Entertainers and
Performers, Sports and Related Workers

Includes 2010-02, Actor or
Entertainer; 2010-27, Musician
(performer, composer)

17

Arts, Design, and
Media Workers

27-1, 273, 27-4

Includes 27-1, Art and Design Workers; 273, Media and Communication Workers;
27-4, Media and Communication
Equipment Workers

Includes 2010-04, Artist; 201022, Interior Decorator (including
designer); 2010-41, Writer or
Journalist

HIGHLY CONFIDENTIAL – ATTORNEYS’ EYES ONLY

Page 111

Card Category

18

Health
Diagnosing and
Treating
Practitioners

BLS
Major
or
Minor
Group
29-1
(excludin
g 291070, 291140, 291150, 291160, 291170)

BLS, "99-XXXX", and "00-XXXX"
Codes

“2010-XX” Codes

Includes 29-1, Health Diagnosing and
Treating Practitioners, except for 29-1070,
Physician Assistants; 29-1140, Registered
Nurses; 29-1150, Nurse Anesthetists; 291160, Nurse Midwives; and 29-1170, Nurse
Practitioners

Includes 2010-16, Dentist
(including orthodontist); 201031, Physician; 2010-37, Therapist
(physical, occupational, speech)

Includes 29-2, Health Technologists and
Technicians; 29-9, Other Healthcare
Practitioners and Technical Occupations;
29-1070, Physician Assistants; 29-1140,
Registered Nurses; 29-1150, Nurse
Anesthetists; 29-1160, Nurse Midwives;
and 29-1170, Nurse Practitioners

Includes 2010-28, Nurse; 201023, Lab Technician or Hygienist

19

Other Healthcare
Occupations Incl.
Nurses

29-2, 299, 291070, 291140, 291150, 291160, 291170

20

Protective Service
Occupations

33

Includes 33, Protective Service
Occupations

Includes 2010-24, Law
Enforcement Officer

21

Sales and Related
Occupations

41

Includes 41, Sales and Related Occupations

Includes 2010-08, Business
Salesperson or Buyer

22

Office and
Administrative
Support
Occupations

43, 23-2

Includes 43, Office and Administrative
Support Occupations; and 23-2, Legal
Support Workers

Includes 2010-05, Business
(clerical)

23

Military Specific
Occupations

55

Includes 55, Military Specific Occupations

Includes 2010-26, Military
service (career)

Source: Augmented Arcidiacono Data
Note: BLS, “99-XXXX”, and “00-XXXX” codes are used by applicants to the classes of 2014 – 2019, and is the only code used by applicants
to the class of 2014. "2010-XXXX” codes are used by applicants to the classes of 2015 – 2019, and are used by the majority of applicants to
the classes of 2015 – 2019.

9.2. Appendix B.2: Error in Prof. Arcidiacono’s difference-in-difference estimates
219. In Table 6.2N of his rebuttal, Prof. Arcidiacono makes a critical error in calculating the
standard error of a “double difference.” Recall that the standard error is a measure of the precision of
an estimate—a smaller standard error means the estimate is more precise, (i.e., more certain) and a
larger standard error means the estimate is less precise (i.e., more uncertain). This mistake leads Prof.
Arcidiacono to erroneously conclude that there was a change in the difference in the academic indices
and the admission rates for single and multi-racial African-American students between the classes of
2014 – 2016 and the classes of 2017 – 2019. In this section, I outline the technical nature of Prof.
Arcidiacono’s mistake, and describe how this mistake leads Prof. Arcidiacono to make assertions the
data does not support.
HIGHLY CONFIDENTIAL – ATTORNEYS’ EYES ONLY

Page 112

220. The key error in the calculation takes place when Prof. Arcidiacono performs the
following step:218

. .

. .

.

.

. .

.

2

2

221. In this formula, . .
.
is the standard error on the difference in group
means from 2014 to 2016. Similarly, . .
.
is the standard error on the difference
in group means from 2017 to 2019. Both of these values are correctly calculated in Prof.
Arcidiacono’s report. Prof. Arcidiacono errs, however, when he divides each of the separate standard
errors by two in this step. This is the approach to calculating the standard error of an average, rather
than the standard error of a difference.
222. The correct approach would have been to use the formula to calculate the distribution of
a linear combination of normally distribution random variables. The proper application of the formula
in this instance is:
. .

.

. .

.

. .

.

223. As shown, Prof. Arcidiacono’s mistake has the effect of making his estimates appear
twice as precise as the data can actually support. After correcting this error, Prof. Arcidiacono’s
conclusions can no longer be supported.
9.3. Appendix B.3: Using absolute deviation to measure the importance of unobserved
characteristics is appropriate
224. In this appendix, I discuss a technical difference between how Prof. Arcidiacono and I
account for the relative importance of “unobserved factors.” In section 6 of my original report, my
approach for estimating this was straightforward. I used the “absolute deviation” which is the
absolute value of the difference between an applicant’s predicted probability of admission according
to the model and the applicant’s admissions decision. For example, if the model said a given
applicant had a 25% chance of admission, and that applicant was actually admitted, unobserved
factors would explain 75% of the admissions decision, and thus the absolute deviation would be 0.75.
If instead that applicant were actually denied admission, then unobserved factors would explain 25%
of the decision, and thus the absolute deviation would be 0.25. By comparing the average marginal
218

Prof. Arcidiacono also makes an additional smaller mistake by failing to weight different years in the data by the
correct number of observations that I have corrected. As this is less consequential than the key mistake outlined above, I
do not detail the effects of that mistake here.
HIGHLY CONFIDENTIAL – ATTORNEYS’ EYES ONLY

Page 113

effect of race to the absolute deviation, we can learn about the relative importance of race, as
compared to the importance of unobserved factors.
225. This is, I believe, the most clear and straightforward way to conduct this type of analysis.
That is, in assessing whether race is determinative, race should be compared to all of the other factors
that determine admission, both those observed and those that aren’t observed, and these factors
should be contextualized in terms of how much of the admissions decision they explain. Professor
Arcidiacono raises a technical objection to my approach.219 He states that the role of unobserved
factors can only be properly assessed by looking at the background machinery of the logit model,
rather than the probability of admission produced by the model. His objection relies on the fact that
we both use a logistic model, and this model utilizes an unseen “latent” variable which allows us to
estimate the probabilities and marginal effects we use in our reports. Specifically, the latent variable
estimates an index of “admission strength” for each applicant that varies from negative infinity to
positive infinity. This latent variable is then mapped into probabilities, which necessarily must be
between 0 and 1, through the logistic function. Higher admission strength translates to a higher
probability of admission but the probability of admission can never go below 0 or above 1.
226. I disagree with Prof. Arcidiacono’s characterization of my approach. My approach is
based on commonly accepted methods, is more transparent, and works directly with the object of
interest in this setting (that is, the probability of admission). While there are many ways in which it is
helpful to have an understanding of the inner workings of the logit model, the latent variable that
underlies the logit model is a tool which is meant to inform us of the real world, rather than an object
of interest in and of itself. Prof. Arcidiacono errs in failing to consider the practical aspects of the
problem, and instead takes the machinery of the model too literally.
227. The absolute deviation I calculate in my report is referred to in the economics literature
as a “generalized residual.”220 These residuals are widely used in econometrics, and in fact one
popular approach is based on this measure.221 Professor Arcidiacono himself has relied upon this
approach in his own academic work.222 The literature describes these values as having a number of
useful properties, even beyond those I describe above. They are “generalized” in the sense that the
information they convey does not depend on the researcher’s modeling assumption. For example,
were Prof. Arcidiacono to have chosen a slightly different model (for example, the linear probability
219

Arcidiacono Rebuttal, pp. 51–52.
Christian Gourieroux et al., “Generalized Residuals,” Journal of Econometrics 34, 1987, pp. 5–32 at pp. 12–14.
221
James J. Heckman, “Sample Selection Bias as a Specification Error,” Econometrica 47(1), 1979, pp. 153–161.
Technically, this paper uses a probit model, which is slightly different than the logit models Prof. Arcidiacono and I use
in our reports. The principle is the same, however.
222
Peter Arcidiacono, Jane Cooley, and Andrew Hussey, “The Economic Returns to an MBA,” International Economic
Review 49(3), 2008, pp. 873–899 at pp. 884–885.
220

HIGHLY CONFIDENTIAL – ATTORNEYS’ EYES ONLY

Page 114

model), he would have no choice but to concede that the absolute deviation is the right way to
measure the importance of unobserved factors.

HIGHLY CONFIDENTIAL – ATTORNEYS’ EYES ONLY

Page 115

10. APPENDIX C
10.1. List of variables included in model of admission

Variable Name

Variable Description

Constructed
by
Arcidiacono

Card
Initial
Model

Card
Updated
Model

Race Variables

race

Mutually exclusive race categories, based on
ethnic_group_cde field with categories: “White,”
“Black,” “Hispanic, Mexican, or Puerto Rican,”
“Asian,” “Native American,” “Hawaiian or Pacific
Islander,” “Race Missing.”

racecoll

Mutually exclusive race categories, based on
ethnic_group_cde field with categories: “White,”
“Black,” “Hispanic and Other,” “Asian,” “Race
Missing.” “Other” includes Mexican, Puerto Rican,
Native American, Hawaiian, and Pacific Islander.

























Base Controls
year
female

Harvard class to which applicant applies: 2014 to
2019.
Indicator for whether applicant indicated “Female”
in a sex code entry field.

disadvantaged

Indicator for whether applicant was flagged by
admissions staff, based on application, as likely
socioeconomically disadvantaged or HFAI eligible.







fgcl

Indicator for first generation college applicant.







earlyDecision

Indicator for Early Action applicant.







athlete

Indicator for athletic profile rating of 1.







legacy

Indicator for whether at least one of applicant’s
parents attended Harvard.







double_legacy

Indicator for whether both of applicant’s parents
attended Harvard.







faculty_or_staff_kid

Indicator for whether applicant is child of Harvard
faculty and staff.







deanDirectorPref

Indicator for whether applicant is on Dean’s or
Director’s interest lists.







waiver_tot

Indicator for whether applicant requested a fee
waiver.







finaid

Indicator for whether applicant applied for
financial aid







meduc

Categories for mother’s level of education: “Less
than college,” “College graduate,” “Master’s,”
“MD/JD/PhD,” “Missing.”







feduc

Categories for father’s level of education: “Less
than college,” “College graduate,” “Master’s,”
“MD/JD/PhD,” “Missing.”







intendedMajor

Categories for applicant’s intended major: “Social
sciences,” “Humanities,” “Biological sciences,”
“Physical sciences,” “Engineering,” “Mathematics,”
“Computer Sciences,” “Unspecified.”







docketFE

Docket to which applicant’s high school is assigned.







HIGHLY CONFIDENTIAL – ATTORNEYS’ EYES ONLY

Page 116

Variable Name

Variable Description

Constructed
by
Arcidiacono

Card
initial
model

Card
updated
model

Academic Variables
SACTmath_std
Normalized ACT/SAT math score.







SACTverb_std

Normalized ACT/SAT verbal score.







SAT2avg_std

Normalized average SAT II subject test score.







gpa_converted_std

Normalized converted GPA.







academic_index_std

Normalized Academic Index.







academic_index2p

Normalized Academic Index quadratic multiplied
by indicator for positive normalized academic
index.







academic_index2m

Normalized Academic Index quadratic multiplied
by indicator for negative normalized Academic
Index.







flaggpa

Indicator for converted GPA equal to 35.







m_SAT2avg


Indicator for missing average SAT II score.







APEA_combos

Combinations of athletic, personal, extracurricular,
and academic ratings. Each profile rating has
categories: 1, 2, 3, 4, 5, or 6. Exact combinations
are determined at the applicant level (e.g. any
applicant who received four ratings of 3 would have
the exact combination 3333). Combinations that
appear in the sample at least 100 times have their
own control group. The remainder of combinations
are combined with the control group with the
closest admission rate.





teach_combos

Combinations of school support ratings, assigned
by Admissions Committee, based on two teacher
recommendations. Each teacher rating has
categories: 1, 2, 3, 4, 5, and Missing. Combinations
are determined at the applicant level (e.g. any
applicant who received ratings of 1 and 2 would
have the combination 12). Combinations that
appear in the sample at least 100 times have their
own control group. The remainder of combinations
are combined with the control group with the
closest admission rate.





counslor_rat_abbr

School support rating, assigned by Admissions
Committee, based on applicant’s recommendation
from guidance counselor. Categories: 1, 2, 3, 4, 5,
and Missing.





alum_combos

Combinations of alumni interview overall and
personal ratings. Each alumni interview rating has
categories: 1, 2, 3, 4, 5 or 6, and Missing.
Combinations are determined at the applicant level
(e.g. any applicant who received an overall rating of
1 and a personal rating of 2 would have the
combination 12). Combinations that appear in the
sample at least 100 times have their own control
group. The remainder of combinations are
combined with the control group with the closest
admission rate.





Ratings Variables



HIGHLY CONFIDENTIAL – ATTORNEYS’ EYES ONLY

Page 117



Variable Name

Variable Description

Constructed
by
Arcidiacono

Card
initial
model

Card
updated
model



Ratings Variables (Continued)
Academic profile rating with categories:
academic_rat_abbr
1, 2, 3, 4, 5 and 6.
Personal profile rating with categories:
personal_rat_abbr
1, 2, 3, 4, 5 and 6.
xtracurr_rat_abbr
athletic_rat_abbr

Extracurricular profile rating with categories:
1, 2, 3, 4, 5 and 6.
Athletic profile rating with categories:
1, 2, 3, 4, 5 and 6.

























alum1_rat_abbr

Alumni interview personal rating with categories: 1,
2, 3, 4, 5 or 6, and Missing.







alum2_rat_abbr

Alumni interview overall rating with categories:
1, 2, 3, 4, 5 or 6, and Missing.







m_alum_rat

Indicator for missing alumni interviewer ratings.







rat2_*

Indicators for having ratings of 2 or better for each
pair of profile ratings (e.g. academic and personal,
athletic and extracurricular, etc.).







teacher1_rat_abbr

School support rating, assigned by Admissions
Committee, based on applicant’s recommendation
from Teacher 1. Categories: 1, 2, 3, 4, 5, and
Missing.







teacher2_rat_abbr

School support rating, assigned by Admissions
Committee, based on applicant’s recommendation
from Teacher 2. Categories: 1, 2, 3, 4, 5, and
Missing.

























alum_twos
school_twos

Count of alumni interview ratings (personal and
overall) of 2 or better.
Count of school support ratings (teacher 1, teacher
2, and guidance counselor) of 2 or better.

HIGHLY CONFIDENTIAL – ATTORNEYS’ EYES ONLY

Page 118

Constructed
by
Arcidiacono

Card
initial
model

Card
updated
model

Variable Name
Contextual Factors

Variable Description

father_occ_cat

Mother’s occupation category





mother_occ_cat

Father’s occupation category













father_deceased_yn
mother_deceased_yn

Indicator for whether father is marked as deceased;
defaulted to false for missing entries.
Indicator for whether mother is marked as
deceased; defaulted to false for missing entries.

parent_ivy

Indicator for whether at least one parent attended
an Ivy League school (not counting Ivy sister
schools); defaulted to false for missing entries





rural

Indicator for whether applicant’s high school
county is not in a Metropolitan or Micropolitan
Statistical Area; for applicants missing high school
city field, permanent address city is used.





intendedCareer

Intended career indicated by applicant, from a
choice of 15 career categories, "Other,"
"Undecided," or "Unknown."





school_type

School type (public, private, Catholic, or missing)





legacy_grad

Indicator for whether at least one of applicant’s
parents went to Harvard Graduate School.













primcoll_*

Indicators for applicant’s primary extracurricular
activities (collapsed into the following groups: (1)
Varsity, JV, or Club athletics; (2) Computer,
Speech and Debate, Journalism, Science, Math,
Robotics, or Academic; (3) Volunteer or Religious;
(4) Environmental, Family, LGBT, School spirit, or
Other; (5) Dance, Drama, or Vocal music; (6)
Instrumental music; (7) Politics; (8) Work; (9)
Career; (10) Cultural, Foreign exchange, or Foreign
language; (11) Missing; and (12) Junior ROTC). A
primary activity is defined as an activity the
applicant lists in the first or second activity field of
her application.





r_staff_yn

Indicator for whether applicant received a staff
interview rating.













perm_res
total_work

born_USA
outside_US_yn

Indicator for whether applicant is a United States
permanent resident.
Total hours of work reported in activity
description.

Indicator for whether applicant was born outside of
United States.
Indicator for whether applicant lived outside of
United States.

HIGHLY CONFIDENTIAL – ATTORNEYS’ EYES ONLY

Page 119

Variable Name

Variable Description

Constructed
by
Arcidiacono

Card
initial
model

Card
updated
model

High School Characteristics
The College Board aggregates applicant-level data to the high school level, based on student’s AICODE. All high school
variables are interacted with the SAT state indicator unless denoted with †.
Indicator for whether applicant’s state has more
SAT takers than ACT takers that applied to


sat_state
Harvard (a student is marked as an SAT/ACT taker
if the corresponding composite score is available
for that student).
hs_sat_math
hs_sat_cr
hs_sat_w

Average score on the math section of the SAT I for
all students at applicant’s high school.
Average score on the verbal section of the SAT for
all students at applicant’s high school.
Average score on the writing section of the SAT for
all students at applicant’s high school.













hs_english

Percent of students at applicant’s high school who
report that they speak only English.





hs_app_outofstate

Percent of students at applicant’s high school who
applied to an out of state college.





hs_avg_num_ap

Average # of AP tests taken by students at
applicant’s high school.













hs_fin_aid
hs_avg_hon

Percent of students at applicant’s high school who
require financial aid for college.
Average # of honors courses taken by students at
applicant’s high school.

hs_parent_ed

Percent of students at applicant’s high school who
reported that no parent had education beyond high
school.





hs_avg_sat_sends

Average number of scores sends for students at
applicant’s high school.





hs_coll_admit_rate

Average rate of admission for colleges receiving
score sends from students at applicant’s high
school.





hs_black†

ACS-based percent of students at applicant’s high
school who are Black.





hs_white†

ACS-based percent of students at applicant’s high
school who are White.





hs_hispanic†

ACS-based percent of students at applicant’s high
school who are Hispanic.

















hs_med_income†
hs_pov_line†
hs_house_val†

ACS-based median family income of students at
applicant’s high school.
ACS-based percent of students at applicant’s high
school who are below the poverty line.
ACS-based median value of home for students at
applicant’s high school, as a percentage of average
state value.

HIGHLY CONFIDENTIAL – ATTORNEYS’ EYES ONLY

Page 120

Constructed
by
Arcidiacono

Card
initial
model

Card
updated
model

Variable Name
Variable Description
Neighborhood Characteristics
The College Board aggregates applicant-level data to the educational neighborhood (one or more contiguous census
tracts). All neighborhood variables are interacted with the SAT state indicator unless denoted with †.
Average score on the math section of the SAT for all


n_sat_math
students in applicant’s neighborhood.
n_sat_cr

Average score on the verbal section of the SAT for
all students in applicant’s neighborhood.





n_sat_w

Average score on the writing section of the SAT for
all students in applicant’s neighborhood.





n_english

Percent of students in applicant’s neighborhood
who report that they only speak English.





n_app_outofstate

Percent of students in applicant’s neighborhood
who applied to an out of state college.





n_avg_num_ap

Average # of AP tests taken by students in
applicant’s neighborhood.





n_fin_aid

Percent of students in applicant’s neighborhood
who require financial aid for college.





n_avg_hon

Average # of honors courses taken by students in
applicant’s neighborhood.





n_parent_ed

Percent of students in applicant’s neighborhood
who reported that no parent had education beyond
high school.





n_avg_sat_sends

Average number of score sends for students in
applicant’s neighborhood.





n_coll_admit_rate

Average rate of admissions for colleges receiving
score sends from students in applicant’s
neighborhood.





n_black†

ACS-based percent of students in applicant’s
neighborhood who are Black.





n_white†

ACS-based percent of students in applicant’s
neighborhood who are White.





n_hispanic†

ACS-based percent of students in applicant’s
neighborhood who are Hispanic.





n_med_income_imp†

ACS-based median family income of students in
applicant’s neighborhood, missing values filled
with mean.





n_pov_line_imp†

ACS-based percent of students in applicant’s
neighborhood who are below the poverty line,
missing values filled with mean.





n_house_val_imp†

ACS-based median value of home for students in
applicant’s neighborhood, as a percentage of
average state value, missing values filled with
mean.









HIGHLY CONFIDENTIAL – ATTORNEYS’ EYES ONLY

Page 121

Variable Name
m_n_pov_line†
m_n_med_income†
m_n_house_val†

Variable Description

Constructed
by
Arcidiacono

Indicator for missing neighborhood poverty line
variable.
Indicator for missing neighborhood median income
variable.
Indicator for missing neighborhood house value
variable.

Card
initial
model

Card
updated
model













Note: I assign parents to be mothers or fathers using the father/mother_type variables for years before 2017, and the
parent1/2_type variables from 2017 and on due to data availability. I assign parents to be “mother figures” (e.g., “mother”,
“aunt”) or “father figures” (e.g., “father”, “grandfather”) using the variables father/mother_type for years before 2017, and
using parent1/2_type from 2017 and on due to data availability. When the parental type variable is gender neutral (e.g.,
“guardian”), I use gender information from the parent1/2_gender variable in my assignment.

HIGHLY CONFIDENTIAL – ATTORNEYS’ EYES ONLY

Page 122