AMERICAN EDUCATIONAL RESEARCH ASSOCIATION, INC. et al v. PUBLIC.RESOURCE.ORG, INC.
Filing
60
MOTION for Summary Judgment Filed by AMERICAN EDUCATIONAL RESEARCH ASSOCIATION, INC., AMERICAN PSYCHOLOGICAL ASSOCIATION, INC., NATIONAL COUNCIL ON MEASUREMENT IN EDUCATION, INC. (Attachments: #1 Statement of Facts Points of Authority, #2 Statement of Facts Statement of Undisputed Facts, #3 Declaration Declaration of Jonathan Hudis, #4 Exhibit Ex. A, #5 Exhibit Ex. B, #6 Exhibit Ex. C, #7 Exhibit Ex. D, #8 Exhibit Ex. E, #9 Exhibit Ex. F, #10 Exhibit Ex. G, #11 Exhibit Ex. H, #12 Exhibit Ex. I, #13 Exhibit Ex. J, #14 Exhibit Ex. K, #15 Exhibit Ex. L, #16 Exhibit Ex. M, #17 Exhibit Ex. N, #18 Exhibit Ex. O, #19 Exhibit Ex. P, #20 Exhibit Ex. Q, #21 Exhibit Ex. R, #22 Exhibit Ex. S, #23 Exhibit Ex. T, #24 Exhibit Ex. U, #25 Exhibit Ex. V-1, #26 Exhibit Ex. V-2, #27 Exhibit Ex. W, #28 Exhibit Ex. X, #29 Exhibit Ex. Y, #30 Exhibit Ex. Z, #31 Exhibit Ex. AA, #32 Exhibit Ex. BB, #33 Exhibit Ex. CC, #34 Exhibit Ex. DD, #35 Exhibit Ex. EE, #36 Exhibit Ex. FF-1, #37 Exhibit Ex. FF-2, #38 Exhibit Ex. FF-3, #39 Exhibit Ex. FF-4, #40 Exhibit Ex. FF-5, #41 Exhibit Ex. FF-6, #42 Exhibit Ex. GG, #43 Exhibit Ex. HH, #44 Exhibit Ex. II, #45 Exhibit Ex. JJ, #46 Exhibit Ex. KK, #47 Exhibit Ex. LL, #48 Exhibit Ex. MM, #49 Declaration Declaration of Marianne Ernesto, #50 Exhibit Ex. NN, #51 Exhibit Ex. OO, #52 Exhibit Ex. PP, #53 Exhibit Ex. QQ, #54 Exhibit Ex. RR, #55 Exhibit Ex. SS, #56 Exhibit Ex. TT, #57 Exhibit Ex. UU, #58 Exhibit Ex. VV, #59 Exhibit Ex. WW, #60 Exhibit Ex. XX, #61 Exhibit Ex. YY, #62 Exhibit Ex. ZZ, #63 Exhibit Ex. AAA, #64 Exhibit Ex. BBB, #65 Exhibit Ex. CCC, #66 Exhibit Ex. DDD, #67 Exhibit Ex. EEE, #68 Exhibit Ex. FFF, #69 Exhibit Ex. GGG, #70 Exhibit Ex. HHH, #71 Exhibit Ex. III, #72 Exhibit Ex. JJJ, #73 Declaration Declaration of Lauress Wise, #74 Exhibit Ex. KKK, #75 Exhibit Ex. LLL, #76 Declaration Declaration of Wayne Camara, #77 Exhibit Ex. MMM, #78 Declaration Declaration of Felice Levine, #79 Exhibit Ex. NNN, #80 Exhibit Ex. OOO (Public Version), #81 Exhibit Ex. PPP, #82 Exhibit Ex. QQQ, #83 Exhibit Ex. RRR, #84 Exhibit Ex. SSS, #85 Exhibit Ex. TTT-1, #86 Exhibit Ex. TTT-2, #87 Exhibit Ex. UUU, #88 Declaration Declaration of Kurt Geisinger, #89 Declaration Declaration of Dianne Schneider, #90 Text of Proposed Order Proposed Order, #91 Certificate of Service Certificate of Service)(Hudis, Jonathan). Added MOTION for Permanent Injunction on 12/22/2015 (td).
EXHIBIT AA
Case No. 1:14-cv-00857-TSC-DAR
IN THE T]NITED STATES DISTRICT COURT
FOR THE DISTRICT OF COLUMBIA
AMERICAN EDUCATIONAL RESEARCH
ASSOCTATION, INC., AMERICAN
PSYCHOLOGICAL ASSOCIATION, INC., and
NATIONAL COUNCIL ON MEASUREMENT IN
EDUCATION,INC.,
Case
No.
I : I 4-cv-00857-TSC-DAR
Plainti ffs/Counter-defendants,
V
PUBLIC.RESOURCE.ORG, INC.,
Defendant/Counterc laimant.
EXPERT REPORT OF JAMES R. FRUCHTERMAN
A
øsxg¡srr_4
Rptr.klù
Table of Contents
What Does Accessibility Mean for a Person Who
Locating an Accessible Version
is Blind?
ofthe 1999 Standards.
Testing the Public.Resource.Org Website's Accessibility.
Today..........
The Public.Resource.Org Version of the 1999 Standards
Making the 1999 Standards Accessible
Confirming Accessibility for People Who Are Blind
The Archive.org Version of the 1999
Standards
3
5
7
..,.................... g
............,....9
9
.........11
Introduction
As an expert in accessibility of written materials for people who have disabilities that
affect using standard print (people who are print disabled), I have been retained by
Public.Resource.Org to evaluate the accessibility of certain content that had been available on
the website of the defendant in this case. As someone dedicated to improving accessibility
for
the benefit of people with disabilities and in the public interest, I agreed to evaluate the
accessibility to people who are blind of this specifrc commonly used standard document.
This expert report is a summary of certain opinions that I intend to give,
if
asked, at
trial
regarding the accessibility of specific documents to people who are blind or print disabled. This
report also states the bases for my opinions, and it discloses the data or other information
considered in forming those opinions. I reserve the right to change or supplement this report
if
additional evidence comes to my attention, and to prepare demonstratives and/or exhibits to
illustrate or explain my opinions, as appropriate.
A copy of my curriculum vitae, including
a list
of my publications and presentations, is
Exhibit A to this report. I provide my expertise in this case pro bono, and I am not receiving
compensation for my time researching, writing this report, or
testi$ing. I previously
an expert in The Authors Guild, Inc. et al. v. HathiTrust, et al. , Case
No. I : I I -cv-063
served as
5I
-HB
(S.D.N.Y.) (case filed September 12,20ll) and I am serving as an expert in American Society of
Testing and Materials, et al. v. Public.Resource.Org, Case No. 1:13-cv-01215-TSC-DAR,
although I have not testified in either case. I have not given deposition or trial testimony in the
past four years.
L
Background and Qualifications
I
serve as Founder and Chief Executive Officer of Benetech, a nonprofit dedicated to
creating new technology solutions that serve humanity and empo\ /er people to improve their
lives. In 1980 I earned a B.S. in Engineering and an M.S. in Applied Physics from California
Institute of Technology. I co-founded Calera Recognition Systems in 1982. Calera developed
optical character recognition (OCR) technology that allowed computers to read virtually all
printed text.
ln 1989, I founded Arkenstone, a nonprofit
social enterprise, which produced reading
machines for the print disabled community based on the Calera technology, and was at one time
the largest maker of affordable reading systems for the blind. The Arkenstone product line was
sold in 2000 and the resulting capital funded the next phase of Arkenstone under its new name,
Benetech. I have been the CEO of Benetech/Arkenstone since 1989.
I have served on three U.S. federal government advisory committees for disability issues:
the Section 255 Telecommunications Access Advisory Committee, the Section 508 Electronic
Information and Technology Access Advisory Committee, and the Advisory Commission on
Accessible Instructional Materials in Postsecondary Education for Students with Disabilities.
I have received numerous other awards and recognition for my work making print materials
accessible to people who are blind or otherwise print disabled. In 2006I received a MacArthur
Fellowship. I was named an Outstanding Social Entrepreneur in 2003 by the Schwab
Foundation and have frequently participated in the World Economic Forum Annual Meetings in
Davos, Switzerland. Benetech received the Skoll Award for Social Entrepreneurship under my
leadership. I also received the Migel Medal from the American Foundation for the Blind, the
Robert F. Bray Award from the American Council of the Blind, and the American Library
2
Association's Francis Joseph Campbell Award in recognition of my successfi¡l efforts to make
literary works more accessible to people who are blind or visually impaired.
What Does Accessibility Mean for a Person Who is Blind?
Accessibility is usually defined in a functional way: can a person with a disability
independently access the same information and perform the same tasks as a person without a
disability? When it comes to accessing materials traditionally available as print, such
as
standards, there are many groups of print disabilities. The most severe is blindness, where a
person cannot perceive the printed text at
all. The next is vision impairment, where
a person
generally cannot perceive the text directly or with corrective lens, but may be able to use
magnifiers of different types to read the text. Another group is learning disabilities that interfere
with reading, such
as
dyslexia. A closely related group of disabilities involve brain injuries that
affect reading or the retention of material read. Another group is physical disabilities that
interfere with the holding or seeing of books or the turning of pages.
In this report, I focused on the accessibility challenges that would be experienced by
btind people, because they are generally the most severe print disabilities. The other groups
of
people with print disabilities use similar technologies to access print (such as having it read
aloud), and experience similar challenges as blind people. In the accessibility field, it is
generally understood that if you make information accessible to a blind person, it
will probably
also meet the accessibility needs of the great majority of people with other print disabilities.
The most common technology used by a blind person for accessibility is called a screen
reader. As the name suggests, a screen reader is a program that runs on a personal computer or a
smartphone that reads the information on the screen aloud (using a computer-synthesized voice)
3
to a blind person. The screen reader runs 'oon top
of'
other programs, figuring out not only what
text is on the screen, but also the controls that are displayed: items such as buttons, menus, textentry boxes and the
like.
Because of the amount of information on a complete screen, and its
complexity, blind people need to be able to focus on the most important information so that they
do not waste time listening to everything on the screen.
For the purpose of this report, measuring the accessibility of standards, I am assuming
that the blind user is using a screen reader on top of a web browser or word processor program
on a personal computer. Based on the information the screen reader can glean from the pages
displayed on the screen, can a blind person locate the standard and read it?
The accessibility tasks I tested were designed to assess whether a blind user with basic
assistive technology skills could perform the same kind of tasks one might expect a user without
a disability to perform in accessing a given standard, without requiring the intervention of a
third
party. This functional approach is the most common method of assessing accessibility.
The specific tasks
¡
I investigated were:
Could a blind user with basic assistive technology skills independently access a
specific standard of interest?
¡
Could a blind user independently read the entire standard using assistive technology?
.
Could a blind user independently navigate to a specific place in the standard and read
the content in that place?
¡
Could a blind user independently do a full text search and find specific mentions of
terms of interest?
4
I conducted these tests on a standards document that was represented to me as having been
available on the Public.Resource.Org website. I primarily used the Window-Eyes screen reading
software and the ABBYY FineReader optical character recognition software to perform my tests.
Locating an Accessible Version of the 1999 Standards
I was asked to review the accessibility of the 1999 edition of The Standards for
Educational and Psychological Testing (hereafter, the
"l999 Standards") for people who are
blind or otherwise print disabled. The first step in determining the accessibility of a document is
to try to locate a version of the 1999 Standards that would be accessible to people who are blind
or have print disabilities. I attempted to locate an accessible version of the 1999 Standards
through two separate avenues: by searching the catalogs of the main libraries that serve people
with print disabilities, and also by doing a standard Google search to try to locate an electronic
version of the 1999 Standards. From my work with people who are blind or print disabled,
I
know that this would be the typical procedure that people who are blind or print disabled would
perform when looking for an accessible version of a document.
The four main libraries that serve people with print disabilities are the American Printing
House for the Blind, Bookshare (which
I founded), Learning Ally, and the National Library
Service for the Blind and Physically Handicapped, Library of Congress.
search
I performed
a
thorough
of all four of these catalogs and found that the 1999 Standards were not available through
any of these resources, either in an electronic form, or in mail-delivery braille or audio recording.
I then performed a Google search to attempt to locate an electronic version of the 1999
Standards online. I was unable to find an electronic version of the 1999 Standards online, but
did locate a used print version for sale on Amazon.com. I have been informed by counsel for
5
I
Public.Resource.Org that although Public.Resource.Org previously hosted an electronic version
of the 1999 Standards on its website, it had been taken down during the course of this litigation.
From my research I believe that a version of the 1999 Standards that is accessible to people who
are blind or print disabled is currently unavailable to the public.
The unavailability of a version of the 1999 Standards that is accessible to people who are
blind or print disabled is problematic because the 1999 Standards are important references for
those making tests that are accessible to students who are print disabled, as well as those
impacted by these tests. For instance, the 1999 Standards were referred to in several works
concerning test accessibility for blind students, specifically: Test Access; Making Tests
Accessible
for
Students with Visual Impairments: A Guide
þr
Test Publishers, Test Developers,
and State Assessment Personnel, Second Edition, by Carol B. Allman, Ph.D., published by the
American Printing House for the Blind ß¡hiÞ¡!_e), and an online resource published by the
American Foundation of the Blind, Building Assessment Initiatives
for Schools; Guidelines to
Support the Contract Development Process Between Test Publishers and Stares
(Sþlþj!.B).
As
an expert in the field, this means that the 1999 Standards are important references today for those
making tests accessible to students with disabilities such as blindness. This also means that it is
an important resource to any students or other individuals with print disabilities that want to
assess compliance
with the 1999 Standards. The unavailability of the 1999 Standards means that
some of those who are most impacted, people who are blind or print disabled, are unable to
independently access the 1999 Standards.
6
Testing the Public.Resource.Org Website's Accessibility
Because the 1999 Standards are no longer hosted on the Public.Resource.Org website
during the course of this litigation, I was not able to locate the full text of the 1999 Standards on
the Public.Resource.Org website while performing my Google search referenced above.
However, searching the terms "1999 Standards for Educational and Psychological Testing" on
Google (for me) shows the page where the 1999 Standards had been located on the
Public.Resource.Org website, located in the frrst page of links in the search results. However,
the
file I found there was a placeholder noting the voluntary takedown of the file. I have also
searched for other standards that Public.Resource.Org has posted on its website, such as NFPA
l0l-2000, and I have found that it would be relatively easy for
a person who is blind or
disabled to use screen reader software and perform a Google search to locate a standard
print
if it was
available on Public.Resource.Org's website. Therefore, when the 1999 Standards had been
hosted on the Public.Resource.Org website, a person who is blind would have been able to locate
the 1999 Standards through a simple Google search, with the assistance of screen reader
software.
The Public.Resource.Org website has no required sign-up procedure.
It
is possible to go
directly to a specific standard either by using a direct weblink or by navigating the text-oriented
website. This is important because sign-up procedures can often have the effect of preventing
people who are blind or print disabled from accessing certain parts of websites due to the fact
that many sign-up procedures use unlabeled buttons or other elements that screen reader software
cannot read. Therefore, a person who is blind or print disabled would have been able to locate a
version of the 1999 Standards on the Public.Resource.Org website when it was still hosted there,
7
and that person would then have been able to gain access to that electronic version of the 1999
Standards.
Making the 1999 Standards Accessible Today
Because an accessible version of the 1999 Standards is not currently available,
person needed to have an accessible version
ofthe 1999 Standards, they would
if
a blind
need to create
it
themselves or request that their employer, educational institution, or a specialized library for the
blind create
it.
Generally, most blind people themselves do not have the ability to convert books.
Some blind people have their own home scanners, and if they purchased a used copy online,
would be able to scan the 1999 Standards page by page on a home scanner, which would take at
least two hours of labor, and then perform optical character recognition on the
title. Optical
character recognition is the process by which a computer converts images of printed text into
machine-encoded text that can be read aloud by a screen reader. If the scanning quality wasn't
very good, signifrcant numbers of errors would be introduced through the optical character
recognition process. The resulting word processor file of recognized text could then be read
using a screen reader.
If Bookshare were to make the 1999 Standards accessible to a blind person, we would
purchase a used copy ofthe printed version, chop
offthe bindings and then process it through
high speed scanner to obtain a high quality scan of the book in less than fifteen minutes. We
would then perform optical character recognition on the image scans of all of the pages of the
book, which typically creates a Microsoft Word file version of the text, and then send it to an
outside service (or a volunteer) to have it proofread, correcting errors introduced by the
limitations of optical character recognition. Public.Resource.Org has already performed the
8
a
great majority of the most expensive and time consuming steps needed to create an accessible
version of this document, specifically purchasing a print version of the title, waiting a few days
to receive it, chopping off the binding and scanning it with a high speed production scanner, or
utilizing a library-grade nondestructive book scanner. This is a valuable contribution to anyone,
individual or organization, that wanted to ensure that the 1999 Standards are accessible to people
who are blind or print disabled, ifthat file were still available.
The Public.Resource.Org Version of the 1999 Standards
I was supplied with a version of the 1999 Standard in PDF format. It was represented to
me that this file had been available online at the Public.Resource.Org website. I examined the
file, and found it to be
a
high quality image scan of the 1999 Standards. If the file was still
online, this would have meant that a blind person wanting to have an accessible version of the
1999 Standards would be able to do so by performing optical character recognition on the
Fublic.Resource.Org image file, creating an accessible text version of the 1999 Standards in
minutes.
Confirming Accessibility for People Who Are Blind
I then performed the steps of taking the Public.Resource.Org version of the 1999
Standards and making it accessible, while using Window-Eyes screen reading software to read
the words on the computer screen aloud. For the version of the 1999 Standards that was on the
Public.Resource.Org website, I used ABBYY FineReader optical character recognition software
to recognize page images, and it converted those pages into a Microsoft V/ord document. In
addition, the process of using the ABBYY software and reading the document was something a
9
blind person could do independently using Window-Eyes software to perform the tasks in an
accessible way, because the program speaks the menus and converted text aloud. Because the
image scan by Public.Resource.Org was high quality, there were few optical character
recognition errors. In addition, I also tested a typical page image from the Public.Resource.Org
version using the website Free Online OCR (http://www.onlineocr.net/), and confirmed that it
also recognized the text
well.
In my opinion, most of the OCR solutions that would be available
to people who are blind should be able to convert this image PDF document into accessible text.
r!1
E
ge'
-
--d!3
p¡J-Ai¡r¡tt Fr'¿,.i.i
,.",
!1¡ilù.!\
v:1 lJüå,,bf{ I -l
*!!
IL
T
[Elf: t¿
ìrre
m
||
'-
aullColor
@rn."un"e.
-r
--{
;1.'
:f ri.!r: i rsv'
EL'-J
fs
Qo"ç.n"e.
rrF
dào
.)
l¿
N¿n
!i
rrdirlbh
ri*r"e.
c;ry .
,f, optou
I
AI
RÉq{¡ft{ th &ùÉilsd?lwrdryd
i
I
r
f..rt
O P1:2.13.4. rb.æFTdqT&¡sdhþ.
.--*
$ ¡¡¡(e)
ud
bør dtly
lcg¡lly
¿
of Am¡ie.
-10ã.+
iÍ¡rÈT"r¡.¡,'r'F'''ri;;,1-"-TüT¡l¡tli-ls iËüìlq
tf
:t
,!r)
!r¿:
È 10d
^ç.,c'i{râ
I then examined in Microsoft Word several pages of the standard as processed by
ABBYY FineReader, and confirmed that Window-Eyes could read the text aloud in logical
reading order. I also successfully performed fulltext searches on a key word, a standards
number, and apage number, using Window-Eyes.
L0
ø
6nñ
¡1¡| t. !r.if
fllA'q.rrùiGliwe
'
;¡¡r$*r.d
o rl
r¡
'.t '¡f ^' ¡' Ð i:'l:"ç' r|l tl I AåBsc ArttC¡ À¡Bbcc AâB ¡o¡¡c. Å¿B!\-. Atæc.. 4{ l1Ï:
tr i...."" ffi
¡.¡¡&r ¡.C-A'lllr
þ- ¡!-À. 'þrF- e¡ûo¡ eúo¡ k ffi *r.... :o* '¡g'¡ ilfl
"¡oJrå*l¿
ñ¡ttflt¡trttlttutrtfttrt
EIrlmr[E
ffiu
rbrd ¡Éa¡.aÞ .g Låi¡Êa
¡-.iL.åi¡FJ
-.¡J*
r.br-rú.'{¡r¡dh
.a.n-.*b¡aixe¡i*
[r¡¡,¡l
dr¡.*5.1ù
lrt
htaãdt¡ú
¡çùaç¡¡ùir
rá¡d¡¡t
d.g¡tu¿.tùkd
ldç,'¡1F,.-å,ù,
tdrËÈhL.¡t¡,t5.
krl¡*m¡heÉ.ùd{.
ñ.t ñi*é.d¡.¡-a-¡
*rñ*dFlL.1
bE¡¡¡ry¡-¡
-t-¡-É¡.ÉÈd
C
affi¡¡
lì.tÉañr¡¡úF.3ã'.,4
-¿.h.. -tb - ù ñ¡
úa
blrrr*Çihkù
dÉ¡Ç--¡t.brF-r-.-bòbrlF-F-*ã¡-¡.-ad¡
lqt r ¿É.4 Þ.{.ia t¡ ñ È
* ¡&a r.r5r. b *
n
-F-rùrr-4.id¡
ah- -dLa.l&...i-d
-.-h-ir¡¡adú
My tests therefore indicated that
a
blind person using a screen reader would be able to
perform all of the functional tasks: reading the entire standard, navigating to a speeific place in
the standard, or searching on key terms. Because the text is provided in a standard format, such
as
Microsoft Word, a blind person is able to listen to the text, or access it using a digital braille
device. This kind of text content is also hìghly accessible to people with other print disabilities
and the assistive technology they use to access
print. For example, people with low vision or
with dyslexia often use a screen reader to read text aloud.
The Archive.org Version of the 1999 Standards
I was also supplied with
a version
person at the Internet Archive, operator
of the 1999 Standard in TXT (text) format by a staff
ofthe Archive.org website. It was represented to me by
this person that this file, aera.standards.1999_djvu.txt, had been available online at the
Archive.org website. According to thc Internet Archive's "Derivatives" page located at
11
https://archive.org/help/derivatives.php, when a PDF frle is uploaded to the Intemet Archive
website, the website automatically creates derivative file types that are also accessible on that
website, including TXT format. The deposition testimony of Christopher Butler from the
Internet Archive, as well as the deposition testimony of Carl Malamud from Public.Resource.Org
indicate that when Public Resource uploaded the PDF fìle of the 1999 Standards to the Internet
Archive website, the Internet Archive automatically created this text file of the 1999 Standards,
which was publicly accessible on the Internet Archive website.r I examined the file, and found it
to be a text version of the 1999 Standards, preceded by informational material about the Internet
Archive in HTML format. It appeared to me that the text version had been created by optical
character recognitiono because there were a few uncorrected errors typical ofthat process.
As established in my discussion of the 1999 Standards on the Public.Resource.Org
website above, once the 1999 Standards are available in an electronic text format, a blind person
using a screen reader would be able to perform all of the functional tasks: reading the entire
standard, navigating to a specifìc place in the standard, or searching on key terms. I confirmed
that this was the case with the aera.standards.l999_djvu.txt frle. Because the text is provided in
a standard and compatible format, a blind person is able to listen to the text, or access
it using a
digital braille device. This kind of text content is also highly accessible to people with other
print disabilities and the assistive technology they use to access print.
Conclusion
I was asked to review the accessibility of the 1999 edition of The Standards for
Educational and Psychological Testing. I was unable to find an accessible version of the
I
Deposition of Christopher Butler of the Internet Archive, December 2, 2014, at pp. 48-49,
deposition of Carl Malamud of Public.Resource.Org, }l{ay 12,2015,atpp.2Sl-284.
72
87
, 102-105;
document online.
If
the document provided to me by Public.Resource.Org had been online on
their website, I believe a blind person of ordinary technical skill would have been able to
independently use that document and commonly available optical character recognition
technology to create an accessible version ofthe 1999 Standards, and carry out reading and
reference tasks similar to those
version of the standard.
If
a
person without a disability would be able to do with a print
the document provided to me by Archive.org had been online on their
website,I believe a blind person of ordinary technical skill would have been able to
independently use that document directly to carry out reading and reference tasks similar to those
a person
without a disability would be able to do with
a
print version of the standa¡d.
Dated: June 13,2015
James R.
13
V ilqIqXfl
James R. Fruchterman
Founder and CEO
Benetech
Education
Califomia Institute of Technology
B.S. Engineering, 1 976-80
M.S, Àpplied Physics, 1978-80
Stanford University, 1 980-81
Ph.D. Srudies in Electrical Engineering
Ptofessional Expetience
I
CEO and Founder,20L5-present
President, CEO, Chaitman, Foundet, 2000-2014
Benetech (name changed from Arkenstone in 2000)
Palo Alto, Cahfornia
I
a
Ptesident, CEO, Chairman, Foundet, 1989-2000
Arkenstone, Inc.
Moffett Field, California
Director, 1 989-present
Vice President Finance, CFO, 1989-2004
Ptesident & CEO, Founder, 1989-95
RAF Technology, Inc.
Palo Alto, Cahfornia and Redmond, Washington
Vice President, Marketin g, 1.987 -89
Foundet, Vice Ptesident, Finance, 1982-88
Calen Recognition Systems, Inc.
Santa Clara, C alifotnia
t
Ptior engineering positions with:
I Phoenix Engineering, Inc.
r G.C.H,, Inc.
r IBM TJ. Watson Research Center
r General Motors Company
r NASA
Propulsion Laboratory
-J.t
¡ Fetmi National Accelerator Labotatory
Publications
I
¡
Technology Seruing Humanity (chapter). In Schultz, R. (editor) Creatin¿Good lYork, Palgrave
Macmillan, February 201 3
Guest Editot's Page, AFBJoumal of Visual Impaitment & Blindness, October-November
2072
I
I
¡
¡
I
¡
I
'
I
r
r
An Interview With Technology Guru Geotge I{erscher, AFBJoumal of Visual Impaitment
& Blindness, October-November 201 2
For Love or Lucre, Stanford Social Innovation Review, Spring 2011
Developing Information TechnologJ¡ to Meet Social Needs
.In
Innomtiozs,
MIT
Press, 2008
Accessing Books and Documents, a chapter in the book, ,{.ssistive Technology for VisionImoaired and Blind Peoole. Sorinser Verlas 2008
I
O
O
Everyone Desewes ,{.ccess to Technology. OpEd in The Satameato
and Gregg Vanderheiden, June 77,2007
Bee
by Jim Fruchterman
Document Recognition Serving People With Disabilities, Praa: SPIE 6S00,International
Society for Optics and Photonics, 2007
Pattetn Recognition Technology Helps Disabled People Access Books, SPIE
International Society fot Optics and Photonics, May 14,2007
Newsroom,
Nothing Venrured Nothing Gained. Addressing the Critical Gaps in Risk-Taking Capital for
-,
Social Enterprise, byJed Emetson, Tim Freundlich andJim Fruchterman, published by
Oxford Said Business School, 2006
Build Great Companies. Then l{elp Build a Great !7orld, OpEd in Tlte San Jon Merctry News,
November 1,3,2006
Comments on Accessibility of Google Print and Google's Ubrary Project, white paper,
February 2005
Technologv Benefiting Humanitv, published in the Association for Computing Machines
U bi q u i fl magazine, }r'f:arch 200 4
r
The Power of Technology Social Enterprises, published in the N-TEN forecast serìes,
February 2004
¡
In the Palm of Your Hand: A Vision of the Future of Technology for People with Visual
Impairments, published in the American Foundation for the Blind's lournal of Vision
Imþainnent and Blindness, October 2003
I
'
r
The Chafee Amendment: Improving Access to Information, published in InJòtwation
TechnoloEt and Disabilitiq a journal published by Equal Access to Software and Information
(EASI), co-authored u¡ith Bookshare Senior Product Manager Alison Lingane, October 2003
The Soundproof Book: Exploration of Rights Conflict and Access to Commercial EBooks
for People with Disabilities, published in First Mondq. co-authored with George l(erscher,
the Intetnational Project Manager of the DAISY Consortium,May 2002
Bookshare. Books without Barriers, at the Closing the Gap conference, Minneapolis, MN,
October 2001
Two presentations given at the IT Accessibility 2001 Conference, }rf:ay 200L at the National
Institute of Standards and Technology
I I Dream of Software
r The Business Case fot ¡\daptive Technology
Humanizing the Voice of the Machine, with Prof. Mari Ostendorf pnivetsity of
Washington), Annual Meeting of the American Association for the Advancement of the
Machine, Boston, MA, February 2000
The Many Facets of Open Book: Ruby Edition, Califomia State Univetsity, Northridge
(CSUN), 15th Technology and Persons with Disabilities Conference, March 2000
Coryorate Responsibiliry for Adaptive Technology, Califomia State University, Northridge
(CSUN), 14th Technology and Persons with Disabilities Conference, March 1999
I
DevelooinE Patnershios for Âssistive and Universallv Designed Technolow for Persons
with Disabilities, Testimony before United States House of Representatives, Committee on
Science, Subcommittee on Technology, August 4,7998
¡
Access to Maps and Location Information through Virtual Reality Techniques and GPS
Satellite Receivers, 3rd International Technical Aids Seminar, Tokyo, Japan, July 1994
Invited Talks
|
I
"Innoaation ir America: The Ro/e of Tecltnologt," August 1,2073, Testimony before U.S.
House of Representatives, Judiciary Commìttee's Subcommittee on Coutts, Intellectual
Property, and the Intemet.
"social Change at Scale
CA.
.
t
t
.
r
I
r
r
r
-
That's Innovation!" May 2012, TEDxSanJoseCA 2012,
San Jose,
'Tlte Pouer of Failure, Peoþle afld Kama Banking." Mry 20, 20'l 2, Commencement speech, St.
Mary's College, Moraga, CA.
Keyon Speech, Association for Education and Rehabilitation
of the Blind and Visually Impaited Confetence, Cleveland OH.
'ß,øising the Floor," October, 201 'l ,
Kelnote tpeech,
IEEE
"Making the Book
Keynote Speech
Secrions Congress, AaguCI 201
Trul@,"
'l
, San Francisco, CA.
Tools of Change Conference, NewYork, NY,2011
UBS-Ashoka Visionaris Award, I(eynote Speech, Social Entrepreneur of the Year Award,
Meúco City, Mexico, September, 2010
A
series of three invited speeches on Bookshare and accessible books, in Tokyo, Shizuoka
and Osaka, Japan, February, 2009
l(eynote Speech, Social Enterprise Wodd Forum, Edinburgh, Scotland, September, 2008
"Raising the Floor: Providing Accessible Technology and Content to Every Person with
Disability on the Planet," International Conference on Computers Helping People with
Special Needs, Linz, Austria, July, 2008 l(eynote Speech
"Raising the Floor," CSUN Conference on Technology and Petsons with Disabilities,
March, 2008 I(eynote Speech
a
t
Extensive speaking engagements to students about technology serving people with
disabilities. Have done invited talks at:
Stanford University
Univetsiry of Califomia at Berkeley
Brigham Young Universiry
University of the Pacifìc
Santa Clara University
California Instirute of Technology
SanJose State
University of California at Santa Cruz
University of California at Davis
Loyola Marymount University
Pepperdine University
University of Washington
Columbia University
Harvard University
University of Geneva
Oxford Universiry
r
I
r
¡
Inflection Point Opportunities in Social Investment, Closing I(eynote for the UBS
Philanthropy Forum, Lisbon, Portugal, July 2007
ft's Not Rocket Science: Building Social Enterprises, I(eynote for the 7th Gathering of the
Social Enterprise Alliance, Atlanta, Georgia, March 2006
Opening l(eynote for the Global Social Venture Competition, New York, April, 2006
I(eynote for the 7th IAPR workshop on Document Analysis Systems, Nelson, New
Zealand, F'ebruary 2006
'
¡
I
'
I
Building a Global Library for People with Print Disabilities, a speech for the World Summit
on the Information Society, Tunis, Tunisia, November 2005
Innovating Information Technologies to Protect Human Rights, a speech for the World
Affairs Council of Northem California, February 2004
Setting the 2004 Agenda: Technology, speaker at the Wodd Economic Forum, Davos,
Switzerland, J anuary 200 4
Seizing Market Failure as an Investment Opportunity, I(eynote for the Business for Social
Responsibility Annual Conference, Los Angeles, November 2003.
In the Palm of Yout Hand, I{eynote for the World Blind Union Asi¿ Pacific conference,
Singapore, November, 2003
r
¡
Technology and Human Rights, Universiry of Peradeniya, Sri Lanka, November ,2003
$(/hen Markets Fail, Who Responds? Discussion Leader at the Wodd Economic Forum,
Davos, Switzedand, Januxy 2003
r
'
Technology for Nonprofits, with Michael Gilbert, National Gathering for Social
Entrepreneurs, Minneapolis, MN, December, 2002
Bookshate: Large Scale, Web-Based Accessible Books, TechShare conference organized by
the Royal National Institute of the Blind, Birmingham,
¡
I
.
I
I
I
r
I
r
r
UI{, November 2002
Putting Technology to Work for Development, speech at the United Nations to the joint
meeting of the rü(/odd Technology Ne¡work and UNOPS, July 2002
Bookshare: The Project for Creating Accessible Books through Computers, at the General
Session of the National Federation of the Blind 2002 Annual Convention,July 2002
Stanford Social Entrepreneurship Conference, January 2002
The Once and Future rù(/eb: Tenth Anniversary of the First U.S. lüeb Page at the Stanford
Linear Accelerator Laboratory, December 2001
Netlmpact Annual Conference at l(enan-Flagler Business School, November 2001
American Council of the Blind Annual Convention, July 2001
Brinsins Sociallv Benefìcial Technolosv into the Service of Humaniw. EE380 at Stanford
University, April 2001
Information Technology in the Service of Human Rights at the Computets, Freedom and
Privacy Conference, March 2001
Rank Prize Fund Symposium, Grâsmere, England
Guest Lecturer for CSUN program in disability leadership
Ptofessional Associations
¡
r
¡
¡
Association for Computing Machinery
Institute of Elecuical and Electronics Engineers
American Association for the Advancement of Science
Social Entetprise Alliance
Awatds and Public Service
.
¡
r
Head of Benetech Delegation, Diplomatic Conference to Conclude aTrczty to Facilitate
Access to Published Works by Visually Impaired Persons and Persons with Print Disabilities,
Wodd Intellectual Property Organization, Marrakesh, Morocco (2013)
Member, Global Agenda Council on Measuring Sustainability, Wodd Economic Forum
(20r2-20r4)
Member of the Board of Directors, ZeroDivide, foundation investing in community
enterprises that leverage technology to benefìt people in low-income and other underserved
communitie s Q007 -201, 3)
I
r
.
Cornmissioner, Federal Advisory Commission on Accessible Instructional Materials in
Postsecondary Education for Students with Disabilities, 2010-2071
Duke University, CASE Award for Enterprising Social Innovation, 201L
Bdgham Young University, Center for Economic Self-Reliance Social Innovator of the Year,
2009
!
AT&T Technology Innovation Award from
the Alliance for Technology Access, March
2008
I
r
r
¡
¡
I
I
I
¡
r
r
¡
r
¡
¡
.
I
r
¡
Strache Leadership Award from the Califomia State University, Northridge,2007
John D. and Catherine T. MacArthur Foundation Fellowship,2006
Technical Advisory Committee Member, National Instnrctional Materials Accessibiliry
Standard, U.S. Department of Education (2005-2008)
Advisory Committee Member, National Instructional Materials Accessibility Center, U.S.
Department of Education (2006-ptesent)
Skoll Award for Social Enuepreneurship ,2004 and 2006
Fast Company Social Capitalist Award: Top 20 Groups Changing the World, 2004
Laureate, The 2003 and 2001 Tech Museum,twards
,{,metican Library Association Francis Joseph Campbell Award, 2003
Schwab Foundation Outstanding Social Entrepreneur of 2003 Award
Member, the Community Partnership Committee, which oversees a diversity and disability
agreement with SBC, Inc.
Runner-up, Yale-Goldman Sachs National Nonprofit Business Plan Competition, 2003
American Foundation for the Blind Access Award, 2003
Robert
S. Brav
Award. The American Council of the Blind
Winner, Education Category,2002 Stockholm Challenge
Fast 50 Chamoion of Innovao'on2002
Tudse.2002 National Social Venture Comoetition
Member, Board of Directors of the Social Enterprise ,l,lliance (2000-2010, chair 2008-2010)
Membet of the Advisory Board, Telecommunications ,{.ccess Rehabilitation Engineering
Research Center, a joint effort of the Trace R&D Center of the University of WisconsinMadison and the Technology Access Program of Gallaudet University, 2001
Panelist, National Science Foundation Small Business Innovation Research Program, L998,
2000,2003
r
¡
I
r
Participant, 1998 NSF Wotkshop fot Discussing Research Priorities and Evaluation
Strategies in Speech Synthesis, August, 1998
Member, Electronic Infotmation and Technology Access Advisory Committee, a federal
advisory committee responsible for drafting federal acquisition standards for accessibility
under Section 508, 1998-1999
Member, Telecommunications Access Advisory Committee, a federal advisory committee
responsible for making recommendations to the U.S. Access Board and Federal
Communications Commission on implementing portions of the 1996 Telecommunications
Act,1996-1997
U.S. Patent Number 5,470,223: System and Method for Tracking a Pedestrian
I
.
Finalist, 1996 DiscoverMagazine A,wards for Technological Innovation
1996 Access Award, American Foundation
for the Blind
Maiot Wotks and Ateas of Expertise
r
Founder and CEO of Benetech, a highly innovative nonprofit company focused on using
the power of technology to address social needs in areas such as disability, literacy, human
rights and the environment.
r
Founder of Arkenstone, Inc., a leading nonprofit organization ptoviding adaptive
technology for education and employment for people with disabilities and the largest maker
of reading systems for people with blindness, vision impairment and learning disabilities.
Developer of the Arkenstone Reader, the first affordable teading system for the blind.
Designer of Open Book, the Frst talking Windows program for the blind. Co-inventor of
Atlas Speaks, the fìrst accessible map software for the blind, and of Suider, a talking GPS
locator for the blind.
r
Cofounder of R {,F Technology- Inc., the nation's leading company in optical character
recognition technology for processing forms in postal and medical applications. RAF's
software is used to route the United States mail.
r
Cofounder of Calera Recognition Systems, Inc., the first company to develop omnifont
optical character recognition that works without uset training.
Exhibit B
Documents, Facts, or Data Considered in Forming My Opinions:
o
o
o
o
o
The Bookshare website, at www.bookshare.org/cms
o
Test Access: Making Tests Accessible
The Public.Resource.Org website, at www.public.resource.org
The American Printing House for the Blind website, at www.aph.org/
The Learning Ally website, at www.learningally.org
The National Library Service for the Blind and Physically Handicapped, Library
Congress website, at http://www.loc.gov/nls/catalog/?loclr:blognls
of
Students with Visual Impairments: A Guide for
Test Publishers, Test Developers, and State Assessment Personnel, Second Edition, by
for
Carol B. Allman, Ph.D., published by the American Printing House for the Blind, and an
online resource published by the American Foundation of the Blind, available at
www.aph. org/tests/access2
o
o
¡
o
o
o
o
Building Assessment Initiatives þr Schools: Guidelines to Support the Contract
Development Process Between Test Publishers and States, available at
www.afb.org/info/afb-national-education-program/jltli-2005-educationsummary/checklist-for-rfp-bu i ldin g/2 3 5
Window-Eyes screen reader software
ABBYY FineReader optical character recognition software
The 1999 Standards in image only PDF format, as provided to me by
Public.Resource.Org
The 1999 Standards in TXT format, as provided to me by the Internet Archive
The deposition of Christopher Butler of the Internet Archive, December 2,2014
The deposition of Carl Malamud of Public.Resource.Org, May 12,2015
f, ilqlqxf,
å
/
tlc(ess
Making Tests Accessible for
Students with
Visual Impairments:
A Guide for Test Publishers,
Iesf Developerq and
State Asses sme
Large Print
oo
OO
oo
Braille
nt Perso n n el x
)))
Audio
TEST ACCESS:
Making Tests Accessible
for Students With
Visual Impairments
A Guide for Test Publishers,
Test Developers, and State
Assessment Personnel*
Fourth Edition
Carol B. Allmâr, Ph.D.
Pu blished
by
American Printing House for the Blind
Louisville, Kentucky
July 2009
ffh
AMERICAN PRINTING HOUSE
FOR THE BLIND, INC.
*Book Number two in the TEST ACCESS Series, promoting
accessibility of testing materials for persons who are blind or
visually impaired
@2OO4, 2OO7, 2OO8, 2OO9 American Printing House for the Blind,
Inc. With the exception of ETS Guidelines for a Test Reader, which is
material presented in Appendix G, this document may be copied in
whole or in part and distributed free of charge for educational and
nonprofit use as long as appropriate credit is given to the author and
publisher, and the "Work in Progress" notice is included on each copy.
No other use of this material is allowed wíthout written permission.
Work in Progress: This document represents a set of guidelines
for making tests accessible to students with visual impairments.
These guidelines are a "work in progress" and will be routinely
updated and revised as additional information is collected and
research results are learned. Please address questions, concerns,
and suggestions regarding these guidelines to the director of APH's
Accessible Tests Department at L-8OO-223-1839 or e-mail them to
tests@aph.org.
Disclaimer: Web links in this document were current as of the
date of publication, but may have become deactivated or modified
since then. These links are for informational purposes only and do
not constitute an endorsement or approval of policy, views,
products, or servíces of the publishing organization.
Preferred Citation:
Allman, C. (2009). Making Tests Accessible for Students with Visual
Impairments: A Guide for Test Publi-çhers, Test Developers,
and State Assessment Personnel. (4'n edition.) Louísville, KY:
American Printing House for the Blind. Available from
http://www.aph.org.
Trademarks are of their respective companies
American Printing House for the Blind
1839 Frankfort Avenue
P.O. Box 6085
Louisville, KY 40206-0085
Toll Free: 800-223-1839
Fax: 502-899-22L9
www.aph.org
^t
TABLE OF CONTENTS
Acknowledgements
V1
ntrod uction
General Guidelines for Accessible Test Formats
1
5
Braille and Tactile Graphics
11
I
Large Print Test Formats and Graphics
Uses of Color for Signage, Graphics, Text, Tests and Power Point
Presentations to be Viewed by Persons Who Are Color Blind or
.....26
Color Vision Deficient......
Guidelines for Audio Versions of
Tests
Guidelines for Oral Reading or Signing of a Test
Accommodations in Testing Students with Visual
Impairments
Guidelines for Braille and Large Print Test Response
.... 29
......32
....... 35
40
Transcription
Reporting Test Results of Students with Visual Impairments....41
...43
Alternate Assessments...
References
46
Resources
51
Appendix A: Braille versus Auditory Access: A Discussion...
Appendix B: Template for Test Administration Braille Tests
Appendix C: Template for Test Administration Large Print
54
Tests
59
Appendix D: Position Paper: Use of an Abacus in Test-Taking
Situations
61
Appendix E: Position Paper: Use of Extended Time
Appendix F: Position Paper: Accommodations for Testing
Students with Visual Impaírments
Appendix G: EIS Guidelines for a Test Reader.
V
57
63
67
78
ACKNOWLEDGEMENTS
The American Printing House for the Blind (APH) and the author
wish to acknowledge and express appreciation to all the individuals
who contributed information, guidelines, feedback, editing, reformatting, and graphic design expertise in order to develop and
make this document available.
VI
I NTRO D U CTIO N
Purpose of Document
The American Printing House for the Blind (APH) is committed to ensuring
that educational materials are accessible to students with visual
impairments. Students with visual impairments include those with some
usable vision, as well as students with no usable vision. This document is
provided as a guide for making tests accessible in tactile, large print, and
audio formats. It is anticipated that this guide will be used as a tool for
implementing appropriate guidelines as test publishers, test developers, test
editors, and state assessment personnel are developing and adapting tests
and assessments. Prior planning using the contents of this document will
help ensure that tests are accessible and will reduce the need to retrofit a
test. Questions concerning this document, the specific guidelines, or resources discussed can be addressed to APH's Accessible Tests Department at
L-8OO-223- 1839 or tests@aph.org.
Federal and State Mandates
Federal and many state laws requíre that all students be assessed through
state assessment procedures using the appropriate accommodations. In the
school year 2005-2006, states were required to assess all students'
progress annually in mathematics and reading in grades 3-8 and once in
grades 9-L2. In addition, by 2OO7-2008, states were required to assess all
students' progress in science, at least once in grades 3-5,6-9, and 10-12.
Assessment results of all students must be reported publicly, and these
results are to be incorporated into the state's accountability plan.
Satisfying these federal requirements necessitates careful attentíon to
making tests accessible for all students. Students with visual impairments
have some unique communication needs that must be addressed as tests
and assessments are made accessible for them. Converting test items into
braille, tactile graphics, large print, or audio format fails to guarantee that
the items are accessible. For example, test items that instruct the student
to "draw the results of the following" or "write a story based on the
picture" are not truly accessible to braille readers. Other examples of this
misconception are discussed within each media-specific section of this
document. Careful planning during test development can help ensure that
tests are accessible, while maintaining the rigor intended.
L
Expectations for students with visual rmpairments
If students with visual impairments are to participate effectively in state and
national testing programs, they must have opportunities to learn academic
skills that will be assessed. These opportunities often are overshadowed by
special skills training to such students, who may miss all or part of academic
classes in order to obtain the skills essential for using braille, assistive
technology, and/or independent living skills, including orientation and
mobility.
In addition to providing the training of special skills, school personnel must
be aware of each student's need for instruction in all academic areas. This
may entail extended days or school years or supplemental instruction by
other agencies that serve students with visual impairments.
Students with visual impairments must spend their educational time
working toward academic content standards, learning special skills needed
for independent living, exploring appropriate media for access to printed
material, and evaluating ways of communication that are effective for them.
Access to printed material may include braille, tactile graphics, regular print
with magnification devices, large print, the use of a human reader, auditory
access, or technology access that provides braille, print, or auditory information. No single method will work for every student, in all situations.
Educational personnel must ensure that students are exposed to and have
opportunities to try all options of access. A student's communication mode
must be based on what works for him or her. see Appendix A for a
discussion of braille versus auditory access.
School personnel must maintain high expectations for the education of
students with visual impairments. If opportunities to learn are present Ín
the curricula, students will have the experiences needed to learn difficult
skills such as map and graph reading, production of graphs and charts,
reading technical materials, or computation of advanced mathematics.
Students cannot be denied their right to learn difficult skills just because
they have a visual impairment. It is these more difficult skills that are being
assessed routinely on most state and national assessments of student
progress in reading, mathematics, and science.
2
Universal Design
In the construction and administration of tests, the process of universal
design helps to ensure accessibility for a multitude of students. Universal
design provides the widest range of students the ability to demonstrate
adequately their skills and knowledge. This process should retain the
validity of inferences drawn from the test results.
The concepts of universal design apply to instruction as well as assessment.
During instruction, universal design enables investigating appropriate
methods, practicing skills and knowledge using appropriate methods,
experiencing trial and error to determine the best methods, and
discovering the success of knowledge and skills learned using the best
methods for each individual student. During assessment, universal design
becomes the process of ensuring that the majority of students can
demonstrate their knowledge and skills. Both aspects of learnÍng,
instruction and assessment, are dríven by the standards of each individual
state.
To ensure that an assessment system is fair and accessible to all students,
states are required to document how they include the principles of universal
design in the item review process. Generally the principles of universal
design include (Thompson & Thurlow, 2OO2, Thompson, Johnstone, &
Thurlow, 2002):
Attention to an inclusive assessment population,
Constructs, including content and cognitive complexity, that are
precisely defined either through states'standards or the test item
specifications,
Accessible test items, as determined by item writers and review teams
that include personnel familiar with various media (braille, tactile
graphics, large print, regular print, and audio),
Non-biased test items, as determined by item writers and review
teams,
Test formats, response options, and scoring policies that are amenable
to various approved accommodations needed by students,
Simple, clear, and intuitive instructions and procedures,
Comprehensive and relevant language that provide needed distracters
in test item foils but are not designed to confuse the student, and
Maximum legibility of print formats including formats that are free from
clutter and void of grayscale.
3
Suggestions for applying universal design to item development include:
a
a
a
a
a
Ensuring that test item writers are trained in concepts of universal
design,
Providing test item writers and reviewers with construct and constructrelated information during the construction and review of test items,
Examining each test item for universal design principles(linguistic
complexity; cognitive complexity; formatting; bias issues; modalities
of braille, large print, and audio; and response formats to be allowed),
Recommending allowable accommodations for test administration,
Re-examining all test items for fidelíty to the construct, and
Field testing all test items wÍth intended populations.
The principles of universally designed assessments are the basis for many of the
guidelines provided in this document. Additionally, the references listed at the
end of this document have been written by individuals involved in ensuring the
accessibility of materials for students with visual impairments. This document
describes guidelines that support braille, tactile graphics, large print, and audio
production of test items.
4
GENERAL GUIDELINES FOR
ACCESSIBLE TEST FORMATS
Students with visual impairments may require testing materials in regular
print, large print, braille, tactile graphics, audio formats, or some
combination of these formats. The provision of a test and related materials
in braille, large print, or audio provided an individual student should be
based on the medium used by the student, as identified in the
Individualized Education Program (IEP) document. Alternate format tests
should be used only by students who use that medium to access printed
textbooks and other instructional materials.
Students with visual impairments can be, and must be, made part of the
state's assessment program through use of accommodations that allow
them to demonstrate their knowledge and skill acquisition, as outlined in
each state's standards and assessment system specifications. Regardless of
the media chosen, students may need access to special materials such as
braille paper, bold line writing paper, talking calculators, abacuses, raised or
bold line rulers, braillewriters, slates and styluses, word processors, or other
materials and devices. A more thorough discussion of accommodations is
provided in the section on Accommodations in Testing Students with Visual
Impairments and in Appendix F.
The following general guidelines are recommended for all formats that are
developed for accommodating students with vísual impaírments. Various
aspects of test construction and implementation are addressed in this
section.
Contract Development
1.
2.
3.
Contracts between states and test publishers/producers must include
provisions for state approved alternate media (braille, large print,
audio editions of tests, and scripts for oral presentation of tests)
including answer sheets and practice tests.
Test publishers need to have the capability of providing the test
administration manual in braille, large print, or audio for test
administrators who are visually impaired and need accessible media.
The contract should state if test administration manuals are needed in
accessible media.
Contracts must include timelines for development, proofreading,
revising, and production of braille, tactile graphics, large print, and
audio test formats and accompanying practice materials.
5
4.
5.
6.
Contracts regarding accessible media should guarantee that each
medium of test materíals and practice materials is produced by the
same entity to ensure consistency in format and graphic production
techniques. Every effort should be made to ensure consistency of
presentation from one year to the next, and from one level of the
test to the next.
Contracts may need to include specifications on tools and materials
that need to be developed or provided to test takers using alternate
media, e.g. a braille ruler, a tactile or large print protractor, or
periodic table of the elements, real money for money related test
items, or some actual objects such as a ball or cube.
Contracts may need to include plans to ship special versions of tests
separately from regular print versions so that distribution of the
accessible formats occurs in a timely manner.
Test Development
1.
2.
3.
4.
5.
Test development must ensure that test score inferences reflect
intended constructs and not disability characteristics (AERA,
2000).
The construct to be measured must be specified in documents and
made available to test item writers and reviewers and to accessible
media producers.
Availability of item specifications is essential in determining
appropriate accommodation use and in the reproduction of test
items to be presented in braille, tactile, large print, or audio
formats.
Test publishers must maintain access to experts in the media of
braille, tactile graphics, large print, and audio, who can provide
information concerning test development and transcription and
tactile graphic design, and who are able to proofread test materials
before mass duplication, and otherwise ensure that materials are
provided in a timely and accurate manner. Proofreading the braille,
large print, or audio version of the test before multiple copies are
made confirms that the material is readable and that the adapted
test follows the print copy in numbering and lettering of test items
and answer choices, and that the graphics are readable and located
correctly. The proofreader must also check for proper formatting.
Validity issues concerning all accessible formats and accommodation
needs should be discussed during test development (Phillips, 1994).
The provision of a test in accessible media should be considered a
valid accommodation as long as it retains the construct that the test
was designed to measure. If a performance item requires drawing,
consider allowing an explanation or description as a valid response
6
6.
7.
B.
opt¡on. If such a revision is allowed, scoring criteria must include
information on this option.
All directions on a test should be worded to allow for alternate
response methods. For example, use of directions like "circle the
answer" should be replaced with "indicate or mark the answer."
Specific auidelines on any test format changes, allowable
accommodations (including time allowances), and general
assistance that can be provided to the student must be stated in
the test administration manual or supplemental administration
materials.
Test item development and review committees should be made aware
of alternate media issues regarding the use of either complicated or
nonessential pictures and graphics.
Item Development and Review
1.
2.
3.
4.
5.
6.
7.
L
Educators with specialization in the field of visual impairments
must be included in the test item development process.
All test items must be reviewed by persons familiar with visual
disability issues to ensure that no test item is biased or
discriminatory toward persons with visual impairments.
It is recommended that as much information as possible be
included in the text of a test item. This will help prevent the
introduction of pictures that contain information necessary for
selection of the correct answer, but which cannot be adequately
brailled, presented in large print or tactile graphics, or described in
audio format.
In general, use of "vision specific" language can be maintained,
e .g., "Look at the following list of animals."
The test item pool must be large enough for bias and item review
committees to replace items determined to be biased or inaccessible
in braille, large print or audio formats, or tactile graphics.
A representative sample of students with visual impairments
must be included in any field-testing of the assessment, as
prescribed in Standard 10.3 (p. 106) of the Standards for
Educational and Psychological Testing (L999).
All practice materials must be provided in accessible format at the
same time that print practice materials are provided. Allow
sufficient time for accessible format preparation.
Provisions should be made to conduct item analyses for accessible
format test items.
7
Accessible Test Development
1.
2.
3.
4.
5.
6.
7.
B.
9.
To ensure that quality materials are developed, state assessment
programs should contract with an agency or persons experienced
in producing braille, tactile graphics, large print, and audio formats.
If a multimedia presentation is to be used by test takers, it is
important that the accessible media producer(s) coordinate presentation of the test items between each of the media.
Production of the alternate format test includes the editing, transcription, reformatting, design, and proofreading of the
alternate media.
Holding a conference call with all parties involved before the
accessible media producer begins to review/ edit the test items helps
to maximize a successful experience and end product.
The name and phone number of the customer's primary contact
person needs to be provided to the producer of accessible media
to facilitate timely production.
Accessible format producers will need access to a primary contact
person, as well as item specifications that include information
about the skíll and construct being assessed.
Test items should be deleted or substituted only if the item cannot
be provided in braille, tactile graphics, large print, or audio format
without significantly changing the item and the intent of the
question. Although not recommended, some test items may need
to be omitted if they are not adaptable as determined and advised
by item reviewers with expertise in the format under consideration.
The deletion or substitution of items should happen infrequently,
particularly if educators with specialization in visual impairments
have been involved in the item development process. Attention to
universal design during test development will also reduce the
probability that a test item will have to be deleted.
If items are omitted in alternate versions, the test score
must be rescaled so that braille, large print, and audio format users
are not unfairly penalized and so that scores can be obtained for
diagnostic and accountability use. The original numbering system
should be maintained and the word "omitted" inserted in place of any
item that had to be omitted.
Responses from the primary contact person regarding questions and
requests for substitutions require a quick turn-around time in order to
ensure accuracy and timeliness of delivery of accessible media.
Substituted items should assess the same skill and have equal
value and validity. Substituted items must maintain the correct
answer in the same position as that of the original test item.
8
10. All field test items and sample questions must be included in
accessible format test versions.
11. Test contracts must indicate preferred publication strategies, such
as brailling on both sides of the braille paper (referred to as
interpoint braille), preferred methods of producing tactile graphics,
and binding of the braille test materials. Assistance with determining
these specifications is available from APH.
L2. Braille tests are generally produced using contracted braille, the
typical method for producing braille in which short forms of words are
used. If the test is for a young child, a new braille reader, or
someone struggling to learn braille, a test may be needed in
uncontracted braille, whereby every letter of every word is
represented by an individual braille cell.
13. The format of an accessible media test edition must follow the
print format as much as possible. That is, ideally the number of
test items and test sections should match that of the print format,
as should the order of the test items and test sections. Deviations
from the print version of the test must be outlined in a print copy of
Test Administration Notes for the altered format. Test Administration
Notes must include reference to print versions wíth associated
accessible format page numbers, identify passages and items by page
(print and alternate format), and provide indication of any changes
made to the alternate format. Appendices B and C contain templates
for creating Test Administration Notes for braille and large print
formats.
14. Special requirements, such as an independent proofreading of test
materials, exact print reproductions of the braille/tactile test items,
or any print labels to be included on braille or tactile graphics need
to be considered and included in the contract.
15. APH's policy in accessible test production includes close collaboration with, and approval from, test publishers and content
specialists to ensure that edited items are acceptable as edited.
16. Test security and confidentiality standards must be upheld during
the process of developing accessible formats.
Test Administration
1
Computers and adaptive technology, electronic note takers,
cassette player/recorders, the cassettes, CDs, etc., must be
inspected for proper functioning prior to their use during a test.
The test administrator or proctor should be instructed on how to
proceed if equipment fails or malfunctions during administration of
the test.
9
2.
3.
4.
5.
6.
Each test administrator or proctor of a student using an alternate
medium test or a combination of media should be assigned a
testing packet that includes a list of materials needed (approved
technology or other manipulatives, such as a talking calculator,
braille or large print ruler, braille paper, bold line writing paper,
raised line graph paper, etc.)
The test administrator or proctor must ensure that special
tools and materials noted on the student's IEP and used for
instructional purposes as accommodations are available, as
needed, to students in the test-taking environment. For example,
if a visually impaired student routinely uses an abacus in the
classroom when sighted students are allowed to use a pencil and
paper for computational purposes, then an abacus must be
available during a test. See Appendix D for further explanation on
the use of an abacus in test-taking situations. Specialized tools
and materials should not be provided if their use presents an
unfair advantage.
In preparation for test administration, the test administrator needs
to review the original test(s), the alternate format/s of the test/s,
the original test administratíon manual(s), the test administration
manual/s for accessible media, and the test administration notes
for the special format/s. These materials should be provided to the
test administrator under secure and confidential means two full days
prior to test administration. This time is needed so the test
administrator can plan appropriately for administration of the test(s)
in alternate media.
Prior to testing, the test administrator or teacher must ensure
that the test is available in a student's primary or preferred reading
medium or combination of media, and that the student has sufficient
proficiency in use of this medium and related tools such as
computers, assistive devices, CD players, or braillewriters.
If students are expected to bring select tools and materials to the test
environment, they need to be notified of this ahead of time.
10
BRAILLE AND TACTILE GRAPHICS
The information in this section describes methods for developing and
implementing assessments for students with vísual impairments who
require braille text or tactile graphics. While some technology provides
auditory access to print, braille is critical to literacy and must be an
option for those students who routinely use it. See Appendix A for a
discussion of braille versus auditory access.
Generally, learning to read braille is no more difficult than learning to
read print. The tactile process is different from the visual process and
creates the following considerations:
Braille (tactile reading) consumes more time than does visual
reading, as students who read braille typically read at fewer words
per minute than do students who read print (Trent & Truan, L997),
and
Braille reading requires tactile training in page orientation and
reading and interpretation of tactile graphics.
Designing Tactile Graphics
The following are aspects of test items that need special consideration
when revíewing and designing for production as tactile graphics:
Complicated graphics that contain multiple layers or pages of
information
Three-dimensional objects from a particular visual perspectivê, ê.g.,
a top view of a house or pyramid
Rotation items that use letters of the alphabet (print letters
rotated or flipped)
Science items that use pictures to demonstrate
experiments and other scientific concepts or processes (cell,
digestive or muscle systems, etc.)
Map reading items that depend on visually recognizable and
unlabeled continents, countries, or states, e .g., Africa, Italy, or
Florida
Visual recognition ítems (interpreting a picture without supporting
text)
Items that require interpretation of complicated drawings, ê.g.,
cross-sectíons of d iagrams
Optical illusions
IL
These types of items frequently require extensive revision during the
production process. For example, a text-based description in addition to
the tactile graphic may be needed. The accessible media producer may
request a test publisher to substitute such items with those that can be
made more accessible and which will retain similar, if not identical,
concepts and have the same weighted score.
Braille Translating (Transcr¡ption) Process
Consideration of the following points will facilitate the production of test
materials in braille format for students with visual impairments:
1.
2.
3.
4.
Test developers and publishers must ensure that contracts for braille
materials specify the use of braille transcribers who are certified by
the National Library Service (NLS), experienced at transcribing tests,
and knowledgeable of braille formats. Braille formats must be
modeled after those of the Braille Authority of North America (BANA)
Guidelines, found in Braille Formats: Principles of Print to Braille
Transcription, L997.
As a test is edited for braille transcription, necessary changes will
be made to make the material accessible to braille readers.
Correct braille transcription also requires that BANA specifications
be observed. Simplification and/or labeling of some graphic material
will likely be necessary. Simplification entails the elimination of some
artistic features, removal of some superfluous material (without
eliminating distracters and other text material that is necessary to
maintain the validity of the test item), or movement on the braille
page of some text or graphic components for more efficíent
readability by the braille reader (moving a scale, legend, or compass
rose on a map to a different location). Even simple tactile graphics
can be very difficult to interpret; some additional labeling may be
needed for the test taker to read and understand the tactile graphic.
Note that simplification and labeling are done relative to the
construct being tested. If during the test editing process, it is not
clear what is being tested, the test publisher will be consulted for
cla rification.
Reproduced references, such as tables of content, dictionary pages or
indices, may need to be shortened in the braille test version while
maintaining correct answer choices and foils. This is done to contain
the braille item to one pâ9€, if possible.
Provision of open-ended items in braille format must indicate to
the braille reader the amount of space provided for the answer.
Directions must specify the space provided by suggesting the time
L2
needed to complete the item or by indicating the approximate
page area or the number of lines or paragraphs. Generally, one page
of print is equal to about two pages of braille unless graphics are
involved which will add to the page length. Directions may indicate
that there are four print lines or eight braille lines available for
responding.
Unnecessary boxes and framing of material may be omitted
unless the framing provides a separation of graphic material from
text or encloses a group of scattered or randomly placed objects.
Specific braille codes exist for transcribing literary works,
mathematics, and science materials into braille. When brailling the
5.
6.
content of the print version of the test, braille transcribers must
follow the standards of code for braille transcription. These codes
are provided in English Braille American Edition, 1994; The Nemeth
Braille Code for Mathematics and Science Notation, 1972 Revision;
and Braille Formats: Principles of Print to Braille Transcription, 1997.
All three manuals are available from the American Printing House for
the Blind.
7. An experienced braille proofreader must be utilized for proofreading
all materials and, in particular, examining all tactile graphics to
ensure readability and accuracy.
8. Experienced braille readers might also need to transcribe students'
braille responses into print for scoring. See the section on Guidelines
for Braille and Large Print Test Response Transcription.
L Braille versíons of a test may include transcriber's notes (notes to the
braille reader from the braille transcriber about the use of special
symbols, and use of any special formats). Transcriber's notes must
be written in print within the Test Administration Notes for Braille
Edition (Appendix B). The number of transcriber's notes in tests
should be kept to a minimum.
10. Test security and confidentiality standards must be upheld by braille
test transcribers, tactile graphic designers, and proofreaders. This
includes the following:
a
a
o
Keeping testing materials in a secure place to inhibit access by
unauthorized persons,
Not sharing information or implying content contained in the
testing materials with other persons,
Maintaining discretion about the work being performed,
Returning all materials to the contracting source, and
Maintaining confidentiality of test content.
13
Tactile Graphics
This section offers information regarding the use of tactile graphics when
testing students with visual impairments. Graphic material, which
includes maps, charts, graphs, diagrams, and íllustrations, frequenily
contains information that is difficult to present in a tactile format.
Research supports the use of tactile graphics and "the idea that visual
experience and visual imagery are not required for the perception of
simple tangible pictures. ." (Heller, et al.,2oo2, p.349). It is possible
to provide many types of graphic material in braille or raised line
drawings. However, certain types of graphic materials either cannot be
provided in braille or tactile formats, or they are so complex that doing
so produces a graphic that cannot be read and interpreted by the test
ta ker.
Most maps, charts, graphs, and diagrams can be translated into tactile
form if the test publisher will allow some editing. Editing could involve
eliminating the shading used solely for visual effect, reducing the number
of distracters, providing two or three charts to present the same
information as one complex print chart, using text based descriptions to
supplement or replace graphics, or using symbols and words with a key
to provide information. Edits needed to convert print graphics to tactile
graphics need to be approved by test developers or publishers.
Most print materials use graphics to emphasize a point, provide another
format for information, or provide visual appeal. Because graphics are
common in text, training in reading graphic material and interpreting a
written description of a graphic are important skills for the student with
a visual impairment to learn. Guidelines for tactile graphic materials are
described on the next few pages in terms of general guidelines, design,
symbols, lead lines, labels, and indicators and scale.
General Guidelines:
1.
2.
3.
Graphics in mathematics tests must follow provisions of The
Nemeth Braille code for Mathematics and science Notation, 7972
Revision, BANA (1983).
Decide if a tactile graphic is needed. omit the graphic if it is
purely decorative. consider using a text based description to
either supplement or to replace all or part of a graphic.
Graphics should be tactually clear and contain only relevant
information based on an understanding of what is being taught
1,4
4.
5.
and what the student's task is. Visual information that is
irrelevant to the meaning or purpose should be omitted.
Graphic material should be simplified without omitting needed
information or creating an unfair advantage by alluding to the
answer.
Picture descriptions should be presented concisely within the
student's test booklet if information in the picture is vital to
answering one or more test items.
6. Picture descriptions will appear as needed in transcriber's notes
in appropriate places throughout the test and must be included in
the Test Administration Notes.
7. Some graphics are best handled by supplementing the image
with a heading, label, description, or key. Edits must be made
carefully so that the braille reader is not unintentionally given an
advantage or cue to the correct response.
8. Consider splitting complex graphics into separate drawings
showÍng layers of information, unless this adds complication for
the test taker.
9. In general, use texture to add information and draw attention to
select parts of a tactile graphic.
10. When necessary, to avoid confusion and accentuate important
information, use different areal symbols (texture) to differentiate
between bodies of water and land on maps.
11. Charts and graphs should be confined to one page when possible.
If graphics and the accompanying test item require more than
one page, use facing pages to present graphics and the
accompanying test item if possible.
12. If a braille test taker is asked to produce a graphic as part of the
test item, such a task can be achieved through the use of tactile
graphic materials that are familiar to the braille reader. Another
option that may be acceptable to test developers is for the
student to describe or explain data or other information. This
option must be approved by the test contractors and included in
the scoring criteria. The test administrator and the braille
reader's teacher, using the braille reader's current IEP, must
collaborate prior to the administration of the test to ensure that
appropriate materials are provided. For the purpose of scoring,
student-produced graphics will need to be hand-scored or
transcribed into a print graphic by persons familiar with braille,
braílle readers, and the content area being tested. See section
on Guidelines for Braille and Large Print Test Response
Transcription.
L5
13. An experienced braille reader must proofread all tactile graphics
prior to mass production of the braille test to ensure readability
and accuracy.
(Kapperman, G., Heinze, T. & Sticken, J., 2000; Poppe, K. & Otto.,
B. & Robinson, M. C.,2OOO; Spence, D. & Osterhaus, S., 2000)
F.
2OO2; Ross, D.
Design:
1.
2.
3.
4.
5.
6.
7.
Avoid clutter and simplify.
"clutter" occurs when different symbols and lines are so crose or so
similar that they become hard to distinguish. spacing is the key to
avoiding clutter.
Symbols and lines closer than 1/+" may be difficult to differentiate,
depending on the medium and tools being used.
Shapes with sides less than 1/2" long may not be recognizable.
Use different textures for lines so that test takers know which part of
a line to follow when two or more line segments cross or meet.
"Simplify" means to eliminate unnecessary elements of the origínal
picture. Focus on the relevant parts and omit details that are purely
decorative or distracting.
when the print picture includes people, animals, objects, etc., replace
them with simple lines, symbols, and/or labels (e.g., use the label
t'hand" instead of drawing a hand or
use a triangle instead of a cat or
dog).
Symbols (Lines, Poinþ and To