J.T. Colby & Company, Inc. et al v. Apple, Inc.

Filing 135

DECLARATION of Claudia T. Bogdanos in Opposition re: 120 MOTION in Limine to Exclude any Testimony, Argument or Evidence Regarding the Expert Reports and Opinions of Susan Schwartz McDonald.. Document filed by Ipicturebooks LLC, J.Boyston & Company, J.T. Colby & Company, Inc., Publishers LLC. (Attachments: # 1 Exhibit A, # 2 Exhibit B, # 3 Exhibit C, # 4 Exhibit D, # 5 Exhibit E, # 6 Exhibit F, # 7 Exhibit G, # 8 Exhibit H, # 9 Exhibit I Part 1, # 10 Exhibit I Part 2, # 11 Exhibit J, # 12 Exhibit K, # 13 Exhibit L, # 14 Exhibit M, # 15 Exhibit N, # 16 Exhibit O, # 17 Exhibit P, # 18 Exhibit Q, # 19 Exhibit R)(Chattoraj, Partha)

EXHIBIT P

Panel Quality
Our Values

Introduction
This document has been prepared by
Research Now to provide some of the
essential information needed by those
considering using online access panels.
It aims to follow ESOMAR’s 26 questions
to ask of a panel provider but also provides
answers to some of the wider questions
currently posed by international
organisations looking at quality control.
For more detailed information on any
aspect of Research Now’s online fieldwork
and panel capabilities, please contact our
panel excellence team:
Email: [e-mail]

Contents
Company Profile
Sample Source
Panel Recruitment
Panel and Sample Management
Policies and Compliances
Partnerships and Multiple Panel Membership
Data Quality and Validation

Company Profile

1. What experience does your company have with providing online
samples for market research?
Research Now has been providing high quality,
proprietary, research-only online panel sample since 2001.
We are an independent, single source for international
online fieldwork across Europe, Middle East, the Americas
and Asia-Pacific.
We manage around 2000 projects per month, providing
both sample-only and full-service studies.
These are executed by our industry-leading programming
and project management teams. We are experienced,
research-literate, multi-lingual online fieldwork specialists
working with our clients across 22 offices worldwide.

Sample Source

2. Please describe and explain the types of the source(s) for online sample
that you provide (are these databases, actively managed panels, direct
marketing lists, web intercept sampling, river sampling/other?)
All of our panels are actively-managed online access
panels that include the Valued Opinions panels and the
e-Rewards opinion panels. The panels are proprietary
and built from scratch. All panels are localised, not just
translated, with native language panel support and
country-specific reward choices.
We utilise multiple source recruitment. Our Valued
Opinions panels are recruited by email and online
marketing with over 300 diverse online affiliate partners
and targeted website advertising.
Our e-Rewards opinion panels utilise a “by-invitationonly” methodology, partnering with a diverse set of
globally recognised consumer and business-focused
brands such as AIRMILES, Best Buy®, BLOCKBUSTER®,
Air France/KLM, Borders®, British Airways, Continental
Airlines®, Delta Air Lines®, Hilton HHonors®, Macy’s®, Pizza
Hut®, Virgin Atlantic Airways, to name a few, that invite
their customers to join the e-Rewards opinion panel.

3. What do you consider to be the primary advantage of your sample
over other sample sources in the marketplace?
Our panels are research-only, wholly owned and built in
a consistent manner to ensure that they are responsive,
balanced and reliable – the foundations for quality research.

4. If the sample source is a panel or database, is the panel or database
used solely for market research? If not, please explain.
Yes, absolutely. Panels are used for market research
purposes only. This applies to all of our proprietary
panels across the world without exception.

5. How do you source groups that may be hard-to-reach on
the internet?
Due to the fact that we have large panels we can drill down to
incidences as low as 1-2%. To facilitate this process, our programme
of pre-screeners collects information for niche sample targets such as
Automotive and Healthcare.
For our e-Rewards business panels we utilise a “closed” or
“by-invitation-only” online panel recruitment method, a method of
exclusively inviting pre-validated individuals or individuals who share
known characteristics to enroll into our market research panels.
We tailor our recruitment partner strategy specifically to target
groups that are not already represented in our panel.

6. What are people told when they are recruited?
When people join our panel, they are asked to join an online market
research panel which is research-only, with no selling. Panellists
are rewarded for taking part in surveys with a structured incentive
scheme. They are supported by a panel support team with an option to
unsubscribe at any time. Terms and conditions can be viewed at
each panel web site, for example www.valuedopinions.co.uk or
www.e-rewards.co.uk.
Panel management is compliant with all relevant market research
industry standards, data protection and privacy laws.

Panel Recruitment

7. If the sample comes from a panel, what is your annual panel
turnover/attrition/retention rate and how is it calculated?
Our average attrition rate across all Valued Opinions
panels is between 10% and 50% per annum. Attrition
rate is defined as the percentage of panellists who have
unsubscribed themselves or have been “unsubscribed” by
our panel team (bounce-backs or panel offences). Our
e-Rewards opinion panels retain approximately 85% of its
panel members each year.

8. Please describe the opt-in process.
New panellists who enroll into our panels complete our
website panel registration form, providing their personal
details (such as name, email address, etc), and then
complete a registration survey.
As part of the enrollment process new panellists are sent
an email with a link to confirm their desire to opt-in to
the panel.  New panellists who confirm their opt-in in this
manner are counted as part of the active panel and are
eligible to be invited to participate in surveys.  Panellists
who do not confirm their opt-in are excluded from the
active panel and are not counted as active panellists.

9. Do you have a confirmation of identity procedure?
Do you have procedures to detect fraudulent respondents at
the time of registration with the panel? If so, please describe.
Panel registrations are automatically checked at
the time of registration for duplicate email address.
Following initial registration, there are additional
checks to identify potentially fraudulent or duplicate
panellists including a check for matches across several
demographic variables.
We also have quality management systems in place in
order to help confirm identities on all of our panels.
We further our commitment to identity validation
with continual review of member data and analysis of
suspicious member profiles to ensure members are who
they say they are.
On client request we offer TrueSample® as an optional,
value-added service. The TrueSample service is available
in Canada, France, Germany, the United Kingdom, and
the United States.

10. What profile data is kept on members? For how many respondents
is this data collected and how often is this updated?
Basic demographic information (e.g. age, gender, region,
household demographics) is collected upon sign up to
the panel. Panellists also have the option (either at the
sign up or later stage) to enter further profile information
for specific areas (e.g. health, mobile phone habits etc).
We also run occasional pre-screeners or profiling surveys,
where information is collected when Research Now is
screening for a particular sub-panel (e.g. automotive or
financial services) or for a particular project. All e-Rewards
panels are extensively profiled for B2B related surveys.
In terms of updating the information, panellists are
reminded at the end of every survey to update their
profile information. Non-active panellists who do not take
part in surveys or pre-screeners and panel mail addresses
that bounce back are taken off the panel. The majority
of Valued Opinions panellists (around 70%) receive their
incentives via post, so address (and therefore regional)
information needs to be up-to-date for panellists to
receive their rewards.
Panel Maintenance is maintained by our panel support
team and monitored for performance by our panel
analytics team.

11. What is the size and/or the capacity of the panel based on
active panel members on a given date? Can you provide an
overview of active panellists by type of source?
An active panel member is defined here as a member
who has participated in at least one survey, or updated
his/her profile data or registered to join the panel within
the last 12 months.
Panel Size/Capacity
As of 8th June 2010, panel membership across the Valued
Opinions and e-Rewards family of country panels totalled
6,253,609 – comprising members in the UK, Germany,
France, Spain, Italy, Ireland, Austria, Portugal, Denmark,
Norway, Sweden, Finland, Russia, Czech Republic, Poland,
Hungary, Greece, the Netherlands, Belgium, Switzerland,
the UAE, the USA, Canada, Brazil, Mexico, Argentina, Chile,
Australia, New Zealand, China, Japan, Korea, Malaysia,
Taiwan, Singapore, Hong Kong and India. These are
individual members and not households.
Source of Panellists
For Valued Opinions panels, panellists are recruited
via multiple source recruitment, by email and online
marketing affiliate partners. e-Rewards panel members
are enrolled by-invitation from various panel recruitment
partners, which include a diverse set of categories such
as many different retailers, hospitality and travel (airline
and non-airline).

Panel & Sample
Management

12. Please describe your sampling process including your exclusion
procedures if applicable. Can samples be deployed as batches/replicates,
by time zones, geography etc? If so, how well is this controlled?
Sample selection is based on the sample needs and
client requirements for each individual survey. Ideally,
pre-profiled sample is used to minimise screen-outs and
provide a better quality panellist experience. Customised
sampling, e.g. nat-rep outgo is also available.
Once sample has been selected, email invites are
automatically randomised to eliminate any potential
biases. Sample can also be subject to category
de-duping. This is a process whereby all projects
are coded with subject identification code so that
respondents who have taken part in surveys on one of the
pre-coded subjects over a stated time period, e.g. a survey
on alcohol in the last 3 months, can be excluded. As a rule,
we apply major category exclusions to surveys on a threemonth basis. Criteria can be more restrictive on demand.
Our proprietary panel management system enables us
to have total control over the way in which we invite
respondents to surveys. Panellists access the survey
with unique links stated in the invite. The time of sample
deployment can be controlled by sending out invites
at a specific time.

13. Explain how people are invited to take part in a survey.
What does a typical invitation look like?
Invitations contain key information for panellists to understand the
commitment required of them for each survey, e.g. broad survey topic,
survey length, incentive, and the length of time the survey will be open for.
The invitation also contains a link to our privacy policy, an opportunity to
unsubscribe from the panel and a link to a member of Research Now staff
for any queries. We have a strict policy on avoiding “leading” invitations
and screen invitations carefully before sending them to our panellists.

14. Please describe the nature of your incentive system(s).
How does this vary by length of interview, respondent
characteristics, or other factors you may consider?
As we have no open survey areas, Valued Opinions panellists have to
be invited to take part in each survey in order to qualify for answering
a survey to receive an incentive. Panellists receive a cash reward for
participating – the amount varies for each survey but is clearly stated
in the invitation email and related to the survey length, interest and
complexity. Typically, incentives for consumer surveys range between
50p-£5. Once a panellist’s reward balance reaches £10, they redeem a
voucher. Vouchers are typically for national retailers in member’s country
and there is always a multiple choice. The Web Perspectives panel in
Canada rewards panellists who participate in longer surveys (i.e. over 20
minutes) with individual incentives. Panellists who participate in surveys
less than 20 minutes in length are typically entered into prize pools (i.e.
chance to win cash or prizes). e-Rewards panel members earn e-Rewards
currency/points for time spent participating in market research activities.
Members can redeem their currency/points for a variety of valuable
rewards that are of interest to them.

15. How often are individual members contacted for online surveys
within a given time period? Do you keep data on panellist participation
history and are limits placed on the frequency that members are
contacted and asked to participate in a survey?
As each panellist is assigned an individual ID, we hold the
entire survey participation history of every respondent on
our panel. This means that we can carefully select each
panel member for each survey to ensure that they are not
over contacted.
As not all panellists respond to every survey or profile
invitation – we always invite more panellists to a
survey than will end up completing it. For this reason,
when measuring panellist activity, we distinguish
between contact and completion rate. In terms of
completion rates, our average panel member completes
approximately 6 full surveys per year.
We conduct monthly audits on panel utilisation to
monitor survey participation rates among various
demographic groups. This gives us a clear picture of
which panel groups are in highest demand and informs
panel recruitment strategy.

Policies & Compliances

16. Is there a privacy policy in place? If so, what does it state?
Is the panel compliant with all regional, national and local laws
with respect to privacy, data protection and children, e.g.
EU Safe Harbor, and COPPA in the US? What other research
industry standards do you comply with, e.g. ICC/ESOMAR
International Code, CASRO guidelines etc?
Yes – see www.valuedopinions.co.uk or www.e-rewards.com for an
example. We follow all regional, national and local laws with respect
to privacy, data protection and children to the extent required for the
country panels which we own.
Our panels also comply with, or exceed, all applicable industry standards
published by: ESOMAR, the MRS, the AMSRS (Australia), BVM (Germany),
CASRO (US), MRA (US), MRIA (Canada).
Amongst others, this includes observing the following guidelines:
• Ensuring the voluntary co-operation of panellists
• Following rules to protect the researcher’s and respondent’s identities
• Respecting the respondent’s right to anonymity
• Ensuring a privacy policy is in place on all our panels worldwide
which is compliant with EU and local laws as appropriate
• Ensuring that data security policies and measures are in place and
constantly reviewed in line with technological developments
• Ensuring that all data procedures are reliable and valid
• Ensuring that we respect all rules governing the interviewing of
children and young people

17. What data protection/security measures do you have in place?
Data Protection
Research Now follows all local data protection procedures for each
country we have panels in.
Security
We have secure servers for collecting survey data. Sampling is carried
out with highly encrypted links to the database servers. It is done without
the use of personal information, just demographic information specific to
each survey. We also use randomisation routines to ensure that there is no
bias towards one part of the database or the other. Samplers do not have
access to the database to reveal the identity of users.
Survey data remains anonymous and is linked to the panel database using
numeric IDs so the identity of the end-user (panellist) is always protected.

18. Do you apply a quality management system?
Please describe it.
Research Now has established a defined project management process
for each of the 2000 + projects managed each month globally. This is
supported by a carefully-documented procedure form which is managed
and maintained by our project managers from project kick-off to delivery.

19. Do you conduct online surveys with children and young people?
If so, please describe the process for obtaining permission.
We do occasionally conduct online surveys with children and young
adults. In the cases where target respondents are below the legal age
to be approached to conduct market research, we follow the standard
industry guidelines which require that respondents are recruited via
their parents and interviewed only with parental permission.
Parents are provided with full details about each survey and its
objectives and can view the survey prior to the child if they wish.

20. Do you supplement your samples with samples from other
providers? How do you select these partners? Is it your policy to
notify a client in advance when using a third party provider?
Do you de-duplicate the sample when using multiple sample providers?
We occasionally use other panel providers to provide samples in countries
where we do not yet operate our own panels. We only use trusted panel
partners and operate on the basis of a preferred supplier list with first time
use under controlled circumstances. We will always tell our clients when
we are using another panel and whether we have used them before.
We are open to taking part in initiatives for independent audits of
panel members to check for duplication across industry panels. While
we cannot pass on personally-identifiable information to a third party,
subject to the appropriate confidentiality clauses being in place, we can
contribute profile data to third party data clearing houses for analysis
purposes to check for duplicates across several panels.

Partnerships & Multiple
Panel Membership

21. Do you have a policy regarding multi-panel membership?
What efforts do you undertake to ensure that survey results are
unbiased given that some individuals belong to multiple panels?
If clients are using multiple suppliers, there is a risk
of duplication. We would always advise using only
one source per project to avoid this. Multiple panel
membership shifts over time and response/completion
rates vary across panels.
We would advise asking a past survey participation
question as a screening question at the start of a survey
to screen-out those who may have taken part in similar
surveys over a recent time period.
To avoid too much duplication, at the time of recruiting
onto our panels, we try to avoid using the same recruitment
sources at the same time as other panel suppliers.

Data Quality
& Validation

22. What are the likely survey start rates, drop-out and
participation rates in connection with a provided sample?
How are these computed?
Survey start rates (or “response rates”) naturally vary, depending on
factors such as length and interest in the survey, country the survey is
fielded in, the demographic group being surveyed and the time of the year.
Average survey response rates range from 8% - 40% and vary by market.
The “completion” rate is classified by Research Now as the percentage
of people who have provided answers to all of the questions that have
been asked of them.
The “completion” rate is typically around 80% - 85% of those who have
started (or “responded”) to the survey. Start, drop-out and participation
rates are calculated by our panel management software and can be run
on a project-by-project basis.

23. Do you maintain individual level data such as recent participation
history, date of entry, source, etc., on your panellists? Are you able to
supply your client with a per job analysis of such individual level data?
We hold the entire survey participation history of every respondent
on our panel.
Each panellist is assigned an individual ID number – this stays with them
throughout their panel membership and allows us to track their history,
date of entry, source etc.
We can also provide a per-job analysis to clients.

24. Do you use data quality analysis and validation techniques to
identify inattentive and fraudulent respondents? If yes, what techniques
are used and at what point in the process are they applied?
Research Now employs a range of measures, including:
Speeding - our project managers apply ad hoc sense checks on data and
we can implement more detailed exclusions on request.
Flatlining - we are able to incorporate survey-specific controls and algorithms
on flatliners in accordance with our clients’ prescribed level of exclusion.
Multiple Survey Completions - to avoid this, we avoid open survey
environments and use a unique URL for each survey.
Non-Response to Open-Ended Questions - we can flag non-responders
and assess on a project-by-project basis who to exclude.
Dummy Answers - it is possible to add dummy questions into a survey
to check whether respondents are answering the survey in a consistent
manner. We can recommend which questions to use on a survey-bysurvey basis.
Research Now Three Strikes Policy - The ‘3 strikes and out’ process flags
an ‘offence’ code with panellists who are identified as not completing
a survey correctly. Survey offences include agreed cases of “speeding”,
“flatlining”, “inattentiveness”, poor quality open ends, answering
inconsistencies and selecting dummy answers. Panellists accumulating
three such offences are disqualified from panel membership, meaning
they cannot be selected for any more surveys.
This process happens as a result of a series of manual checks for
each project and is carried out across all of our full-service projects.
For sample-only studies, we are reliant on our clients to provide the
IDs of any panellists they consider to be offenders (together with an
explanation of how the “offence” is detected) so that these can be duly
flagged in our panel. On our e-Rewards opinion panels our Veracity360®
quality management system evaluates and removes inattentive and
suspected fraudulent respondents from the panel.

25. Do you measure respondent satisfaction?
Yes – on our Valued Opinons panels we run a short panellist project
satisfaction survey for every project (both sample-only and full-service).
Data is collected monthly and is available to clients on aggregate as a
benchmark and for each of the surveys they run. We also run a half-yearly
panel satisfaction survey in the UK, Germany, France, USA, Canada and
Australia. The sample size is 500 per country and the sample is balanced
by panel activity levels. The survey includes questions on panel experience,
survey experience, panel incentives and panel communication.
In order to further enhance panellists’ survey experience, we also
actively encourage survey designers to incorporate the latest
technologies, e.g. Flash programming into surveys to keep respondents
engaged and interested.
Research Now runs an ongoing programme of research into the impact of
Flash programming on the respondent experience, and a white paper is
available on the subject.
On our e-Rewards opinion panels we conduct a member satisfaction
survey among panellists annually. We measure their satisfaction on
elements such as frequency of invitations, value and diversity of
incentives and redemption choices, their willingness to complete various
lengths of surveys, and e-Rewards’ level of responsiveness to any
questions or concerns they share with our Member Services team.

26. What information do you provide to debrief your client after
the project has finished?
Research Now captures all participation history, the date
of entry and panel recruitment source for each member
and, provided this does not involve the disclosure of
personally-identifiable information, is able to provide this
to clients as a report on request.
For full service projects where Research Now hosts the
survey, it is also possible to see a full survey activity
report for each project.
The key statistics from such a report detail:
• Fieldwork dates
• Total number of survey invitations sent
• Number of surveys started
• Number of screen-outs/quota-fulls and
survey drop-outs
• Number of completes

Contact us:
Athens
t: [phone]
Dubai
t: [phone]
[e-mail]
Frankfurt
t: [phone]
[e-mail]
Hamburg
t: [phone] - 0
[e-mail]
London
t: [phone]
[e-mail]
Madrid
t: [phone]
[e-mail]
Milan
t: [phone]
[e-mail]
Munich
t: [phone] - 0
[e-mail]

Chicago
t: [phone]
[e-mail]

Auckland
t: [phone]
[e-mail]

Dallas
t: [phone]
[e-mail]

Melbourne
t: [phone]
[e-mail]

Los Angeles
t: [phone]
[e-mail]

New York
t: [phone]
[e-mail]

New Delhi
t: [phone]
[e-mail]

San Francisco
t: [phone]
[e-mail]

Singapore
t: [phone]
[e-mail]

Seattle
t: [phone]
[e-mail]

Sydney
t: [phone]
[e-mail]

Shanghai
t: [phone]
[e-mail]

Toronto
t: [phone]
[e-mail]

Paris
t: [phone]
[e-mail]

www.researchnow.com