Coleman, et al v. Schwarzenegger, et al

Filing 8078

ORDER signed by Chief District Judge Kimberly J. Mueller on 12/5/2023 ORDERING that The Special Master's 6/21/2023 Report and Recommendation Regarding Third-Level Data Remediation Dispute Regarding Timely Compliance Methodology, 7954 , as amended by 8033 and 8033 -1 is ADOPTED in full; The Special Master's Recommendation shall be implemented forthwith through the data remediation process under his supervision.(Mena-Sanchez, L)

1
2
3
4
5
6
7
8

UNITED STATES DISTRICT COURT

9

FOR THE EASTERN DISTRICT OF CALIFORNIA

10
11

RALPH COLEMAN, et al.,

12
13
14

No. 2:90-cv-0520 KJM DB P

Plaintiffs,

ORDER

v.
GAVIN NEWSOM, et al.,

15

Defendants.

16
17

On September 21, 2023, the Special Master filed a Report and Recommendation

18

Regarding Third-Level Data Remediation Dispute Regarding Timely Compliance Methodology.1

19

ECF No. 7954.2 Defendants have filed objections to the Report and Recommendation. ECF No.

20

8025. With leave of court, ECF No. 8049, plaintiffs have filed a response to defendants’

21

objections, at ECF No. 8064.

1

The Special Master filed his Report and Recommendation in accordance with the
original process established for resolving disputes that arise during ongoing data remediation.
ECF No. 7556-2. On October 11, 2023, the court modified the data remediation dispute
resolution process “to allow for timely and effective presentation of disputes to the court.”
October 11, 2023 Order, ECF No. 8008, at 2.
2
On October 24, 2023, the Special Master filed a Notice of Errata and an Amended
Report and Recommendation. ECF Nos. 8033, 8033-1. This filing removes one key indicator
from the list of indicators involved in the present dispute. For the sake of clarity, this order will
cite to the Report and Recommendation at ECF No. 7954, with reference to the amendment as
appropriate.
1

1

I.

BACKGROUND

2

A.

3

More than three years ago, after extended investigation by a neutral expert followed by an

4

evidentiary hearing, the court found defendants had knowingly presented misleading information

5

to the court and the Special Master. See generally Coleman v. Newsom, 424 F.Supp.3d 925

6

(E. D. Cal. 2019). The court proceedings then uncovered an absence of transparency, clarity, and

7

accuracy in defendants’ mental health data reporting system, which prompted the court to require

8

extensive remediation. May 24, 2023 Order, ECF No. 7847, at 1. The “main purpose” of data

9

remediation “is to ensure the transparency and accuracy of defendants’ mental health data

10
11

General Background

reporting.” Id.
Data remediation in this action is taking place alongside defendants’ development of

12

“their mental health quality management system, also known as the Continuous Quality

13

Improvement (CQI) process.” November 16, 2023 Order, ECF No. 8069, at 2. An adequate

14

quality management system is a required part of the remedy in this action. See December 17,

15

2020 Order, ECF No. 6996, at 2, and orders cited therein. The quality management system

16

consists of “‘quality assurance [which] focuses on quantification of system performance . . . [and]

17

quality improvement [which] focuses on the quality of that same system’s performance.’” ECF

18

No. 8069 at 2 (quoting ECF No. 6996 at 2).

19

The continuous quality improvement tool (CQIT), which “is part of the larger mental

20

health quality management system,” ECF No. 8069 at 2, has two chief purposes: “to accurately

21

measure [all degrees of compliance with] remedial requirements and transition monitoring from

22

the Special Master to the defendants.” ECF No. 7847 at 7; see also id. at 5 (quoting July 12, 2018

23

Order, ECF No. 5852, at 3 (“CQIT must be completed so it can be used to ‘report all degrees of

24

compliance with monitored [remedial] requirements, from zero percent to 100 percent.’”). On

25

July 1, 2021, the court provisionally approved a list of key CQIT indicators. July 1, 2021 Order,

26

ECF No. 7216. These “‘key indicators’ . . . ‘signify the material provisions of the Program

2

1

Guide3 . . . that must be durably implemented’ in order to satisfy the Eighth Amendment.” Id. at

2

4 (internal citations omitted).

3

The ongoing data remediation process and finalization of the list of key CQIT indicators

4

are “‘progressing jointly.’” January 4, 2023 Order, ECF No. 7695 at 1 (quoting ECF No. 7693 at

5

2)). The finalized list of key CQIT indicators “will be presented to the court for final approval

6

once data remediation is complete.” ECF No. 7847 at 6. The key CQIT indicators “must

7

substantively track the areas the Special Master has monitored so the court can assess defendants’

8

compliance with the Eighth Amendment”; “[d]efendants acknowledge the design of CQIT’s key

9

indicators ‘should be “informed by the Special Master’s monitoring of . . . remedial

10

requirements.”’” Id. at 7 (quoting ECF No. 7805 at 5).

11

B.

The Dispute Before the Court

12

The dispute before the court is over whether defendants should be required to augment

13

their data reporting system with an additional methodology to report data and summary statistics

14

for twenty-five key CQIT indicators that measure compliance with timeframes required by the

15

Program Guide. Defendants prefer to report data on these indicators using only a methodology

16

that focuses on the service requirement that is the subject of the particular indicator.4 Defendants’
3

“The Program Guide is defendants’ plan, approved by the court, to remedy identified
violations in the delivery of mental health care to the plaintiff class.” September 3, 2020 Order,
ECF No. 6846, at 2.
4
The terminology for these service requirements varies. In some instances, the Program
Guide requires that a treatment service be offered to inmate-patients on a particular schedule.
See, e.g., ECF No. 7333-1 at 58 (“[E]ach [Enhanced Outpatient Program [EOP]] inmate-patient
shall be offered at least ten hours per week of scheduled structured therapeutic activities. . .”), 160
(each inmate patient in a psychiatric services unit “will be offered at least ten hours per week of
scheduled structured therapeutic activities . . .”). Other Program Guide timelines require
specified acts, including mental health screening, treatment planning meetings with
interdisciplinary treatment teams (IDTT), contacts with primary clinicians (PC) and transfers to
levels of care within the Mental Health Services Delivery System (MHSDS) to be completed
within a particular timeframe. See, e.g., id. at 19 (timelines to complete inmate-patient transfers
between levels of care in the MHSDS), 23, 25 (all inmates must receive initial mental health
screening for certain indices “within 24 hours of arrival” at a reception center, “within the first
seven calendar days” for other indices; inmates with “possible mental health needs” must “receive
a standardized mental health evaluation within 18 calendar days of arrival, and prior to any
placement decision”), 59 (Enhanced Outpatient Program (EOP) inmate-patients must have IDTT
meetings “at least every 90 days” and “[w]eekly clinical contact with PC (primary clinician)
either individually or in group therapy; individual clinical contact at least every other week”). In
3

1

approach is referred to, alternatively, an “events-wise,” “opportunity-based,” or “last due date”

2

methodology. ECF No. 7954 at 3.

3

In his pending Report and Recommendation, the Special Master recommends defendants

4

be required to use a second methodology in addition to their preferred methodology to report data

5

and summary statistics for these indicators. The methodology recommended by the Special

6

Master focuses on how many individual inmate-patients subject to a specific service requirement

7

timely receive the service requirement. This proposed methodology is referred to as “the patient-

8

wise methodology.” Id.

9

The patient-wise methodology can be developed in two different ways. One way, referred

10

to as “[t]he ‘all-or-none’ variation,” “would require a patient to receive all required services in a

11

review period to be scored as compliant” with the requirement. Id. at 26 (emphasis in original).

12

The other way, referred to as “the flexible scoring variation,” allows “cases featuring some

13

deviation from the minimum standard to be scored as compliant.” Id. The Special Master

14

recommends the patient-wise methodology be developed with the flexible scoring approach,

15

explaining that reports prepared using the patient-wise methodology with the flexible scoring

16

approach would most closely reflect his historic approach to monitoring. Id.

17

The Special Master also recommends that the reports prepared using the patient-wise

18

methodology be supplemented with “degree of impact” statistics. “Degree of impact” statistics

19

provide additional information on the extent of the delay when timelines were not met, for

20

example, whether an inmate-patient was seen one day past a treatment service due date or thirty

21

days past a treatment service due date. Id. at 5. Use of degree of impact statistics “would help

22

users understand the nature and extent of deviations from the minimum [compliance] standard.”

23

Id. at 23. The Special Master does not recommend replacing defendants’ preferred methodology

24

with the patient-wise methodology. Id. at 22. Rather, he recommends defendants augment their

25

data reporting system with the patient-wise methodology to more fully report summary statistics

26

on compliance with Program Guide timelines requirements. Id.
this order the court uses the phrase “service requirement” or “required service” to cover each
requirement of the twenty-five indicators covered by the Special Master’s recommendation.
4

1

C.

Defendants’ Objections

2

Defendants object to adding the methodology proposed by the Special Master to the

3

reporting tools in their mental health data system. Defendants’ objections are argumentative and

4

mischaracterize both the methodology proposed by the Special Master and the effort he and his

5

team have expended in working with defendants to explicate the proposed methodology and

6

accommodate defendants’ concerns.

7

Defendants incorrectly characterize the two methodologies discussed in the Special

8

Master’s report as “competing timely compliance methodologies,” and they presume, without

9

foundation, that the Special Master’s proposed methodology, if adopted, “will become the de

10

facto method for monitoring and reporting purposes.” ECF No. 8025 at 2, 3. But the patient-wise

11

methodology proposed by the Special Master does not compete with defendants’ preferred

12

methodology: it fills a critical gap in defendants’ data reporting system and allows for

13

comprehensive assessment of whether inmate-patients in defendants’ custody are timely offered

14

necessary mental health services. As the examples offered below show, if that gap is not filled,

15

defendants likely will end up presenting an incomplete picture to the court when it assesses

16

whether and to what extent the members of the plaintiff class timely receive service requirements.

17

The gap must be filled for the court to be fully informed in fulfilling its role.

18

Defendants’ objections are marked by strident hyperbole5 that detracts from the proper

19

focus required to address the questions before the court, namely, whether the Special Master’s

20

/////
5

See, e.g., ECF No. 8025 at 2 (“The Special Master’s proposed methodologies are
incomplete, confusing, and vague, and if adopted will obfuscate relevant data that would
otherwise be plainly and transparently presented through Defendants’ methodology. In fact, the
Special Master’s proposed methodologies seem specifically designed to hide relevant information
rather than present it in a straightforward and transparent manner.”), 7 (“the Special Master’s
proposed methodology. . . remains a half-baked, inappropriate, and potentially misleading
compliance measure for the delivery of mental health care.”), 9 (“[l]abeling this methodology
with a term that superficially implies greater patient-centricity serves only to obfuscate the actual
mechanics and implications of the approach . . . [t]he Special Master’s preference for a
methodological approach based on its title rather than its functionality is troubling because it
creates the false impression that the methodology provides useful-patient level data when it in
fact does not.”).
5

1

Report withstands review for clear error and whether his Recommendation serves the purposes of

2

data remediation and finalization of CQIT key indicators set out above.

3

Notwithstanding these observations, the court has considered all relevant arguments made

4

by the parties in the objections and the response. For the reasons explained below, the court will

5

overrule defendants’ objections and adopt in full the Special Master’s Report and

6

Recommendation.

7

II.

8

LEGAL STANDARD
The Special Master’s factual findings are reviewed for clear error. See December 11,

9

1995 Order of Reference, ECF No. 640, at 8; see also ECF No. 7847, at 8-9. “A finding is

10

‘clearly erroneous’ when although there is evidence to support it, the reviewing court on the

11

entire evidence is left with the definite and firm conviction that a mistake has been committed.”

12

United States v. U.S. Gypsum Co., 333U.S. 364, 395 (1948) (quoted in Anderson v. City of

13

Bessemer City, N.C., 470 U.S. 564, 573 (1985)).

14

III.

DISCUSSION

15

A.

The Timely Compliance Method Indicators

16

As noted, this dispute focuses on key indicators that measure whether inmate-patients

17

timely receive service requirements in the Program Guide. As listed by the Special Master in his

18

report and the notice of errata, these indicators are:

19
20
21
22
23
24
25
26
27
28
29
30
31
32
33








AC1: Timely MH Referrals
AC2.1: Timely PC Contacts
AC2.4: Daily PC Interactions – ASU EOP Hub High Refusers
AC3: Timely Psychiatry Contacts
AC4: Timely IDTTs
AC5: Treatment Offered
MM11.3: Diagnostic Monitoring Lithium (created from MM11)


 MM10: Diagnostic Monitoring-QT Prolongation EKG 12 Months
 MM11.1: Diagnostic Monitoring Carbamazepine (created from MM
11)
 MM11.2: Diagnostic Monitoring Lamotrigine (created from MM 11)
 MM11.4: Diagnostic Monitoring Oxcarbazepine (created from MM
11)
 MM11.5: Diagnostic Monitoring Valproic (created from MM11)
 MM14: Diagnostic Monitoring-Antidepressants
6






MM3: Diagnostic Monitoring-Antipsychotics
MM4: Diagnostic Monitoring-Clozapine
MM5: Continuity of Meds Upon Inter-Institutional Transfer at R&R
MM6: Continuity of NA/DOT Meds with Intra-Institutional
Transfers (Excluding:
 ASU/SHU/PSU)
 MM7: Continuity of Meds: Mental Health Crisis Bed (MHCB) Transfers
 MM8: Continuity of Meds with Intra-Institutional Transfers to

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

ASU/SHU/PSU


MM9.1: Continuity of Meds: Discharge/Transfer from a Community



Hospital and/or
DSH

 NM1: Timely response to Critical Med non-adherence notification
 NM2: Timely response to Non-critical Med non-adherence
notification
 SC8: Timely Submission of MH RVR Assessment Results
 SP9: Mental Health Observation Tool

19

ECF No. 7954 at 11-12, as amended by ECF No. 8033, 8033-1. It is undisputed that the

20

indicators on this list all measure timeliness of service requirements required by the Program

21

Guide.

22

B.

The Special Master’s Proposed Patient-Wise Methodology

23

In general, statistical data is calculated using a defined numerator and a defined

24

denominator. See ECF No. 7954 at, e.g., 13-14. The patient-wise method proposed by the

25

Special Master uses as the numerator the number of patients who timely receive the required

26

service and as the denominator the number of patients who were required to receive the service

27

within a given reporting period. Id. at 13. The Special Master provides the following illustration:

28

if ten patients in a mainline Correctional Clinical Case Management System (CCCMS) require

29

two routine primary clinician (PC) contacts during the period under review, using the patient-wise

30

methodology the denominator is ten, representing the number of patients who required two

31

routine PC contacts during the review period. Id. If seven patients timely receive the two

32

required contacts, the numerator is seven. Id. In this example, the summary statistic would show

33

7/10 or 70 percent of patients timely received the routine PC contacts during the review period.

34

/////

7

1

In response to concerns raised by defendants that use of the patient-wise methodology

2

would “require[ ] ‘perfection,’”6 the Special Master’s data expert has demonstrated how the

3

patient-wise methodology can be developed with a “flexible scoring” approach that, as its name

4

suggests, allows the methodology to “be applied flexibly to adjust to any threshold the court may

5

ultimately set as the benchmark for constitutional minima.” Id. at 5. It does this by allowing

6

“limited amounts of variation from the minimum standards laid out in the Program Guide to be

7

scored as compliant.” Id. at 23.

8
9

In response to concerns raised by defendants that the patient-wise methodology “would
hide how well their most needy patients are being treated,” the Special Master’s expert has

10

demonstrated that the patient-wise methodology can also be supplemented with calculation of

11

“degree of impact” statistics that show the extent of delay in any missed timeframe, thus

12

providing a more nuanced understanding of the extent of delays in particular contexts. Id. at 5.7

13

While the Special Master recommends the court direct defendants to augment their data-reporting

14

system with the patient-wise methodology, he intentionally chose not to seek court approval of, or

15

“include granular instructions to defendants” on, how this approach will be operationalized “for

16

each affected indicator” because such granular instructions at this stage “would circumvent a

17

critical st[e]p of the data remediation process, stakeholder review.” Id. at 8-9. He expresses

18

confidence “the parties will work together cooperatively to operationalize” the recommended

19

approach if the court adopts his Recommendation. Id. at 9.

20

In short, the Special Master’s proposed patient-wise approach supplemented by degree of

21

impact statistics and with flexible scoring will allow for organization and presentation of the

22

underlying data in a manner congruent with any compliance threshold ultimately set by the court.

6

This concern appears to reflect defendants’ tendency to conflate their concern regarding
ultimate compliance standards with a description of what a particular set of statistics might show.
Ultimate compliance standards are for the court; those must be set with reference to transparent,
clear, and accurate data.
7
It appears that by “most needy” patients, defendants refer to those who for a variety of
reasons present obstacles to meeting service requirements in a timely manner.
8

1

C.

The Defendants’ Proposed Events-wise/Last Due Date Methodology

2

Defendants’ preferred methodology uses “a numerator of the number of routine PC

3

contacts that were timely delivered and a denominator of the number of routine PC contact

4

deadlines that occurred during the review period.” Id. at 14 (emphasis in original). To illustrate,

5

the Special Master uses the same ten CCCMS patients requiring two PC contacts in the period

6

under review, this time with seven patients timely receiving two required PC contacts and the

7

other three patients timely receiving one required PC contact. Id. Using defendants’

8

methodology, the numerator is 17 timely PC contacts and the denominator is 20 routine PC

9

contacts. Id. The summary statistic reported in this illustration would show that 85 percent of PC

10

contacts were timely delivered. Id. Prepared using defendants’ preferred methodology, the report

11

does not show the number or percentage of inmate-patients who timely received the required PC

12

contacts, namely seven or 70 percent, as explained above.

13

D.

Eighth Amendment and Program Guide Requirements

14

As relevant here, the Eighth Amendment requires defendants to provide seriously

15

mentally ill inmates in their custody with access to adequate mental health care. Coleman v.

16

Wilson, 912 F. Supp. 1282, 1298 (E. D. Cal. 1995). This duty extends to all seriously mentally ill

17

inmates. See, e.g., id. at 1305 (duty to provide access to adequate mental health care includes

18

obligation to systematically identify inmates with mental illness), 1306 (Eighth Amendment

19

requires defendants to “employ mental health staff in ‘sufficient numbers to identify and treat in

20

an individualized manner those treatable inmates suffering from serious mental disorders”)

21

(emphasis added) (internal citation omitted). The remedial work in this action has been and

22

remains to develop structures and systems for sustainable delivery of adequate mental health care

23

to all seriously mentally ill inmates in California’s prisons. The court will determine compliance

24

with the Eighth Amendment in large part by the degree to which defendants comply with “the

25

material provisions of the Program Guide and Compendium” as measured by CQIT’s key

26

indicators and assessed at compliance thresholds the court has yet to confirm. See September 3,

27

2020 Order, ECF No. 6846, at 24.

28

/////
9

1

The Program Guide, as the remedial plan for the delivery of mental health care to class

2

members, reflects the Eighth Amendment requirement that defendants provide all seriously

3

mentally ill inmates in their custody with access to adequate mental health care. The twenty-five

4

indicators covered by the Special Master’s pending recommendation all measure compliance with

5

timelines established in the Program Guide for provision of treatment services to individual

6

inmate-patients. See ECF No. 7954 at 9-10 (citing 2021 Program Guide Update, ECF No. 7333-

7

1), as amended by ECF No. 8033, 8033-1. There is no dispute about what is to be measured by

8

the indicators.

9

E.

Comparison of the Two Methodologies

10

There is no dispute the Special Master has accurately described the way the two

11

methodologies operate and the information each provides. There is also no dispute that the two

12

methodologies could provide significantly different statistical information from the same set of

13

data. The example in defendants’ Level 2 Dispute Statement, ECF No. 7954-1 at 3, is illustrative.

14

Defendants posit an Enhanced Outpatient Program (EOP) unit with two patients, each of

15

whom must be offered four PC contacts per month. Id. In defendants’ example, neither patient

16

was timely offered four contacts: one patient was offered three contacts timely, and one patient

17

was offered two contacts timely. Id. The summary statistic derived using the patient-wise

18

methodology shows 0 percent compliance: 0 (number of patients who were timely offered all

19

four required contacts) / 2 (number of patients who were to be timely offered four contacts) = 0

20

percent. This summary statistic shows neither patient in this EOP unit was timely offered all four

21

contacts during the reporting month. In this example, the flexible scoring adjustment could be

22

used to show that 50 percent of the patients received timely PC contacts 75 percent of the time,

23

and 50 percent of the patients received timely PC contacts 50 percent of the time. In addition, the

24

degree of impact statistics could be used to report on the extent of the delay in each instance.

25

The summary statistic derived using defendants’ events-wise methodology shows 62.5

26

percent compliance: 5 (number of contacts timely offered) / 8 (total number of timely contacts

27

required). This statistic provides no information about whether any inmate-patients timely

28

received all the required contacts. This example illustrates clearly why the patient-wise
10

1

methodology is a useful and important addition to defendants’ preferred methodology, to ensure

2

both an adequate quality management system and compilation of an adequate set of data for the

3

court to use, ultimately, in assessing defendants’ compliance with their Eighth Amendment

4

duties.

5

In other words, application of the patient-wise methodology results in summary statistics

6

that show the percentage of inmate-patients who timely receive the service requirement measured

7

by particular key indicators. The addition of degree of impact statistics to this methodology will

8

help users of the data to “understand the nature and extent of deviations from the minimum

9

standard.” ECF No. 7954 at 23. And, as discussed above, the use of flexible scoring will allow

10

modification of the patient-wise methodology to incorporate variations to reflect compliance

11

thresholds required by the court. Id. at 23.8

12

Application of defendants’ preferred methodology shows the number of times service

13

requirements are provided within required timeframes, which also has utility particularly in a

14

system as large as CDCR’s Mental Health Services Delivery System (MHSDS). There is no

15

dispute defendants’ methodology also uses data derived from services provided to inmate-

16

patients. As discussed, defendants’ methodology does not measure or report the percentage of

17

inmate-patients in their custody who receive the minimum services required by the Program

18

Guide: it measures and reports how many times these services are provided in a timely manner

19

across the population in question. In this way, defendants’ methodology operating alone could

20

mask serious deficiencies in the delivery of constitutionally adequate mental health care to class

21

members. As the above example in this section shows, defendants could report 62.5 percent

22

compliance with timely PC contact requirements in an EOP unit when in fact the underlying data

23

shows none of the inmate-patients timely received all service requirements. The patient-wise

24

statistics reported using the same example are required to present a more complete picture,
8

Defendants object that the Special Master has failed to adequately define a flexible
scoring approach by failing to propose any internal thresholds. ECF No. 8025 at 6. But the point,
rather, is that the flexible scoring approach can allow adjustments to the reporting parameters to
reflect compliance thresholds set by the court. Id. at 23. Presumably the flexible scoring
approach will also allow the stakeholders to generate reports the court can use as it considers
what those compliance thresholds should be.
11

1

showing – if it is the case -- that none of the EOP patients timely received all required PC

2

contacts in the review period. On the other hand, the same information assessed using

3

defendants’ methodology, namely, the percentage of time PC contacts occurred timely, could

4

show the requirement was timely met in a number of instances. This latter point supports the

5

importance of adding the nuance provided by the degree of impact statistics and flexible scoring

6

options to the patient-wise methodology to make it more transparent, comprehensive and useful,

7

particularly for the court’s purposes in making ultimate compliance determinations.

8

F.

9
10

How the Methodologies Fit with Purposes of Data Remediation
1.

Tranparency and Accuracy

The addition of the patient-wise methodology to the events-wise methodology

11

substantially amplifies the transparency of defendants’ data. The events-wise methodology alone

12

does not show how many inmate-patients timely receive service requirements. Importantly, the

13

events-wise approach counts all requirements timely met without distinguishing when an inmate

14

patient receives more timely services than the minimum required by the Program Guide. See,

15

e.g., id. at 13 n.17 (events-based “summary statistics report the percentage of requirement

16

deadlines that were met during the review period but, importantly, does not aggregate its results

17

per patient”) (emphasis in original). A variation on the EOP example from above illustrates the

18

way in which the events-based approach should be paired with a patient-wise approach,

19

supplemented as required, to present a complete and accurate picture of services provided. In an

20

EOP unit with four patients, each of whom must be offered four PC contacts per month, if two

21

patients are offered (or receive) five PC contacts in a month and two patients are offered (or

22

receive) three contacts in that month, the events-wise methodology shows a score of 100 percent:

23

16 contacts offered (received) /16 contacts required. Viewed through the patient-wise approach,

24

2 patients are offered (or receive) the required number of timely PC contacts / 4 patients were

25

required to receive PC contacts, so only 50 percent of the patients in the unit received the

26

minimum number of required PC contacts. The two approaches paired together will allow for a

27

more comprehensive understanding of the timeliness of service delivery in a unit during the

28

review period.
12

1

2.

Monitoring

2

Throughout the Report, the Special Master explains the ways his monitoring has focused

3

on class members and the mental health treatment they receive. See ECF No. 7954 at, e.g., 2-3,

4

11, 29-30, 33. Defendants assert that incorporation of the proposed patient-wise methodology

5

into their data system represents a shift away from the Special Master’s historical monitoring

6

practices. See, e.g., ECF No. 8025 at 19. The precise methodology now proposed by the Special

7

Master is new: he acknowledges at the beginning of the Report that the proposed methodology is

8

“a more refined and comprehensive way of measuring and reporting what the Special Master has

9

done historically.” ECF No. 7954 at 2. What is monitored, however, is not new: the Special

10

Master has monitored and continues to monitor whether inmate-patients timely receive the

11

services required by the Program Guide. The proposed patient-wise methodology does not

12

change the substance of what the Special Master has monitored historically; it changes, improves,

13

and increases the efficiency of this aspect of his monitoring practice.9

14

G.

Feasibility of Incorporating the Patient-Wise Methodology

15

Finally, it is important to note there is no dispute that defendants already collect the raw

16

data required for the patient-wise methodology and degree of impact statistics. See ECF No.

17

7954-1 at 2 n.2. Moreover, defendants expressly acknowledge “there is no workload issue in

18

developing” the summary statistics using the patient-wise methodology and degree of impact

19

statistics. Id. With these acknowledgements, and noting the important gap that will be filled

20

using the patient-wise methodology and degree of impact statistics, the court finds no reason not

21

to include the Special Master’s proposed methodology in defendants’ mental health data system

22

and several important reasons to include it.

23
24

The Special Master specifically recognizes that defendants’ preferred methodology has
utility and he plainly “does not seek to prevent defendants’ [sic] from using data derived from this
9

Defendants suggest that the Special Master should test this methodology during a
monitoring period before proposing it to the court. See, e.g., ECF No. 8025 at 4 & 10 n.6. But the
data remediation process includes comprehensive steps for testing and validating all indicators
and methodologies to ensure their accuracy. A preliminary field test by the Special Master is
neither required nor useful in this context.
13

1

methodology.” ECF No. 7954 at 22. However, as the Special Master explains the focus of the

2

Program Guide requirements underlying the key indicators at issue here is on whether patients

3

timely receive service requirements. See, e.g., id. at 9-10. Defendants’ methodology focuses on

4

the number of times service requirements are timely met, rather than the number of inmate-

5

patients who timely receive the service requirements. Defendants acknowledge they collect the

6

latter data, and the Special Master’s proposal simply requires defendants to be able to create

7

reports on compliance with timeline requirements using both methodologies.

8

H.

Conclusion

9

Defendants have not shown the Special Master’s findings are clearly erroneous. The

10

findings in the Special Master’s Report show the data remediation process has opened the door to

11

a more comprehensive and accurate way for defendants to measure and report whether class

12

members timely receive services required by the Program Guide, key data that the Special Master

13

historically has monitored in a more general and labor-intensive way. While nothing in the

14

Special Master’s Report suggests the patient-wise methodology will completely supplant other

15

patient-centered monitoring tools on which he has historically relied, the improvements described

16

in the Report are significant and laudatory. Defendants’ preferred more limited methodology for

17

analyzing compliance with required timelines provides little insight into what percentage of

18

inmate-patients timely receive required services.10 Addition of the patient-wise methodology

19

recommended by the Special Master will close this gap and foreclose the possibility of

20

misleading inferences that could be drawn from reports that are exclusively service-centered

21

rather than patient-centered. Defendants acknowledge there is no workload issue with developing

22

the summary statistics using the Special Master’s proposed methodology, see ECF No. 7954-1 at

23

2 n.2, and they have shown no other reason why the Special Master’s Report and

24

Recommendation should not be adopted in full.

25

/////

10

This is true even though, as defendants attest, their methodology allows them to identify
“patterns in missed timelines” and individual patient data is “on demand reports” or individual
patients’ electronic health records. Decl. of Cartwright, ECF No. 8025-1, at 7.
14

1

Accordingly, IT IS HEREBY ORDERED that:

2

1. The Special Master’s September 21, 2023 Report and Recommendation Regarding

3

Third-Level Data Remediation Dispute Regarding Timely Compliance Methodology, ECF No.

4

7954, as amended by ECF Nos. 8033 and 8033-1 is ADOPTED in full;

5

2. The Special Master’s Recommendation shall be implemented forthwith through the

6

data remediation process under his supervision.

7

DATED: December 5, 2023.

15