Coleman, et al v. Schwarzenegger, et al

Filing 7609

ORDER signed by Chief District Judge Kimberly J. Mueller on 8/29/2022 ADOPTING Special Master's Expert's Analysis of CDCR's annual suicide report for 2015 and the Special Master's expert's annual suicide reports from 2016-2020, with amendments as noted in this order. (Donati, J)

1
2
3
4
5
6
7
8

UNITED STATES DISTRICT COURT

9

FOR THE EASTERN DISTRICT OF CALIFORNIA

10
11

RALPH COLEMAN, et al.,

12
13
14

No. 2:90-cv-0520 KJM DB P

Plaintiffs,

ORDER

v.
GAVIN NEWSOM, et al.,

15

Defendants.

16
17

On July 25, 2022, the court issued an order to the parties to show cause within fourteen

18

days why the court should not adopt in full the Special Master’s expert’s reports on completed

19

inmate suicides in the California Department of Corrections and Rehabilitation (CDCR) for

20

calendar years 2016 through 2020 and why the court should not adopt the Special Master’s report

21

on his expert’s summary of findings on inmate suicides in CDCR from January 1, 2013 through

22

December 31, 2020 (hereafter “Summary Report”). July 25, 2022 Order, ECF No. 7591. Both

23

parties have filed timely responses to the court’s order. ECF Nos. 7599 (Defendants) & 7600

24

(Plaintiffs).

25

I. BACKGROUND

26

The Special Master’s specific responsibilities with respect to suicide reporting began on

27

December 24, 1998. See Order filed July 23, 1999, ECF No. 1054, at 1, 5. On July 23, 1999, the

28

court issued an order implementing all matters covered by a stipulation filed by the parties on
1

1

December 24, 1998, making the implementation retroactive to the December date. ECF No.

2

1054. The order included a provision “that this Court and the special master have the power and

3

duty to report on defendants’ suicide prevention program, the incidence of suicide and any related

4

issues concerning suicides in the CDC[R].” Id. at 5. In accordance with that order, the Special

5

Master has filed reports on completed inmate suicides for every calendar year since 2001, as well

6

as a summary report for calendar years 1999 through 2004.1 The Special Master has also filed a

7

number of reports related to suicide prevention. See, e.g., Dec. 3, 2020 Order, ECF No. 6973, at

8

1-2 (listing reports filed by Special Master on his expert’s audits of CDCR suicide prevention

9

practices).

10

The reports now pending before the court cover calendar years 2015 through 2020 as well

11

as a summary for calendar years 2013 through 2020. The report filed January 28, 2021, ECF No.

12

7038, consists of a report by the Special Master on his expert’s analysis of a report prepared by

13

CDCR on completed inmate suicides for calendar year 2015 and a report by the Special Master’s

1

See June 27, 2016 Order, ECF No. 5459 (adopting in full Special Master’s Expert’s
Report on Completed Inmate Suicides for Calendar Year 2014, ECF No. 5428); March 25, 2016
Order, ECF No. 5426 (adopting in full Special Master’s Expert’s Report on Completed Inmate
Suicides for Calendar Year 2014, ECF No. 5396); September 14, 2015 Order, ECF No. 5361
(adopting in full Special Master’s Expert’s Report on Completed Inmate Suicides for July 1,
2012-December 31, 2012, ECF No. 5325); July 12, 2013 Order, ECF No. 4693 (Order on Special
Master’s Expert’s Report on Completed Inmate Suicides for January 1, 2012-June 30, 2012, ECF
No. 4376); March 15, 2013 Order, ECF No. 4394 (Order on Special Master’s Expert’s Report on
Completed Inmate Suicides for Calendar Year 2011, ECF No. 4308); ECF Nos. 4109, 4110
(Special Master’s Report on His Expert’s Report on Completed Inmate Suicides for Calendar
Year 2010 and the Expert’s Report); ECF Nos 4008, 4009 (Special Master’s Report on His
Expert’s Report on Completed Inmate Suicides for Calendar Years 2008 and 2009 and the
Expert’s Report); ECF No. 3677 (Report Completed Inmate Suicides for Calendar Year 2007);
ECF No. 3030 (Report on Completed Inmate Suicides for Calendar Year 2006); ECF No. 2566
(Report Completed Inmate Suicides for Calendar Year 2005); ECF No. 2339 (Review of
Completed Inmate Suicides in Calendar Years 1999 through 2004); ECF No. 1806 (Report on
Completed Inmate Suicides for Calendar Year 2004); ECF No. 1658 (Report on Completed
Inmate Suicides for Calendar Year 2003); January 12, 2004 Order, ECF No. 1559 (citing Special
Master’s Expert’s Report on Completed Inmate Suicides for Calendar Year 2002); July 25, 2003
Order, ECF No. 1536 (citing Special Master’s Expert’s Report on Completed Inmate Suicides for
Calendar Year 2001); Dkt. No. 1373 (Report on Completed Inmate Suicides for Calendar Year
2000 appended to Special Master’s Ninth Round Monitoring Report); Dkt. No. 1213 (Report on
Completed Inmate Suicides from October 1, 1998—December 31, 1999 appended toe Special
Master’s Sixth Round Monitoring Report).
2

1

expert on completed inmate suicides for calendar year 2016. The reports for calendar years 2017

2

through 2020 and the summary report were all prepared by the Special Master’s expert. See ECF

3

Nos. 7038 (amended by ECF No. 7404), 7077 (amended by ECF No. 7404) (calendar year 2017),

4

7161 (amended by ECF No. 7440 (calendar year 2018), 7239 (amended by ECF No. 7404)

5

(calendar year 2019), 7405 (calendar year 2020) and 7511 (summary report). Defendants have

6

filed objections to all of the reports. See ECF Nos. 7052, 7098, 7182, 7249, 7409 and 7522.

7

On July 25, 2022, in addition to issuing the order to show cause, the court issued an order

8

adopting in full CDCR’s proposal to assume responsibility for filing annual reports on completed

9

inmate suicides, subject to the provisional approval process outlined in the proposal. ECF No.

10

7592. Specifically, CDCR’s proposal provides for provisional approval of CDCR’s assumption

11

of suicide reporting through completion of the annual suicide report for calendar year 2021,

12

followed by a meet and confer of the parties supervised by the Special Master and submission of

13

a further recommendation from the Special Master as to whether the proposal should become

14

final. Id. at 9-10.

15

The July 25, 2022 order to show cause contemplates that adoption of the annual suicide

16

reports currently pending before the court will create a consistent record of annual suicide reports

17

through calendar year 2020 and allow a clear demarcation point beginning with the filing of

18

CDCR’s first report for calendar year 2021. ECF No. 7591 at 2. The order also contemplates

19

acceptance of the summary report on the docket as an additional tool for reference by the parties

20

and the court, without resolution of any objections thereto. Id.

21

Plaintiffs agree with the direction suggested by the order to show cause. See ECF No.

22

7600 passim. Defendants do not, instead requesting that the court resolve certain objections to

23

“guide future monitoring and reporting,” ECF No. 7599 at 2, and others to “correct the record,”

24

id. at 5.

25

I.

26

STANDARD OF REVIEW
This court’s review of the pending reports is guided by the Order of Reference, ECF No.

27

640, filed contemporaneously with the court’s Order Appointing Special Master, ECF No. 639.

28

/////
3

1

Paragraph C of the Order of Reference provides in relevant part:

2
3
4
5
6
7
8
9
10

[A]ny compliance report of the special master filed in accordance with
paragraph A(5) above shall be adopted as the findings of fact and conclusions
of law of the court unless, within ten days after being served with the filing of
the report, either side moves to object or modify the report. . . . The objecting
party shall note each particular finding or recommendation to which objection
is made, shall provide proposed alternative findings or recommendations, and
may request a hearing before the court. Pursuant to Fed. R. Civ. P. 53(e) (2),
the court shall accept the special master’s findings of fact unless they are
clearly erroneous.

11

ECF No. 640, at 8. As required, the Special Master’s findings of fact are adopted unless those

12

findings are “clearly erroneous.” Id. “A finding is ‘clearly erroneous’ when although there is

13

evidence to support it, the reviewing court on the entire evidence is left with the definite and firm

14

conviction that a mistake has been committed.” United States v. U.S. Gypsum Co., 333 U.S. 364,

15

395 (1948) (quoted in Anderson v. City of Bessemer City, N.C., 470 U.S. 564, 573 (1985)).

16

II.

ANALYSIS

17

Defendants preface their objections with the observation that, while they “anticipate[ ] that

18

the Special Master will be even-handed and consistent in their assessment of CDCR’s monitoring

19

and reporting”, “[t]he issues raised in these objections are issues that could be raised in the future

20

by the Special Master’s team in their criticisms of CDCR’s future suicide reports under the court-

21

approved proposal to assume suicide reporting” and that “leaving the objections unresolved will

22

cause future disagreements over established practices in suicide reporting.” ECF No. 7599 at 2-3.

23

The court declines to make orders to resolve issues that may or may not arise in the future

24

particularly where, as here, the court-approved proposal specifically contemplates a process for

25

vetting and resolution of such disputes should they arise.

26
27
28
29
30
31

The specific objections defendants preserve in response to the July 25, 2022 order to show
cause are addressed in turn below.
A.

Objections Focused On Future Monitoring and Reporting
1.

Deficiencies in Suicide Reporting

Defendants reassert their objection to the expert’s 2018 Report on the grounds that the
report “included a finding that 76% of the individual suicide case reviews [were] ‘deemed
4

1

adequate by the expert’ but the report did not identify which suicide reports were deemed

2

deficient and why.” ECF No. 7599 at 3 (citing ECF No. 7182 at 10). Defendants contend the

3

Special Master’s response, that the information could be found in Appendix A to the expert’s

4

2018 Report, is too vague. Id. (citing ECF No. 7161 at 6). They request an order directing the

5

Special Master to “provide a detailed list of the deficiencies identified by this objection” and

6

assert they “should not be deemed to have waived the right to object to such non-specified

7

deficiencies.” Id.

8

The Special Master’s response to this objection to the draft report – that the information

9

defendants seek “is discussed in detail in Appendix A of the Report,” ECF No. 7161 at 6 – is fully

10

supported by the record. To the extent defendants want a summary or a list of deficiencies and/or

11

inadequacies identified in the expert’s 2018 Report, Appendix A contains all the information

12

necessary for them to prepare such a summary or list.2 Furthermore, nothing in the court’s

13

adoption of this or any other report precludes either party from raising proper, focused evidentiary

14

objections at such time that any report or finding might be offered into evidence in specific court

15

proceedings.

16

This objection is overruled.

17
18

2.

Incongruence between Terminology Used in Reporting vs.
Terminology Used by Monitors at Suicide Case Review Conferences

19

Defendants reassert an objection they raised in response to the 2019 and 2020 Reports,

20

that the Expert’s Report’s critique of suicide case reviews is inconsistent with the views of the

21

Special Master’s monitors who attend the suicide case review conferences and that those monitors

22

do not have input into the report. Secondly, defendants also object to the term “marginally” as

2

It is entirely reasonable for the Special Master to rely on the detailed format his expert
uses to discuss individual case reviews, and to expect defendants to be able to distill necessary
remedial information from those reviews. It appears to the court defendants could deploy basic
word searching techniques in the course of preparing such a distillation. The court’s own cursory
word search of Appendix A reveals 99 times where the word “adequate” appears, 13 times where
the word “adequacy” appears, 13 times where the word “adequately” appears, 42 times where the
word “inadequate” appears, 3 times where the word “inadequacies” appears, 1 time where the
word “inadequacy” appears, 31 times where the word “deficiencies” appears, and 17 times where
the word “deficits” appears.
5

1

used to criticize suicide case reviews, and they request the language in suicide case reviews be

2

changed, as appropriate, “to reflect that they are ‘adequate,’ or alternatively, ‘adequate with

3

opportunities for improvement,’ which would be consistent with the Special Master’s own

4

findings.” ECF No. 7599 at 3-4.

5

Experts on the Special Master’s team attend CDCR suicide case review (SCR)

6

conferences. The Special Master’s experts are provided draft SCRS in advance of every

7

conference. In response to the 2019 and 2020 Reports, defendants objected that the 2019 and

8

2020 Expert Reports include critiques of individual SCRs that were not raised by the Special

9

Master’s experts at the time of the relevant SCR conferences. See Carda Decl., ECF No. 7248-1,

10

at 2 (cited in ECF Nos. 7248 at 8 & 7409 at 13). The fact that the Special Master’s Expert’s

11

Reports provide additional information concerning the adequacy of particular SCRs beyond that

12

provided at the time of the SCR conferences does not, without more, demonstrate inconsistency

13

on the part of the Special Master’s team.

14

Defendants’ assertion that the monitors who attended the SCR conferences do not have

15

input into the Expert’s Reports is, as the Special Master explains in his 2020 Report on his

16

Expert’s Report, “factually incorrect.” ECF No. 7405 at 5 n.6. Defendants identify Drs. Hughes

17

and Metzner as the leaders of the Special Master’s team of experts who attend the SCR

18

conferences. ECF No. 7248-1, Carda Decl. at 2. As the Special Master explains, “Drs. Metzner

19

and Hughes, who participate in CDCR’s Suicide Case Review meetings, are key contributors to

20

the Special Master’s internal suicide report writing workgroup, as has been noted in each of the

21

Special Master’s reports filed in the last year. See ECF No. 7038 at 2; ECF No. 7077 at 1-2; ECF

22

No. 7161 at 1-2; ECF No. 7239 at 1-2.” In light of the Special Master’s explanation, the

23

defendants’ assertion is completely unsupported.3

24

/////

3

Going forward, the court will consider imposing a certification requirement on all
counsel who sign objections to the Special Master’s monitoring or other reports. Such a
requirement, if imposed, will require all counsel who sign such objections to certify they have
read in full the Special Master’s responses to all objections and responses raised regarding any
draft monitoring report, as well as the final monitoring report filed with the court.
6

1

The second part of the objection, regarding the use of the term “marginally adequate” to

2

critique case summaries, borders on the frivolous. The Special Master’s Expert’s Reports provide

3

a wealth of nuanced information to defendants for their ongoing remedial work. Given the

4

court’s approval of CDCR’s Proposal to take over the filing of annual suicide reports, defendants

5

will have an opportunity in the coming year to make adjustments to achieve rhetorical congruence

6

if doing so aids the work that remains to be done.

7

These objections are overruled.

8
9

3.

Conclusions Regarding a History of Trauma

Defendants reassert their objection to the expert’s suggestion in the 2020 Report that a

10

positive history for trauma correlates with increased suicide risk. ECF No. 7599 at 4 (citing ECF

11

No. 7409 at 11). Relying on a general quote from a 2019 paper entitled “Prisoner Post Traumatic

12

Stress,” which does not purport to be based on a study of California’s prisons, defendants contend

13

“trauma is not necessarily an indicator of increased risk for suicide.” The quote from the paper

14

states that trauma is “ubiquitous” in male prison populations. Defendants also contend the

15

expert’s findings “lack foundation, are irrelevant” and that any probative value is outweighed by

16

the significant danger that the findings will “mislead the court.” Id. Defendants apparently urge

17

rejection of the expert’s recommendation that “CDCR increase staff awareness of the relationship

18

between trauma and suicide risk as a means to reduce suicides.” Id. Defendants have not

19

presented sufficient evidence to suggest the findings they challenge in the 2020 Report are clearly

20

erroneous.

21
22
23

This objection is overruled.
4.

Misunderstanding of CDCR’s Operations

Defendants re-assert two objections raised in their objections to the Special Master’s

24

expert’s 2020 Report concerning “the adequacy of CDCR’s proof of practice following quality

25

improvement plans (QIP) for training, and the QIP process as it relates to referrals to the Office of

26

Internal Affairs (OIA).” ECF No. 7599 at 4-5 (citing ECF No. 7409 at 18-20). They are

27

concerned about potential issues between defendants and the Special Master going forward

28

regarding CDCR’s proof of practice or the process of referring issues to the OIA. This objection
7

1

illustrates the purpose behind the court’s acceptance of the provisional approval process agreed to

2

by the parties and the Special Master. Any potential issues that remain following filing of

3

CDCR’s annual suicide report for 2021 can and will be addressed by the parties, the Special

4

Master and, as necessary, the court.

5

This objection is overruled.

6

B.

Objections to Correct the Record

7

1.

Incorrect Citations in 2018 Report on Completed Inmate Suicides

8

Defendants request correction of two typographical errors in footnote 8 of the Special

9

Master’s Report on his expert’s report on suicides completed in CDCR in calendar year 2018,

10

ECF No. 7161 at 6 n.8. ECF No. 7599 at 9. These objections were included in defendants’

11

objections to the 2018 Report. ECF No. 7182 at 9. Specifically, defendants contend:

12
13
14
15



The Special Master’s citation to ECF No. 7161-1 at 16 for the following
parenthetical “(clarifying discussion of suicide by security level)” should be
corrected to cite ECF No. 7161-1 at 15 where that information actually
appears;

16
17
18
19



The Special Master’s citation to ECF No. 7161-1 at 26 for the following
parenthetical “(clarifying discussion of deficiencies in mental health
treatment and evaluation)” should be corrected to cite ECF No. 7161-1 at 25
where that information actually appears.

20

Id. (cited in ECF No. 7599 at 5-6). The first request is well-taken and the Special Master’s

21

Report will be deemed amended accordingly. The second request also is well-taken in substantial

22

part. After review, the court finds the second citation should be corrected to cite to ECF No.

23

7161-1 at 23-26; the Special Master’s Report will be deemed amended accordingly.

24

2.

Incorrect Citations in 2019 Report on Completed Inmate Suicides

25

Defendants request correction of two typographical errors in the Special Master’s

26

expert’s report on suicides completed in CDCR in calendar year 2019, ECF No. 7239. ECF No.

27

7599 at 6. Defendants included these objections in defendants’ objections to the 2019 Report.

28

ECF No. 7248 at 9-10. Specifically, defendants contend (1) in a table of suicides presented in the

29

expert’s 2019 report, a decimal point in the 1999-2019 average for Mule Creek State Prison

30

“inadvertently shifted to the left” and “should be shifted to the right by one space for the sake of
8

1

accuracy” and (2) the table covers the period from 1/1/1999 through 12/31/2019 and therefore

2

reports on averages over 21 years, not 20 years. Id. at 9-10 (cited in ECF No. 7599 at 6). Both

3

requests are well-taken and the expert’s 2019 report, ECF No. 7239 at 13-14, will be deemed

4

amended accordingly.

5

C.

Redundancy

6

Defendants next re-assert their general contention that the six pending reports are

7

“redundant because they report on annual suicides that are already reported by CDCR, or merely

8

re-state data that is already in the record.” ECF No. 6599. This objection misses the mark.

9

As discussed above, the annual reports pending before the court are filed by the Special

10

Master as required by the court’s July 23, 1999 order. At some point, CDCR started publishing

11

on its website its own reports related to completed suicides and suicide prevention efforts. These

12

reports are identified on CDCR’s website under the auspices of California Correctional Health

13

Care Services (CCHCS), as follows:

14



2020 Report on Suicide Prevention and Response within CDCR

15



2017-2019 Aggregate Suicide Report

16



2019 Report on Suicide Prevention and Response within CDCR

17



2018 Report on Suicide Prevention and Response within CDCR

18
19



Annual Report on Suicides in the California Department of Corrections and
Rehabilitation January 1, 2016 – December 31, 2016

20
21
22

Annual Report on Suicides in the California Department of Corrections and Rehabilitation
January 1, 2015 – December 31, 2015. See https://www.cdcr.ca.gov/reports/.
The Special Master explains in his report that in 2016 he “began meeting with the then-

23

Secretary of CDCR about ways to reduce the Special Master’s monitoring responsibilities.” ECF

24

No. 7038 at 3. As part of those discussions “the Special Master offered to permit CDCR to write

25

the 2015 annual report under his supervision and in coordination with his expert” and, if that

26

effort were successful, to “consider recommending to the court the delegation of his authority to

27

draft the annual suicide report to CDCR going forward.” Id. The final version of CDCR’s 2015

28

report is attached as Appendix A to the Special Master’s January 28, 2021 report. ECF No.
9

1

7038-1. Appendix B to the January 28, 2021 report is the Special Master’s expert’s analysis of

2

that report. Id. The Special Master ultimately did not recommend ongoing delegation because he

3

determined the delegation had proved unsuccessful in large part due to defendants’ ongoing

4

refusal to apply long-established definitions that had been used in this case to guide analysis of

5

completed inmate suicides. ECF No. 7038 at 3-4. Moreover, defendants themselves did not

6

follow through until July of this year to request leave of court to assume responsibility for annual

7

reporting on completed suicides. In addition, the titles of the reports on CDCR’s website alone,

8

without more, show those reports are not redundant of the reports provided by the Special Master.

9

Only two are annual reports; one is an “aggregate” report covering three years from 2017 to 2019,

10

and the other three are focused on CDCR’s suicide prevention and response.

11

It was not until this past Spring, under the leadership of the Special Master and CDCR

12

Secretary Kathleen Allison, that discussions resumed to address the disputes that had previously

13

precluded successful delegation of annual suicide reporting from the Special Master to CDCR.

14

Those discussions proved successful and, as noted above, resulted in the Proposal agreed to by

15

the parties and the Special Master and approved by this court. In relevant part, the Proposal

16

resolves key disputes, including whether or not the Special Master’s annual suicide reports were

17

necessary in light of annual suicide reports CDCR has published on its website for a number of

18

years. The court’s July 25, 2022 order adopting CDCR’s Annual Suicide Report Proposal signals

19

that, subject to the provisional approval process incorporated within the approved Proposal, the

20

Special Master’s responsibility for annual suicide reporting that began with the July 23, 1999

21

order will end with the report prepared for calendar year 2020, and CDCR’s responsibility for

22

annual suicide reporting will begin with the report for calendar year 20214

23

no authority for the proposition that a court should revisit previously disputed issues that have

24

/////

4

Defendants present

The content and analysis of the annual suicide reports will continue to cover the same
topics and, similar to the practice followed by the Special Master, each year CDCR will circulate
the annual suicide report in draft form to the Special Master and plaintiffs for review and
comments, and will review any feedback before finalizing the report and filing it with the court.
See ECF No. 7574 at 13, 15.
10

1

been resolved to the satisfaction of all stakeholders and the court, and the court declines to do so

2

here.

3

With respect to the summary report, as noted in the order to show cause, that report is a

4

tool available to the court and to the parties if and when trend analysis is useful in the remedial

5

process. ECF No. 7591 at 2 (internal citations omitted). The court will accept it as part of the

6

record, with defendants’ objections noted.

7

III.

8

CONCLUSION
For the reasons explained in this order, the court will adopt the Special Master’s Expert’s

9

Analysis of CDCR’s annual suicide report for calendar year 2015 and the Special Master’s

10

expert’s annual suicide reports from calendar year 2016 through calendar year 2020, with

11

amendments as noted. Accordingly, IT IS HEREBY ORDERED that:

12

1. Footnote 8 of the Special Master’s Report on His Expert’s Report on Suicides

13

Completed in the California Department of Corrections and Rehabilitation for Calendar Year

14

2018, ECF No. 7161 at 6, is deemed amended as followed:

15
16
17

a. The citation to ECF No. 7161-1 at 16 for the following parenthetical “(clarifying
discussion of suicide by security level)” is deemed amended to cite to ECF No.
7161-1 at 15; and

18
19
20

b. The citation to ECF No. 7161-1 at 26 for the following parenthetical “(clarifying
discussion of deficiencies in mental health treatment and evaluation)” is deemed
amended to cite to ECF No. 7161-1 at 23-26.

21

2. The Special Master’s Expert’s Report on Suicides Completed in the California

22

Department of Corrections and Rehabilitation for Calendar Year 2019, ECF No. 7239 at 8-40, is

23

deemed amended as follows:

24
25

a. The average of inmate suicides at Mule Creek State Prison included in the table at page
14 is changed from .095 to 0.95; and

26

b. The parenthetical in the last full sentence on page 13 is changed from 20 to 21 years.

27

2. Except as amended by this order, the Special Master’s expert’s reports on completed

28

inmate suicides in the California Department of Corrections and Rehabilitation (CDCR) from

29

Calendar Year 2016 through Calendar Year 2020, ECF Nos. 7038 (amended by ECF No. 7404)

11

1

(Calendar Year 2016), 7077 (amended by ECF No. 7404) (Calendar Year 2017), 7161 (amended

2

by ECF No. 7440 (Calendar Year 2018), 7239 (amended by ECF No. 7404) (Calendar Year

3

2019), 7405 (Calendar Year 2020), are adopted in full.

4

3. The Special Master’s Report on his expert’s summary of findings on inmate suicides in

5

CDCR from January 1, 2013 through December 31, 2020, ECF No. 7591, is accepted for filing as

6

part of the record, with defendants’ objections noted, ECF No. 7522.

7

DATED: August 27, 2022.

12