Artigos

Conteúdo (article):

LOGICAL MODEL AND MATRIX OF CRITERIA FOR ASSESSING CARE TO PEOPLE WITH MENTAL DISORDERS IN CONFLICT WITH THE LAW

Lannuzya Veríssimo e Oliveira1. Universidade Federal do Rio Grande do Norte.
E-mail: lannuzya@ufrn.edu.br. ORCID: 0000-0002-6881-898X
Ardigleusa Alves Coelho2. Universidade Estadual da Paraíba.
E-mail: ardigleusacoelho@gmail.com. ORCID: 0000-0001-8869-3793
Severina Alice da Costa Uchôa3. Universidade Federal do Rio Grande do Norte.
E-mail: alicedacostauchoa@gmail.com. ORCID: 0000-0002-2531-9937
Pétala Tuani Cândido de Oliveira Salvador4. Universidade Federal do Rio Grande do Norte.
E-mail: petalatuani@hotmail.com. ORCID: 0000-0002-3208-6270
Cláudia Helena Soares de Morais Freitas5. Universidade Federal da Paraíba.
Email:chsmfreitas@hotmail.com. ORCID: 0000-0003-0265-5396

RESUMO
Objetiva-se descrever a construção e validação do modelo lógico e da matriz de critérios para avaliação da assistência à pessoa com transtorno mental em conflito com a lei. Trata-se de pesquisa metodológica, desenvolvida de fevereiro a dezembro de 2019, seguindo três procedimentos:1) teóricos – composto por estudo de avaliabilidade, realizado a partir de pesquisa documental e entrevista com informantes-chave; 2) empíricos – construção do instrumento de coleta e seleção de experts para o processo de validação do instrumento de avaliação proposto; e 3) analíticos – realização de duas etapas Delphi. O estudo de avaliabilidade resultou na elaboração e pactuação do modelo lógico; na análise e comparação entre a realidade da política e o modelo lógico; e na elaboração da matriz de critérios. A adequabilidade do conteúdo da matriz de critérios foi avaliada por 16 experts na etapa Delphi 1 e por 12 na etapa Delphi 2, a qual teve seu conteúdo validado com Coeficiente de Validade de Conteúdo de 0,93. Espera-se que o modelo lógico e a matriz de critérios propostos neste estudo direcionem profissionais de saúde, pesquisadores, trabalhadores e demais atores sociais para a avaliação da assistência a esta clientela.
Palavras-chave: Avaliação de programas e projetos de saúde; Avaliação de Programas e Instrumentos de Pesquisa; Estudo de validação; Transtornos mentais; Prisioneiros.

ABSTRACT
We aim to describe the construction and validation of the logical model and the matrix of criteria for assessing care to people with mental disorders in conflict with the law. This is a methodological research, carried out from February to December 2019, following three procedures: 1) theoretical – composed of an evaluability study performed from documentary research and interviews with key informants; 2) empirical – construction of the collection instrument and selection of experts for the validation process of the proposed evaluation instrument; and 3) analytical – performing two Delphi steps. The evaluability study resulted in the elaboration and agreement of the logical model; the analysis and comparison between the reality of the policy and the logical model; and the elaboration of the matrix of criteria. The criteria matrix content adequacy was assessed by 16 experts in the Delphi 1 step and 12 experts in the Delphi 2 step, whose content was validated with a Content Validity Coefficient of 0.93. The logical model and the matrix of criteria proposed in this study are expected to direct health professionals, researchers, workers, and other social actors to assess this clientele’s care.
Keywords: Evaluation of Health Programs and Projects; Evaluation of Research Programs and Tools; Validation study; Mental disorders; Inmates.

INTRODUCTION
In Brazil, the rights to dignity, autonomy, and social inclusion of people with mental disorders and the gradual deinstitutionalization of mental health users, along with the prohibition on prolonged hospitalization in asylum or asylum-like institutions were established under Law N° 10.216/01 – the landmark of the Psychiatric Reform1.
Thus, the hospitalization of people with mental disorders started to be indicated only when the treatment outside the hospital proved to be ineffective. Three options are available: voluntary hospitalization, with the patient’s consent; involuntary, without consent and at the request of another person; and compulsory, determined by Justice2.
Offenders with mental disorders are included in the compulsory hospitalization modality. In these cases, the Security Measures established by articles 96 to 99 of the Brazilian Criminal Code (CP) are applied to those considered unimputable or semi-imputable, who are unable to understand the illegality of their acts, and have an indication of hospitalization in Custody and Psychiatric Treatment Hospital (HCTP), or are subject to outpatient treatment3.
However, a mismatch is observed between the current National Mental Health Policy, published from Law N° 10.216/2001, and the application of Security Measures, since legal issues centered on the ability to understand and self-determination in implementing the illicit act, combined with the specific and exclusive assessment of the psychiatrist, culminate sometimes in life imprisonment of the “crazy offender” in HCTPs3,4.
The last survey carried out by the National Penitentiary Department in 2011 identified 26 Custody and Psychiatric Treatment Establishments (ECTP) in Brazil, namely, 23 HCTPs and three Psychiatric Treatment Wards (ATPs)6. A survey conducted by the Geopenitentiary System of the National Council of Justice (CNJ) in 2014 identified 35 asylum-like spaces for the confinement of people with mental disorders in conflict with the law in Brazil7. It is worth considering that such establishments remain connected to the security systems, and, despite being called “hospitals”, they have historically been governed according to the criminal execution principles since Law N° 7.210, of 19845.
Such spaces had the following characteristics: the absence of a Singular Therapeutic Project (PTS); containment model determined by criminal law to the detriment of health policies; low participation of the health and social assistance networks; inadequate, insufficient, or nonexistent treatments; reinforced stigma, social prejudice, and eternalization of institutionalizations of patients; almost always irreversible loss of family and social ties; and improper consumption of public resources8.
Attentive to such demands, the Ministries of Justice and Health have implemented actions aimed at improving mental health assistance offered to people in conflict with the law, such as Resolution N° 113, of the National Council of Justice, which, in April 2010, included the Psychiatric Reform Law in the rules governing the application of the Security Measure, together with the Criminal Execution Law9.
Again, in 2010, the National Council for Criminal and Penitentiary Policy, in Resolution N° 4, highlighted the need to observe the principles established by Law N° 10.216/2001 in the implementation of the Security Measure. It proposed important guidelines, such as the intersectoral approach, continuous psychosocial monitoring, individualizing the imposed measure, concerning the psychological, social, and biological individual singularities, the social inclusion, among others10.
More recently, in 2014, through Ordinance N° 94, in line with the National Policy for Comprehensive Health Care for People Deprived of Liberty (PNAISP), the Ministry of Health established the service of evaluation and monitoring of therapeutic measures applicable to people with mental disorders in conflict with the law within the Unified Health System (SUS)11.
This study is justified by the scarce literature, especially in the health sciences, on studies that evaluate the care to people with mental disorders in conflict with the law in the Brazilian context, the limited investments in evaluative research on health programs and policies associated with the difficulty of assessing a complex intervention, with a multidimensional character, involving services, systems, and multiple activities, and the lack of an instrument guiding the assessment of mental health care to people in conflict with the law continuously and systematically, and producing changes in the production of care for this group.
From this perspective, we aim to describe the process of constructing and validating the logical model and the matrix of criteria for assessing care to people with mental disorders in conflict with the law.
METHODS
This is a methodological research, with a mixed approach, developed from February to December 2019, based on the methodological framework adapted from the three-step Psychometry of Pasquali et al.12, as follows: 1) theoretical procedures, to identify the underlying contents of the logical model and the evaluation criteria matrix; 2) empirical procedures, when the content validation of the criteria matrix with experts in the field was outlined; and 3) analytical procedures, when the validation was analyzed with the Delphi technique to confirm or refute the validation of the proposed criteria matrix.
As for the theoretical procedures, a qualitative evaluability study that precedes the evaluation step itself was carried out to build the logical model and evaluation criteria matrix. It allows identifying the extent a program/policy is in a position to be subjected to an evaluation13.
It is a four-step cyclical and dynamic process, as follows: I) clarifications about the objectives and goals of the program/policy; II) development and agreement of the logical model with the interested parties; III) analysis and comparison between the reality of the program/policy and the logical model; IV) recommendations for carrying out the program/policy evaluation13.
Initially, the following documents were read in order to list relevant data and establish appropriations about PNAISP with regard to mental health care for people in conflict with the law: Interministerial Ordinance N° 1, of 2014, which establishes the PNAISP within the SUS14; Ordinance N° 94/GM/MS, of 2014, which establishes the service of evaluation and monitoring of therapeutic measures applicable to people with mental disorders in conflict with the law, within the SUS11; Law N° 10.216, of 2001, which provides for the protection and rights of people with mental disorders and redirects the care model in mental health2; Ordinance N° 95, of January 14, 2014, which provides for the financing of the service of evaluation and monitoring of therapeutic measures applicable to the judicial patient, within the Unified Health System (SUS)15; and Ordinance N° 142, of February 28, 2014, which establishes rules for the registration of the Teams for the Assessment and Monitoring of Therapeutic Measures Applicable to People with Mental Disorders in Conflict with the Law (EAP)16.
Then, the texts were read once more, and the relevant information for the description of the PNAISP was highlighted in the context of mental health care for people in conflict with the law and the contribution to the construction of the logical model. Thus, it was possible to identify PNAISP’s objectives, resources, activities, products, results, and impact. As a result, the logical model was built to systematize the functioning of the PNAISP concerning mental health care for people in conflict with the law.
The logical model was presented to the key informants – five actors who were directly involved with the implementation of mental health care for people in conflict with the law, within the State Health Secretariat (SESAP) and the State Penitentiary Administration Secretariat (SEAP) of the State of Rio Grande do Norte – interviewed by the responsible researcher, in their work environment, according to previous scheduling. At the time, they identified the proposed model’s adequacy to describe the policy and proposed adjustments.
Noteworthy is that, in the State of Rio Grande do Norte, locus of this study, care to people with mental disorders in conflict with the law occurs, primarily, at the Custody and Psychiatric Treatment Hospital (HCTP/RN). According to data provided by SEAP, it has a population of 45 inmates who occupy all the vacancies available at the institution. Also, the institution is equipped with the following health team: psychiatrist (1), nurses (2), nursing technicians (16), psychologist (1), and social worker (1).
The interviews adopted a semi-structured roadmap, as follows: Were the elements of PNAISP presented? Are there any other plausible ways to achieve the expected results? Have all relevant context factors been identified, and their potential influences described? If not, what other aspects can be added? What are the difficulties or facilities to operationalize the proposed objective?
They were recorded in audio, transcribed per the participants’ consent and, subsequently, an exhaustive reading-analysis of the transcriptions was carried out to extract information for readjustments of the proposed logical model. A consensus meeting was held after the suggested adjustments were made, where the logical model was presented to the key informants and validated through the Nominal Group Technique (TNG)17. In light of the validated logical model, a matrix of criteria was constructed to evaluate care to people with mental disorders in conflict with the law.
Subsequently, empirical and analytical quantitative procedures were used to validate the content of the criteria matrix. Experts selected through the analysis of curricula submitted to the Lattes Platform participated in this validation process, according to the following search strategy: Search mode – subject (title or keyword of the production) – “Mental health”; “Prison Health”; at the bases – doctors and other researchers; academic background/degree – Master.
Sixty-six experts were selected and, subsequently, their curricula analyzed according to the adaptation of Fehring’s criteria18, with a minimum score of five points being established to select experts in the area of the construct, identifying 48 experts with the established score (Box 1), which was followed by the search for the e-mails of the selected experts through their resumes, on the websites of their institutions, or in published papers. The e-mail address of four experts was not found, and these professionals were excluded.
BOX 1
Six to twenty experts are recommended to compose the content validation process12. However, considering the possible losses due to non-responses, an invitation e-mail was sent with the research’s presentation and its objectives, and the link that directed to the collection instrument for the 44 selected experts previously.
The collection instrument was built using the electronic tool Google Forms, consisting of four sections: 1) characterization of experts; 2) guidance to experts on how to complete the form; 3) logical model and criteria matrix; and 4) 20 multiple-choice questions (items) according to the Adequate, Partially Adequate or Inadequate options, with an open space for “comments or suggestions for inadequacies” about each item in the matrix mentioned above.
The multiple-choice questions referred to the items addressed in the criteria matrix, organized according to the dimensions of analysis: structure (1-8), process (9-14), and results (15-20). The experts only accessed the other sections of the form after signing the ICT.
The analytical procedures occurred through two Delphi steps. In the Delphi 1 step, 16 experts evaluated the matrix of criteria. In the Delphi 2 step, 12 completed the assessment instrument. The data were analyzed using simple descriptive statistics, using absolute and relative frequencies. The item with more than 70% agreement between the experts (assessed as Adequate) and a Content Validity Coefficient (CVC) > 0.812 was considered valid.
The ethical precepts established by Resolution N° 466/2012 of the National Health Council were followed, so that the study proposal was assessed and approved by the Research Ethics Committee, through Consubstantiated Opinion N° 3.084.950, of 15/12/2018, CAAE nº 01975418.9.0000.5292. Participants signed the Informed Consent Term (ICT), and their anonymity and voluntary nature were preserved.
RESULTS
I- Construction of the Logical Model and the Evaluation Criteria Matrix
The logical model of care to people with mental disorders in conflict with the law was elaborated (Figure 1) from the consulted documents and interviews with key informants. It aims to identify the components, resources, activities, and results expected with the implementation of the program (product and final result), and the causal chain of premises that articulate such elements. Two components (political-managerial and tactical-operational) and three subcomponents were defined, as proposed by the experts consulted.
Figure 1.
The discrepancies between the proposed logical model and the study’s loci’s reality were pointed out by the key informants and are shown in Box 2.
BOX 2
Based on the logical model and the comparison between what is legally established and the reality, the components, subcomponents, dimensions of analysis, the indicators, and the different investigation techniques that resulted in an evaluation criteria matrix were described. Noteworthy is that the analysis dimension was organized from the triad proposed by Donabedian19: structure, process, and results.
II- Validation of the Criteria Matrix
Twelve experts participated in the final sample of the research, with a predominance of women (10; 83.3%); with a mean age of 42.6±8.9 years; graduates in psychology (7; 58.3%) and with a doctorate (7; 58.3%). Table 1 shows the result of evaluating the items in the criteria matrix, in Delphi steps 1 and 2. Eleven items were considered valid in their content in the first evaluation step and had CVC=0.89 (Table 1).
Changes were made to items not validated during the Delphi 1 step, according to the experts’ suggestions, which were in item 1-To add community participation in the working committee; item 2-To subdivide into two indicators: elaboration and implementation of deinstitutionalization strategies; item 4-Details of the purpose of realizing the mental transfer; items 5, 8, 12, 14, 18 and 20-To add a field diary and interview with users as collection techniques to verify the established indicators.
Then, the matrix was submitted to a new evaluation step (Delphi 2). All items were considered valid in their content from this evaluation round, with experts’ agreement higher than 70% and CVC=0.93 (Table 1).
TABLE 1
The matrix validated by experts is shown in Box 3.
BOX 3
DISCUSSION
The complexity, plurality, and intersectoriality of the mental health field denote the need to implement assessment processes in this field, in the light of theoretical and methodological rigor under current legislation1,20.
In this context, the study of evaluability emerges as a fundamental step, as it provides a favorable environment for evaluation, builds understandings among those involved about the nature and objectives of the program/policy, seeks agreement on the interest in carrying out the evaluation, and potential study users, increasing the likelihood of using the evaluation results12.
We identified that the policy’s objectives and goals concerning the context studied are poorly defined. However, the development of the logical model clarified issues regarding PNAISP’s direction regarding care to people with mental disorders in conflict with the law, emphasizing deinstitutionalization based on the following actions: implementing the Singular Therapeutic Project (PTS); favoring the articulation between SUS services and the Unified Social Assistance Service (SUAS); and contributing to the progressive disinternation of people under security measures in penal or hospital institutions13. Notwithstanding, enacting legal norms is an insufficient condition for realizing that their practical consequences must be materialized in systematic and integrated actions12.
In this context, comparing the logical model and the evaluated policy’s reality allows appreciating essential aspects not outlined in the logical model and already subsidize the subsequent evaluation process12. Regarding the structure, the key informants pointed out the difficulty of operationalizing the logical model related to the non-detailing of official documents, a finding similar to the study by Silva et al.21. On the normative production in the right to health of people with mental disorders in conflict with the law, a study by Borges and Moura22 points out the need for rectification in the legislation on the subject that prioritizes the link with the Justice sector the country.
The key informants pointed out the structural deficiency in providing the recommended care, which can be explained by the history of negligence experienced by the “crazy offender”23 and increasing hospitalizations due to the Safety Measure of users of alcohol and other drugs24. Besides the structural deficiency, restructuring the care to people with mental disorders in conflict with the law collides with the struggle of implementing intersectoriality, both among the health team professionals, and these professionals and legal practitioners25.
The medical hegemony-based statements, to the detriment of the expanded clinic, and based on actions’ punitive character, to the detriment of resocialization, hinder everyone’s intersectoriality in conflict with the law, regardless of their mental health status23. In turn, such discourses are maintained, among other reasons, by the insufficient continuing and permanent education projects focusing on psychosocial care in the services26, which contributed to the disarticulation between the devices of the Psychosocial Care Network (RAPS) and the courts of law, and the lack of shared accountability by the PTS of subjects under SM27.
The construction of the matrix of evaluation criteria elucidated the absence of goals defined in the consulted legal documents, which denotes that aspects addressed in the PNAISP are not adequately explained regarding care to people in conflict with the law. However, the matrix presented in this study can favor systematic monitoring by the actors involved with such a clientele12. Also, the validation of the content of the evaluation criteria matrix is a fundamental step to verify the relevance of the items underpinning the material proposed to the construct they represent18.
Delphi is the most appropriate instrument validation technique because it seeks the consensus of professionals with significant expertise in knowledge on screen and obtains quality answers and opinions for a given question addressed to these professionals28. Concerning suggestions from experts in the field, we underscore the importance of including users in the evaluation process since, as research by Lanbecker et al.29 points out, people with mental disorders’ voice is sometimes invisible in the Brazilian research scenario. Noteworthy is that the validation of health assessment tools must be systematic and continuous, given the health field\'s dynamics.
FINAL CONSIDERATIONS
The institutionalization of evaluation in the daily life of services is a valuable tool to support processes of change in the evaluated scenarios organization, insofar as they provide those involved or interested, directly or indirectly, with conditions to decide how to address and solve everyday problems.
From this perspective, the elaboration of the logical model and its confrontation with the reality experienced by the key informants allowed the identification of gaps regarding the structure, process, and results of the implementation of care to people with mental disorders in conflict with the law. Also, the elaboration of the criteria matrix and subsequent validation of its content by experts in the field can favor the proposed evaluation.
An important limitation of the study is the reduced number of critical informants carrying out the evaluability study.
Finally, the logical model and the criteria matrix constructed in this study can be studied and revised at any time, as new guidelines and implementation strategies are innovated or as other analytical or evaluative perspectives direct to the PNAISP. We recommend establishing achievable goals, and that the criteria matrix proposed in this study to guide the evaluation of care to people with mental disorders in conflict with the law.

REFERENCES
1. Amarante P, Nunes MO. A reforma psiquiátrica no SUS e a luta por uma sociedade sem manicômios. Ciênc. saúde coletiva 2018; 23: 2067-2074.
2. Brasil. Lei nº 10.216, de 6 de abril de 2001. Dispõe sobre a proteção e os direitos das pessoas portadoras de transtornos mentais e redireciona o modelo assistencial em saúde mental. Diário Oficial da União 2001; 30 abr.
3. Brasil. Decreto-Lei 2.848, de 7 de dezembro de 1940 alterado pela Lei nº 9.777 em 26/12/98. Código Penal. Diário Oficial da União 1940; 31 dez.
4. Cunha CC, Boarini ML. A medicina com o voto de minerva: o louco infrator. Psicologia & Sociedade 2016; 28,442-452.
5. Oliveira AS, Dias FMV. Andando na contramão: o destino dos indivíduos com transtorno mental que cometem crimes no Brasil. Physis 2018; 28: e280305.
6. Diniz D. A custódia e o tratamento psiquiátrico no Brasil: Censo 2011. Brasília: Letras Livres: Editora Universidade de Brasília 2013; 382.
7. Geopresídios [internet]. Sistema Geopresídios do Conselho Nacional de Justiça (CNJ). [citado em 20 mar 2020]. Disponível em: https://www.cnj.jus.br/inspecao_penal/mapa.php.
8. Brasil. Serviço de Avaliação e Acompanhamento de Medidas Terapêuticas Aplicáveis à Pessoa com Transtorno Mental em Conflito com a Lei. Brasília: Ministério da Saúde, 2014.
9. Conselho Nacional de Justiça. Resolução nº 113, de 20 de abril de 2010. Dispõe sobre o procedimento relativo à execução de pena privativa de liberdade e de medida de segurança, e dá outras providências. Diário da Justiça Eletrônico 2010; 26 abr.
10. Conselho Nacional de Política Criminal e Penitenciária. Resolução nº 4, de 30 de julho de 2010. Dispõe sobre as Diretrizes Nacionais de Atenção aos Pacientes Judiciários e Execução da Medida de Segurança. Diário Oficial da União; 2010; 2 ago.
11. Brasil. Portaria nº 94 de 15 de janeiro de 2014. Dispõe sobre a avaliação e acompanhamento de medidas terapêuticas aplicáveis à pessoa com transtorno mental em conflito com a Lei, vinculado à Política Nacional de Atenção Integral à Saúde das Pessoas Privadas de Liberdade no Sistema Prisional. Diário Oficial da União; 2014, 15 jan.
12. Pasquali L. Instrumentação psicológica: fundamentos e práticas. Porto Alegre: Artmed; 2010
13. Mendes MFM, Cazarin G, Bezerra LCA, Dubeux LS. Avaliabilidade ou pré-avaliação de um programa. In: Samico A, Felisberto E, Figueiró AC, Frias PG, colaboradores. Avaliação em Saúde: bases conceituais e operacionais. Rio de Janeiro: MedBook; 2010. p. 57-64.
14. Brasil.Secretaria de Atenção à Saúde. Departamento de Ações Programáticas Estratégicas. Coordenação de Saúde no Sistema Prisional. Política Nacional de Atenção Integral à Saúde das Pessoas Privadas de Liberdade no Sistema Prisional. Brasília, 2014.
15. Brasil. Portaria nº 95, de 14 de janeiro de 2014. Dispõe sobre o financiamento do serviço de avaliação e acompanhamento às medidas terapêuticas aplicáveis ao paciente judiciário, no âmbito do Sistema Único de Saúde (SUS) . Brasília: Ministério da Saúde, 2014.
16. Brasil. Portaria nº 142, de 28 de fevereiro de 2014. Estabelece normas para o cadastramento das equipes de Avaliação e Acompanhamento das Medidas Terapêuticas Aplicáveis à Pessoa com Transtorno Mental em Conflito a Lei (EAP) Brasília: Ministério da Saúde, 2014.
17. Pope C, Mays N. Pesquisa Qualitativa na Atenção à Saúde. 1ª ed. Porto Alegre: Artmed; 2009.
18. Fehring RJ. The Fehring model. In: Carrol-Johnson RM, Paquete M. Classification of nursing diagnoses: proceeding of the tenth conference. Philadelphia: Lippincott Company; 1994. p 55-62.
19. Donabedian A. Basic approaches to assessment: structure, process and outcome. In: Donabedian A. Explorations in quality assessment and monitoring. Michigan (US): Health Administration Press; 1980. p. 77-125.
20. Bessa MS, Araújo Filho JD, Azevedo DM, Salvetti MG, Torres GV. Avaliação de Resultados da Assistência em Saúde Mental por Usuários. Rev bras ciênc saúde 2019; 23:89-96.
21. Silva RN, Guarda FRB, Hallal PC, Martelli PJL. Avaliabilidade do Programa Academia da Saúde no Município do Recife, Pernambuco, Brasil. Cad. Saúde Pública 2017; 33: e00159415.
22. Borges AAB, Moura LM. Análise da produção normativa no direito à saúde dos inimputáveis - doentes mentais infratores (2011-2014). Cad. Ibero-Amer. Dir. Sanit 2015; 4:83-102.
23. Soares Filho MM, Bueno PMMG. Demografia, vulnerabilidades e direito à saúde da população prisional brasileira. Ciênc. saúde coletiva 2016; 21:1999-2010.
24. Lhacer PMV. Transinstitucionalização: Caminhos e Descaminhos na Dinâmica de Internações e Desinternações de Pessoas com Transtorno Mental em Conflito com a Lei no Estado de São Paulo [tese]. São Paulo (SP): Universidade de São Paulo; 2019.
25. Oliveira TTSS, Abrici EP, Santos MA. Estrutura e funcionamento de uma equipe de saúde mental de Trieste na perspectiva de seus integrantes: um estudo qualitativo. Psicol. Pesq 2018; 12:24-35.
26. Ferreira TPS, Sampaio J, Souza ACN, Oliveira DL, Gomes LB. Produção do cuidado em Saúde Mental: desafios para além dos muros institucionais. Interface 2017;21:373-384.
27. Cortez PA, Souza MVR, Oliveira, LFA. Princípios de uma política alternativa aos manicômios judiciais. Saude soc 2018; 27: 1206-1217.
28. Scarparo AF, Laus AM, Azevedo ALCS, Freitas MRI, Gabriel CS, Chaves LDP. Reflexões sobre a técnica delphi em pesquisa na enfermagem. Rev Rene. 2012; 13(1):242-51.
29. Langbecker A, Castellanos MEP, Neves RF, Catalan-Matamoros D. Journal coverage on issues of interest to Brazilian Collective Health: a literature review. Interface 2019; 23: e1800095.