Artigos

Conteúdo (article):

Cross-cultural adaptation of the Recovery Self-Assessment RSA-R Family/Brazil: Validity evidence based on test content

Authors:
Leidy Janeth Erazo Chavez (Erazo-Chavez, L. J), University of Campinas, leidye32@gmail.com, https://orcid.org/0000-0003-3715-7864
Ehideé Isabel Gómez La-Rotta (La-Rotta, E.I.G), University of São Paulo, larottaehidee@gmail.com, https://orcid.org/0000-0003-1194-9898
Rosana Teresa Onocko Campos (Onocko-Campos, R. T), University of Campinas, rosanaoc@unicamp.br, https://orcid.org/0000-0003-0469-5447

Abstract:
In the cross-cultural adaptation of instruments, the qualitative component of adaptation is generally poorly reported, sometimes being superficially addressed. In this study we aimed to describe the qualitative component of the cross-cultural adaptation process and to demonstrate
validity evidence based on test content of the Recovery Self-Assessment-RSA-R Family/Brazil. We conducted a qualitative study that included the steps of preparation, translation, back-translation, expert’s assessment, workshop with a researcher from Yale University, and two pilot studies involving family members of patients attended at mental health services. Among the results, we found considerable validity evidence based on test content with a percentage of agreement above 80%. Pilot studies contributed to accentuating this evidence, assisting in the cultural adequacy of the statements and in the operational equivalence of the instrument. The adaptation process of the RSA-R Family/Brazil proved to be complex. From this experience, we concluded that presenting validity evidence based on test content is important to ensure the applicability tools to the target culture. The instrument will still be evaluated as for psychometric characteristics through statistical techniques.

Keywords: Recovery; Mental Health Services; Family; Validation studies; Qualitative analysis.

Introduction
According to the Standards for Educational and Psychological Testing, established by the American Educational Research Association, validity evidence based on test content, also generally known as content validity, is one of the five sources of validity evidence of an instrument. Test content refers to themes, words and format of the items, administration, and test scores1,2.
Validity evidence based on test content, in turn, refers to the level at which the content is congruous with the purposes of a test1, i.e., “it evaluates the degree to which each element of an assessment instrument is relevant and representative for a specific construct with a particular evaluation purpose”3 (p. 3063, free translation).
Overall, cross-cultural adaptation studies are focused on providing statistical evidences for demonstrating the validity and reliability of an instrument. Therefore, they lack emphasis on validity evidence based on test content and on the qualitative process of adapting a questionnaire to a new culture. However, this process must be equally important to statistical evidence, considering that a good adaptation cannot disregard differences arising from language, cultural context, and lifestyle, since results of this process will be reflected on the statistical results3.
In this study we address the cross-cultural adaptation of an instrument for assessing the recovery-oriented practices of mental health services, with special emphasis on the qualitative process involved in adapting it to the Brazilian context. Within this context, recovery, in the field of mental health, is understood “as a deeply personal, unique process of changing one’s attitudes, values, feelings, goals, skills, and/or roles. It is a way of living a satisfying, hopeful, and contributing life even with limitations caused by illness”4 (p. 525).
We chose the Recovery Self-Assessment-RSA-R Family instrument to involve family members of users of mental health services in their care, since, after the psychiatric reform in Brazil, the family became the most responsible people for providing care in recovery. Thus, providing tools that include them would allow establishing a partnership between relatives and services, enabling them to participate in decisions, and sharing the time and responsibility dedicate to care with them5.
RSA is one of the most used scales to assess the recovery-oriented practices of mental health services6. Such use favors the reflection on strengths and on limitations of the services within this scope7. This scale has good psychometric properties and is commonly used in evaluations of mental health services in other countries8–18.
In our study we aimed to describe the qualitative component of the cross-cultural adaptation process of the RSA-R Family/Brazil and to demonstrate validity evidence based on test content of the instrument. We present the main results of the instrument adaptation process by analyzing how the qualitative component was treated. Our study becomes relevant for reviewing methodological processes in cross-cultural adaptation research, based on the experience reported in each step, thus showing the considerable bonding potential of the target population in this type of research.
In addition, investing in the adaptation of instruments for assessing mental health services, within the recovery-oriented focus, shall bring gains in the field of public health. Among these gains we can mention that the instrument allows knowing the degree to which mental health services involve users and family members in the treatment; likewise, there is the gain of enabling the implementation of services that promote the autonomy of people with mental disorders, since authors of studies conducted so far19,20,21 point to reducing the chronicity of the disease and the disabilities arising from it, thus improving health conditions and increasing the quality of life of people with mental disorders and their family members.
Methods
This is a qualitative study that involved family members of persons in recovery attended at Psychosocial Support Centers (Centros de Atenção Psicossocial – CAPS) and Community Centers (Centros de Convivência – CECOs) located in Campinas, state of São Paulo – Brazil, from June 2016 to December 2017. Family members of users attended at these services for a period longer than three months, aged over 18 years, who were able to communicate in Brazilian Portuguese, without any cognitive impairment, and who agreed to participate were eligible for the study. In this convenience sampling, individuals were invited to participate in the study as voluntaries and did not receive financial incentives. The study was approved by the Research Ethics Committee of the University of Campinas CAAE no. 60826616.6.0000.5404.
Assessment Instrument
RSA-R was designed by Yale University, from the United States of America, and contains 32 items for each stakeholders (person in recovery, family member, provider, and CEO and directors). However, in the versions concerning family members and CEO and directors, the instrument contains appendixes with 8 and 4 additional items, respectively, for assessing 6 domains of recovery7,22.
The RSA-R Family version consists of 32 items and 8 appendixes, with five response options to be chosen: 1 (strongly disagree) to 5 (strongly agree), and two additional options, D/K (do not know) and N/A (not applicable). It is divided into six factors: life goals, involvement, diversity of treatment options, choices, individually-tailored services, inviting, and the family only appendix7,22.
Procedure
The cross-cultural adaptation of the RSA-R Family scale for the Brazilian context was carried out mainly following the Principles of Good Practice for the Translation and Cultural Adaptation Process for Patient-Reported Outcomes23 and other guidelines proposed in the literature2,3. It involved a total of six steps, as described next.
Preparation
Authorization was obtained from the main author of the scale, in such a way it could be translated and adjusted to Brazilian Portuguese. A literature review was also conducted on the concept of “recovery” and its equivalence with the notion of psychosocial rehabilitation, generally used in Brazil24.
Ideally, the process of cross-cultural adaptation of an instrument should achieve the maximum equivalence between the original instrument and its adapted version. According to Gorenstein and Wang25, “equivalence implies that the effectively observed differences between samples from different cultures result from cultural differences, which are not caused by the form or the evaluation of the constructs of interest” (p. 13, free translation). Equivalence is divided into several categories. There is no consensus in the literature on the categories and their denominations; nevertheless, overall, equivalences associated with conceptual definitions, instrument translation, application, and psychometric properties are included25.
Translation and Back-translation
Initially, a bilingual translator, whose mother tongue was Brazilian Portuguese and who was aware of the research objectives, translated the scale from English to Portuguese. Then, a second bilingual translator, whose mother tongue was English and who did not know the objectives of the research, did the back translation, translating the instrument back to English. From these two versions, the research team compared the back-translation with the original instrument, in order to identify inconsistencies and make corrections.
Experts’ Evaluation
The version resulting from the previous steps was evaluated by five bilingual experts in the field of mental health, whose mother tongue was Brazilian Portuguese. There were four psychologists and one occupational therapist, who worked in mental health services and were aware of the recovery construct. Such professionals were familiar with the target population and the construct (Chart 1). They aimed at verifying the validity evidence based on test content by assessing conceptual, semantic, idiomatic, and experiential equivalence, concerning the title, instructions, response options, and items. The agreement percentage in these aspects was above 80%. After agreement analysis and, considering the experts’ suggestions, the research team made a qualitative analysis and adapted the instrument.
Pilot I – Focus Group Assessment
A first pilot study was carried out with the participation of nine family members, based on the Focus Group (FG) technique, divided into three meetings. In the first two meetings, the objectives of the research were explained, the Informed Consent Form was filled in, and the items present in the instrument were discussed. In the third meeting, the already modified instrument was applied to the same participants, and aspects regarding the layout of the instrument (title, instructions, response options) were discussed, in addition to the items, in such a way to guarantee conceptual, semantic, and operational equivalences of the instrument27.
Workshop with a researcher from Yale University
The “Workshop: Recovery-Oriented Mental Health Systems of care: assessment, indicators and meaning” was held with the participation of Professor PhD. Maria O’Connell, from Yale University – United States of America, the main author of the original instrument. In this space were discussed items identified in the FG and deemed as challenging, which were previously back-translated, in order to guarantee the equivalence between the original instrument and the target version.
Pilot II – Interviews
The pre-final questionnaire was applied to 10 family members, using the Cognitive Debriefing Interviews technique. This technique implied an in-depth interview, during which the participants were able to explain the questions out loud and comment on any difficulties in understanding23,28. An education professional (graduated in Pedagogy), who works with Popular Education in the city of Campinas, participated in the discussion about the instrument to analyze the material and propose changes.
A harmonization step was carried out after each step, from which, according to the findings and the consensus of the research team, the instrument was adjusted seeking to maintain similarities to the original content, without disregarding the singularity to the cultural context of the Brazilian Portuguese language23. These steps are illustrated in Figure 1.
Data Analysis
The qualitative analysis of these steps followed the critical hermeneutic approach, according to which several perceptions are sought to be articulated in an intelligible way, obtaining a more complex and reliable result of the object under study at the end of the analysis, thus establishing a dialogical relationship between participants and researchers, assuming that any analysis will only be performed within this articulation29.
Results
Validation involves careful attention to possible distortions of meanings resulting from the inadequate representation of the construct and measurement aspects, such as the test format and administration conditions or language level, which may limit or qualify the interpretation of test scores in material terms1. The contribution of experts and family members in this study was paramount to identify these aspects.
As a result of the experts’ evaluation step, we obtained an agreement percentage above 80%, both for the layout of the instrument and for the items. Thus, for the RSA-R Family/Brazil, the minimum agreement percentage between raters was 87.5% (items 26 and 38), and a maximum agreement percentage of 100% (items 7, 8, 10, 11, 12, 19, 20, 28, 35, and 36). Regarding the layout, there was an agreement of 98.75% regarding the instrument title; of 96.25% regarding the instructions; and of 98.75% regarding the scale response options (Table 1).
Nineteen family members also participated in this study, 9 in the Pilot I and 10 in the Pilot II. We present the sociodemographic data in Table 2. Most participants were women, with a mean age of 53.22 years (± 18.1) in Pilot I, and of 57.60 (± 18.4) in Pilot II. Regarding the bond established with the user, most of the participants were parents, or spouses, of people diagnosed with schizophrenia or depression and who had been in treatment for more than 12 years in mental health services.
We analyzed and adopted the observations and suggestions of experts and family members throughout the process of cross-cultural adaptation, which resulted in five modifications to the RSA-R Family/Brazil scale (Chart 2).
From the analysis of Pilot studies I and II, we identified limitations in the operational equivalence, specifically concerning the way the questionnaire was applied to the target population, especially with participants with low level of education. The first modification involved applying the scale in the form of an interview, and not as a self-administered questionnaire, as in the original scale. We also deemed necessary to create a document with general guidelines containing instructions and response options, which the interviewer should read and hand in to the participants to facilitate their answers. The interviewer should also reciprocate the reading of the statements and response options, noting down each of the responses given by the participants.
Throughout the cross-cultural adaptation process, we observed difficulties in adapting the sentence “my loved one” which, in the original version, was used to refer to the person in recovery. Initially, we chose to translate it, into Portuguese, as “meu familiar” (“my relative”). However, in the pilot studies, we verified some difficulties at the time of answering the questions, mainly on the part of participants whose level of education was low. Thus, taking advantage of the fact that the scale would no longer be self-administered, the second modification consisted in indicating, in the sentences, that the interviewer could say the user’s name instead of “my relative.” This modification was made in items 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 34, 35, and 36. This made the sentences more specific and easier to understand, preventing them from having more than one interpretation.
The third modification refers to the division of the scale into two parts, considering that 32 items were related to the family member’s perception of the assistance provided to the person in recovery, and the remaining 8 items were formulated in the first person singular, investigating their own experience as a relative, which generated misunderstandings at the time of responding the questions. Therefore, we chose to insert in the first part of the scale the 8 items regarding the family member’s own experience and, in the second part, the remaining 32 items related to the participant’s perception of the user’s experience.
The fourth modification was made based on the experts’ suggestion and improved, later, with the popular education professional in the pilot study step. According to the experts, denominating all the response options would facilitate the understanding of the scale. In the original scale, only the response options “1=strongly disagree” and “5=strongly agree” were denominated. In the Brazilian version of the scale, all the response options were designated, “1=Strongly disagree,” “2 =Disagree,” “3=Undecided,” “4=Agree,” and “5=Strongly agree.” We also added graphic elements to facilitate the understanding.
Sentences in the original scale were long and had examples to improve the understanding. However, in both pilot studies, this became a problem, because, overall, participants were only focused on understanding some examples and not the sentence as a whole. In this sense, as a fifth modification, we deemed necessary to shorten and reformulate the writing of the sentences in a simpler language, making them clearer and facilitating their understanding on the part of participants. We also excluded the examples of items 2, 9, 16, 19, and 22.
Furthermore, some items have become challenging throughout the cross-cultural adaptation process (items 13, 21, 25, 29, 30, and 32) due to cultural differences and, in some cases, the lack of certain practices in mental health services in Brazil. For example, item 21, which refers to the practice of “peers support,” was divided into two items that question, in general, the existence of these and other similar strategies in mental health institutions.
Finally, the use of the “recovery” construct – to which the scale refers – was challenging throughout the cross-cultural adaptation, since according to the experts’ evaluation, the literal translation into Portuguese as “recuperação” could evoke a simplistic interpretation of the term. Initially, we considered to keep the term in English, an option that has been discussed in other scientific articles30,31. Nevertheless, in the first pilot study, the need for finding an appropriate term in Brazilian Portuguese remained evident. We discussed this aspect with the research team and the researcher responsible for the aforementioned workshop. We reached the consensus on keeping the word “recuperação” as a literal translation, but inserted a brief explanation of the concept in the general guidelines that interviewers must read before filling in the questionnaire with the participants.
Discussion
Providing instruments for the assessment of mental health services in Brazil, which are not centered only on the managerial perspective, is an imperative task. The RSA-R Family/Brazil allows including a long-forgotten stakeholder in the country’s evaluative research. This insertion can contribute to improving the quality of mental health services by incorporating the collected information into the treatment32.
The validity evidence based on test content was supported by the fact that, fundamentally, the instrument was evaluated and discussed by experts on the mental health field, who analyzed it and considered conceptual, semantic, idiomatic, and experiential equivalences, which resulted in substantial agreement between the items and the construct to be measured.
The equivalence between the original instrument and its version adapted to Brazil was maintained by including family members as a stakeholder in both pilot studies. There was a special emphasis on using techniques that allowed face-to-face meetings with participants, involving them not only as spectators who filled in the questionnaires, but also encouraging them so they could talk about the strengths, weaknesses, and applicability of the instrument to the Brazilian context.
We made this choice aiming at maintaining the procedures adopted in the creation of the original RSA, in which there was involvement of stakeholders (person in recovery, family member, provider, and CEO and directors) who assessed the scale in terms of content and understanding7. This strategy was repeated in most of the cross-cultural adaptations of RSA performed in other countries, especially in the versions concerning providers and users10,15,33,34. In the process of cross-cultural adaptation of the RSA Family carried out in Canada and China, no involvement of family members in the process was identified, only the participation in answering the questionnaire9,11.
In Brazil, as pointed out by Vasconcelos31, there are socioeconomic and cultural differences (low education, difficulty in accessing citizen’s rights and exercising citizenship) that challenge the dialogue about recovery and its implementation. This statement was evidenced in the cross-cultural adaptation of the RSA-R Family/Brazil in four factors.
The first one was the participants’ difficulty in filling in the instrument without guidance, in such a way we made the Brazilian version of the RSA-R Family/Brazil to be applied through an interview. In specific studies on the family member’s version, we found no evidence of the use of face-to-face strategies to fill in the questionnaire9,11. In the aforementioned Chinese study, the questionnaire was filled in by mail, which resulted in great loss of data9.
However, we observed other strategies adopted to support participants in completing the questionnaire – specifically in the RSA Person in Recovery version – such as the research team reading the questions to illiterate users in China17. In addition, we observed the possibility of answering the questions through face-to-face or telephone interviews with researchers in the United Kingdom12,15. Also, we verified the hiring of providers from the peer support program, for the recruitment and provision of support to users, who helped users to fill in the questionnaire in Sweden and Canada11,14. These strategies resulted in positive outcomes, mainly in Sweden, since users reported they felt safe with the peer support work and appreciated the received support14.
In the original RSA-R questionnaire, item 21 refers to the existence of peer support programs in mental health services. However, such programs have not yet been implemented in Brazil and, therefore, throughout the process of cross-cultural adaptation, this was a challenging topic. After the evaluation of experts and family members, we decided to keep it, considering the challenge of implementing practices that directly include persons in recovery and their families. This discussion reinforces the importance of using instruments internationally developed, which not only evaluate the currently available practices and strategies, but also those that mobilize the creation of new recovery-oriented strategies, knowledge, and projects in mental health services and that enable an international dialogue.
According to our investigation, we can state that applying the RSA-R Family/Brazil in the form of an interview is a valid strategy. Furthermore, this strategy has already been employed in other study whose authors adapted another mental health scale for users and family members in Brazil35,36. Nevertheless, this does not prevent researchers from thinking about new strategies in the future – such as the implementation of peer support programs, both for users and family members –, since there are several studies whose authors argue that the implementation of peer programs (and their support in research) generates positive results for users and family members, who usually feel less embarrassed when responding to a peer rather than to a professional34.
The second factor that made the adaptation of the RSA-R Family to the Brazilian context challenging was the number of items and the language used in the original version, which requires great literacy skills. For the Brazilian version, we needed to reduce the extent of the items, removing examples and adapting the items to a simpler language, with the assistance of a popular educator. This factor was acknowledged in a literature review6 and in previous studies carried out in Canada and in the United States of America11,34. Authors of such studies recommended that, in future attempts to develop new items for the RSA, one must consider the formulation of simple and clear questions concerning part of the construct, which may not only improve the structure of the scale, but also facilitate the interpretation of future outcomes on the part of participants, researchers, and clinicians8. In this sense, the Brazilian version of the RSA-R Family would be the precursor in enabling the scale in a more accessible language, broadening comparison possibilities with other countries with similar social standards.
RSA-R, in all its versions, has a five-point Likert-type scale for measuring agreement. Adapting this scale of responses was the third challenging factor in the Brazilian version of the RSA-R Family. To do so, we needed to write simpler words in each number of the scale and insert graphic elements. Although we did not verify similar experiences in other countries, in Sweden, for instance, the Likert scale was repeated as the header on each new page of the questionnaire14. In a study conducted in the United States of America, the authors created an instrument based on the RSA, with the alteration to a five-point Likert-type scale for measuring frequency34. We believe that changes made in the Brazilian version of the RSA-R Family will help family members in filling in the questionnaire and in its applicability, which will allow future studies to provide psychometric evidence of variability and distribution between responses. Nevertheless, we do not disregard the possibility of adapting, in the Brazilian version of the RSA-R Family, the frequency scale for future questionnaires or to reduce the number of response categories. According to the study on the brief version of the RSA, the reduced number of categories may be more favorable for mental health users, as well as for the general population, and encourages further research for testing the RSA with fewer categories of responses to improve the valid recovery-oriented measurement potential in people with mental illness8.
The adaptation of the recovery concept was the last challenging factor; it was inserted in most items as “recuperação” after consensus in the FG with family members. As reported in the literature, it was difficult to find an equivalent term in Brazilian Portuguese. Overall, the term “recuperação” (Portuguese term for “recovery”) is mistaken for “cura” (Portuguese term for “cure”). As well as researchers from other countries (China, Germany, and Sweden) who have adapted this scale, we believe that recovery, as a paradigm of mental health care, may broaden the spectrum of intervention and stimulate the creation of activities and practices that enable us to overcome the biomedical model, enhancing the autonomy of users and their family members in activities, such as job creation, educational and cultural access, and the implementation of strategies such as peer support programs9,10,14,18.
Although in this study we sought to adapt the RSA-R Family instrument to the Brazilian context, we brought to light elements of discussion about the cultural differences in Brazil and the applicability possibilities of the construct, when compared with other countries. In addition, we sought to maintain, in this version, the highest level of equivalence with the original questionnaire, seeking to respect, at all steps, the culture to which we were adapting it. Therefore, it is worth stating that, in order to demonstrate evidence of the reliability and validity of this scale in the present context, future psychometric studies shall be necessary.
Acknowledgment
The authors thank Espaço da Escrita – Pró-Reitoria de Pesquisa – UNICAMP - for the language services provided.
Bibliographical References
1. American Educational Research Association, American Psychological Association, National Council on Measurement in Education. Standards for educational and psychological testing. Washington: American Educational Research Association; 2014.
2. Sireci S, Faulkner-Bond M. Validity evidence based on test content. Psicothema [jornal on the Internet] 2014 Oct [cited 2019 Aug 6]; 26 (1): [about 7 p.]. Available from: http://www.psicothema.com/PDF/4167.pdf
3. Alexandre NMC, Coluci MZO. Validade de conteúdo nos processos de construção e adaptação de instrumentos de medidas. Cien Saude Colet [jornal on the Internet] 2011 Jul [cited 2017 Mar 28]; 16 (7): [about 7 p.]. Available from: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S1413-81232011000800006
4. Anthony WA. Recovery from mental illness: The guiding vision of the mental health service system in the 1990s. Psychosoc Rehabil J [jornal on the Internet] 1993 Apr [cited 2019 set 19]; 16 (4): [about 2 p.]. Available from: https://psycnet.apa.org/buy/1993-46756-001
5. Rosa LC dos S. Transtorno mental e o cuidado na familia. 2nd Edition. São Paulo, SP: Cortez; 2008.
6. Williams J, Leamy M, Bird V, Harding C, Larsen J, Boutillier C Le, et al. Measures of the recovery orientation of mental health services : systematic review. Soc Psychiatry Psychiatr Epidemiol [jornal on the Internet] 2012 Nov [cited 2019 May 28]; 47 (11):[about 8 p.]. Available from: https://link.springer.com/article/10.1007%2Fs00127-012-0484-y
7. O’Connell M, Tondora J, Croog G, Evans A, Davidson L. From rhetoric to rutine: assessing perceptions of recovery-oriented practices in a state mental health and addiction system. Psychiatr Rehabil J [jornal on the Internet] 2005 Nov [cited 2017 Feb 2]; 28 (4): [about 8 p.]. Available from: https://psycnet.apa.org/record/2005-04063-010
8. Barbic SP, Kidd SA, Davidson L, Connell MJO. Validation of the brief version of the recovery self-assessment (rsa-b) using rasch measurement theory. Psychiatr Rehabil J [jornal on the Internet] 2015 Dec [cited 2019 May 28]; 38 (4): [about 9 p.]. Available from: https://psycnet.apa.org/record/2015-26457-001
9. Bola J, Chan THC, Chen EH, Ng R. Cross-validating chinese language mental health recovery measures in Hong Kong. Res Soc Work Pract [jornal on the Internet] 2016 Jan [cited 2019 May 28]; 26 (6): [about 6 p.]. Available from: https://journals.sagepub.com/doi/abs/10.1177/1049731515625326?journalCode=rswa
10. Burgess P, Pirkis J, Coombs T, Rosen A. Assessing the value of existing recovery measures for routine use in Australian mental health services. Aust N Z J Psychiatry [jornal on the Internet] 2011 [cited 2019 May 28]; 45 (4): [about 13 p.]. Available from: https://doi.org/10.3109/00048674.2010.549996
11. Kidd SA, George L, O’Connell M, Sylvestre J, Kirkpatrick H, Browne G, et al. Fidelity and recovery-orientation in assertive community treatment. Community Ment Health J [jornal on the Internet] 2010 Apr [cited 2019 May 28]; 46 (4):[about 8 p.]. Available from: https://journals.sagepub.com/doi/abs/10.3109/00048674.2010.549996?rfr_dat=cr_pub%3Dpubmed&url_ver=Z39.88-2003&rfr_id=ori%3Arid%3Acrossref.org&journalCode=anpa
12. Leamy M, Clarke E, Boutillier C Le, Bird V, Choudhury R, Macpherson R, Pesola F, Sabas K, Williams P, Slade M. Recovery practice in community mental health teams: national survey. Br J Psychiatry [jornal on the Internet] 2016 Oct [cited 2019 May 28]; 209 (4):[about 6p.]. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5046739/
13. Petrakis M, Brophy L, Lewis J, Stylianou M, Scott M, Cocks N, Buckley L, Halloran K. Consumer measures and research co-production: a pilot study evaluating the recovery orientation of a mental health program collaboration. Asia Pac J Soc Work Dev [jornal on the Internet] 2014 Apr [cited 2019 May 28]; 24 (1–2):[about 14 p.]. Available from: https://www.tandfonline.com/doi/abs/10.1080/02185385.2014.885212
14. Rosenberg D, Svedberg P, Schon U-K. Establishing a recovery orientation in mental health services: evaluating the recovery self-assessment (rsa) in a Swedish context. Psychiatr Rehabil J [jornal on the Internet] 2015 Dec [acessado 2019 May 28]; 38 (4): [about 7 p.].328–35. Available from: https://psycnet.apa.org/record/2015-25437-001
15. Simpson A, Hannigan B, Coffey M, Barlow S, Cohen R, Jones A, Všetečková, J, Faulkner A, Thornton A, Cartwright. Recovery-focused care planning and coordination in England and Wales: a cross-national mixed methods comparative case study. BCM Psychiatry [jornal on the Internet] 2016 May [cited 2019 May 28]; [about 7 p.]. Available from: https://bmcpsychiatry.biomedcentral.com/articles/10.1186/s12888-016-0858-x
16. Waldemar AK, Esbensen BA, Korbek L, Petersen L, Arnfred S. Recovery orientation in mental health inpatient settings: Inpatient experiences? Int J Ment Health Nurs [jornal on the Internet] 2018 [cited 2019 May 28]; 27 (3):[about 10 p.]. Available from: https://onlinelibrary.wiley.com/doi/abs/10.1111/inm.12434
17. Ye S, Pan J, Wong DFK, Bola JR. Cross-validation of mental health recovery measures in a Hong Kong chinese sample. Res Soc Work Pract [jornal on the Internet] 2013 Jan [cited 2019 May 28]; 23 (3):[about 14 p.]. Available from: https://onlinelibrary.wiley.com/doi/abs/10.1111/inm.12434
18. Zuaboni G, Kieser LD, Glavanovits K, Utschakowski J, Behrens J. Recovery self assessment-übersetzung und kulturelle anpassung eines recovery-orientierten einschätzungsinstrumentes. Pflege [jornal on the Internet] 2015 Aug [cited 2019 May 28]; 28 (4):[about 10 p.]. Available from: https://econtent.hogrefe.com/doi/abs/10.1024/1012-5302/a000436
19. Thomas EC, Despeaux KE, Drapalski AL, Bennett M. Person-oriented recovery of individuals with serious mental illnesses: a review and meta-analysis of longitudinal findings. Psychiatr Serv [jornal on the Internet]. 2018 Mar [cited 2019 Oct 15]; 69 (3): [about 7 p.]. Available from: https://doi.org/10.1176/appi.ps.201700058
20. Zipursky RB, Reilly TJ, Murray RM. The myth of schizophrenia as a progressive brain disease. Schizophr Bull [jornal on the Internet]. 2012 Nov [cited 2019 Oct 15]; 39 (6): [about 9 p.]. Available from: https://doi.org/10.1093/schbul/sbs135
21. Leonhardt BL, Huling K, Hamm JA, Roe D, Hasson-Ohayon I, McLeod HJ, et al. Recovery and serious mental illness: a review of current clinical and research paradigms and future directions. Expert Rev Neurother [jornal on the internet]. 2017 Nov [cited 2019 oct 15]; 17(11): [about 13 p.]. Available from: https://doi.org/10.1080/14737175.2017.1378099
22. O’Connell M, Tondora J, Kidd, Stayner, Hawkins, Davidson L. RSA-R Family Member/Significant Other Version [Internet]. Yale University. 2007 [cited 2017 Feb 14]. Available from: http://medicine.yale.edu/psychiatry/prch/tools/RSA_Family_Advocate_tcm800-204209_tcm800-284-32.pdf
23. Wild D, Grove A, Martin M, Eremenco S, McElroy S, Verjee-Lorenz A, Erikson P. Principles of good practice for the translation and cultural adaptation process for patient-reported outcomes (pro) measures. Value Health [jornal on the Internet] 2005 Mar-Apr [cited 2017 Feb 2]; 8(2):[obout 9 p.]. Available from: https://linkinghub.elsevier.com/retrieve/pii/VHE4054
24. Onocko Campos RT, Costa M, Pereira MB, Ricci EC, da Silva TEG, Erazo CLJ, Reis G, Davidson L. Recovery, citizenship, and psychosocial rehabilitation: A dialog between Brazilian and American mental health care approaches. Am J Psychiatr Rehabil [jornal on the Internet]. 2017 Aug [cited 2017 Aug 17]; 20 (3):[about 15p.]. Available from: https://doi.org/10.1080/15487768.2017.1338071
25. Gorenstein C, Wang Y, Hungerbühler I. Fundamentos de mensuração em saúde mental. In: Gorenstein C, Wang Y, Hungerbühler I, organizers. Instrumentos de avaliação em saúde mental. Porto Alegre: Artmed; 2016. p. 1–52.
26. Pereira MB. Avaliação de serviços de saúde mental: validação da versão para trabalhadores de instrumento de avaliação de Recovery [thesis]. Campinas: Faculdade de Ciências Médicas Universidade Estadual de Campinas; 2019. Available from: http://repositorio.unicamp.br/jspui/handle/REPOSIP/334852
27. Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine (Phila Pa 1976) [jornal on the Internet] 2000 Dec [cited 2017 Mar 28];25 (24):[about 5 p.]. Available from: https://insights.ovid.com/crossref?an=00007632-200012150-00014
28. Eremenco SL, Cella D, Arnold BJ. A comprehensive method for the translation and cross-cultural validation of health status questionnaires. Eval Health Prof [jornal on the Internet] 2005 Jun [cited 2017 Aug 28]; 28 (2): [about 20 p.]. Available from: https://journals.sagepub.com/doi/10.1177/0163278705275342
29. Onocko-Campos R. O planejamento no labirinto: uma viagem hermeneutica. São Paulo: Hucitec Editora; 2003.
30. Baccari IOP, Onocko-Campos R, Stefanello S. Recovery: revisão sistemática de um conceito. Cien Saude Colet [jornal on the Internet] 2015 Jan [cited 2017 Jan 20]; 20 (1): [about 11 p.]. Available from: http://dx.doi.org/10.1590/1413-81232014201.04662013
31. Vasconcelos EM. As abordagens Anglo-Saxônicas de empoderamento e recovery (recuperação, restabelecimento) em saúde mental II: uma avaliação crítica para uma apropriação criteriosa no cenário brasileiro. Cad Bras Saúde Ment [jornal on the Internet] 2017 Feb [cited 2019 May 28]; 9 (21): [about 17 p.]. Available from: https://periodicos.ufsc.br/index.php/cbsm/article/view/69535
32. Dantas CDR, Oda AMGR. Cartografia das pesquisas avaliativas de serviços de saúde mental no Brasil (2004-2013). Physis Rev Saúde Coletiva [jornal on the Internet] 2014 Dec [cited 2017 Mar 15]; 24 (4):[about 52p.]. Available from: http://dx.doi.org/10.1590/S0103-73312014000400008.
33. Mcloughlin KA, Wick A Du, Collazzi CM, Puntil C. Recovery-oriented practices of psychiatric-mental health nursing staff in an acute hospital setting. J Am Psychiatr Nurses Assoc [jornal on the Internet] 2013 Jun [cited 2019 May 18]; 19 (3):[about 7p.]. Available from: https://doi.org/10.1177/1078390313490025
34. Lodge AC, Kuhn W, Earley J, Manser SS. Initial development of the recovery-oriented services assessment : a collaboration with peer-provider consultants. Psychiatr Rehabil J [jornal on the Internet]. 2018 Jun [cited 2019 May 28]; 41 (2):[about 10p.]. Available from: http://dx.doi.org/10.1037/prj0000296
35. Bandeira M, Calzavara MGP, Costa CS, Cesari L. Avaliação de serviços de saúde mental: adaptação transcultural de uma medida da percepção dos usuários sobre os resultados do tratamento. J Bras Psiquiatr [jornal on the Internet] 2009 May [cited 2017 Jan 20];58 (2): [about 7 p.]. Available from: http://dx.doi.org/10.1590/S0047-20852009000200007
36. Bandeira M, Felicio CM, Cesari L. Validation of the perception of change scale- family version (emp-f) as a treatment outcome measure in mental health services. Rev Bras Psiquiatr [jornal on the Internet] 2010 Sep [cited 2017 Aug 24]; 32 (3): [about 4 p.]. Available from: http://dx.doi.org/10.1590/S1516-444620100050000