Artigos

Conteúdo (article):

MHEALTH IN THE PREVENTION OF SEXUALLY TRANSMITTED INFECTIONS STIs
Rafaela Oliveira de Sales
ORCID ID: 0000-0002-5725-279X
rafaelasalesunifor@gmail.com
Universidade de Fortaleza-UNIFOR

Raimunda Magalhães da Silva
ORCID: 0000-0001-5353-7520
rmsilva@unifor.br
Universidade de Fortaleza-UNIFOR
Abstract
The objective of this study was to analyze in the literature how the applications were developed and how they influence the prevention of Sexually Transmitted Infections (STIs). PubMed, CAPES Periodicals and Bireme databases were searched using the descriptors sexually transmitted diseases and mHealth combined by the Boolean connector AND. The search considered studies published between 2013 to 2017 available online in Portuguese and English. The results of the studies showed formative research was the most used method and focus group was the most used technique for data collection in the development of an application. This technique aims at the exchange of experience and allows discussion about issues inherent to infections. The applications were built with methodological rigor with the participation of users and with instructional resources that influence the management of the prevention and control of infections.
Keywords Sexually Transmitted Infection, mHealth, Technology.

Introduction
The expansion of mobile health (mHealth) and the Internet in recent years has made the health-related applications market evolve rapidly, thus enhancing the potential of this sector1.
Applications “apps” are computer programs designed for smartphones, tablets and other mobile devices that offer games, location-based services, access to sites for social and sexual relations and health care2.
Thus, “apps” have great potential to disseminate health information for the population, especially among patients with stigmatizing diseases, such as sexually transmitted infections (STIs)3. There are an estimated 357 million new cases of curable STIs every year among people aged 15-49. The rate is similarly high for viral infections, with an estimated 417 million people infected4.
This type of technology provides the user with many opportunities, such as the notification of sexual partners, the location of specialized health services, obtaining information on the modalities of transmission and on how to get treatment and on preventive measures5,6.
With the popularization of applications for the promotion of sexual health, there has also been a growth in gay apps such as Grindr® and Tinder® designed to facilitate social and sexual encounters that can provide risky sexual behavior in certain contexts. On the other hand, they are also spaces where professionals can promote actions aimed at sexual health7.
Latin American countries have approximately 332 million users with Internet access, 114 million of whom are Brazilians. The application most used by Brazilians is WhatsApp®, with 46 million users. Currently, the two largest application distribution platforms are the App Store and Google Play8,9.
The result of this growth mobilizes the researchers’ efforts to carefully evaluate the methodological and scientific aspects of the applications in order to ensure that their components can reflect on behavior change and maximize their effectiveness in controlling STI. However, producing research using mHealth is a complex process of intervention that involves specific characteristics of the Information and Communication Technology (ICT) field in all phases of the project10,11.
Due to the importance of mobile technology, the use of this tool by a large number of people in the world and the scarcity of studies on applications for prevention, diagnosis, treatment and control of infectious diseases, question has been raised regarding the methodological approaches used for the development of the applications and on how they influence prevention of STIs.
In this context, this study sought to analyze in the literature the methods used in the design of the applications and their influence on the prevention of STIs.
Methodology
This is an integrative review as it allows the synthesis of multiple published studies and general conclusions regarding a particular area of study12. This study was conceived by defining a research question and formulating a hypothesis, establishing inclusion and exclusion criteria, categorizing studies, evaluating studies, interpreting results and synthesizing knowledge12.
The following research question was selected for the study: How are applications developed and how can they influence STI prevention?
The electronic search was conducted in December 2017 on the databases of PubMed, CAPES Periodicals and Bireme. Articles published in the last five years (2013 to 2017) in English and Portuguese and fully available were selected. As there are no specific descriptors for this theme in the Health Sciences Descriptors (Descritores em Ciências da Saúde – DeCS), the following terms in English were used: sexually transmitted diseases and mHealth combined by the Boolean connector AND.
Inclusion criteria were qualitative studies that addressed the methodological process and the contributions of focus groups for sites and application for the prevention of STIs considering the reading of the titles, abstracts and published full-text articles. Quantitative studies were excluded as they did not use focus groups to analyze the participants’ opportunities to design the technology. Review studies, theses, dissertations, and essays that were not fully available were also excluded.
The studies were organized using a chart containing data about the authors, year of publication, objective, method and results.
The level of scientific evidence of the studies included in the research was measured using the Evidence Based Medicine (EBM) instrument, which seeks to obtain relevant studies and critically analyze the quality of publication13 classifying the studies in:
• Grade A: studies supported by good evidence (level of evidence 1- systematic review of randomized controlled trials or meta-analyses of randomized clinical trials) favorable or contrary to the recommended intervention.
• Grade B: studies supported by reasonable evidence (level of evidence 2 - prospective comparative studies, meta-analyses of level 2 studies or level 1 studies with inconsistent results or level 3 studies - retrospective cohort studies, case-control studies or observational studies with control).
• Grade C: studies with conflicting or poor quality evidence (level 4 - observational studies without control or extrapolations of levels 2 or 3 studies).
• Grade D: studies without sufficient evidence for recommendations and often derived from level 4 evidence or inconsistent or inconclusive studies of any level.
Results
The search for studies in December 2017 in the selected databases yielded 416 studies (20 in Bireme, 148 in CAPES and 248 in Medline), 20 of which were duplicates and were hence withdrawn, thus remaining 396 studies for the general evaluation. A total of 364 studies were excluded based on the title and because the subjects addressed were unrelated to the subject of interest. Thus, 32 studies were fully read to assess their eligibility for inclusion in the study. After reading the 32 studies, 17 were excluded because they did not describe the qualitative approach used for the design of the applications. Therefore, a total of 15 studies were selected for analysis as shown in Figure 1.
Figure 1
The selected studies were carefully organized and evaluated with regard to their methodological quality and relevance of the information. The 15 studies selected for review are described in Chart 1.
Chart 1
Of the 15 studies selected, one was published in 201714, five were published in 201615-19, six were published in 201520-25, one was published in 201426 and two were published in 201327,28. Most studies were conducted in the United States14,15,16,18,19,20,21,23,24,25,26,27, the United Kingdom17,22 and in Africa28. All the studies were published in English.
With regard to the methodological processes used in the development of the 15 studies analyzed in this review, nine studies used formative research15,16,18,19,20,22,23,27,28, one study used mixed methods25, and five studies used qualitative approaches14,17,21,24,26.
All the studies use the focus group technique to obtain data14-28. Two studies deepened the findings by carrying out individual interviews21,28 and six studies also collected quantitative data15,16,19,20,25,28.
It should be noted that 14 studies were about apps with information about HIV prevention and treatment14,15,16,17,18,19,20,21,23,24,25,26,27,28. mHealth interventions have been promising to improve patient-professional communication in addition to providing health education and supporting self-management of HIV27,28.
One of the studies describes the development of an application for the prevention of chlamydia22. Chlamydia infection is highly prevalent among the young population aged 15 to 24 years. It is frequently asymptomatic in 70% of women and 50% of men and can lead to serious reproductive health-related morbidities29.
A total of eight applications are for men who have sex with men (MSM)14,15,16,19,20,23,26,27. Gays or bisexuals and MSM are among the groups in which HIV is out of control worldwide. These are 24 times more likely to be infected with HIV than men in the general population. The prevalence of HIV among MSM is 15% higher30,31.
Of the analyzed studies, three were designed for the prevention of HIV among black MSM14,20,27. HIV has disproportionately affected MSM since the onset of the epidemic. Currently, almost 40% of people living with HIV in the United States are African American32.
However, HIV prevention and treatment strategies do not delay the epidemic because they require significant involvement of a health system that addresses the needs of black MSM by removing barriers such as racism and homophobia during consultations and the discomfort and mistrust associated with the disclosure of sexual behavior by health professionals20,32.
Discussion
The applications analyzed used different methods with comprehensive and convergent information that influence the prevention of STIs. Some of the methods that should be highlighted were the qualitative approach for the development of the studies and the focus group technique for the design of the applications. In this regard, formative research consisted of the most appropriate method for the effective conception and implementation of mHealth with the participation of the population and health professionals at all phases of the project16,27,28.
In the initial phases of the development of the intervention, the formative research used focus groups and interviews with populations and stakeholders to determine users’ barriers, capacities and motivations33. In the final phases, this method could be used to evaluate material acceptability, literacy level, content, and forms of implementation34. These steps facilitate the effective development of the intervention and allow specialists to analyze whether the intervention can be adapted to new settings or populations35.
All the studies used the focus group technique in order to understand the preferences for functions and interface15,16,22,23,26,27 of applications for HIV prevention, treatment and testing promotion14,17,18,20,21,24,28 and adherence to antiretroviral therapy19,25.
Focus groups consist of the ability to share experiences with others and thus identify barriers to treatment and assess the positive and negative aspects of applications for STIs14,17,24. However, given the stigmatizing and discriminatory nature of the topic, the lack of anonymity and the sharing of sexual experiences with community members can generate discomfort that interferes with the participants’ responses17,21,24.
Maximizing the benefits and minimizing the harm to the participants involved in the study is one of the research team’s ethical responsibilities. Stakeholders can bring valuable contributions regarding potential social harms from participating in clinical trials. This is of particular importance in the case of vulnerable, marginalized, stigmatized individuals or groups who hold less power in society.
Even so, the participation of users allows understanding the resources and functions that should be featured in application for HIV/ STI prevention15,18,20,22,23,27, facilitates communication among health professionals20, encourages adherence to antiretroviral therapy16,19 and provides information on the care of patients with HIV/AIDS28.
The Community-Based Participatory Research (CBPR), which is based on the Health Beliefs Model construct, allowed community involvement in building the design and content of applications for HIV prevention and treatment of patients receiving primary care21 and young Africans living in the United Kingdom17. This model addresses environmental vulnerabilities for the implementation of new technologies for HIV prevention37.
The Information, Motivation, and Behavioral Skills (IMB) model used in designing an application for MSM has identified the content, characteristics and functions of an application for HIV prevention. This approach was designed to facilitate the development of technology for a vulnerable demographic group23.
The IMB includes three features that influence behavior change for adherence to antiretroviral therapy: (1) information - knowledge about drugs; (2) motivation - personal and social motivations; and (3) behavioral skills - the person’s ability to perform the tasks required for adherence38.
However, software building integrates users’ contributions for the design of the application interface, which provides the research team with opportunities to share conceptual models with participants from the beginning of the design process in order to collect feedback and make the necessary changes21.
Thus, qualitative evaluation, social learning theory and software development can complement each other and are important components for the development of a culturally personalized and clinically relevant application39.
In order to obtain methodological rigor for the design of the applications, Information Systems Research (ISR) associated with Human Computer Interaction (HCI) incorporated the feedback from members of the target population into the design preferences, barriers and facilities that encourage or limit the use of technology for the prevention and treatment of HIV. The structure of ISR consists of three interrelated cycles: (1) relevance, (2) rigor and (3) project40.
Design guidelines play an important role in HCI as they guide the development of the system and provides data for heuristic evaluation through which HCI specialists eliminate remaining flaws in the system22. It is an interdisciplinary area of research and practice based on the traditions of Humanities, Computer Science, Information Systems, Psychology, Sociology and Visual Design. HCI is about designing technologies centered on the users and their daily needs and routines41.
Another aspect analyzed was the low-fidelity prototype, which allowed exploring users’ preferences regarding content, tool performance and information for the elaboration of functional requirements16. The prototype usability test aims to identify bugs, provides feedback on interface and navigability19. Prototypes are essential tools in the product design process and can help minimize design errors that may occur at the beginning and end of the process42.
The applications presented similar features and functions related to HIV/STI prevention and treatment14,16,19,23,28, interaction with health professionals14,18,20,23,27, online forum with other users14,18,23,24,27, reminders of the dates to take the HIV test15,20 and days of consultations27, medication schedules19,20,27, side effects of antiretroviral therapy16,19,28, information about safe sex23,27,26,21,20 and use of illicit substances18,27, and the location of health care facilities14,15,16,20,23,27.
One feature that should be highlighted is the Short Message Service (SMS), which is an interactive, quick, easy and personalized intervention15,17,24,27. The benefits of this intervention include the convenience and omnipresence24 that allow sending reminders for consultations and medication time and the provision of tips for reducing the risk of getting HIV and interaction with the health professional27. However, frequent messages are unnecessary and irritating and hence increase the likelihood of interrupting the use of the application19. Therefore, it is necessary that the user can control the frequency and type of messages received27.
On the other hand, messages intended to motivate adherence to antiretroviral therapy may have undesired effects among HIV patients who have not yet accepted the diagnosis19. Thus, positive and reassuring messages should be included to avoid exclusive focus on HIV17. SMS has been used in South Africa for health promotion among patients living in rural and urban areas with the aim of improving HIV care43.
Self-monitoring and self-management of sexual behavior via SMS offer new opportunities to involve HIV patients in the period between clinical visits and the daily routine25. However, this component should be considered educational and not just a record of sexual activities15. SMS offers users a source of information about HIV, adherence to antiretroviral therapy and communication with health professionals44.
The language should be professional, fun and allow customized adaptations made by the users. However, this option may not be feasible as it would require twice the work to adjust the application. A cost-effective option may be the incorporation of two formal and informal languages26. Programmers need to understand the limitations related to the population’s perception as the information presented in the application can be intelligible45.
The studies pointed out that in order to involve users applications must incorporate innovative ideas26 using a visually attractive design16,20, be intuitive, contain games18, incorporate social media platforms26,28 with clear texts18, and be cheap or free15,20.
The requirements for downloading an application consist of gratuity, data security and privacy, intuitive interface, proper functioning, control of sounds and other resources, good ratings given by other users and absence of constant system updates14,15,27. These findings are in line with other studies which refer to the gratuity and positive evaluation by other users as requirements for downloading an application46,47.
Social privacy allows users to control access to personal information in their devices through passwords, intimate partner notification and visual identity without any association to the theme. Institutional privacy is the unauthorized collection and disclosure of users’ personal data by outsourced organizations22.
Researchers should protect participants’ data from potential privacy violations, including disclosure of participation in the study or personal health information, especially on delicate topics such as HIV status and sexual behavior and orientation48. Applications should be designed to have a discreet, neutral, non-theme-linked, and password-protected interface. These requirements are necessary to avoid stigma related to STIs and to encourage the use of technology49.
Regarding the personal information inserted in an application, studies point to the need for a credible source that clarifies the terms of description and privacy of users’ data, otherwise, it could generate users’ distrust and hesitation to download it14,15,17,18, 20,21,22,23,24,26,27,28. Therefore, before extending interventions using mHealth, it is essential for researchers to demonstrate that applications keep the privacy of patient data27.
Clearly defined privacy policies on the handling of user information are important to safeguard users’ privacy47. The involvement of health professionals during application development does not necessarily guarantee their effectiveness; it probably provides a greater insight into patients’ needs rather and suggests that the content is trustworthy50.
Applications can be integrated into care routines for the prevention and treatment of STIs14,22 among hard-to-reach populations27 such as MSM who have occasionally reported racist and homophobic judgments made by health professionals working in primary care20,14.
Tackling HIV-related stigma in health care settings and addressing related effects will require more than interventions targeted at health professionals or individuals with HIV. Although interventions to reduce stigma among people with HIV can mitigate some individual effects on health, they do not address their social insertion that promotes such promulgated stigma51.
Smartphones are tools used by MSM on a daily basis for SMS, online video chatting, status postings, blogs and more. Many men use the tool to find sexual partners and also HIV/STI information27. There is no consensus in the literature about the association of sexual encounters using applications and increased exposure to HIV. It is believed that applications facilitate multiple partnerships and act as a catalyst for risky sexual behavior52.
Websites and applications allow users to create profiles with images on a social network to chat, share media files, and locate sexual partners using the global positioning system27. Some of the most accessed social networks are Facebook®, Twitter®14,27 and Instagram®14. Jack’d®, Grind® and Adam4Adam®14,27 are applications of geosocial networks. Social or sexual sites are Craigslist®14 and Black Gay Chat®27 and text messaging and chat applications such as Oovoo®, Tango®, and Skype®27.
Recently, Facebook® and Grindr®, social media platforms, partnered with researchers to disseminate information on HIV prevention, promote testing, and provide information about medical care for MSM53,54. Studies conducted in the United States and the United Kingdom have shown that promotion of HIV testing can be conducted through geosocial applications55,56.
Final Considerations
The studies analyzed prioritized formative research, which uses a qualitative and quantitative approach for a comprehensive understanding of cultural, social, gender, sexuality, political and structural factors aspects of the population. It establishes important planning for the development of applications for the prevention of STIs which can reduce the spread of infection and guide the identification and treatment of sexual partners.
The technique most used in the studies was the focus group, which broadened the context experienced by the health professionals and the users’ expectations. This technique aims at the exchange of experience during the intervention and allows the discussion of issues inherent to the human being with HIV/STIs. This technique helps researchers to elaborate applications focused on the needs of the users and also allows to understand the prevention, follow up and treatment of infected people.
The studies showed the importance of applications for self-monitoring and self-management of care for people with HIV/STIs. The participants identified the need to incorporate information on prevention, antiretroviral therapy, online forum with health professionals and other users, reminders of consultations and testing and location of health care facilities. In addition, applications must contain a discreet interface, be interactive, free or inexpensive, protect the privacy of users’ data and use trustworthy sources.
Researchers integrate features and information from geospatial networks that use the Global Positioning System (GPS) to locate sexual partners for the purpose of promoting health surveillance measures aimed at the prevention and treatment of STIs.
Therefore, obtaining mHealth technology to assist health professionals in providing care to STI patients is an important step in the control, prevention and treatment and in the design of STI epidemiological surveillance programs.
References
1.Xu W, Liu Y. mHealthApps: a repository and database of mobile health apps. JMIR Mhealth Uhealth. 2015;3(1):e28. DOI:10.2196/mhealth.4026

2.Muessig KE, Nekkanti M, Bauermeister J, Bull S, Hightow-Weidman LB. A systematic review of recent smartphone, Internet and Web 2.0 interventions to address the HIV continuum of care. Curr HIV/AIDS Rep. 2015 Mar;12(1):173-90. DOI:10.1007/s11904-014-0239-3

3.Harding-Esch E, Jofre-Bonet M, Dhanjal JK, Burr S, Edwards T, Holland M, et al. Costs of testing for ocular Chlamydia trachomatis infection compared to mass drug administration for trachoma in the Gambia: application of results from the PRET study. PLoS Negl Trop Dis. 2015;9(4):e0003670. DOI:10.1371/journal.pntd.0003670

4.World Health Organization (WHO). Global Health Sector Strategy on Sexually Transmitted Infections 2016–2021.Towards Ending STIs. 2016. Disponível em: http://apps.who.int/iris/bitstream/handle/10665/246296/who-rhr-16.09-eng.pdf;jsessionid=FE6F73393DD8F8CD07A6C3E481967ADF?sequence=1Acesso em 11 de abril de 2019.

5.Rana AI, van den Berg JJ, Lamy E, Beckwith CG. Using a Mobile Health Intervention to Support HIV Treatment Adherence and Retention Among Patients at Risk for Disengaging with Care. AIDS Patient Care STDS. 2016 Apr;30(4):178-84. DOI:10.1089/apc.2016.0025

6.Estcourt CS, Gibbs J, Sutcliffe LJ, Gkatzidou V, Tickle L, Hone K, et al. The eSexual Health Clinic system for management, prevention, and control of sexually transmitted infections: exploratory studies in people testing for Chlamydia trachomatis. Lancet Public Health. 2017 Apr;2(4):e182-e190. DOI:10.1016/S2468-2667(17)30034-8

7.Grov C, Breslow AS, Newcomb ME, Rosenberger JG, Bauermeister JA. Gay and bisexual men’s use of the Internet: research from the 1990s through 2013. J Sex Res. 2014;51:390–409. DOI:10.1080/00224499.2013.871626

8.Avazu Holding. Global Internet Industry Research Brazil. 2016. Disponível em: http://avazuinc.com/wordpress/wp-content/download/en/Global%20Internet%20Industry%20Research%20-%20Brazil(Full%20Edition).pdf Acesso em 11 de abril de 2019.

9.Instituto Brasileiro de Geografia e Estatística (IBGE). Pesquisa nacional por amostra de domicílios: síntese de indicadores 2015/IBGE, Coordenação de Trabalho e Rendimento. Rio de Janeiro: IBGE, 2016. 108p. Disponível em: https://biblioteca.ibge.gov.br/visualizacao/livros/liv98887.pdf . Acesso em 11 de abril de 2019.

10.Ben-Zeev D, Schueller SM, Begale M, Duffecy J, Kane JM, Mohr DC. Strategies for mHealth research: lessons from 3 mobile intervention studies. Adm Policy Ment Health. 2015 Mar;42(2):157-67. DOI:10.1007/s10488-014-0556-2

11.Gibbs J, Gkatzidou V, Tickle L, Manning SR, Tilakkumar T, Hone K, et al. \'Can you recommend any good STI apps?\' A review of content, accuracy and comprehensiveness of current mobile medical applications for STIs and related genital infections. Sex Transm Infect. 2017 Jun;93(4):234-235. DOI: http://dx.doi.org/10.1136/sextrans-2016-052690

12.Mendes KDS, Silveira RCCP, Galvão CM. Revisão integrativa: método de pesquisa para a incorporação de evidências na saúde e na enfermagem. Texto Contexto Enferm, Florianópolis, 2008 Out-Dez; 17(4): 758-64. DOI: http://dx.doi.org/10.1590/S0104-07072008000400018.

13.Esteitie R. Fundamentos da pesquisa clínica. Porto Alegre: Artmed; 2015.
14.Holloway IW, Winder TJ, Lea CH, Tan D, Boyd D, Novak D. Technology Use and Preferences for Mobile Phone-Based HIV Prevention and Treatment Among Black Young Men Who Have Sex With Men: Exploratory Research. JMIR Mhealth Uhealth. 2017 Apr 13;5(4):e46. DOI:10.2196/mhealth.6436

15.Mitchell JW, Torres MB, Joe J, Danh T, Gass B, Horvath HJ. Formative Work to Develop a Tailored HIV Testing Smartphone App for Diverse, At-Risk, HIV-Negative Men Who Have Sex With Men: A Focus Group Study. JMIR Mhealth Uhealth. 2016 Nov 16;4(4):e128. DOI: 10.2196/mhealth.6178

16.Schnall R, Rojas M, Bakken S, Brown W, Carballo-Dieguez A, Carry M, et al. A user-centered model for designing consumer mobile health application (apps). J Biomed Inform. 2016 Apr; 60: 243-251. DOI:10.1016/j.jbi.2016.02.002

17.Evans C, Turner K, Suggs LS, Occa A, Juma A, Blake H. Developing a mHealth intervention to promote uptake of HIV testing among African communities in the conditions: a qualitative study. BMC Public Health. 2016; 16:656. DOI:10.1186/s12889-016-3278-4

18.Sheoran B, Silva CL, Lykens JE, Gamedze L, Williams S, Ford JV, et al. YTH StreetConnect: Development and Usability of a Mobile App for Homeless and Unstably Housed Youth. JMIR Mhealth Uhealth. 2016 Jul-Sep; 4(3):e82. DOI: 10.2196/mhealth.5168

19.LeGrand S, Muessig KE, McNulty T, Soni K, Knudtson K, Lemann A, et al. Epic Allies: Development of a Gaming App to Improve Antiretroviral Therapy Adherence Among Young HIV-Positive Men Who Have Sex With Men. JMIR Serious Games. 2016 Jan-Jun; 4(1): e6. DOI:10.2196/games.5687

20.Levy ME, Watson CC, Wilton L, Criss V, Kuo I, Glik SN, et al. Acceptability of a Mobile Smartphone Application Intervention to Improve Access to HIV Prevention and Care Services for Black Men Who Have Sex with Men in the District of Columbia. Digit Cult Educ. 2015;7(2);169-191. Disponível em: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4652936/ Acesso em 11 de abril de 2019.

21.Cordova D, Bauermeister JA, Fessler K, Delva J, Nelson A, Nurenberg R, et al. Youth Leadership Council. A Community-Engaged Approach to Developing an mHealth HIV/STI and Drug Abuse Preventive Intervention for Primary Care: A Qualitative Study. JMIR Mhealth Uhealth. 2015;3(4):e106. DOI:10.2196/mhealth.4620

22.Gkatzidou V, Hone K, Sutcliffe L, Gibbs J, Sadiq ST, Szczepura A, et al. User interface design for mobile-based sexual health interventions for young people: Design recommendations from a qualitative study on an online Chlamydia clinical care pathway. BMC Med Inform Decis Mak. 2015;15:72. DOI:10.1186/s12911-015-0197-8

23.Aliabadi N, Carballo-Dieguez A, Bakken S, Rojas M, Brown W, Carry M, et al. Using the Information-Motivation-Behavioral Skills Model to Guide the Development of an HIV Prevention Smartphone Application for High-Risk MSM. AIDS Educ Prev. 2015 Dez; 27(6):522-237. DOI:10.1521/aeap.2015.27.6.522

24.Broaddus MR, Marsch LA. Risks and benefits of text message-delivered and small group-delivered sexual health interventions among African American women in the Midwestern U.S. Ethics Behav. 2015 Mar;25(2):146-168. DOI:10.1080/10508422.2014.948957

25.Swendeman D, Ramanathan N, Baetscher L, Medich M, Scheffler A, Comulada WS, et al. Smartphone self-monitoring to support self-management among people living with HIV: perceived benefits and theory of change from a mixed-methods randomized pilot study. J Acquir Immune Defic Syndr. 2015 May 1;69 Suppl 1:S80-91. DOI:10.1097/QAI.0000000000000570

26.Goldenberg T, McDougal SJ, Sullivan PS, Stekler JD, Stephenson R. Preferences for a Mobile HIV Prevention App for Men Who Have Sex With Men. JMIR Mhealth Uhealth. 2014 Oct-Dec; 2(4): e47. DOI:10.2196/mhealth.3745

27.Muessig KE, Pike EC, Fowler B, LeGrand S, Parsons JT, Bull SS, et al. Putting Prevention in Their Pockets: Developing Mobile Phone-Based HIV Interventions for Black Men Who Have Sex with Men. AIDS Patient Care STDS. 2013 Apr; 27(4): 211–222. DOI:10.1089/apc.2012.0404

28.Chang LW, Njie-Carr V, Kalenge S, Kelly JF, Bollinger RC, Alamo-Talisuna S. Perceptions and acceptability of mHealth interventions for improving patient care at a community-based HIV/AIDS clinic in Uganda: a mixed methods study. AIDS Care. 2013 Jul; 25(7):874-880.
DOI:10.1080/09540121.2013.774315

29.O’Connell CM, Ferone ME. Chlamydia trachomatis Genital Infections. Microb Cell. 2016 Sep 5; 3(9): 390–403. DOI:10.15698/mic2016.09.525

30.Ayala G, Santos GM. Will the global HIV response fail gay and bisexual men and other men who have sex with men? J Int AIDS Soc. 2016 Nov 21;19(1):21098. DOI: 10.7448/IAS.19.1.21098

31.Programa Conjunto das Nações Unidas sobre HIV/AIDS 2017(Unaids).
Disponível em: https://unaids.org.br/2017/11/homens-sao-menos-propensos-buscar-tratamento-para-hiv-e-tem-mais-chances-de-morrer-por-causas-relacionadas-aids-diz-unaids/ Acesso em 11 de abril de 2019.
32.Centers of Disease Control and Prevention (CDC). HIV among African American gay and bisexual men. Disponível em: https://www.cdc.gov/hiv/group/msm/bmsm.html. Acesso em 11 de abril de 2019.
33.Donovan R, Henley N. Social marketing and social change. In Principles and Practice of Social Marketing: An International Perspective. Cambridge: Cambridge University Press. 2010:1-22. https://doi.org/10.1017/CBO9780511761751.002 Acesso em 11 de abril de 2019.
34.Freimuth VS, Mettger W. Is there a hard-to-reach audience? Public Health Rep. 1990 May-Jun;105(3):232-8.Disponível em: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1580016/ Acesso em 11 de abril de 2019.

35.Bartholomew LK, Parcel GS, Kok G, Gottlieb NH, Schaalma H, Markham C, et al. Planning health promotion programs: An intervention mapping approach. San Francisco, CA, US: Jossey-Bass. 2 ed. 2006.

36.Programa Conjunto das Nações Unidas sobre HIV/Aids (UNAIDS). Boas práticas de participação. Diretrizes para ensaios clínicos de prevenção biomédica do HIV 2011. Disponível em: https://www.avac.org/good-participatory-practice. Acesso em 06 de abril de 2019.

37.Aggleton P, Parker R. Moving Beyond Biomedicalization in the HIV Response: Implications for Community Involvement and Community Leadership among MSM and Transgender People. Am J Public Health. 2015 Aug;105(8):1552-8. DOI: 10.2105/AJPH.2015.302614.

38.Fisher JD. Fisher WA. Amico KR. Harman JJ. An information-motivation-behavioral skills model of adherence to antiretroviral therapy. Health Psychol. 2006;25:462–473. DOI:10.1037/0278-6133.25.4.462

39.Whiteley L, Brown L, Lally M, Heck N, van den Berg JJ. A Mobile Gaming Intervention to Increase Adherence to Antiretroviral Treatment for Youth Living With HIV: Development Guided by the Information, Motivation, and Behavioral Skills Model. JMIR Mhealth Uhealth 2018;6(4):e96. DOI: 10.2196/mhealth.8155.

40.Hevner A. A three cycle view of design science research. Scand J Inf Syst 2007;19(2):4. Disponível em: https://www.uio.no/studier/emner/jus/afin/FINF4002/v13/hefner-design.pdf Acesso em 11 de abril de 2019.

41.Norman D, Draper P. User centered design: new perspectives on human–computer interaction. Hillsdale, NJ: Lawrence Erlbaurn Associates; 1986.

42.Deininger M, Daly SR, Sienko KH, Lee JC. Novice designers\' use of prototypes in engineering design. Des Stud. 2017 Jul;51:25-65. doi: 10.1016/j.destud.2017.04.002.

43.Lippman SA, Shade SB, Sumitani J, DeKadt J, Gilvydis JM, Ratlhagana MJ et al. Evaluation of short message service and peer navigation to improve engagement in HIV care in South Africa: study protocol for a three-arm cluster randomized controlled trial. Trials. 2016 Feb 6;17:68. DOI: 10.1186/s13063-016-1190-y.

44.Ruan Y, Xiao X, Chen J, Li X, Williams AB, Wang H. Acceptability and efficacy of interactive short message service intervention in improving HIVmedication adherence in Chinese antiretroviral treatment-naïve individuals. Patient Prefer Adherence. 2017 Feb 10;11:221-228. DOI: 10.2147/PPA.S120003.

45.Ioannis A. Chatzipavlou , Sofia A. Christoforidou e Vlachopoulou M. A recommended guideline for the development of mHealth Apps. Mhealth. 2016; 2: 21. DOI: 10.21037/mhealth.2016.05.01.

46.Pereira-Azevedo N, Osório L, Cavadas V, Fraga A, Carrasquinho E, Cardoso de Oliveira E, et al. Expert Involvement Predicts mHealth App Downloads: Multivariate Regression Analysis of Urology Apps. JMIR Mhealth Uhealth. 2016 Jul 15;4(3):e86. DOI: 10.2196/mhealth.5738.

47.Loy JS, Ali EE, Yap KY. Quality Assessment of Medical Apps that Target Medication-Related Problems. J Manag Care Spec Pharm. 2016 Oct;22(10):1124-40. DOI: 10.18553/jmcp.2016.22.10.1124.

48.White Y, Sandfort T, Morgan K, Karen Carpenter, Russell Pierre. Family relationships and sexual orientation disclosure to family by gay and bisexual men in Jamaica. Int J Sex Heal. 2016; 28: 306–17. DOI: 10.1080/19317611.2016.1227015.

49.Goedel WC, Mitchell JW, Krebs P, Duncan DT. Willingness to Use Mobile Phone Apps for HIV Prevention Among Men Who Have Sex with Men in London: Web-Based Survey. JMIR Mhealth Uhealth. 2017 Oct 11;5(10):e153. DOI: 10.2196/mhealth.8143.

50.Ahmed EU, Ahmad NS, S Ali, Ali S, George A, Saleem Danish H, et al. Medication Adherence Apps: Review and Content Analysis. JMIR Mhealth Uhealth 2018; 6 (3): e62. DOI: 10.2196 / mhealth.6432.

51.Chambers LA, Rueda S, Baker DN, Wilson MG, Deutsch R, Raeifar E, et al. Stigma, HIV and health: a qualitative synthesis. BMC Public Health. 2015 Sep 3;15:848. doi: 10.1186/s12889-015-2197-0.

52.Zou H, Fan S. Characteristics of men who have sex with men who use smartphone geosocial networking applications and implications for HIV interventions: a systematic review and meta-analysis. Arch Sex Behav. 2017 May;46(4):885–894. DOI: 10.1007/s10508-016-0709-3.

53Cao B, Gupta S, Wang J, Hightow-Weidman LB, Muessig KE, Tang W, et al. Social media interventions to promote HIV testing, linkage, adherence, and retention: systematic review and meta-analysis. J Med Internet Res. 2017 Nov 24;19(11):e394. DOI: 10.2196/jmir.7997.

54.Young LE, Fujimoto K, Schneider JA. HIV prevention and sex behaviors as organizing mechanisms in a Facebook group affiliation network among young black men who have sex with men. AIDS Behav. 2018 Mar 13;22(10):3324–3334. DOI: 10.1007/s10461-018-2087-4.

55.Lampkin D, Crawley A, Lopez TP, Mejia CM, Yuen W, Levy V. Reaching suburban men who have sex with men for STD and HIV services through online social networking outreach: a public health approach. J Acquir Immune Defic Syndr. 2016 May 01;72(1):73–8. DOI: 10.1097/QAI.0000000000000930.

56.Elliot E, Rossi M, McCormack S, McOwan A. Identifying undiagnosed HIV in men who have sex with men (MSM) by offering HIV home sampling via online gay social media: a service evaluation. Sex Transm Infect. 2016 Dec;92(6):470–3. DOI: 10.1136/sextrans-2015-052090.