Artigos

Conteúdo (article):

DO FAMILY DOCTORS REFER LESS? IMPACT OF FCM TRAINING ON THE RATE OF PHC REFERRALS
Autores:
Ricardo Collar Rebolho, Secretaria Municipal de Saúde de Florianópolis, ricardo_rebolho@hotmail.com, https://orcid.org/0000-0002-7797-8037;
Paulo Poli Neto, Universidade Federal do Paraná, ppolineto@gmail.com, https://orcid.org/0000-0003-1073-6583;
Lucas Alexandre Pedebôs, Secretaria Municipal de Saúde de Florianópolis, lucas.enfermagem@gmail.com, https://orcid.org/0000-0001-6663-352X;
Leandro Pereira Garcia, Secretaria Municipal de Saúde de Florianópolis, lpgarcia18@gmail.com, https://orcid.org/0000-0002-8601-7166;
Ana Cristina Vidor, Secretaria Municipal de Saúde de Florianópolis, vidoranacris@gmail.com, https://orcid.org/0000-0003-2518-6414.

RESUMO
Objetivo: Medir o impacto da formação em medicina de família e comunidade no percentual de encaminhamentos a partir da atenção primária. Métodos: Estudo transversal descritivo de 375.645 consultas e 34.776 encaminhamentos realizadas por 123 médicos da atenção primária no ano de 2016 relacionando o percentual de encaminhamentos com características dos médicos (sexo, idade, formação em MFC), dos pacientes (sexo e idade) e do serviço (população pelo IBGE e população ativa). Resultados: A formação em MFC por meio de residência médica apresentou uma significativa redução no percentual de encaminhamentos a partir da atenção primária (2,86%), IC:(1,55;4,17), p<0,0001. Essa redução manteve-se na análise multivariada mesmo ajustando-se para todas as possíveis variáveis confundidoras. Não houve diferença na comparação do percentual de encaminhamentos entre médicos sem formação em MFC e médicos com titulação em MFC. Na análise das especialidades para as quais foram realizados os encaminhamentos, os médicos com residência em MFC encaminharam menos para ginecologia, psiquiatria e pediatria e mais para oftalmologia. Conclusão: O estudo mostrou que a formação em MFC por meio de residência médica acarretou significativa redução no percentual de encaminhamentos a partir da atenção primária.
Palavras-chave: Medicina de família e comunidade, atenção primária à saúde, encaminhamento e consulta e residência médica.
ABSTRACT
Objective: To evaluate the impact of family medicine residence on the PHC referral rate. Methods: This is a cross-sectional descriptive study on 375.645 visits and 34.776 referrals by 123 PHC physicians in 2016, linking the referral rate to the characteristics of doctors (gender, age, family medicine training), patients (gender and age) and service (general population and working population). Results: Family and community medicine residency training had a significant reduction in PHC referral rate (2.86%), CI:(1.55;4.17), p<0,0001. This reduction persisted in the multivariate analysis, after adjusting for all the possible confounding variables. No difference was found between the referral rates of doctors with and without family and community medicine (FCM) degree. Concerning referral to specialties, doctors with FCM residence training had lower rates of referral to gynecology, psychiatry and pediatrics and higher rates of referral to ophthalmology. Conclusion: The study showed that FCM residency significantly reduced PHC referral rates.
Keywords: Family and Community Medicine, primary health care, referral and visit, and medical residency.
1. INTRODUCTION
It is known that the organization of health systems with better health outcomes for its population and better cost-effectiveness is one that is based on robust and resolute primary care (1). Primary care is a gateway to the health system and a care coordinator, with the attribution of addressing 85 to 95% of the population’s health problems. One of the essential functions of primary care is to play the role of filter, or gatekeeper, that is, to define which patients should be referred for specific or continuous assessments in specialized care and which may have all of their demand met by primary care at that moment (2). Such a function plays a fundamental role in the safety and quality of the treatment provided to the user since follow-up by focus specialists alone can generate excessive, unnecessary and potentially harmful prescriptions, mainly when such monitoring occurs with doctors from different specialties concomitantly. Also, specialized care is usually characterized by its higher costs, making a number beyond the ideal level of referrals contribute to lower system efficiency. On the other hand, depriving the user of such assessment can generate below-ideal treatment, and primary care must be attentive to this balance (3).
Based on this principle, the Brazilian Unified Health System (SUS) is organized from the levels of primary care and specialized services. At the primary level, the Family Health Strategy (ESF) is the primary care program. In this program, the family health teams, consisting of doctors, nurses, nursing technicians, and community health workers, are spread throughout the national territory in the PHC Facilities (UBS), which are the gateway to the health system. It is the responsibility of these teams to care for the health of the attached population, which in the Brazilian system is divided on a territorial basis and each team is responsible for serving 2,000 to 3,500 people (4). The medical professionals of these teams should define which patients require referrals to specialized services, and only from this referral will the patient access the other levels of the health system (2).
Professionals should be adequately trained and knowledgeable to perform their gatekeeping function satisfactorily and sufficiently respond to 85-95% of the users’ health demands; otherwise, the health system organization becomes more expensive and uncoordinated (5). Brazil only requires medical graduation to work in primary care, which means that many professionals without this specific training end up occupying this essential function in the system. It is known that the health systems with the best results for their populations are those in which primary care is effectively resolute and performs well as a gateway, referring the urgent cases (2). Thus, professionals who work there have been trained for this through residency programs after graduation. In Brazil, family and community medicine is the specialty that gathers all the desirable knowledge to work in primary health care, and family and community medicine residency is the program through which graduate students acquire this knowledge (6). However, because the availability of these professionals in the labor market is still well below the ideal level, due to several factors, the professionals who work in PHC with this training are a minority (7).
Florianópolis is considered the capital with the best primary health care network in Brazil, since, among other factors, it has 100% population coverage by ESFs and 70% of doctors trained in family and community medicine working in the network (8). Despite this, 30% of the doctors who work in the network do not have this training.
Several studies around the world have evaluated the rate of PHC referrals and the factors that influenced their occurrence (9-15). The numbers ranged from 5 to 30% of visits that resulted in referrals (9-15). Several factors were investigated to try to explain this considerable variability, some of which can be explained by variations of the patients, such as age, race, gender, and number of comorbidities (16). Another part is due to doctor’s factors, such as age, family medicine training, payment method, length of stay in the same team, and gender (17-19). Some studies have also investigated the influence of local resources, such as the number of specialized care services available and distance to services (17-19).
Although several ongoing studies are evaluating the rate of referral to specialized care in Brazil, our search did not return any published study so far. While there is no scientific evidence of the ideal number of PHC referrals, a study published in 1993 with more than 1,500 doctors in 15 Western European countries showed a referral rate ranging from 2.6 to 8.2% (20). This percentage may represent a parameter of primary care quality, as well as generate a huge financial impact on the health system.
Thus, this study aims to identify the influence of factors related to doctors, user and services on the rate of referrals to specialized care from the visits performed in primary health care in the municipality of Florianópolis-SC.
2. METHODS
Study population
The referral rate of 123 primary care doctors who worked at the Florianópolis Municipal Health Secretariat (SMS) in the year 2016 was analyzed. All the doctors who performed at least 500 visits in 2016 and were responsible for some Family Health Strategy team were included, and this number of visits was chosen as the cutoff point because it is the mean number of monthly visits performed by doctors. Doctors from the Family Health Support Center (NASF) or resident doctors were not included. The doctor’s team was considered the one in which most of the patients he attended in 2016 were registered.
Data extraction
Data collection was based on two strategies: extracting data from the electronic health record system of the SMS and searching for complementary data on the Internet. Data were retrieved from all the visits held at all UBS facilities in Florianópolis in 2016 from the electronic health record system, by attendance unit, professional’s name, municipal registration, professional registration number, visit code, referral code (when performed) and to which specialties, code, age, and gender of users, and the assigned area/team in which they are registered.
Also, the gender and age of doctors were collected in the human resources sector of the Municipal Health Secretariat. We did not access the patients’ medical records, and third parties of the institution itself provided all the mentioned data. The population estimate for each team was obtained from a projection carried out by the team of the Municipal Health Secretariat based on the 2010 IBGE demographic census (21). Three methods were employed to search for data on the doctor\'s training: 1) List of graduates in all the editions already held of the Family and Community Medicine Specialist Degree (TEMFC) on the website of the Brazilian Society of Family and Community Medicine (SBMFC) (22); 2) Register of family medicine and community specialty in the Regional Council of Medicine (CRM), searching for doctors on the CRM website (23); 3) The National Medical Residency Commission System (SisCNRM) to search for Certificates of Medical Residency in Family and Community Medicine (24). All doctors who completed the FCM medical residency by March 2016 were considered as with residency, regardless of whether or not they hold an FCM specialist degree. Doctors without medical residency registration in FCM and who were approved in the TEMFC by March 2016 were considered as having a degree. Doctors who had neither residence nor degree in FCM by March 2016 were deemed to be non-FCM.
Data analysis
The referral rate was calculated by dividing the total number of referrals by the total number of visits. Only referrals to medical specialties were counted. The mean age of the population was calculated by adding the age of people served by the doctor, divided by the total number of patients attended by the same doctor, in 2016. The proportion of women attended by each doctor was calculated similarly. The working population of each team was defined as the total number of people attended in medical and nursing visits from 01/01/2015 to 31/12/2016 based on data collected from the health information sector.
The characteristics of doctors and patients were described according to medical training. Pearson’s chi-square test was used to compare the proportions of the categorical variables. ANOVA was used to compare means between two or more groups. Student’s t-test was used for the correlation between the referral rate and the categorical variables, and Pearson’s correlation coefficient was used for the correlation of the referral rate with the quantitative variables. Values of r<0.25 were considered as unrelated, and as a weak relationship when between 0.25 and 0.5. Values of p<0.05 were significant. The group of non-FCM doctors was used for all comparisons of the standard group. In all analyses, the dependent variable was the referral rate, while the independent variables were FCM training, doctor’s gender and age, the proportion of women in the population served, mean age of the population served, IBGE population and working population.
Multivariate regression was performed to adjust for possible confounding variables in the referral rate. For this analysis, all variables with a p ≤ 0.25 in the bivariate analysis entered the multivariate model, as recommended by Hosmer and Lemeshow (25), and only resident and non-resident doctors were compared, where the latter group also included doctors only with a degree, since the bivariate analysis showed no difference between non-resident doctors and doctors who only had a degree. Several variables were no longer statistically significant after analyzing the initial model. A new model was elaborated only with the variables with p ≤ 0,05, and this was the model chosen. These analyses were performed using statistical software R v.3.3.2 (26) and R-Studio v.1.0.44 (27).
Ethical considerations
Data collection was authorized by the Municipal Health Secretariat via the Monitoring Committee for Health Research Projects through Internal Circular OE 049/SMS/GAB/ESP/2017 of September 19, 2017, and approved by the Research Ethics Committee of the State Health Secretariat of Santa Catarina under opinion Nº 2.839.829 of August 23, 2018.
3. RESULTS
Table 1 shows the characteristics of doctors, patients, and service by medical training. We observed that 47.97% of doctors included in the study had FCM residency, 13.82% had FCM degrees, and 38.21% had neither residence nor FCM degree. P-values refer to the comparison with the standard population (non-FCM).
Total referrals arrived at 9.57%. Table 2 shows the bivariate analysis of the impact of each variable on the referral rate. The comparison between doctors with FCM residency and doctors without an FCM degree evidenced that the referral rate was significantly lower among doctors with residency (8.06% versus 11.04%, 95% CI (1.09;4.85), p=0.002). The analysis of the continuous variables with the referral rate revealed a weak, but significant correlation between the referral rate and the age of doctors, the mean age of patients and the proportion of women in the population; that is, the higher the age of doctors, patients or the greater the proportion of women in the population, the higher the referral rate. Of the variables related to the service, both the population per team by IBGE and the working population did not correlate with the referral rate.
Table 3 shows the regression coefficients and the crude and adjusted confidence intervals of the linear regressions. In this analysis, only the mean age of doctors and the population and FCM residency significantly affected the referral rate, and the latter was responsible for the higher stand-alone impact on the referral rate.
Table 4 shows the proportion of referrals to each specialty by medical training. The specialties with the highest proportion of referrals in each of the groups are similar, but a significant difference was found between the groups in ophthalmology, gynecology, psychiatry, and pediatrics. In the group-to-group comparison, this difference remained significant between the FCM residency and non-FCM residency groups in these same specialties, with a higher rate of referral to ophthalmology in the group with FCM residency, and a lower rate of referral to gynecology, psychiatry, and pediatrics also in this group. The groups with FCM degree versus non-FCM degree showed a significant difference only in the pediatrics/otorhinolaryngology specialty, with a lower proportion of referrals in the FCM degree group. The comparison between residency versus degree groups showed a significant difference only in the specialties of gynecology and psychiatry, both with a lower proportion in the group with FCM residency.
4. DISCUSSION
The study showed that FCM training through medical residency significantly reduced the rate of referral from primary care to specialized care. This impact persisted even with the adjustments for the possible confounding variables analyzed. Some hypotheses can be raised to explain this difference, among them the fact that the residence in FCM considers a two-year period, under the supervision of another more experienced FCM doctor, in which the resident acquires the capacity to handle most of the problems of the population, increasing their resolving power. Also, residents learn the principles of FCM and can apply them in their practice, such as longitudinality, in which they develop a bond with the patient and can follow it over a more extended period, which may be useful in defining which cases require a referral or can be addressed in primary care. Also, the principle of access, widely discussed in FCM residency programs, where residents value the importance of patients having easy and timely access to their doctor, and that in situations of deteriorated health problems, they may be serviced promptly, when necessary.
The specialties with the highest proportion of referrals are similar, but there is a significant difference in some specialties. The FCM residency group had a significantly higher proportion of referrals to ophthalmology, which may represent a parameter of a better quality of care for chronic patients in this group, due to the need for annual referral of hypertensive and diabetic patients to ophthalmology. On the other hand, the lower proportion of referrals to gynecology, psychiatry, and pediatrics in this same group may also be related to a higher resolution of complaints related to these specialties, perhaps due to the skills developed during residency training. It is worth emphasizing that these three specialties can be part of the Family Health Support Center (NASF), and it would be interesting to evaluate their presence in the NASF teams of the health centers where these doctors worked, and also whether the training in FCM affects the work with these professionals.
The study also showed that the FCM degree did not have an impact on the referral rate. It is also relevant to note that there was no significant difference in the referral rate to each specialty in the comparison between the group with FCM degree versus the non-FCM group, except in the pediatrics/otorhinolaryngology specialty, which may mean that the FCM degree has no impact on the FCM quality of care provided by these doctors.
The main strengths of the study are as follows: 1. The evaluation of a large number of visits (375,645) and referrals (34,776); 2. Data reliability, since all queries and referrals are necessarily performed through the computerized system; 3. The presence of a good number of doctors in the different groups by training method; 4. The possibility of gauging some confounding variables of doctors, patients, and service.
Some limitations of the study are as follows: 1. The non-evaluation of other possible confounders such as doctor’s training time, length of stay in the same team, population income, and number of comorbidities of the population; 2. Failure to evaluate whether or not the referrals were appropriate. A Dutch study (28), for example, compared the proportion of adequate referrals of two doctors with high referral rates and two physicians with low referral rates and found no difference between them. This shows that a high referral rate is not necessarily related to inadequate or avoidable referrals.
The referral rate found in this study was similar to that described in the literature, ranging from 5 to 30% (9-15,29,30). As in other studies (11,13,17,18), the highest mean age and the highest proportion of women in the population are related to a higher referral rate. Other studies have also shown that female doctors have a higher referral rate (9,13,14,17). Increased referral rate with the increased age of doctors found in this study has already been described in the literature, but the findings are conflicting, and there is no plausible explanation (9,16,17).
A Norwegian study (14) found similar results for FCM training, with a lower referral rate among doctors with FCM residency compared to non-FCM residency doctors (12.5% versus 14.9%). Another Thai study (31) also found a lower referral rate by doctors with FCM residency compared to doctors without this training (14% versus 24%).
In this study, a Canadian paper (19) found that specialties with the highest number of referrals from primary care were Gastroenterology (11%), Gynecology (10%), Dermatology (9%), General Surgery (9%), Orthopedics (7%), Otorhinolaryngology (6%), Ophthalmology (4%), Allergology and Immunology (4%), Cardiology (4%), and Plastic Surgery (4%). These numbers may be influenced by the fact that Canadian patients directly access the optometrist for refraction, which significantly reduces the number of referrals to ophthalmology.
This study reaffirms the importance of family and community medicine training in primary care. The lower referral rates of these professionals comply with the principles of the organization of the health system in levels of complexity, in which 85-95% of complaints are resolved in primary care. These data are essential for Brazilian municipal managers, so that they may perceive the financial and quality impact of doctors with FCM training working in PHC. In Florianópolis alone, in 2016, when these data were collected, the difference in the absolute number of referrals between physicians with FCM residency and those without residency was 5,420 referrals, which is equivalent to 15% of the total. Considering data collected by the recent study on the PHC costs in Florianópolis, conducted by the Pan American Health Organization (PAHO) (32), every visit to the municipal polyclinics, where most of the secondary care of the municipality operates, has a cost of R$ 125; thus, only the cost of the visits generated by these referrals would represent a financial impact of 682,000 Brazilian Reais in the annual budget, besides all the requested tests and medications prescribed in these visits and subsequent consultations in the specialized care.
Extrapolating these data to the national level, where the vast majority of physicians working in PHC have no FCM training, we can imagine the impact of a high percentage of referrals to the health system, burdening the secondary and tertiary levels with a high number of patients who could often be followed-up exclusively in primary care, which prevents patients who really need specialized care from reaching specialists, generating a significant elevation in the health system’s costs.
Furthermore, as there was no difference between the percentage of referrals and the proportion of referrals to the different specialties among doctors with and without FCM degree, the criteria considered for obtaining the specialty title were called into question. In the future, it would be interesting to investigate other factors that may influence the referral rate, such as doctor’s length of stay in the same team, training time, income and number of comorbidities of the population. It would also be of great interest to study the proportion of appropriate referrals among doctors with and without FCM training. The search for quality indicators related to FCM training would also be of fundamental importance, such as the number of admissions due to PHC-sensitive causes in the populations assisted by doctors with FCM training versus doctors without this training, and indicators of the cost of care of patients assisted by doctors with FCM training, such as number of drugs prescribed, number of tests requested, compared to doctors without this training.
5. References

1. Starfield B. Primary Care and Health. JAMA [Internet]. 1991 Oct 23 [cited 2017 Apr 2];266(16):2268. Available from: http://jama.jamanetwork.com/article.aspx?doi=10.1001/jama.1991.03470160100040
2. Starfield B. Primary care Is primary care essential? Lancet [Internet]. 1994 [cited 2017 May 25];344(October,22):1129–33. Available from: http://ac.els-cdn.com/S0140673694906343/1-s2.0-S0140673694906343-main.pdf?_tid=c3fafab6-4192-11e7-8ded-00000aacb35f&acdnat=1495748606_cb46e76b91b2326a5c0201e92dd0f0a4
3. Coulter A. Managing demand at the interface between primary and secondary care. BMJ [Internet]. 1998 Jun 27 [cited 2017 May 25];316(7149):1974–6. Available from: http://www.ncbi.nlm.nih.gov/pubmed/9641944
4. Ministério da Saúde. Política Nacional de Atenção Básica [Internet]. PORTARIA No 2436 de 21 de setembro de 2017 2017. Available from: http://www.foa.unesp.br/home/pos/ppgops/portaria-n-2436.pdf
5. Macinko J, Starfield B, Shi L. Quantifying the Health Benefits of Primary Care Physician Supply in the United States. Int J Heal Serv [Internet]. 2007 Jan 22 [cited 2018 Jan 28];37(1):111–26. Available from: http://journals.sagepub.com/doi/10.2190/3431-G6T7-37M8-P224
6. SECRETARIA DE EDUCAÇÃO SUPERIOR. RESOLUÇÃO No 1, DE 25 DE MAIO DE 2015 [Internet]. 25 DE MAIO DE 2015. Available from: http://portal.mec.gov.br/index.php?option=com_docman&view=download&alias=17661-res01-25052015-sesu-residencia-medica&category_slug=junho-2015-pdf&Itemid=30192
7. DEMOGRAFIA MÉDICA NO BRASIL 2015. [cited 2018 Jan 28]; Available from: https://www.usp.br/agen/wp-content/uploads/DemografiaMedica30nov2015.pdf
8. MP, Andrade; ITS, Chaves; R, Zonta; JP S. O Serviço Público de Saúde em Florianópolis. 2016.
9. Franks P, Williams GC, Zwanziger J, Mooney C, Sorbero M. Why do physicians vary so widely in their referral rates? J Gen Intern Med [Internet]. 2000 Mar [cited 2017 Apr 1];15(3):163–8. Available from: http://www.ncbi.nlm.nih.gov/pubmed/10718896
10. Barnett ML, Song Z, Landon BE. Trends in physician referrals in the United States, 1999-2009. Arch Intern Med [Internet]. 2012 Jan 23 [cited 2017 Apr 1];172(2):163–70. Available from: http://www.ncbi.nlm.nih.gov/pubmed/22271124
11. BCS TCI for I, Ryan B, Maddocks H, Thind A. Patterns of referral in a Canadian primary care electronic health record database: retrospective cross-sectional analysis. J Innov Heal Informatics [Internet]. 2011 [cited 2017 Apr 1];19(4):217–23. Available from: https://hijournal.bcs.org/index.php/jhi/article/view/816/828
12. Franks P, Zvanziger J, Mooney C, Sorbero M. Variations in Primary Care Physician Referral Rates. HSR Heal Serv Res [Internet]. 1999 [cited 2017 Apr 1];34(1):323–9. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1089004/pdf/hsresearch00017-0125.pdf
13. Liddy C, Singh J, Kelly R, Dahrouge S, Taljaard M, Younger J. What is the impact of primary care model type on specialist referral rates? A cross-sectional study. BMC Fam Pract [Internet]. 2014 Feb 3 [cited 2017 Feb 14];15:22. Available from: http://www.ncbi.nlm.nih.gov/pubmed/24490703
14. Ringberg U, Fleten N, Deraas TS, Hasvold T, Førde O. High referral rates to secondary care by general practitioners in Norway are associated with GPs’ gender and specialist qualifications in family medicine, a study of 4350 consultations. BMC Health Serv Res [Internet]. 2013 Apr 23 [cited 2017 Feb 14];13:147. Available from: http://www.ncbi.nlm.nih.gov/pubmed/23617296
15. Zielinski A, Håkansson A, Jurgutis A, Ovhed I, Halling A. Differences in referral rates to specialised health care from four primary health care models in Klaipeda, Lithuania. BMC Fam Pract [Internet]. 2008 Nov 26 [cited 2017 Feb 14];9:63. Available from: http://www.ncbi.nlm.nih.gov/pubmed/19032796
16. O’Donnell CA. Variation in GP referral rates: what can we learn from the literature? Fam Pract [Internet]. 2000 Dec 1 [cited 2017 May 25];17(6):462–71. Available from: https://academic.oup.com/fampra/article-lookup/doi/10.1093/fampra/17.6.462
17. Chan BTB, Austin PC. Patient, Physician, and Community Factors Affecting Referrals to Specialists in Ontario, Canada A Population-Based, Multi-Level Modelling Approach. [cited 2018 Jan 27]; Available from: https://insights.ovid.com/pubmed?pmid=12665714
18. Ryan BL, Shadd J, Maddocks H, Stewart M, Thind A, Terry AL. Methods to Describe Referral Patterns in a Canadian Primary Care Electronic Medical Record Database: Modelling Multilevel Count Data. J Innov Heal Informatics [Internet]. 2017 Nov 17 [cited 2018 Jan 27];24(4):311. Available from: https://hijournal.bcs.org/index.php/jhi/article/view/888
19. Liddy C, Arbab-Tafti S, Moroz I, Keely E. Primary care physician referral patterns in Ontario, Canada: a descriptive analysis of self-reported referral data. [cited 2018 Jan 27]; Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5567435/pdf/12875_2017_Article_654.pdf
20. Fleming D. The European Study of referrals from primary to secondary care [Internet]. 1993 [cited 2018 Feb 3]. Available from: https://www.nivel.nl/sites/default/files/bestanden/1001281.pdf
21. Weblet Importer [Internet]. [cited 2018 Jan 28]. Available from: http://www.pmf.sc.gov.br/sistemas/saude/unidades_saude/populacao/uls_2015_index.php
22. Titulados | SBMFC [Internet]. [cited 2018 Jan 28]. Available from: http://www.sbmfc.org.br/default.asp?site_Acao=mostraPagina&paginaId=18
23. CREMESC [Internet]. [cited 2018 Jan 28]. Available from: http://www.cremesc.org.br/buscamedico.jsp
24. SisCNRM - Sistema da Comissão Nacional de Residência Médica [Internet]. [cited 2018 Jan 28]. Available from: http://siscnrm.mec.gov.br/certificados
25. Hosmer D, Lemeshow S. Applied Logistic Regression. 2nd ed. Hoboken: Wiley-Interscience; 2000.
26. R Core Team. R - A language and environment for statistical computing. Vienna: R Foundation Statistical Computing; 2016.
27. RStudio Team. RStudio: Integrated Development for R. Boston: RStudio, Inc; 2016.
28. Knottnerus JA, Joosten J, Daams J. Comparing the quality of referrals of general practitioners with high and average referral rates: an independent panel review. Br J Gen Pract [Internet]. 1990 May [cited 2017 May 25];40(334):178–81. Available from: http://www.ncbi.nlm.nih.gov/pubmed/2114131
29. Gwynne M, Page C, Reid A, Donahue K, Newton W. What’s the Right Referral Rate? Specialty Referral Patterns and Curricula Across I3 Collaborative Primary Care Residencies. [cited 2018 Jan 27]; Available from: http://www.stfm.org/Portals/49/Documents/FMPDF/FamilyMedicineVol49Issue2Gwynne91.pdf
30. Shea D, Stuart B, Vasey J, Nag S. Medicare Physician Referral Patterns. HSR Heal Serv Res [Internet]. 1999 [cited 2018 Jan 27];34(1). Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1089005/pdf/hsresearch00017-0132.pdf
31. Jaturapatporn D, Hathirat S. Does family medicine training affect the referral pattern of primary care doctors in Thailand? [2]. Family Medicine. 2006.
32. Organização Pan-Americana da Saúde. Centro de custos da Prefeitura Municipal de Florianópolis [Internet]. 2017 [cited 2018 Jan 28]. Available from: http://189.28.128.100/dab/docs/portaldab/documentos/gestao_custos_dab_abril_2017.pdf