Abstract(resumo):

The aim was to compare the performance of the Oral Health Teams participating in the 2nd and 3rd cycles of the PMAQ-AB. A longitudinal study with data from the external evaluation phase of the second (18,115 OHT) and the third (22,994) cycles of PMAQ-AB was carried out. The OHT participating in the two cycles were matched using the National Team Identifier, resulting in a sample of 15,734 OHT. Eleven procedures related to preventive, surgical-restorative, prosthetic, and oral cancer prevention or diagnosis procedures were selected and analyzed by a questionnaire. Item Response Theory estimated the performance scores of the OHT based on the items. The Wilcoxon test (p ? 0.05) was used to compare the OHT performance between cycles. The mean score in Brazil went from -0.125 (2nd cycle) to 0.149 (3rd cycle). In the North, the average score went from -0.429 to -0.237; Northeast from -0.203 to 0.222; Midwest from -0.259 to 0.007; South from 0.011 to 0.114; Southeast from 0.067 to 0.199, with a greater difference between cycles in the Northeast, Midwest, and North. Brazil and the Brazilian regions showed improvement in the scores between the two cycles analyzed, with regional differences.

Keywords(palavra-chave):

Primary Health Care, Oral Health, Dental Health Services, Healthcare Quality, Access and Evaluation

Content(conteúdo):

Introduction
From the establishment of the Unified Health System (SUS) onwards, Primary Health Care (PHC) has been the key strategy to ensure the population's access to healthcare services. This has facilitated the provision of services grounded in health promotion, prevention, and recovery. Furthermore, it has expanded healthcare coverage, promoting both comprehensiveness and equity 1. The implementation of the National Oral Health Policy and the integration of oral health teams (OHT) into PHC 2 marked a reorganization of the strategy to broaden oral healthcare across all levels (primary, secondary, and tertiary care), with PHC acting as the coordinator of care and the primary point of access for users to the health system. This organizational framework has increased the likelihood of greater service utilization, including dental procedures 3.
Although oral health policies have made significant progress in recent years, particularly in decreasing the prevalence of dental caries 4, barriers to healthcare access remain a challenge in Brazil 5. Furthermore, the implementation of robust and complex strategies and policies necessitates constant planning and evaluation. The systematic analysis of the effectiveness, efficiency, and equity of implemented health actions and services allows for the identification of gaps, potentialities, and challenges in the provision of dental services in PHC. Such rigorous analysis informs decision-making, optimizes the allocation of resources, adapts intervention strategies and public policies to the specific needs of the population, and, ultimately, contributes to the reduction of oral health inequities and the improvement of the quality of life for individuals and the community.6
In this context, the National Program for Improvement of Access and Quality of Primary Care (PMAQ-AB, in Portuguese) was launched in 2011 to increase access and improve the quality of PHC, institutionalizing evaluation within the context of SUS 7.
Historically, the PMAQ-AB marked the first national initiative to integrate a culture of evaluation into the routine operations of Primary Health Care, linking financial transfers to the performance outcomes of a significant portion of SUS health teams. The program aimed for continuous improvement in standards and indicators, transparency in actions, and the definition of a quality parameter to enhance teams' problem-solving capacity 8. However, due to its complex implementation and, primarily, its voluntary nature, its benefits may not have been equitable, and the teams most in need might not have accessed financial resources to improve their health services 9. Thus, after three successful evaluation cycles, PMAQ-AB was replaced by a less complex evaluation and financial transfer program. Nevertheless, the data generated from the PMAQ-AB evaluation cycles are extensive and of great importance to the SUS.
The availability of procedures within PHC constitutes one component of the service evaluation criteria for public health policies, illustrating its interaction with the user 10. Therefore, an analysis of PMAQ-AB is pertinent, as it stands as one of the most extensive global initiatives aimed at enhancing primary care performance, gathering nationwide data to encourage improved access to oral health services in Brazil 5.
Analyzing data from the first cycle of PMAQ-AB, Reis et al. 11 and Fagundes et al. 12 demonstrated that preventive and surgical-restorative procedures were more commonly performed by OHT, including those with lower performance. In contrast, actions concerning the identification of oral cancer and the provision of dental prostheses in PHC were more frequently carried out by high-performing OHT. Subsequent analyses of data from the second cycle conducted by Mendes et al. 13 and the third cycle conducted by Scalzo et al. 14 yielded similar findings.
The current understanding of dental procedures performed by Brazilian OHT in PHC, based on PMAQ-AB data, is limited by studies that analyze evaluation cycles independently, without inter-cycle comparisons. To address this gap, the aim of this study was to compare the PMAQ-AB evaluation cycles in terms of dental procedure performance by Brazilian OHT, and also to compare this performance across Brazilian regions. The formulated hypothesis is that the performance of dental procedures by OHT demonstrated improvement over the PMAQ-AB cycles when analyzed across Brazilian regions.

Methods
This study was approved by the Research Ethics Committee of Universidade Federal de Minas Gerais, logged under protocol number 02396512.8.0000.5149. Written Informed consent was obtained from all subjects involved in the study.
The PMAQ-AB program offered performance-based payments to municipalities that met service qualification criteria and improved the quality of services for their users. Participation was voluntary, and the evaluation process included professionals from primary care teams, managers, and users. The program operated through successive cycles, each structured in four phases: adhesion/contractualization, development of actions, external evaluation, and certification/recontractualization 15. Three complete evaluation cycles were implemented: the first from 2011 to 2013, the second from 2013 to 2015, and the third from 2015 to 2018 16.
This longitudinal study employed secondary data from the external evaluation phases of the second and third cycles of PMAQ-AB. The second cycle involved 18,115 OHT, while the third cycle included 22,994. To facilitate comparison, teams participating in these two cycles were paired using the National Team Identifier (NIT), a unique national code assigned to individual teams for the purpose of tracking their movement and workforce within the municipality's healthcare network. Data from the first cycle could not be incorporated into this analysis due to the absence of the NIT code in its database, rendering team pairing for comparison infeasible. The final analytical sample thus consisted of 15,734 OHT that participated in both the second and third PMAQ-AB cycles.
Data were collected during the external evaluation phase, through evaluator visits to PHC units and document verification to confirm adherence to quality standards established by Brazilian Ministry of Health norms and protocols. The development of the external evaluation process and the instrument used was based on and validated according to essential strategic quality standards of the SUS. Its development involved a National Coordination, comprising the Ministry of Health and Educational and Research Institutions, with support from the PMAQ-AB Working Groups, which included representatives of municipal and state health departments, Councils of Municipal Health Departments, and the Department of Primary Care. The instrument included questions on dental office structure, work processes, management, and service organization.
The present study included a total of 11 mostly dichotomous (yes/no) questions from the instrument regarding primary dental healthcare procedures performed by OHT. These questions covered surgical-restorative clinical procedures (deciduous and permanent tooth extraction, ulotomy/ulectomy, amalgam and composite fillings); preventive procedures (fluoride application, supragingival scaling, root planing, and coronal polishing); actions related to dental prosthesis fabrication (impression and installation); and actions related to oral cancer prevention or diagnosis, including biopsies. For each procedure (item), OHT performance was dichotomized into those performing no procedures and those performing at least one procedure.
The Item Response Theory (IRT) model for graded responses 17 was used to estimate the performance scores of OHT, based on the 11 items. The IRT involves a set of mathematical models that relate an individual's probability of response to an article/item and its latent characteristic, which is a characteristic that cannot be measured directly, such as attitude, satisfaction, and proficiency 18. The latent construct for this study was the performance of the OHT, estimated through the answers obtained on the execution or non-execution of 11 selected dental procedure items.
Descriptive analyses of dental procedures were performed. To confirm the feasibility of IRT application, the first value was calculated from the decomposition of the polychoric correlation matrix, checking its domain. The internal consistency of the scale was also evaluated using Cronbach's alpha. The IRT model allows in-depth analysis of the variables selected to construct the measure, including the level of difficulty and the ability to discriminate. In this sense, the number of dental procedures performed by the teams is not what determines their performance, but rather the difficulty values and the ability to discriminate between items. The discrimination parameter is the ability of the item to differentiate between OHT with different abilities to perform those procedures. The higher the value, the more the item is able to detect differences between teams. The difference parameter is the minimum ability that the OHT needs to have a 50% probability of performing a given procedure.
Although IRT has limitations when applied to dichotomous variables (performance or non-performance of PHC dental procedures), it remains a useful tool for analyzing the quality of test items and comparing respondents' performance. It provides a robust analysis by modeling the relationship between an individual's ability and their probability of endorsing an item, allowing for the estimation of item parameters such as difficulty and discrimination 19.
The data were organized in Stata Software (StataCorp. 2015. Stata Statistical Software: Version 14. University Station, TX: StataCorp LP). The ltem package for Latent Variable Modeling and Item Response Analysis 20 of R software v. 4.0.2 21 was used to analyze the data. Descriptive analysis and correlation matrix to check the IRT hypothesis, and the latent trace model was used to adjust the IRT model.
After obtaining the performance scores of the second and third cycles of the PMAQ-AB, these scores were compared using the Wilcoxon signed-rank test (p ? 0.05), using R software v. 4.0.2 21 to compare the performance of OHT in Brazil and Brazilian regions.

Results
The correlations between the items were positive, which indicates that the items measure the same dimension of the construct. The decomposition of the polychoric correlation matrix according to its eigenvalues indicates the unidimensionality assumption necessary for the IRT application, once the score related to the first component explained 40.96% of the total variance of the analyzed items. The Cronbach alpha coefficient was 0.49.
Table 1 shows the discrimination parameter and the difficulty level of each evaluated procedure performed by the participating OHT in the last two cycles of PMAQ-AB. Difficulty parameters with lower values reveal which items were frequently performed by the OHT, while higher values reveal items that are less frequently performed by the teams. The items with higher difficulty parameters were those related to impression for prostheses (b=7.529) and installation of prostheses (b=7.640).
Table 2 shows the mean of scores obtained through IRT in the second and third cycles according to the Brazilian regions and Z-values obtained from the Wilcoxon signed-rank test.
The difference in scores between the cycles were: 0.273 for Brazil; 0.191 for the North; 0.424 for the Northeast; 0.251 for the Midwest; 0.123 for the South and 0.266 for the Southeast.
The results of the scores obtained for the IRT analysis show that all regions showed a significant increase in the mean of scores between cycles (p<0.001). However, the Northeast, Midwest and North, regions were the ones that presented the greatest evolutions.
Thus, the hypothesis was confirmed. There was a significant increase in OHT performance in Brazil and Brazilian regions, between the second and third cycles of the PMAQ-AB, when compare Brazil and the regions.

Discussion
The findings of this study revealed an improvement in the performance of OHT when comparing the two most recent evaluation cycles of the PMAQ-AB. Notably, both Brazil as a whole and its constituent regions exhibited increased OHT performance scores between the analyzed cycles. These results may indicate a potential relationship between the implementation of systematic service evaluations, such as the PMAQ-AB, and positive developments in indicators related to access to oral health services across Brazil.
Although Brazilian OHT performance scores improved between the two cycles, regional inequalities in oral health within PHC in Brazil continued to exist. The quantity of dental procedures performed is a recognized indicator of service utilization and coverage. Consequently, comparing OHT performance over time is crucial for pinpointing the weaknesses and strengths of the oral health system, ultimately informing strategies to improve oral healthcare for the population and address existing inequalities.
A comparison of the effectiveness of dental procedures between the two cycles revealed a high probability of execution for preventive, restorative, and surgical procedures among the majority of OHT. Conversely, prosthetic procedures exhibited a lower probability of being performed by the teams. The challenges associated with executing and installing dental prostheses suggest a deficiency in the completeness of care and indicate a lack of significant progress relative to earlier assessments 11,13-22.
A study investigated the factors associated with the implementation of dental prostheses by oral health teams in primary health care in Brazil from 2013 to 2014. It was observed that teams with more highly trained professionals, academic involvement, better organizational support, and better work incentives were more likely to implement dental prostheses. Thus, a lack of professional skills for prosthesis execution, insufficient resources, and the absence of reference laboratories for prosthesis manufacturing are determining factors for the accomplishment of these procedures 14, 23.
Our findings align with those of Ribeiro et al. 5, who compared the first and second PMAQ-AB cycles regarding the structure and work processes of Brazilian OHT, including dental procedures. They identified more significant improvements in the Midwest, North, and Northeast regions, contrasting with the less pronounced changes in the South and Southeast. The National Oral Health Policy increased oral health incentives for municipalities with lower Human Development Indices, benefiting the North and Northeast regions with greater access and improved infrastructure 22, which may explain their higher improvement rates. Despite the noted progress, regional disparities persist across Brazil. These disparities may be explained by the necessity of financial investments from management for improvements in health unit infrastructure, and the requirement of workforce involvement for the adequate optimization of teamwork processes 23.
Teams exhibiting lower performance scores are concentrated in Brazilian regions characterized by the most unfavorable socioeconomic indicators. The trajectory of economic development in the North, Northeast, and Midwest regions of the country diverged from that of the South and Southeast regions, a pattern also observed in the targeting of public health funding. Consequently, the gradual improvement observed is indicative of the disparities in social development among these regions, resulting in corresponding differences in oral health epidemiological profiles 24.
Consequently, the structuring and improvement of public policies aimed at expanding and qualifying the work processes of OHT will facilitate a broader provision of oral health services with a focus on mitigating inequities. Furthermore, strengthening the effectiveness of prosthetic procedures in PHC will allow for a better response to the population's epidemiological needs.
The PMAQ-AB program provides financial transfers to municipalities demonstrating improvements in their indicators. Notably, the program does not dictate the application of these transferred resources, granting municipalities autonomy to invest them according to local priorities, which may include team training, infrastructural enhancements, and the acquisition of technology 23. The observed improvement in performance may be explained by this flexibility, given that the quantity and quality of oral healthcare provision are closely associated with the availability of suitable infrastructure, materials for clinical procedures, and qualified human resources 28.
However, it is important to highlight that PMAQ-AB was a complex program to implement, conducting three evaluation cycles in a short timeframe and primarily benefiting teams with better infrastructure and work processes. Thus, even with financial transfers linked to program adherence, resources were not distributed equally, and the short interval between evaluations likely did not allow sufficient time for major changes in the teams, despite the agreed-upon indicator improvements between cycles.
Nevertheless, PMAQ-AB aimed to institutionalize evaluation practices in PHC, identify teams facing greater challenges, and develop strategies to qualify them, thereby reducing barriers to the growth and improved quality of service delivery. PMAQ-AB arguably represents the world's most extensive program for performance evaluation in primary care and has played a crucial role in increasing investment in primary care across Brazil 29.
A limitation of this study is its reliance on secondary databases from the second and third evaluation cycles. Consequently, it is necessary to acknowledge the potential for errors in data recording within these databases, as well as the possibility of overestimating the results, given that adherence to the PMAQ-AB was voluntary and may have been subject to the selective participation of better-organized teams and health units. Also, some variables could not be evaluated due to differences in the program questionnaires across the two cycles. Although the Cronbach alpha estimate indicated a lower internal consistency, the correlations between the items were all positive, indicating that the items measured the same dimension of the construct. The first eigenvalue of the polychoric correlation matrix accounted for 40.96% of the variability in the covariance structure of the items.
Furthermore, for the present study, we opted to define OHT performance based on the execution of 11 oral health procedures within PHC, thus excluding other relevant dimensions, including frequency, population coverage, clinical resolution, effectiveness, continuity of care, problem-solving capacity, user-professional bonding, care coordination, as well as structural and organizational factors of dental facilities. The limitation of determining team performance based on only one dimension of care (execution or non-execution of certain procedures) are acknowledged, although this remains a relevant aspect of primary oral healthcare, thus justifying the scope used. Future research should consider incorporating these additional dimensions.
Despite these limitations, PMAQ-AB represented the most robust national evaluation program ever implemented in Brazil. It generated a significant volume of public data regarding the performance of OHT, thereby promoting a culture of information sharing and transparency. Furthermore, it established a novel funding mechanism, providing supplementary support to the teams while respecting their decision-making autonomy, which translated into increased access to PHC services 30.
The use and comparison of data from two PMAQ-AB evaluation cycles provide important insights into the current state and evolution of Brazilian OHT. These findings can inform the implementation of strategies aimed at further expanding access to and improving the quality of oral health services offered throughout Brazil.
Upon the conclusion of the program, the structuring of a national policy becomes fundamentally important. This policy should continuously evaluate the infrastructure of health units, the work processes of teams, and the experiences of all actors involved—managers, users, and professionals—to generate data that facilitate the identification of difficulties and barriers to healthcare access, with the ultimate goal of reducing inequalities and enhancing access for those with the greatest need.
Conclusion
This study's findings demonstrated an improvement in team performance scores between the second and third cycles of the program in all regions of the country. Nevertheless, regional disparities remain.
The establishment of national-level health evaluation, employing performance indicators, can serve to pinpoint teams with greater challenges and to develop targeted approaches for their qualification, thereby reducing obstacles that impede growth and the improvement of service quality.

Acknowledgements
Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES 001), Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq), Fundação de Amparo à Pesquisa do Estado de Minas Gerais (FAPEMIG PPM 00148-17), and Pró-Reitoria de Pesquisa da Universidade Federal de Minas Gerais (PRPq-UFMG).

References
1. Barros RD, Aquino R, Souza LEPF. Evolution of the structure and results of Primary Health Care in Brazil between 2008 and 2019. Evolução da estrutura e resultados da Atenção Primária à Saúde no Brasil entre 2008 e 2019. Cien Saude Colet. 2022;27(11):4289-4301. doi:10.1590/1413-812320222711.02272022

2. Melo EA, Probst LF, Guerra LM, Tagliaferro EP da S, De-Carli AD, Pereira AC. Indicators for dental appointment scheduling in primary health care: a national cross-sectional study. BMC Public Health. 2021 Dec;21(1).

3. Corrêa GT, Celeste RK. Associação entre a cobertura de equipes de saúde bucal na saúde da família e o aumento na produção ambulatorial dos municípios brasileiros, 1999 e 2011 [Association between coverage by oral health teams in the family health and the increase in dental care output in Brazilian municipalities, 1999 and 2011]. Cad Saude Publica. 2015;31(12):2588-2598. doi:10.1590/0102-311X00000915

4. Souza GC de A, Kusma SZ, Moysés SJ, Roncalli AG. Implementation of the Brazilian National Oral Health Policy and its influence on oral health conditions in Brazilian state capitals in the first decade of the 21st century. Cad Saude Publica. 2021;37(12).

5. Ribeiro AGA, Martins RFM, Vissoci JRN, da Silva NC, Rocha TAH, Queiroz RC de S, et al. Progress and challenges in potential access to oral health primary care services in Brazil: A population-based panel study with latent transition analysis. Fontoura Bastos L, editor. Plos One. 2021 Mar 16;16(3):e0247101.

6. Bueno AS, Celeste RK. Relationship between Professional Training of Dentists and Outpatient Clinical Production. Braz-Silva PH, editor. BioMed Research International. 2022 Mar 20;2022:1–7.

7. Galvão MHR, Roncalli AG. Performance of Brazilian municipalities in the supply of specialized oral health services. Cad Saude Publica. 2021;37(1).
8. Medeiros GAR, Nickel DA, Calvo MCM. Building a model to evaluate use of the National Program for Improving Primary Health Care Access and Quality in Brazil. Epidemiol Serv Saude. 2019 Oct 28;28(3):e2018281. English, Portuguese. doi: 10.5123/S1679-49742019000300006.

9. Eijkenaar F, Emmert M, Scheppach M, Schöffski O. Effects of pay for performance in health care: a systematic review of systematic reviews. Health Policy. 2013 May;110(2-3):115-30. doi: 10.1016/j.healthpol.2013.01.008.

10. Tanaka, OY. Primary health care evaluation: a new approach. Saúde e Sociedade. 2011, 20, 927-934.

11. Reis CM, Matta-Machado AT, Amaral JH, Werneck MA, Abreu MH. Describing the primary care actions of oral health teams in Brazil. Int J Environ Res Public Health 2015;12:667-78.

12. Fagundes DM, Thomaz EBAF, Queiroz RC de S, Rocha TAH, Silva NC da, Vissoci JRN, et al. Diálogos sobre o processo de trabalho em saúde bucal no Brasil: uma análise com base no PMAQ-AB. Cad Saude Publica. 2018 Sep 6;34(9).

13. Brasil. Ministério da Saúde. Secretaria de Atenção à Saúde. Departamento de Atenção Básica. Programa Nacional de Melhoria do Acesso e da Qualidade da Atenção Básica (PMAQ-AB): manual instrutivo. Brasília: Ministério da Saúde, 2015.

14. Wachs, LS, Facchini, LA, Thumé, E, Tomasi, E, Fassa, MEG., & Fassa, AG. Evaluation of the implementation of the School Health Program from the Program for Access and Quality Improvement in Primary Care: 2012, 2014, and 2018. Cad Saúde Pública, 2022; 38.

15. Mendes SR, Martins RC, Matta-Machado ATGM, Mattos GCM, Gallagher JE, Abreu MHNG. Dental procedures in Primary Health Care of the Brazilian National Health System. Int J Environ Res Public Health 2017;14:1480.

16. Scalzo MTA, Abreu MHNG, Matta-Machado ATG, Martins RC. Oral health in Brazil: What were the dental procedures performed in Primary Health Care? Plos One. 2022 Jan 28;17(1):e0263257.

17. Samejima, FA. Estimation of latent ability using a response pattern of gradedscores. Psychometric Monograph. 1968; 1:169. Available in: https://onlinelibrary.wiley.com/doi/10.1002/j.2333-8504.1968.tb00153.x

18. Bhakta B, Tennant A, Horton M, Lawton G, Andrich D. Using item response theory to explore the psychometric properties of extended matching questions examination in undergraduate medical education. BMC Med Educ 2005;5:9.

19. Ogunsakin IB, Shogbesan YO. Item response theory (IRT): a modern statistical theory for solving measurement problem in 21st century. International Journal of Scientific Research in Education (IJSRE). 2018;11:627-35.

20. Dimitris R. ltm: an R package for Latent Variable Modeling and Item Response Analysis. Journal of Statistical Software. 2007; 17:1-25. Available in: https://www.jstatsoft.org/article/view/v017i05

21. R Core Team (2019). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. Available in: https://www.R-project.org/

22. Cunha MAGM, Matta-Machado ATG, Lucas SD, Abreu MHNG. Availability of dental prosthesis procedures in brazilian primary health care. Biomed Res Int 2018;11:4536707.

23. Cunha MA, Vettore MV, Santos TRD, Matta-Machado AT, Lucas SD, Abreu MHNG. The Role of Organizational Factors and Human Resources in the Provision of Dental Prosthesis in Primary Dental Care in Brazil. Int J Environ Res Public Health. 2020 Mar 3;17(5):1646. doi: 10.3390/ijerph17051646.

24. Chaves SCL, Almeida AMFL, Rossi TRA, Santana SF, Barros SG, Santos CML. Oral health policy in Brazil between 2003 and 2014: scenarios, proposals, actions, and outcomes. Cien Saude Colet. 2017;22:1791-803.

25. Lucena EHGD, Lucena CDRXD, Alemán JADS, Pucca GA, Pereira AC, Cavalcanti YW. (2020). Monitoring of oral health teams after National Primary Care Policy 2017. Rev Saude Publica 2020:54:99.

26. da Silva JV, Machado FC, Ferreira MA. Social Inequalities and the Oral health in Brazilian Capitals. Cien Saude Colet. 2015;20(8):2539-2548. doi:10.1590/1413-81232015208.12052014

27. Rodrigues, AWD., Mello, ECDA., Candeia, RMS., Silva, G., Gomes, LB., & Sampaio, J.. Performance payment to the Primary Care Teams: analysis from the PMAQ-AB cycles. Saúde em Debate, 45, 1060-1074.

28. Baumgarten A, Hugo FN, Bulgarelli AF, Hilgert JB. Curative procedures of oral health and structural characteristics of primary dental care. Rev Saude Publica 2018;52:35.

29. Macinko J, Harris MJ, Rocha MG. Brazil's National Program for Improving Primary Care Access and Quality (PMAQ): fulfilling the potential of the world's largest payment for performance system in primary care. The Journal of ambulatory care management, 40(2 Suppl), S4.

30. Dantas Gurgel G Junior, Kristensen SR, da Silva EN, et al. Pay-for-performance for primary health care in Brazil: A comparison with England's Quality Outcomes Framework and lessons for the future. Health Policy. 2023;128:62-68. doi:10.1016/j.healthpol.2022.11.004