Artigos

INTRODUCTION
The Brazilian public health system, SUS (Sistema Único de Saúde - Unified Health System), is based on the principles of universality, equitability, integrability, and focused on meeting the population's needs to be effective and efficient1. According to 2, health systems fit the definition of complex adaptive systems, which translates into many stakeholders, numerous instances of legal regulation, uncertain demand, and the demand for individualized procedures. Acute events, such as pandemics or natural disasters, as well as everyday challenges, may defy these systems’ ability to fulfill their goals2–4.
This perspective supports an approach of managing performance variability of work processes at the sharp-end, as a paramount role of workers5,6. Variability management has been extensively linked to system resilience7–12.
Referral is an important strategy for integrating healthcare services, but it needs to be improved in health systems, especially in low and middle income countries such as Brazil13,14. In Brazil, the Ministry of Health established in 2008 the National Policy for Referral Prioritization of the SUS. This policy aimed to enhance the integration of processes in the health system and rationalize the use of resources15. This policy introduced the concept of referral prioritization, which comprises the organization, control, management, and prioritization (gatekeeping) of access to care within the Brazilian public health system, ensuring fair and timely access to health services16. Instruments and mechanisms used in the SUS’ referral prioritization include: definition of the network providers, contracts for health services, registration of the health care service units, health care programs, hospitalizations authorizations and authorizations for high complexity procedures, central information system about hospital beds, analytical and operational auditing, monitoring of the health public budget, evaluation and monitoring of health care actions17.
However, there are still gaps in coordination and communication between healthcare providers. In studies in different locations across the country, 18–24 found the following issues hindering the effectiveness of the referral process: a limited supply of vacancies, poor reference and counter-reference process, low use/absence of protocols, and poor information technology systems. 25, in a recent study conducted with oral cancer patients, concluded that the flow ordering system and the patient referral process are part, however, do not guarantee the principles of integrability and health care access. These types of issues may result in avoidable visits to specialists, duplicated tests, and delayed diagnoses, potentially leading to inefficient use of resources, patient harm, and higher operations costs26.
These factors reflect on primary and secondary losses in the healthcare network. A primary loss is defined as wasting the first available medical appointment referenced within the healthcare network while a secondary loss or absenteeism refers to the non-attendance of the patient scheduled in the appointment medical or/and medical exams due to absence of transportation, indication unnecessary the procedure, or other reasons27–29.
Referral prioritization may concern urgent care, hospital stays or ambulatory care (specialist appointments, specialized laboratory and imaging tests, and elective surgical procedures). Recent work employing qualitative approaches such as ethnographic-oriented fieldwork and resilience engineering models have shed light on how health systems adapt to everyday disturbances through referral prioritization of ambulatory care22,30. According to 25, monitorization, integration and adjustment to the needs of services and users is a condition for improving health care.
This study aims to analyze the behavior of variability in referral prioritization at a regional level through an Ergonomic Work Analysis31 and the application of the Functional Resonance Analysis Method (FRAM)32, building on previous studies regarding this service33. We intend to contribute to developing management strategies to manage variability in referral prioritization of ambulatory care.

Research context
This study was conducted in the Brazilian state of Rio de Janeiro, which has a population of approximately 16 million people, according to 34, spread across 92 cities. With a large and diverse population, the state was divided into eight regions to improve the quality of healthcare. Each region - Metropolitana I, Metropolitana II, Noroeste Fluminense, Norte Fluminense, Serrana, Baixada Litorânea, Médio Paraíba, and Centro-Sul Fluminense - comprises a set of cities with diverse demographic characteristics.
In Brazil, the referral prioritization process occurs at four levels: municipal, regional (within a state), state, and national. Referral flows across these levels for Rio de Janeiro state are detailed in 35. At the municipal level, referrals are made for low-complexity procedures by health units within that municipality through the Municipal Referral Center. At the regional level, referral is carried out through Regional Centers (CREGs), composed of a group of municipalities that communicate with each other to use their available health resources effectively. At the state level, the High Complexity Ambulatory Referral Center (REUNI) and the State Internal Referral Center (CRECE) refer low, medium and high-complexity patients and mediate the process between CREGs. Finally, at the national level, the National High Complexity Referral Center carries out referrals for highly complex procedures and manages communication among all Brazilian states36. Figure 1 shows the referral flows within and across the municipal / regional / state / national spheres.

FIG. 1: Referral flows within and across spheres (levels of referral prioritization).

In Figure 1, all arrows represent the directions of the referral - from the Requesting Unit (RU - generally a primary care facility), to a referral center, then sometimes between referral centers of different levels, and finally to a specialized care provider (SCP - usually a hospital or specialized clinic). Referral Centers are the units that receive and analyze the sending referrals by RU and directing them to SCP. At the end, the SCP are the health care facilities where the patient will have the requested ambulatory procedure performed. Flows always start (from referral requests) and end (at SCP’s vacancies) at municipalities, being them the state capital, municipalities from one of the state regions, or even municipalities outside Rio de Janeiro state. In case of the latter, flows are directed to the National Referral Center in Brasilia (Brazil’s capital) and subsequently to the destination state.

METHOD
As initially stated, this study aims to contribute to the development of strategies for managing variability in the referral prioritization of ambulatory care by analyzing the behavior of performance variability. To this end, we use Ergonomic Work Analysis (EWA), which aims to create more realistic work representations, enabling better problem solving by examining the context in which activities are performed37. EWA is characterized by a methodology that, through the situated study of work, fosters an understanding of how workers approach problems, identifies potential obstacles to task execution, and promotes positive transformations in the situations analyzed31. To accomplish this, EWA employs various techniques and methods, including observation, documentation, measurement, description, and modeling of work dimensions37. EWA is based on the notion that workers continuously adjust their performance to cope with the constraints imposed by the situations, recognizing variability as a natural characteristic of work systems. Additionally, EWA can be easily combined with other methods for modeling and analyzing work situations.
This was an exploratory cross-sectional study. Data was collected through observations and semi-structured interviews with referral coordinators, gatekeeper doctors, referral nurses, and administrative workers. Data collection was conducted in three CREGs within the Brazilian state of Rio de Janeiro between August and November 2022, amounting to 30 hours of fieldwork. The diversity of participants (with different positions in the referral prioritization process) aimed to bring a more holistic view of the process. Interviews and observations focused on identifying the work system’s main functions and how they relate to each other, as well as the potential variabilities that impact the work system’s performance and the cognitive processes behind workers’ decisions and actions.
Data analysis was conducted by following a series of steps: (1) field notes were transcribed; (2) data collected from interviews and observations were preliminarily analyzed; (3) system functions were identified using the FRAM method, followed by the (4) identification of the couplings between the functions. Finally, (5) the FRAM model was built using the FRAM Model Visualiser (FMV) software; and (6) respective variabilities were identified, with a spreadsheet being used to tabulate the relevant functions and output variability.

The Functional Resonance Analysis Method (FRAM)
The Functional Resonance Analysis Method (FRAM)32 is a system-modeling method oriented at analysis of performance variability in activities performed by human, organizational and technological actors. FRAM allows for analyzing a system’s activities and examining the interactions among its functions to identify how performance variability can affect situations positively or negatively38,39. The use of FRAM has been explored in various domains with ensuing contributions, being healthcare one of the domains with most applications40,41. The interest in this area is likely due to the sector’s social characteristics, high variability conditions, and limited resources42.
In FRAM notation, the necessary activities to produce a specific output are called functions. These activities can be performed by an individual, a group of workers, an organization, or a technological system. Therefore, functions can be classified as human, organizational, or technological43.
Functions’ interrelations are modeled according to six aspects: input, resource, time, control, precondition, and output32. Step zero for an FRAM analysis is defining its focus and scope. The method can be used for retrospective or prospective analyses, i.e., one can analyze the past events of a system, such as in an accident investigation, as well as possible future events, such as in a risk assessment32,44,45. Once the focus is defined, analyzing the system using FRAM involves four steps, as listed32:
1. Characterization of functions: the result of this first step is a model of the system, graphically illustrating the functions and their potential couplings.
2. Characterization of variability: The potential and actual variability in each function’s output is described in this step.
3. Functional resonance analysis (aggregation of variability): The actual variability of functions under certain conditions is analyzed, and functional resonance possibilities are sought (instantiation).
4. Analysis of results and recommendations: FRAM’s fourth and final stage involves analyzing the results and providing recommendations for maintaining system outcomes within an acceptable range. At this stage, the goal is to create barriers that prevent resonance and mitigate variability.
Through the previous steps, FRAM supports the modeling and analysis of complex systems, enabling the development of interventions to manage variability and increase system resilience. Health systems, particularly referral processes, display complex behavior, with many stakeholders, uncertain demand, and legal regulations. FRAM can provide an understanding of how these elements interact and how these interactions can lead to unexpected results. Therefore, FRAM is a suitable method for modeling the referral prioritization process, as it allows for the representation and analysis of its complexity and performance variability.

RESULTS AND DISCUSSION
In this section, we provide a description for referral prioritization according to collected and analyzed data, and model this service using a process flow notation and the FRAM method. Finally, we show the findings from the FRAM analysis and discuss the results.

Ambulatory healthcare access referral prioritization process
The process starts when the patient goes to a health unit - hereafter called requesting unit (RU) - seeking medical attention. After arriving at the unit, the patient reports to a healthcare worker their personal data, which will be recorded in the municipal referral IT system (MRS). Then, the patient waits for the medical appointment.
During the medical appointment, the physician analyzes the patient’s clinical conditions and requests a referral if specialized care is needed, if not, the patient receives treatment in the unit itself. As the municipal referral IT system (MRS) and the Regional Center’s IT system (RCS) are different, the RU worker must manually transfer the patient’s personal and clinical information to submit the referral request. Hence, the CREG becomes an intermediary between the RU and the SCP.
At the CREG, the request is evaluated by gatekeeper doctors or referral nurses with supporting administrative workers according to referral prioritization protocols and procedures to check if the patient’s clinical information and supporting documentation (laboratory and imaging tests, medical reports) are inserted correctly. At this step, the gatekeeper doctor can make three types of decisions regarding the request: accept it and proceed to the next step; refuse the referral, returning it to the RU and justifying the decision; or put it on hold, sending the request back to the RU because of missing information.
When a request is accepted, the next step is to look for an SCP with a vacancy matching the patient’s profile by gatekeeper doctors or referral nurses with help from administrative workers. After this matching phase, the gatekeeper sends the referral to the SCP, which will evaluate it and submit its decision in the Regional Centers’ IT system (RCS). In case of positive feedback from the SCP, the gatekeeper informs the RU about the decision, i.e., the date and time for the medical appointment and if the patient will need to do some preparation for the procedure, such as fasting. The RU then calls the patient to transmit the information. Following that, the patient must find transportation to attend the appointment, as the procedures are often only available in units far from patients’ home addresses. According to the law, local health authorities are responsible for providing transportation to patients. They do so by offering ambulances, cars, or buses to transport them to the medical facilities. If the SCP refuses the request, the gatekeeper will try to find another SCP to execute the procedure.
However, when the gatekeeper or the SCP notice missing or incorrect information, they put the request on hold and highlight the changes needed in the referral thought of the same system. Besides, they can also ask for a change in the procedure requested, considering the patient’s clinical data. Once the RU is notified about a required change in the referral request, they will try to answer it as fast as possible, so the referral prioritization process is not delayed. After altering the request, the RU resends the referral request to the CREG. The referral prioritization process then proceeds, as explained before. However, if the necessary changes are not adequately addressed, the gatekeeper will put the request on hold again. The BPMN model for this process can be found in 33.

FRAM modeling for the referral prioritization process regarding access to ambulatory care
Considering the process previously described, Figure 2 shows its corresponding FRAM model. Forty-one functions were used to represent the process, 18 presenting output variability. These 18 functions have one or more potential instances of variability in their outputs. Some variabilities were not identified in the three CREGs after analyzing data interviews. During data analysis, we identified eight functions (highlighted in red) more frequently mentioned by the participants of the three CREGs whose output variability could hinder the referral prioritization process. Thus, it was crucial to manage the variability in these functions to achieve the desired results. Therefore, we labeled these functions as “critical functions”.
The presence of variability in these functions' outputs could compromise the overall result of the referral prioritization process, hampering patient referral. Table 1 presents the potential variabilities of these eight functions considering the dimensions of precision and time.
The first highlighted function is “Register the patient in the MRS”. In this function, the RU worker records the patient’s personal data into the IT system. This function is executed manually because the municipal referral IT system is not integrated with other software/databases. Therefore, this function’s output is subject to variability regarding precision since the patient can incorrectly inform the worker, or the worker can commit a mistake when filling out the form. The RU can correct the information later, but the time to perform this action can delay the referral prioritization process and the activation of the following function.
The “Migrate data from one system to another” function is relevant in the modeling because municipal referral centers and CREGs utilize different information systems to carry out the referral prioritization process. Therefore, the RU needs to transfer patient’s personal and clinical data from one IT system to another. As a consequence, data can be migrated incorrectly. Besides, this implies an increase of the workload. Regarding the use of several IT systems to perform the referral prioritization process, a participant points: “Most [of the cities] use the National Referral System (SISREG), one or two cities use another system. Here, at the CREG, it is only the State Referral System (SER)”.

FIG. 2. FRAM general model of the referral prioritization process for ambulatory care.

Variability can be found in the function “Insert referral into the RCS” since some RU workers do not know how to properly use the referral platform and other technological resources (e.g., how to compress large imaging files). This situation may cause inaccuracy in the request, delaying the referral prioritization process. Other factors that contribute to imprecisions and omissions include restrictions on the size and quantity of files that can be uploaded into the IT system, and incorrect requests due to the lack of knowledge about protocols, procedures, and workflows of the referral prioritization process.
When the “Receive referral” function is activated, its output may show variability due to instabilities in the referral IT system and internet access. These two issues may compromise the process since it heavily depends on web-based systems. Participant Z explains:

The system crashes a lot. We check our phones before coming to the referral center. And when I’m at home, someone calls asking if I can log in because if I can’t, it means the system is down...

Another point of attention identified by the modeling is related to the mixing of requests in the referral IT system inbox. According to the participants, the information system does not provide any notification when there is an update on a request that is on hold. Moreover, the IT system does not have functionality allowing new requests to be sorted from the old ones being followed up on due to the need for revisions. To manage this problem, the workers have resorted to using paper notes or electronic spreadsheets to sort out the referrals manually. Using this alternative “system” results in a higher cognitive workload. Additionally, some referrals are associated with a legal process, demanding more attention since their answer must be forwarded to judiciary actors within a short deadline.
Regarding the “Find a vacancy in some SCP” function, participant Y says,

There is not a connection between the registered procedure in the referral and registered procedure in the opening of the system offered by the executant. Mainly for exams. I found a “shoulder ultrasound” vacancy, thereabouts I have found a “general ultrasound” vacancy in the small words (restrictions) that show us that accept to shoulders. Sometimes, I permit it to pass....

This statement highlights the difficulty faced by the gatekeeper in finding a suitable vacancy for the patient when the SPC fails to provide an accurate and standardized description, which can lead to delays in sending the request to the appropriate SPC.
When the SPC carries out the “Answer the referral” function, output variability refers to the timing of the output due to the possible delay in responding to the request with its decision. Participant W points out: “The SPC takes a long time to respond, “confirmation waiting”.
The variability identified in the “Contact the patient” function concerns whether or not the patient can be contacted. This step is crucial in the referral prioritization process as the next step is to inform the patient about the appointment, including the time and location of the medical procedure. If the RU cannot contact the patient, absenteeism is highly likely, leading to vacancy loss. However, if communication is successful, one of the following functions is “Get the transportation”.
According to the findings, patients face significant difficulty accessing transportation services, as some municipalities lack the necessary means to provide this service. This issue is particularly prevalent in certain CREGs. Participant V explains, “We run into another bottleneck because the municipality does not offer transportation. The treatment is continuous. There is no point sending [the patient] if there is no transportation.” Without transportation, patients cannot attend appointments with specialized care providers, often far from their homes. Therefore, transportation is a critical factor for patients to access other levels of care.
The variabilities and their identified interactions reveal the potential brittleness of the referral prioritization system. If the output variability of the system’s functions is not handled properly, there may be a breakdown in the implementation of the process. These challenges in everyday work lead the workers to develop strategies to manage the variability. These experiences resulted in proposals for improvements to manage these outputs and mitigate their negative impacts.
The workers have pointed out some aspects that need improvement in their work system. They suggest adding color signals to alert when new requests are received, and pending requests have been responded to. They also recommend providing training on the software, protocols, procedures, and workflows. Another suggestion was to create a checking functionality in the IT system that automatically classifies whether the patient meets the vacancy admission criteria, reducing patient waiting time and rework. They also recommend automatically migrating patient data from the municipal to the regional IT system to minimize cognitive effort. Finally, they suggested increasing the IT system’s capacity for file upload and inserting characters.

FRAM Analysis
Numerous functions present in Figure 2 display output variability, revealing potential brittleness in the system. Workers often develop mechanisms to adapt when faced with variability during task execution. They aim to achieve acceptable results and ensure adherence to referral prioritization guidelines. However, less-than-effective management of variability can lead to disruptions. Additionally, the stress of dealing with these variabilities can result in mental wear. The “Contact with the patient” function (Figure 3) is an example of a function affected by internal and upstream-downstream variability, and is also related to primary losses in referral prioritization.
This function aims to inform the patient about the status of their referral. If the response is positive, the RU will provide the patient with details about the appointment, including the time, date, and location. However, this function can only be executed if two upstream functions are completed: “Register patients in the municipal system” and “Receive referral in the RU”. Of these two functions, “Register patient in the municipal system” has been identified as the most critical due to potential inaccuracies in the patient’s contact information, which could hinder the notification process.
When patients provide their personal information, mistakes can occur. They may provide incorrect phone numbers and addresses or even someone else’s contact information. The latter happens because patients may not have a telephone number or know how to use it. This situation is especially common in rural areas, where addresses can also be misleading.

FIG. 3: Excerpt of the FRAM model highlighting the functions involved in notifying the patient about the authorized referral

Another significant point concerns when the worker registers the patient in the system. When the RU professional collects the patient’s data, they might misunderstand it and incorrectly register it. This situation might happen due to the high cognitive workload of the process, which may affect the worker’s perception and attention.
Incorrect patient personal information can seriously compromise the referral process. Inaccuracies may only become apparent after the referral prioritization process, wasting previous efforts and potentially depriving the patient of necessary treatment. In this context, workers have developed strategies to address this issue. For example, when the RU is unable to reach the patient by telephone, a health worker may visit the address provided by the patient to deliver appointment details or inform a neighbor. However, faulty addresses can also hamper this notification process.
Given the earlier explanations, we can see that if the output is not acceptable, the following functions need to handle both exogenous and endogenous disturbances. If this function cannot absorb the variability, it could impact the downstream functions and potentially lead to a system failure, compromising referral prioritization outcomes.
To minimize instances of primary loss and improve the execution of the “Register patients in the system” function, government databases, such as CadSUS, could be shared with the health units among the patient’s ID number, allowing the forms to be automatically filled with patient data. Direct access to this information could prevent some errors, as suggested by study participants. As for the “Contact the patient” function, a dedicated team could be assigned to manage it, thus reducing the cognitive load on the workers. Since this function requires time and precision to ensure patients are informed about their appointments, establishing this team could be argued as essential. An additional adaptation that could be considered to improve this function is to request the contact and address of a second person who can be neighbors or relatives.
Therefore, to reduce the variabilities identified in this study, Table 2 presents the functions considered critical with their associated problems and your respective proposed interference based on the insights gathered from the interview responses.

CONCLUSION
This study analyzed the impact of the variability in the referral prioritization process for ambulatory care at three Regional Referral Centers in a Brazilian state. For this purpose, we employed Ergonomic Work Analysis and the Functional Resonance Analysis Method (FRAM). This approach allowed for identifying interdependencies and non-linear relationships between the system’s functions.
While the Business Process Modeling Notation (BPMN) is frequently used to comprehend workflow in a process, a FRAM model illustrates how functions are interconnected and interrelated in the same process. Therefore, while BPMN seeks to map processes in a standardized manner, FRAM enables the modeling of complex systems and the comprehension of emerging variability.
In the data analysis phase, the FRAM model helped in identifying variability and its behavior. One of the main issues identified is that the referral IT systems do not communicate with other software used by the referral process actors (Requesting Unit - RU; and Specialized Care Provider - SPC). Another important point concerns the lack of the required knowledge to use the regional IT system and other technological systems effectively.
Furthermore, the participants also demonstrated a need for more knowledge about standard procedures. However, workers were willing to propose ways of improving the system, which is important since they operate the process and, as a result, have a comprehensive understanding of the system’s weaknesses and strengths.
This study’s main contribution is understanding how everyday variability impacts the referral prioritization process at a regional level and, consequently, access to healthcare. In addition, we identified possible improvements to the process based on the participants’ perspectives. As a next step, we intend to expand our understanding of the referral prioritization process and its potential improvements by conducting an analysis focused on hospital stays.

Tab.1

Funding. The research had the support of the Coordination for the Improvement of Higher Education Personnel – Brazil (CAPES) - Funding Code 001 and National Council for Scientific and Technological Development (CNPQ) (Process nº 160.740/2022-3) and Center for Strategic Studies at FIOCRUZ.

Ethics clearance.

Conflicts of interest. Do not declare for the authors.

Collaborations. MC Bruzi: conceptualization, formal analysis, investigation, data curation, writing - original draft, writing - review & editing. R Arcuri: conceptualization, data curation, supervision, investigation, writing - review & editing, project administration. HC Bellas: supervision, investigation, writing - review & editing, project administration. J Vianna: investigation, writing - review & editing. L dos Santos: writing - original draft, writing - review & editing. AS Moraes: supervision, writing - review & editing.

REFERENCES
1. Jatobá A, Bellas HC, Soranz D, Koster I, Arcuri R, Bulhões B, et al. Relatórios de Pesquisa. Rio de Janeiro: Centro de Estudos Estratégicos da Fiocruz; 2019. (FIOCRUZ). Disponível em: https://cee.fiocruz.br/sites/default/files/CEE-Relat%C3%B3rio%20de%20Pesquisa_Analise_Situada_Sistema_Regulacao_Jatob%C3%A1%20et%20al.pdf
2. Braithwaite J, Clay-Williams R, Nugus P, Plumb J. Health care as a complex adaptive system. Em: Hollnagel E, Braithwaite J, Wears RL, organizadores. Ashgate Studies in Resilience Engineering. Ashgate; 2013.
3. Braithwaite J. Changing how we think about healthcare improvement. BMJ. 2018;361. Disponível em: https://www.bmj.com/lookup/doi/10.1136/bmj.k2014
4. Brasil. PORTARIA No 1.559 Institui a Política Nacional de Regulação do Sistema Único de Saúde - SUS. Ministério da Saúde. 2008.
5. Moraes ASP, Arezes PM, Vasconcelos R. Promoting safety during process variability: A multidisciplinary challenge. Saf Reliab Risk Anal Horiz. 2014;
6. Moraes ASP, Arezes PM, Vasconcelos R. Variability management: A still to be noticed role of workers. Occup Saf Hyg II. 2014; Disponível em: https://repositorium.sdum.uminho.pt/bitstream/1822/33693/1/CL52_moraes.pdf
7. Coles E, Anderson J, Maxwell M, Harris FM, Gray NM, Milner G, et al. The influence of contextual factors on healthcare quality improvement initiatives: a realist review. Syst Rev. 2020;9(1):94.
8. Ellis LA, Churruca K, Clay-Williams R, Pomare C, Austin EE, Long JC, et al. Patterns of resilience: A scoping review and bibliometric analysis of resilient health care. Saf Sci. 2019;118:241–57.
9. Gilson L, Ellokor S, Lehmann U, Brady L. Organizational change and everyday health system resilience: Lessons from Cape Town, South Africa. Soc Sci Med. 2020;266:113407.
10. Bellas HC, Arcuri R, Ferreira D de S, Bulhões B, Masson L, Vidal MCR, et al. Complex systems design based on actual system functioning: Coping with variability in a national water ambulances service. Work Read Mass. 2022;73(1):265–77.
11. Bellas HC, Bulhões B, Arcuri R, Vidal MCR, de Carvalho PVR, Jatobá A. Community health workers’ non-technical skills for delivering primary healthcare in low-income areas. Work Read Mass. 2022;72(3):1047–54.
12. Jatobá A, Bellas H, Arcuri R, Bulhões B, Carvalho PVRD. Water ambulances and the challenges of delivering mobile emergency healthcare to riverine and maritime communities. Am J Emerg Med. 2021;47:258–66.
13. Kruk ME, Gage AD, Arsenault C, Jordan K, Leslie HH, Roder-DeWan S, et al. High-quality health systems in the Sustainable Development Goals era: time for a revolution. Lancet Glob Health. 2018;6(11):1196–252.
14. Kruk ME, Myers M, Varpilah ST, Dahn BT. What is a resilient health system? Lessons from Ebola. The Lancet. 2015;385(9980):1910–2.
15. Guabiraba K, Gomes G, Melo EA. Oportunidades, percalços e justificativas: a descentralização da regulação ambulatorial no município do Rio de Janeiro. Saúde Em Debate. 2022;46(132):107–20.
16. Santos L dos, Feichas A, Carvalho PV de, Arcuri R, Vidal MC. Análise da variabilidade nos processos de regulação assistencial: a oferta de vagas em um hospital público de alta complexidade. Em: Anais do Congresso Brasileiro de Ergonomia da ABERGO. 2022.
17. Santos FP dos, Merhy EE. Public regulation of the health care system in Brazil - A review. Interface. 2006;2.
18. Serra CG, Rodrigues PHDA. Avaliação da referência e contrarreferência no Programa Saúde da Família na Região Metropolitana do Rio de Janeiro (RJ, Brasil). Ciênc Saúde Coletiva. 2010;15(suppl 3):3579–86.
19. Silva MVS da, Silva MJ da, Silva LMS da, Nascimento AAM do, Damasceno AKC, Oliveira RM. Regulação do acesso à saúde: o processo de trabalho administrativo da enfermagem. Esc Anna Nery. 2011;15(3):560–7.
20. Soares EP, Scherer MDDA, O’Dwyer G. Inserção de um hospital de grande porte na Rede de Urgências e Emergências da região Centro-Oeste. Saúde Em Debate. 2015;39(106):616–26.
21. Farias ACBD, Barbieri AR. Follow-up uterine cervical cancer: study of continue assistance to patient in a health region. Esc Anna Nery - Rev Enferm. 2016;20(4). Disponível em: http://www.gnresearch.org/doi/10.5935/1414-8145.20160096
22. Arcuri R, Bulhões B, Jatobá A, Bellas HC, Koster I, d’Avila AL, et al. Gatekeeper family doctors operating a decentralized referral prioritization system: Uncovering improvements in system resilience through a grounded-based approach. Saf Sci. 2020;121:177–90.
23. Basto LBR, Barbosa MA, Rosso CFW, Oliveira LMDAC, Ferreira IP, Bastos DADS, et al. Practices and challenges on coordinating the Brazilian Unified Health System. Rev Saúde Pública. 2020;54:25.
24. Bernardino Junior SV, Medeiros CRG, Souza CFD, Kich J, Alves AM, Castro LCD. Processos de encaminhamento a serviços especializados em cardiologia e endocrinologia pela Atenção Primária à Saúde. Saúde Em Debate. 2020;44(126):694–707.
25. Casotti E, Carvalho CPMD. Regulação do acesso ao tratamento do câncer de boca no estado do Rio de Janeiro. Rev Flum Odontol. 2022;1(60):75–87.
26. Tuot DS, Leeds K, Murphy EJ, Sarkar U, Lyles CR, Mekonnen T, et al. Facilitators and barriers to implementing electronic referral and/or consultation systems: a qualitative study of 16 health organizations. BMC Health Serv Res. junho de 2015;15(1):568.
27. Pinto RB, Ferreira IP, Costa RDJPD, Silva Junior MRD. Study on absenteeism and underuse of specialized consultations and exams in the municipalities of the Rio Caetés region, Pará, Amazon. Em: II International Seven Multidisciplinary Congress. Seven Congress; 2023. Disponível em: https://homepublishing.com.br/index.php/cadernodeanais/article/view/94
28. Oliveira SCP. Perdas primárias e secundárias de consultas especializadas de um complexo hospitalar público municipal. Disponível em: https://saude.campinas.sp.gov.br/biblioteca/trabalhos/Curso_Gestao_Saude_2016_2017/Stefane_CHPEO.pdf
29. Universidade Aberta do SUS. Regulação do Acesso Ambulatorial - T3: Monitoramento. 2018. Disponível em: https://moodle.unasus.gov.br/vitrine29/mod/page/view.php?id=2918
30. Kierkegaard P, Owen-Smith J. Determinants of physician networks: an ethnographic study examining the processes that inform patterns of collaboration and referral decision-making among physicians. BMJ Open. 2021;11(1):e042334.
31. WISNER A. Understanding problem building: ergonomic work analysis. Ergonomics. 1o de março de 1995;38(3):595–605.
32. Hollnagel E. FRAM: The Functional Resonance Analysis Method. CRC Press; 2012.
33. Bruzi MCTBG, Arcuri R, Bellas HC, Souza JTVD, Santos LD, Carvalho PVRD, et al. Análise da variabilidade na regulação do acesso à assistência à Saúde ambulatorial em complexos reguladores utilizando o método FRAM. Em: Anais do(a) Anais do Congresso Brasileiro de Ergonomia da ABERGO. Florianópolis: Even3; 2023. Disponível em: https://www.even3.com.br/anais/abergo2023/696820-ANALISE-DA-VARIABILIDADE-NA-REGULACAO-DO-ACESSO-A-ASSISTENCIA-A-SAUDE-AMBULATORIAL-EM-COMPLEXOS-REGULADORES-UTILI
34. Instituto Brasileiro de Geografia e Estatística. Panorama do Censo 2022. 2022. Panorama do Censo 2022. Disponível em: https://censo2022.ibge.gov.br/panorama/
35. Bruzi MCTBG, Arcuri R, Bellas HC, Souza JTVD, Carvalho PVRD, Jatobá A. Fluxos de trabalho em sistemas complexos: uma análise da regulação médica estadual ambulatorial. Em: Anais do(a) Anais do Congresso Brasileiro de Ergonomia da ABERGO. São José dos Campos: Even3; 2022. p. 1–12. Disponível em: http://www.even3.com.br/Anais/abergo2022/539096-FLUXOS-DE-TRABALHO-EM-SISTEMAS-COMPLEXOS--UMA-ANALISE-DA-REGULACAO-MEDICA-ESTADUAL-AMBULATORIAL
36. World Health Organization. Health systems resilience toolkit: A WHO global public health good to support building and strengthening of sustainable health systems resilience in countries with various contexts. World Health Organization; 2022.
37. Másculo FS, Vidal MC. Ergonomia: Trabalho Adequado e Eficiente. Rio de Janeiro: Elsevier; 2011.
38. Haddad AN, Rosa LV. Construction sustainability evaluation using AHP and FRAM methods. Em: Proceedings of the 2015 Industrial and Systems Engineering Research Conference. 2015. p. 556–65.
39. Kaya GK, Ozturk F, Sariguzel EE. System-based risk analysis in a tram operating system: Integrating Monte Carlo simulation with the functional resonance analysis method. Reliab Eng Syst Saf. 2021;215:107835.
40. Bagnara S, Tartaglia R, Albolino S, Alexander T, Fujita Y, organizadores. Proceedings of the 20th Congress of the International Ergonomics Association (IEA 2018): Volume I: Healthcare Ergonomics. Cham: Springer International Publishing; 2019. (Advances in Intelligent Systems and Computing; vol. 818). Disponível em: http://link.springer.com/10.1007/978-3-319-96098-2
41. Sujan M, Pickup L, de Vos MS, Patriarca R, Konwinski L, Ross A, et al. Operationalising FRAM in Healthcare: A critical reflection on practice. Saf Sci. 2023;158:105994.
42. Salehi V, Veitch B, Smith D. Modeling complex socio-technical systems using the FRAM: A literature review. Hum Factors Ergon Manuf Serv Ind. 2021;31(1):118–42.
43. Bellini E, Coconea L, Nesi P. A Functional Resonance Analysis Method Driven Resilience Quantification for Socio-Technical Systems. IEEE Syst J. 2020;14(1):1234–44.
44. França JEM, Hollnagel E, dos Santos IJAL, Haddad AN. FRAM AHP approach to analyse offshore oil well drilling and construction focused on human factors. Cogn Technol Work. 2020;22(3):653–65.
45. Yousefi A, Rodriguez Hernandez M, Lopez Peña V. Systemic accident analysis models: A comparison study between AcciMap, FRAM, and STAMP. Process Saf Prog. 2019;38(2).