Artigos

Conteúdo (article):

Science, Technology, and Innovation Policies and the Brazilian Unified Health System: a study of National Institutes of Science and Technology in Health.

ABSTRACT
This study analyzes the potentialities and limits of science, technology, and innovation policies for engagement between the health research conducted in Brazil and the country’s Unified Health System (SUS). The study is based on an analytical matrix grounded in the field of social studies of science and technology, from a social constructivist perspective. Data were obtained from semi-structured interviews with the coordinators of the National Institutes of Science and Technology in health. The evaluation of participants’ responses was oriented by the analytical categories of ‘motivation’ and ‘influence’, including recommendations addressed to initiatives in research promotion. The results show that Brazil’s NIST Program succeeded in strengthening the country’s scientific base and linkage between researchers and institutions, but the progress was incipient in terms of closer interaction between research and the SUS. The study also revealed gaps in the initiatives by the NIST Program, such as bureaucratic assessment of the research results and lack of dialogue between researchers, policymakers, health administrators, and the population. Consistent assessment and dialogue are necessary to expand the scientific community’s engagement in confronting the country’s public health problems and promoting more effective incorporation of research results in the SUS.
KEY WORDS: National Science, Technology, and Innovation Policy; Research Financing; Instruments for Management of Scientific Activity; Public Health; Unified Health System

RESUMO
O trabalho analisa as potencialidades e os limites das Políticas de Ciência, Tecnologia e Inovação para aproximação da pesquisa conduzida no país e o SUS. É embasado por matriz analítica ancorada no campo dos Estudos Sociais da Ciência e Tecnologia, sob a perspectiva construtivista social. Os dados foram obtidos em entrevistas semiestruturadas realizadas com os Coordenadores dos INCTs da saúde. A avaliação das respostas foi norteada por categorias analíticas de motivação e influência, incluindo recomendações direcionadas às iniciativas de fomento. Os resultados revelam que o Programa INCT foi bem-sucedido em fortalecer a base científica no país e a articulação entre pesquisadores e instituições, mas os avanços foram incipientes em se tratando da aproximação da pesquisa com o SUS. Também deixam entrever lacunas nas iniciativas do Programa INCT, como avaliação burocrática dos resultados e a falta da interlocução entre pesquisadores, formuladores de políticas, gestores de saúde e a população, estratégias necessárias para ampliar o engajamento da comunidade científica no enfrentamento dos problemas de saúde pública e viabilizar a incorporação mais efetiva de resultados de pesquisas no SUS.
Palavras-chave: Política Nacional de Ciência, Tecnologia e Inovação; Financiamento da Pesquisa; Instrumentos para a Gestão da Atividade Científica; Saúde Pública; Sistema Único de Saúde.

INTRODUCTION
The recognition of health as a universal right guaranteed by the State requires governments to deal with numerous challenges, one of which is to ensure the democratization of access to improvements provided by scientific and technological progress. This is because the level of investment and risks involved in generating innovations is greater in health than in other fields, while the resulting breakthroughs have a strong economic appeal, which favors the development of technologies that prioritize the demands of the more economically privileged population, thereby aggravating social inequities and imposing additional obstacles to the sustainability of universal health systems such as Brazil’s Unified Health System (SUS) (2–5).
In this scenario, the current study aimed to analyze the potentialities and limits of induction of science, technology, and innovation policies (STIPs) in Brazil for closer engagement between research and the SUS and the promotion of effective solutions to public health problems, based on a case study of National Institutes of Science and Technology (NISTs) in health. These institutes, qualified as relevant centers for research and innovation in Brazil (6), were created through the NIST Program (9,10) launched by the Ministry of Science, Technology, and Innovations (MCTI)(7,8) and implemented by the National Council for Scientific and Technological Development (CNPq).
The analysis was based on the social constructivist perspective applied to the field of social studies of science and technology (SSSTs), evidencing the influence and role of social factors in the formation of the epistemic content of science and technology(11), especially through the legacy from anthropological studies that revealed science as a social practice similar to other such practices and signaled the invasion of knowledge production by economic and social pressures and interests, where policies act as an instrument capable of materializing research results (12–17). The analysis was also backed by reflections from studies on the dynamics of interaction between various factors that influence research and the knowledge it produces (18–21).
Systematization of the study’s information allows identifying potential strengths and weaknesses in the strategy for induction of NISTs in health, listing perspectives for the refinement of ST&I policies linked to the Unified Health System.

METHODS
This exploratory study (22) was carried out with semi-structured interviews with the coordinators of the NISTs or their proxies. The institutes, totaling 30, were classified by the CNPq as belonging to the health field (23).
From this total of 30 institutes, 26 NISTs participated in the study. A total of 30 individuals were interviewed: 23 coordinators, 2 assistant coordinators, 3 members of the management committees, 1 thematic area joint coordinator, and 1 researcher that works with the institute’s coordination. Based on specific circumstances, 3 NISTs had more than one participant, usually at the interviewee’s request.
The interviews took place in November and December 2020. The great majority were held online and lasted 50 minutes on average. They were based on a script developed to identify the main motivations and influences that oriented the proposals for creating the NISTs, and which affects the direction of the studies. The study aimed to understand the role of the health priorities for the population and the SUS in this process, to verify the alignment of the results produced by the NISTs with the services provided by the SUS, and to seek to capture the participants’ assessment of the NIST Program and suggestions for measures to promote closer engagement between the institutes’ research and the SUS.
The interviews were filmed, and the answers were transcribed, organized, summarized, grouped, and finally systematized according to the units and subunits of analysis established with the theoretical frame of reference that oriented the current study (Table 1).
The study’s limits are due mainly to the institutes’ diverse characteristics and the interviewees’ profiles. The institutes’ research themes vary, as do their mission, territory, structure, and focus. Some conduct basic research, while others perform applied research, and still others work with health services, the support received from the Program, and other aspects. They also result from the authors’ academic profile and professional careers, which may lead to distinct perceptions of the objects of study and the responses received.
The current article represents a cross-section of the principal author’s PhD research, entitled “Science, Technology, and Innovation Policies and the SUS: an analysis of the participation of the National Institutes of Science and Technology in Health in the organization of Brazil’s public universal healthcare system”. The study was approved by the Institutional Review Board of the Institute of Studies in Collective Health, under Universidade Federal do Rio de Janeiro, review number 4.379.869.

RESULTS
MOTIVATIONS AND FACTORS THAT INFLUENCE THE CREATION OF THE NISTs: GENERAL ASPECTS AND THE ROLE OF THE SUS
We began by asking generically about the factors and motivations that had oriented the proposals submitted to the CNPq. Next, we asked specifically about the influence of the SUS in this process. In the former case, fewer than half of the participants cited social problems, public health issues, or services provided by the SUS among the principal factors, as shown in Table 2. An inverse situation was observed with the specific question on the SUS, when the great majority stated that the public health system was an important factor.
Concerning the influence of the SUS, some participants argued that research by the NIST is essential for public health or for the public health system, without voicing the opposite view. This direction indicated the pertinence of delving deeper into the question during the interview. Participants sometimes reworded the answer. In other situations they continued to highlight the project’s importance for the SUS, justifying its social relevance and insisting on the direction ‘from-research-to-the-health-system’, not the other way around.
Some answers added further perspectives to the research question. One participant stated that some public health problems had still not been acknowledged by the SUS when the NIST was created, and thus that the proposal aimed to demonstrate that certain diseases, such as rare genetic diseases, needed to be considered. Another interviewee called attention to difficulties or limitations with the incorporation of the knowledge and technologies generated by the NIST, since the research functioned at “a somewhat higher level than what is possible in the SUS”. Participants acknowledged that no place other than the SUS had the potential to perform clinical research. This characteristic made the demands by the SUS important for the studies, citing the SUS as “the world’s best laboratory for experimentation”.
Nearly all participants who did not identify relevant influence from the SUS stated that their institute had its own important research lines and trajectories, independent of the public health system. For example, one participant cited the correspondence between his institute’s research and current global challenges in the health field. Additional arguments drew on the idea that health needs were part of the research projects’ justification, but that research commitments to the SUS were not priorities.
In addition, the bureaucratization of definitions on research demands and priorities established by the government, according to some participants, would involve establishing guidelines beyond the horizons for contribution by the proposing groups. This would lead to a reduction in the power of the SUS to influence the proposals. Another participant stated that there was no concern with the public services supplied or that the NIST would only support researchers that work in the health field without having a direct affiliation with health.

INFLUENCES IN THE IMPLEMENTATION OF STUDIES BY THE NISTs
General influences
Factors related to budget were the most widely cited. Budget availability, flexibility, stability, and delays were highlighted. All participants stated that the NIST relied on other sources of financing, both domestic and sometimes international. Some participants emphasized that it was inevitable to adapt the researcher’s scientific interest to the financing agency’s convenience, the influence of which varied according to the funds available at the institute. However, some contended that the impacts were marginal, since the adjustments could be made without altering the nature or essence of the research. One interviewee reported that the research was flexible, and that it was possible “to look a little to the side” without changing the direction of the research.
In addition to budget, the team’s nature and composition formed another aspect with a preponderant influence, identified as the factor that most contributed to the studies’ quality performance. Aspects such as team qualifications, training, specialization, experience, commitment, complementary skills, scientific leadership, motivation, optimistic and innovative spirit, focus, and management capacity were reported as important.
The majority acknowledged the influence of the numerous collaborations with universities and research institutes in Brazil and globally, particularly when linked to the possibility of interaction with centers of excellence to access to new technologies and more advanced research settings. Although less frequent, partnerships with companies were also mentioned, and their influence was particularly associated with differences in views on the direction of the research projects, with companies guided by market factors and the researchers by scientific interests, which affected the objectives and priorities.
Few participants referred to public health or the SUS as factors that influenced the direction of studies conducted by the NISTs. However, over the course of the interviews, many reported working with studies related to health emergencies, notably the fight against the Zika epidemic and the COVID-19 pandemic.
Other aspects such as progress in the field of studies and in the research conducted by the NISTs, regulatory requirements, and bureaucracy also affected the studies’ planning and direction.
Influences of the NIST Program
The great majority of participants reported that the NIST Program had changed the course of their research. The reasons featured the funds’ availability, volume, and flexibility. Equal importance was assigned to the Program’s requests for proposals, especially the mandatory structuring of the NISTs in network format and the induction of focus on innovation. The latter factor also appeared in answers that highlighted the importance of conducting research projects concerned with the ultimate destination of the knowledge produced, beyond the scientific field itself.
The Program’s longevity was also emphasized as the first structuring initiative for science and technology in Brazil in decades, allowing the implementation of work proposals aimed at laying the foundations for the research. Participants also cited the incentives for training and incorporation of young scientists, staff training and qualification, the visibility assigned to ‘NIST status’, and the Program’s management format.
Some participants reported that the Program had not changed the direction of their research. However, some cited certain impacts from the conditions for conducting the research, attributing this especially to the greater robustness provided to the research and the possibility of conducting activities that had not been funded previously, related to the transition from basic to applied research (e.g., prototyping).

CONTRIBUTIONS FROM RESEARCH BY NISTs TO THE SUS: EXPECTATIONS AND INCORPORATION OF TECHNOLOGIES AND KNOWLEDGE PRODUCED BY NISTs
Nearly all participants cited at least one example of expected contributions by the NIST to the SUS, but only slightly more than one third stated that their institute had succeeded in ensuring incorporation of the knowledge by the public health system.
Participants sometimes placed greater value on the perceived potential for future and indeterminate contributions. For example, they glimpsed that the knowledge generated by the NIST in the identification of the SARS-CoV-2 genome could be incorporated by third parties in new products (while the NIST’s contribution to this development would not be direct).
We were also able to group the contributions listed by participants into three categories (27): product development, input for clinical practice, and backing for public policies.
As for product development, participants expressed varied views on what they considered an expectation or actual contribution to the SUS. On the one hand, the mere offer of a potential product was considered a deliverable by the institute, such as a molecule’s proof of concept. From this perspective, it would be up to the government to create the conditions for the subsequent development of studies, if such development was in the government’s interest. Meanwhile, the delivery of final products was considered a contribution. Still, the lack of a joint strategy between the NISTs and government, including precarious dialogue with the SUS, was cited as a barrier and disincentivizing factor for the involvement and commitment by researchers in the innovation process. Other difficulties were described, especially the lack of interest by Brazil’s domestic industry in the final development of products generated initially by the NISTs.
Participants also cited the actual contributions from the consolidation of technological platforms in the NISTs as structures equipped for the development and manufacturing of products and inputs demanded by the SUS, as in the urgencies created by the COVID-19 pandemic.
Products that have already been incorporated were listed: (i) online teaching materials for qualification and training adopted by the University of the Unified Health System; (ii) participation in the Brazilian standardization of an antibiogram (BrCAST); (iii) development, production, and supply by the NIST of recombinant proteins for coronavirus diagnostic tests used in the SUS; and (iv) an insulin cream capable of accelerating the healing of wounds caused by diabetes, effective in the prevention of amputations. In the latter case, the product would be manufactured by the NIST itself and only distributed among its collaborators. According to the participant, the initiative still lacked a partner to sponsor the clinical trials and to produce the insulin cream on an industrial scale.
As for contributions to clinical practice, some participants reported that the NIST works directly with the population’s diagnosis and treatment and would be involved in the establishment of treatment strategies, the identification of more rapid diagnostic procedures, and the performance of clinical trials to identify more effective drugs, besides training and qualification of personnel to work directly with the services provided by the SUS. The participants’ responses resulted in a long list of potentialities and expectations for translation of the knowledge into available technologies in the SUS. However, only some of the participants cited results that had already been incorporated.
As for the last category of contributions, namely backing for the development of public policies, some interviewees mentioned participation in the drafting of a bill of law in the mental health field; the design of a policy for rare genetic diseases implemented by the Brazilian Federal government; and the supply of the necessary elements for a ruling on the incorporation of technologies in the Ministry of Health’s decision-making processes in the National Commission for the Incorporation of Technologies in the SUS (CONITEC).
A few participants stated that the respective NIST did not have any project with an immediate impact on the SUS and that the institute did not have any relationship with the public health system.

ASSESSMENT OF THE NIST PROGRAM
Relevance for the S&T activity’s engagement in the country and the SUS
Most participants stated that the NIST Program had contributed to modifying the relationship between their scientific activity and the generation of innovations in Brazil and the SUS, attributed mainly to the premises of the requests for proposals and action by the Ministry of Health.
As for the requests for proposals, participants emphasized the requirement of including such aspects as the project’s social dimension, valuing applied research, responsibility for return on investment, and knowledge transmission. Meanwhile, some warned that the SUS could be cited in the projects’ wording merely to obtain the funding, without any real presence in the proposed study’s objectives.
In relation to the Ministry of Health, participants added that engagement with the SUS was promoted when the Ministry acted as co-financer of the Program, acting in the project selection and monitoring of the studies, especially through participation in biannual or annual meetings to assess the results. However, this follow-up was limited to the exchange of progress reports, thus limiting the actual progress achieved.
Participants also cited reasons involved in the NISTs’ modern R&D structure as provided by the Program, available to obtain solutions to health, as well as to develop and produce cutting-edge technologies for the health system.
For those who claimed that the NIST Program failed to engage researchers with public health demands, half criticized the Ministry of Health and the SUS, emphasizing the difficulty or lack of channels for interaction and questioning the Ministry’s tendency to foment its own agencies almost exclusively.
Participants also emphasized the difference between the approaches by the Ministry of Science, Technology, and Innovation and the Ministry of Health. They reported that interaction was much easier with the Ministry of Science, Technology, and Innovation. As explained, it was not necessary to chase after this Ministry: “They came to me and said, ‘We need this, that, and the other.’” Meanwhile, the relationship with the Ministry of Health was limited to launching requests for proposals, with no direct dialogue or strategies for commissioning technologies. They also warned that the lack of medium- and long-term scientific planning in the SUS, that is, the definition of where the system is intended to go and what is needed to get there, was a factor for estrangement between the studies conducted in the NISTs and the public health system.
Other participants stated that the Program’s prime objective had been missed by the government when it financed NISTs that offered no product or deliverable whatsoever. The criticism was targeted at the Program’s second edition, suggesting that by allowing projects with a basic research profile, the Program had overlooked the priority of producing knowledge and technologies objectively available to the population.
Other participants did not answer or were not conclusive when asked about the relevance of the NIST Program for the research teams’ engagement with the SUS. For example, some said they were unable to take a position when asked why the respective NIST was not doing frontline work and thus lacked a direct connection to the SUS.
Strategies for engagement between research and the SUS.
Given the request to identify strategies for engagement between the NISTs’ research and the SUS, the great majority of the recommendations were addressed to the Ministry of Health and this public health system’s administration. Slightly less than half of the participants made remarks addressed specifically to the NIST Program.
In the case of recommendations to the Ministry of Health and the SUS, participants cited the need for more direct contact between policymakers and health administrators at the various levels of power and the NISTs. Some warned that many research projects might have the potential to generate frontline impact, but that this was jeopardized by the lack of dialogue. As warned, the merely bureaucratic follow-up of the studies was insufficient, since researchers managed to draft progress reports that satisfied the funding agent’s formal requirements.
One suggestion for overcoming the problem called for frequent meetings with the Ministry of Health and the SUS, such as seminars, discussion forums, and specific meetings to debate and operationalize the application of the knowledge to the public health services.
It was also suggested that the MoH Department of Science, Technology, Innovation, and Strategic Inputs should identify what could be financed in the NISTs’ portfolio to meet the population’s health needs or to encourage NISTs to analyze the demands of the SUS. The argument was that it would be in the researchers’ own interest to contribute to studies induced by government, based on actual public health demands, since it would be a way of lending relevance to the research work, with greater odds of success in the incorporation of results (i.e., it would reduce the risk of developing products that would not interest the government). With this purpose, they recommended launching competitive requests for proposals aimed at meeting specific health demands and at commissioning technologies. In the latter case, they also stated that this strategy could be promoted at all levels of Brazil’s federal system, including the state and municipal levels, such as the National Council of State Health Secretaries (CONASS) and the National Council of Municipal Health Secretaries (CONASEMS).
However, while the Ministry of Health and the SUS were criticized for their limited interest in the research conducted by the NISTs, some participants also reported that many researchers themselves have no idea what the SUS is. This ignorance of the public health system’s scope, complexity, and demands was cited as a problem in the scientific community itself, according to some participants.
As for the recommendations addressed specifically to the NIST Program, most responses were targeted to the request for proposals and the need for more intense interaction between the NIST Program and the Ministry of Science, Technology, and Innovation with the Ministry of Health. It was suggested that the request for proposals should require the specification of the relationship between the proposed research and its application to the SUS, contending that this would lead to a mandatory change of behavior in the scientific community (if it were a standard requirement).
This set of suggestions also included an emphasis on the importance of Brazil’s domestic industry participating in the Program’s next edition, as well as the need for a national agenda of innovation coordinated by government, with participation by industry. In terms of joint initiatives, it was suggested that the Ministry of Health (Department of Science, Technology, Innovation, and Strategic Inputs), CNPq, and the Ministry of Science, Technology, and Innovation should map which NISTs would have the possibility to offer products, and that the SUS should be one of the Program’s financers, among others.
There were also recommendations for initiatives focused on the Program’s administration, besides closer follow-up of the activities developed by the NISTs, and that the assessment should be continuous and propositional, with the creation of an interlocutor with the CNPq and the State Research Support Foundations, whose role would be to ensure the NIST’s success, besides the need to support more simplified projects in the Program’s subsequent editions, with more direct application to the SUS.

DISCUSSION
Based on the theoretical analytical framework adopted here, our study allowed identifying less obvious elements in the various decisive elements for the shaping and dynamics of the work by Brazil’s National Institutes of Science and Technology (NISTS) in health, revealing the mosaic of factors that influence the knowledge content produced in these institutes.
The modeling of the proposals was oriented mainly by the opportunity for access to a robust research funding source, associated especially with the interest in the spinoffs from the research lines on the part of the project coordinator and the research groups in the NISTs. Certain guidelines in the NIST Program also influenced this design, potentially considering the budget constraints imposed on scientific research activity in Brazil, making Brazilian researchers potentially sensitive to opportunities for funding, and having favored the incorporation of these dictates into the proposals submitted (18–21).
The relevance of the influences of demands from public health and the SUS was tied to the premises in the Program’s requests for proposals, assuming a merely marginal importance. In addition, many of the clarifications provided by participants showed that the demonstration of the proposed studies’ social relevance was attributed more to the need to make them eligible for funding, increasing the odds of approval by the NIST Program, but without corresponding to the proposed study’s real objectives (a common strategy in the scientific community, as discussed in numerous studies) (18,26,28). Another characteristic that proved applicable to the research conducted by the NISTs was that due to the paradigm differences between policy, management, and science that affect the perception of priorities (29,30), an insufficient dialogue between drafters of the research agendas and the academic community could jeopardize the subsequent uptake of what is considered scientifically relevant, resulting in low adherence (or merely formal adherence) to the proposed research agendas.
Other characteristics for understanding the Program’s impact on studies by research groups in the NISTs were revealed by the participants’ remarks on the factors that impacted the conduction of the proposed studies, which was compared to the comments on the expected- versus-actual contributions by the studies conducted by these public health institutes. However, such comments suffered from lack of clarity and precision in the cause-and-effect relationship between the research results and the actual impact on health conditions and access, since impacts in the health field are not short-term phenomena (24,31–34). In addition, the type of study impacts the capacity to perceive the expectations; the more basic the research, the more difficult it is to demonstrate or raise hypotheses on its contributions to the Unified Health System (32). The current study’s data allowed the identification of (and reflection on) factors that affect researchers’ engagement with public health demands and the social repercussions of the knowledge generated in the process, pointing to the NIST Program’s scope and the limits of its power to influence research.
We found that the induction exerted by the NIST Program coexists with several other influences, especially the research teams’ profiles, budget characteristics, other research agencies’ interests, and the emerging partnerships. In practice, research priority during the conduction of studies is guided by the need to make them feasible and achieve scientific excellence (24,28,35). Factors related to assessment of the institutes’ activities and results, especially the Ministry of Health’s limited participation in the process and the difficulty in transcending the laboratories’ walls to transform the knowledge into products, including the government’s lack of interest in the technologies generated by the NISTs, among others, contributed to reducing the Program’s power to induce the studies’ social impact. Even so, the main type of engagement between the NISTs’ activities and public health demands stemmed from action by the Ministry of Health and the institutes. Participants identified this as extremely important for aligning the studies with the health priorities set by the government, serving to spark interest among NIST researchers in orienting their studies toward actual demands from the Unified Health System. In addition, the participants’ responses suggest that in a scenario with more favorable conditions, especially greater dialogue with health authorities and policymakers at all levels, researchers would be more predisposed to collaborate more effectively to translate the knowledge they produce into improvements in the population’s health.
In relation to the NISTs’ contributions to the Unified Health System, we found that part of the knowledges generated is still highly conceptual, corresponding to primary research impacts (24,33), which are important from the scientific point of view but are still not associated with practical spinoffs. Another part of the knowledge mostly includes results that may face a long path ahead before impacting public health services (where this distance varies according to the type of contribution). The participants’ comments also denote the existence of decisive factors from outside the sphere of action of the NISTs and that limit the studies’ potential to contribute effectively to the desired impacts (36). They reveal that proper use of the potential of the NISTs, acknowledged as an important piece in recovering Brazil’s scientific and technological capability (37) to help solve the country’s public health problems, requires increased interaction between these institutes, administrators, and policymakers in the Unified Health System, which should act to link researchers with other actors such as members of industry.
FINAL REMARKS
We conclude that the NIST Program made important strides in researchers’ engagement with the health demands and services provided by the public health system. This was attributed to the requirements in the Program’s requests for proposals and follow-up of the studies’ conduction, especially during the period with active participation by the Ministry of Health.
We also observed that the strategy of adjusting the proposals’ work to accommodate the interest in the scientific research to the opportunity for budget support (a strategy commonly used in the scientific community) also applies to Brazil’s NISTs. In practice, research priority is guided by the need to achieve scientific excellence rather than social relevance. Bureaucratic follow-up of the studies’ conduction and their assessment centered mainly on scientific aspects contribute to perpetuating this pattern.
Initiatives such as the NIST Program need to be permeated by strong dialogue with policymakers and health administrators to expand the engagement between research agendas proposed by the scientific community and problems confronted by the health system in serving the population and to allow the research results to be transformed into available technologies incorporated by the health system. In addition, it is necessary to establish and value parameters for social evaluation of the studies’ impacts, so that such results become an actual priority for researchers.

REFERENCES
1. Bahia L. Entradas e Bandeiras e o SUS no Século XXI. Ciênc saúde coletiva. November 2012;17:2878–80.
2. Costa LS. Inovação nos serviços de saúde: apontamentos sobre os limites do conhecimento. Cadernos de Saúde Pública [Internet]. 2016 [cited November 28, 2016];32. Available at: http://www.scielosp.org/scielo.php?script=sci_abstract&pid=S0102-311X2016001202004&lng=pt&nrm=iso&tlng=pt
3. Rocha E. Saúde e objetivos de desenvolvimento sustentável. Revista Factores de Risco. December 2017;(45):12–9.
4. Vieira FS. O financiamento da saúde no Brasil e as metas da Agenda 2030: alto risco de insucesso. Rev Saúde Pública. November 30, 2020;54:127.
5. Kickbusch I, Piselli D, Agrawal A, Balicer R, Banner O, Adelhardt M, et al. The Lancet and Financial Times Commission on governing health futures 2030: growing up in a digital world. The Lancet. November 6, 2021;398(10312):1727–76.
6. Ministério de Ciência, Tecnologia, Inovações e Comunicações. Estratégia Nacional de Ciência, Tecnologia e Inovação 2016-2022: Sumário Executivo [Internet]. Ministério de Ciência, Tecnologia, Inovações e Comunicações: Centro de Gestão e Estudos Estratégicos; 2018 [cited April 22, 2019]. Available at: http://www.mctic.gov.br/mctic/export/sites/institucional/ciencia/SEPED/Arquivos/PlanosDeAcao/PACTI_Sumario_executivo_Web.pdf
7. Ministério da Ciência e Tecnologia. Portaria MCT no 429, July 17, 2008 [Internet]. 2008. Available at: http://www.mctic.gov.br/mctic/opencms/legislacao/portarias/migracao/Portaria_MCTI_n_704_de_11082015.html
8. Ministério da Ciência Tecnologia e Inovações. Portaria MCTI no 577, June 4, 2014. [Internet]. 2014. Available at: http://www.mctic.gov.br/mctic/opencms/legislacao/portarias/migracao/Portaria_MCTI_n_704_de_11082015.html
9. Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPQ). Edital No 15/2008 – MCT/CNPq/FNDCT/CAPES/FAPEMIG/FAPERJ/ FAPESP/Institutos Nacionais de Ciência e Tecnologia. Ministério de Ciência, Tecnologia e Inovações; 2008.
10. Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPQ). Chamada INCT – MCTI/CNPq/CAPES/FAPs no 16/2014. Ministério de Ciência, Tecnologia e Inovações; 2014.
11. Bloor D. Conhecimento e imaginário social. Edição: 1. São Paulo: Editora Unesp; 2010.
12. Callon M. Entrevista com Michel Callon: dos estudos de laboratório aos estudos de coletivos heterogêneos, passando pelos gerenciamentos econômicos. Sociologias. June 2008;(19):302–21.
13. Knorr-Cetina KD. The Manufacture of Knowledge: An Essay on the Constructivist and Contextual Nature of Science. Oxford ; New York: Pergamon Pr; 1981. 189 p.
14. Latour B. Science in Action: How to Follow Scientists and Engineers Through Society. Cambridge, Massachusetts, EUA: Harvard University Press; 1987.
15. Latour B, Woolgar S. A vida de laboratório: a produção dos fatos científicos. Rio de Janeiro: Relume Dumara; 1997. 298 p.
16. Trigueiro MGS. O que foi feito de Kuhn? O construtivismo na sociologia da ciência: considerações sobre a prática das novas biotecnologias. Em: Sobral FA da F, Maciel ML, Trigueiro MGS, org. A alavanca de Arquimedes; ciência e tecnologia a virada do século. Brasília, DF: Paralelo 15; 1997. p. 119–41.
17. Whitley RD. Black Boxism and the Sociology of Science: A Discussion of the Major Developments in the Field. The Sociological Review. May 1, 1970;18(1_suppl):61–92.
18. Gläser J. How can governance change research content? Linking science policy studies to the sociology of science. Em: Handbook on Science and Public Policy [Internet]. Massachusetts/USA: Edward Elgar Publishing; 2019 [cited June 11, 2021]. p. 419–47. Available at: https://www.elgaronline.com/view/edcoll/9781784715939/9781784715939.00033.xml
19. Gläser J, Laudel G. Governing Science: How Science Policy Shapes Research Content. European Journal of Sociology / Archives Européennes de Sociologie. April 2016;57(1):117–68.
20. Rip A. Protected Spaces of Science: Their Emergence and Further Evolution in a Changing World. Em: Carrier M, Nordmann A, org. Science in the Context of Application [Internet]. Dordrecht: Springer Netherlands; 2011 [cited October 20, 2021]. p. 197–220. (Boston Studies in the Philosophy of Science). Available at: https://doi.org/10.1007/978-90-481-9051-5_13
21. Whitley R, Gläser J, Laudel G. The Impact of Changing Funding and Authority Relationships on Scientific Innovations. Minerva. May 1, 2018;56(1):109–34.
22. Minayo MC de S, Deslandes SF, Cruz Neto O, Gomes R. Pesquisa social: Teoria, método e criatividade. Edição: 1. Minayo MC de S, org. Petrópolis, RJ: Editora Vozes; 2016.
23. Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPQ). III Seminário de Avaliação dos Institutos Nacionais de Ciência e Tecnologia. Em Brasília, DF; 2019.
24. Bornmann L. What is societal impact of research and how can it be assessed? a literature survey. Journal of the American Society for Information Science and Technology. 2013;64(2):217–33.
25. Chubb J, Reed MS. The politics of research impact: academic perceptions of the implications for research funding, motivation and quality. Br Polit. September 1, 2018;13(3):295–311.
26. Wallace ML, Rafols I. Research Portfolio Analysis in Science Policy: Moving from Financial Returns to Societal Benefits. Minerva. June 1, 2015;53(2):89–115.
27. Kuruvilla S, Mays N, Walt G. Describing the impact of health services and policy research. J Health Serv Res Policy. April 1, 2007;12(1_suppl):23–31.
28. Bozeman B, Sarewitz D. Public Value Mapping and Science Policy Evaluation. Minerva. March 1, 2011;49(1):1–23.
29. Garvin T. Analytical Paradigms: The Epistemological Distances between Scientists, Policy Makers, and the Public. Risk Analysis. 2001;21(3):443–56.
30. Cardinaal E, Dubas-Jakóbczyk K, Behmane D, Bryndová L, Cascini F, Duighuisen H, et al. Governance of academic medical centres in changing healthcare systems: An international comparison. Health Policy. July 1, 2022;126(7):613–8.
31. Kuruvilla S, Mays N, Pleasant A, Walt G. Describing the impact of health research: a Research Impact Framework. BMC Health Serv Res. October 18, 2006;6(1):134.
32. Penfield T, Baker MJ, Scoble R, Wykes MC. Assessment, evaluations, and definitions of research impact: A review. Research Evaluation. January 1, 2014;23(1):21–32.
33. Rivera SC, Kyte DG, Aiyegbusi OL, Keeley TJ, Calvert MJ. Assessing the impact of healthcare research: A systematic review of methodological frameworks. PLOS Medicine. August 9, 2017;14(8):e1002370.
34. Rafols I, Yegros A. Is Research Responding to Health Needs? [Internet]. Rochester, NY: Social Science Research Network; September 2017 [cited April 12, 2020]. Report No.: ID 3106713. Available at: https://papers.ssrn.com/abstract=3106713
35. Weiss AP. Measuring the Impact of Medical Research: Moving from Outputs to Outcomes. AJP. February 1, 2007;164(2):206–14.
36. Sarewitz D, Pielke RA. The neglected heart of science policy: reconciling supply of and demand for science. Environmental Science & Policy. February 1, 2007;10(1):5–16.
37. Guimarães R, Morel CM, Aragão É, Paranhos J, Palácios M, Goldbaum M, et al. Política de Ciência, Tecnologia e Inovação em Saúde (CT&I/S): uma atualização para debate. Ciênc saúde coletiva. December 13, 2021;26:6105–16.