Constructing a would require researchers to consider how the innovation relates to each of the constructs in the model, to identify that make up the constructs and to consider their of the concepts (eg, how they conceive the prevailing work ethic or experience the managerial hierarchy). They may also be able to postulate between different constructs or concepts or decide to focus on particular aspects of the model, which they could explore conceptually using the literature. Their research design would be influenced by their areas of interest, which would, in turn, determine their research methods. The findings could allow them to modify their model with evidence-based relationships and new concepts.
Qualitative research’s “uneasy relationship with theory” 4 may be due to several misconceptions. One possible misconception is that qualitative research aims to build theory and thus does not need theoretical grounding. The reality is that all qualitative research methods, not just Grounded Theory studies focused on theory building, may lead to theory construction. 16 Similarly, all types of qualitative research, including Grounded Theory studies, should be guided by research frameworks. 16
Not using a research framework may also be due to misconceptions that qualitative research aims to understand people’s perspectives and experiences without examining them from a particular theoretical perspective or that theoretical foundations may influence researchers’ interpretations of participants’ meanings. In fact, in the same way that participants’ meanings vary, qualitative researchers’ interpretations (as opposed to descriptions) of participants’ meaning-making will differ. 32 , 33 Research frameworks thus provide a frame of reference for “making sense of the data.” 34
Studies informed by well-defined research frameworks can make a world of difference in alleviating misconceptions. Good qualitative reporting requires research frameworks that make explicit the combination of relevant theories, theoretical constructs and concepts that will permeate every aspect of the research. Irrespective of the term used, research frameworks are critical components of reporting not only qualitative but also all types of research.
In memory of Martie Sanders: supervisor, mentor, and colleague. My deepest gratitude for your unfailing support and guidance. I feel your loss.
Conflicts of Interest: None.
What's the difference.
Research design and research methods are two essential components of any research study. Research design refers to the overall plan or structure of the study, outlining the objectives, research questions, and the overall approach to be used. It involves making decisions about the type of study, the target population, and the data collection and analysis techniques to be employed. On the other hand, research methods refer to the specific techniques and tools used to gather and analyze data. This includes selecting the appropriate sampling method, designing surveys or interviews, and choosing statistical tests for data analysis. While research design provides the framework for the study, research methods are the practical tools used to implement the design and collect the necessary data.
Attribute | Research Design | Research Methods |
---|---|---|
Definition | The overall plan or strategy to answer research questions | The specific techniques or tools used to collect and analyze data |
Objective | To provide a framework for conducting research | To gather and analyze data to answer research questions |
Scope | Encompasses the entire research process | Focuses on data collection and analysis |
Types | Experimental, quasi-experimental, descriptive, exploratory, etc. | Surveys, interviews, observations, experiments, case studies, etc. |
Flexibility | Can be flexible and adaptable based on research needs | Can be rigid or flexible depending on the chosen methods |
Timeframe | Establishes the overall timeline for the research | Varies based on the chosen methods and research goals |
Data Analysis | May involve statistical analysis, qualitative coding, etc. | Includes statistical analysis, content analysis, thematic analysis, etc. |
Validity | Concerned with the overall quality and accuracy of the research | Focuses on the reliability and validity of data collection methods |
Introduction.
Research is a systematic process that aims to gather and analyze information to answer specific questions or solve problems. It involves careful planning and execution to ensure reliable and valid results. Two key components of any research study are the research design and research methods. While they are closely related, they serve distinct purposes and have different attributes. In this article, we will explore and compare the attributes of research design and research methods.
Research design refers to the overall plan or strategy that guides the entire research process. It outlines the structure and framework of the study, including the objectives, research questions, and the overall approach to be used. The research design provides a roadmap for researchers to follow, ensuring that the study is conducted in a systematic and organized manner.
One of the key attributes of research design is its flexibility. Researchers can choose from various research designs, such as experimental, correlational, descriptive, or exploratory, depending on the nature of their research questions and the available resources. Each design has its own strengths and limitations, and researchers must carefully consider these factors when selecting the most appropriate design for their study.
Another important attribute of research design is its ability to establish the causal relationship between variables. Experimental research designs, for example, are specifically designed to determine cause and effect relationships by manipulating independent variables and measuring their impact on dependent variables. This attribute is particularly valuable when researchers aim to make causal inferences and draw conclusions about the effectiveness of interventions or treatments.
Research design also plays a crucial role in determining the generalizability of the findings. Some research designs, such as case studies or qualitative research, may provide rich and in-depth insights into a specific context or phenomenon but may lack generalizability to a larger population. On the other hand, quantitative research designs, such as surveys or experiments, often aim for a representative sample and strive for generalizability to a broader population.
Furthermore, research design influences the data collection methods and tools used in a study. It helps researchers decide whether to use qualitative or quantitative data, or a combination of both, and guides the selection of appropriate data collection techniques, such as interviews, observations, questionnaires, or experiments. The research design ensures that the chosen methods align with the research objectives and provide the necessary data to answer the research questions.
Research methods, on the other hand, refer to the specific techniques and procedures used to collect and analyze data within a research study. While research design provides the overall framework, research methods are the practical tools that researchers employ to gather the necessary information.
One of the key attributes of research methods is their diversity. Researchers can choose from a wide range of methods, such as surveys, interviews, observations, experiments, case studies, content analysis, or statistical analysis, depending on the nature of their research questions and the available resources. Each method has its own strengths and limitations, and researchers must carefully select the most appropriate methods to ensure the validity and reliability of their findings.
Another important attribute of research methods is their ability to provide empirical evidence. By collecting data through systematic and rigorous methods, researchers can obtain objective and measurable information that can be analyzed and interpreted. This attribute is crucial for generating reliable and valid results, as it ensures that the findings are based on evidence rather than personal opinions or biases.
Research methods also play a significant role in ensuring the ethical conduct of research. Ethical considerations, such as informed consent, privacy protection, and minimizing harm to participants, are essential in any research study. The choice of research methods should align with these ethical principles and guidelines to ensure the well-being and rights of the participants.
Furthermore, research methods allow researchers to analyze and interpret the collected data. Statistical analysis, for example, enables researchers to identify patterns, relationships, and trends within the data, providing a deeper understanding of the research questions. The choice of appropriate analysis methods depends on the nature of the data and the research objectives, and researchers must possess the necessary skills and knowledge to conduct the analysis accurately.
Lastly, research methods contribute to the reproducibility and transparency of research. By clearly documenting the methods used, researchers enable others to replicate the study and verify the findings. This attribute is crucial for the advancement of knowledge and the validation of research results.
Research design and research methods are two essential components of any research study. While research design provides the overall plan and structure, research methods are the practical tools used to collect and analyze data. Both have distinct attributes that contribute to the reliability, validity, and generalizability of research findings. By understanding and carefully considering the attributes of research design and research methods, researchers can conduct high-quality studies that contribute to the advancement of knowledge in their respective fields.
Comparisons may contain inaccurate information about people, places, or facts. Please report any issues.
Home » Theoretical Framework – Types, Examples and Writing Guide
Table of Contents
Definition:
Theoretical framework refers to a set of concepts, theories, ideas , and assumptions that serve as a foundation for understanding a particular phenomenon or problem. It provides a conceptual framework that helps researchers to design and conduct their research, as well as to analyze and interpret their findings.
In research, a theoretical framework explains the relationship between various variables, identifies gaps in existing knowledge, and guides the development of research questions, hypotheses, and methodologies. It also helps to contextualize the research within a broader theoretical perspective, and can be used to guide the interpretation of results and the formulation of recommendations.
Types of Types of Theoretical Framework are as follows:
This type of framework defines the key concepts and relationships between them. It helps to provide a theoretical foundation for a study or research project .
This type of framework starts with a general theory or hypothesis and then uses data to test and refine it. It is often used in quantitative research .
This type of framework starts with data and then develops a theory or hypothesis based on the patterns and themes that emerge from the data. It is often used in qualitative research .
This type of framework focuses on the collection and analysis of empirical data, such as surveys or experiments. It is often used in scientific research .
This type of framework defines a set of norms or values that guide behavior or decision-making. It is often used in ethics and social sciences.
This type of framework seeks to explain the underlying mechanisms or causes of a particular phenomenon or behavior. It is often used in psychology and social sciences.
The components of a theoretical framework include:
A theoretical framework is an essential part of any research study or paper, as it helps to provide a theoretical basis for the research and guide the analysis and interpretation of the data. Here are some steps to help you write a theoretical framework:
Here are some examples of theoretical frameworks:
Following are some situations When to Have A Theoretical Framework:
The purposes of a theoretical framework include:
Some of the characteristics of a theoretical framework include:
Here are some of the advantages of having a theoretical framework:
Researcher, Academic Writer, Web developer
Implementation Science volume 19 , Article number: 57 ( 2024 ) Cite this article
198 Accesses
2 Altmetric
Metrics details
Venous thromboembolism (VTE) is a preventable medical condition which has substantial impact on patient morbidity, mortality, and disability. Unfortunately, adherence to the published best practices for VTE prevention, based on patient centered outcomes research (PCOR), is highly variable across U.S. hospitals, which represents a gap between current evidence and clinical practice leading to adverse patient outcomes.
This gap is especially large in the case of traumatic brain injury (TBI), where reluctance to initiate VTE prevention due to concerns for potentially increasing the rates of intracranial bleeding drives poor rates of VTE prophylaxis. This is despite research which has shown early initiation of VTE prophylaxis to be safe in TBI without increased risk of delayed neurosurgical intervention or death. Clinical decision support (CDS) is an indispensable solution to close this practice gap; however, design and implementation barriers hinder CDS adoption and successful scaling across health systems. Clinical practice guidelines (CPGs) informed by PCOR evidence can be deployed using CDS systems to improve the evidence to practice gap. In the Scaling AcceptabLE cDs (SCALED) study, we will implement a VTE prevention CPG within an interoperable CDS system and evaluate both CPG effectiveness (improved clinical outcomes) and CDS implementation.
The SCALED trial is a hybrid type 2 randomized stepped wedge effectiveness-implementation trial to scale the CDS across 4 heterogeneous healthcare systems. Trial outcomes will be assessed using the RE 2 -AIM planning and evaluation framework. Efforts will be made to ensure implementation consistency. Nonetheless, it is expected that CDS adoption will vary across each site. To assess these differences, we will evaluate implementation processes across trial sites using the Exploration, Preparation, Implementation, and Sustainment (EPIS) implementation framework (a determinant framework) using mixed-methods. Finally, it is critical that PCOR CPGs are maintained as evidence evolves. To date, an accepted process for evidence maintenance does not exist. We will pilot a “Living Guideline” process model for the VTE prevention CDS system.
The stepped wedge hybrid type 2 trial will provide evidence regarding the effectiveness of CDS based on the Berne-Norwood criteria for VTE prevention in patients with TBI. Additionally, it will provide evidence regarding a successful strategy to scale interoperable CDS systems across U.S. healthcare systems, advancing both the fields of implementation science and health informatics.
Clinicaltrials.gov – NCT05628207. Prospectively registered 11/28/2022, https://classic.clinicaltrials.gov/ct2/show/NCT05628207 .
This paper provides a study protocol for a new and novel stepped wedge study variation which includes external control sites to take into account external influences on the uptake of traumatic brain injury guidelines nationally
This paper provides a study design for one of the largest trauma pragmatic trials in the U.S. of 9 heterogenous hospitals
This study is also unique and first-in-kind feature as the guideline may change over time during the study due to the “living” nature of the guideline being implemented.
Venous thromboembolism (VTE) is a preventable complication of traumatic brain injury (TBI), which has a substantial impact on patient morbidity, mortality, disability. It is also associated with significant economic burden > $1.5 billion per year [ 1 , 2 ]. VTE is considered a preventable medical condition in the majority of cases [ 2 , 3 ]. Unfortunately, adherence with patient centered outcomes research (PCOR)-informed VTE prevention best practices is highly variable and often poor across U.S. hospitals. Compliance with best practice is especially relevant in the case of TBI as 54% of TBI patients will develop a VTE if they do not receive appropriate anticoagulation [ 4 ]. The delivery of appropriate VTE prophylaxis to TBI patients is such an important quality measure that adherence is tracked nationally and benchmarked by the American College of Surgeons Trauma Quality Improvement Program (ACS-TQIP) [ 5 ]. We have previously shown that instituting a hospital-wide VTE prevention initiative modeled after the Berne-Norwood criteria for VTE prophylaxis in TBI was associated with significantly increased compliance with VTE-related process and improved outcome metrics [ 6 ]. Specifically, we observed improved adherence with the Berne-Norwood criteria [ 7 , 8 ], reduced time to initiation of VTE prophylaxis, and reduced VTE events [ 9 ]. Multiple studies have shown that VTE prophylaxis in trauma patients not only reduces VTE events, but also significantly reduces mortality [ 10 ]. We noted the same reduction in mortality for TBI patients following the initiation of a VTE prophylaxis guideline for patients with TBI [ 11 ]. Unfortunately, despite widely published PCOR-informed best practice, nationally there is reluctance to initiate VTE prevention due to concerns for progression of intracranial hemorrhage. This is despite research which has shown early initiation of VTE prophylaxis to be safe in TBI without increased risk of delayed neurosurgical intervention or death [ 12 , 13 , 14 , 15 , 16 ].
Since approximately 40% of TBI patients do not receive DVT prophylaxis in a timely manner, there is a critical and timely need to close the gap between current PCOR evidence and clinical practice. [ 17 , 18 , 19 , 20 , 21 , 22 , 23 ]. Clinical decision support (CDS) systems are an indispensable solution to close this practice gap; however, design and implementation barriers hinder CDS adoption [ 24 , 25 ]. Another significant challenge to the implementation of CDS is that health information technology (IT) needs a common language for PCOR evidence to translate it into practice across multiple organizations [ 26 ]. Because of these challenges, we will deploy CDS using fast healthcare interoperability resources (FHIR) standards to rapidly implement PCOR evidence into practice [ 27 , 28 ]. We hypothesize that, FHIR standards will reduce CDS development and maintenance costs, increase PCOR uptake in rural and other underserved sites, and speed the development timeline to build a comprehensive suite of CDS for PCOR evidence [ 29 ].
Few studies have investigated specific barriers to and facilitating factors for adoption of interoperable FHIR-based CDS [ 30 ]. For example, many current studies investigating barriers and facilitators for interoperable CDS are limited to expert opinion [ 30 , 31 ] or lack a formal implementation science framework-guided investigation [ 32 , 33 ]. Barriers to and facilitating factors for adoption of interoperable CDS following real-life implementation and multicenter scaling guided by validated implementation science frameworks should be rigorously investigated. This study will facilitate comprehensive exploration of clinician and environmental (internal and external) contextual elements that influence interoperable CDS implementation success. In this study, we will scale and assess the effectiveness of a CDS system for a VTE prophylaxis guideline in patients with TBI and evaluate implementation across 9 sites within 4 U.S. trauma systems.
This trial consists of a stepped wedge hybrid effectiveness-implementation trial to scale the CDS system across 4 trauma systems and in parallel evaluate implementation strategy guided by the Exploration, Preparation, Implementation, and Sustainment (EPIS) implementation framework (Fig. 1 a) [ 34 ]. We anticipate variability in CDS adoption across sites during the implementation trial. This variation represents a unique opportunity to study implementation at each site and understand what strategies, system factors, and engagement of specific stakeholders are associated with improved CDS adoption. We will rigorously evaluate each implementation phase, guided by The EPIS Implementation Framework [ 34 ], our determinant framework (Fig. 1 b). We will apply the EPIS framework to guide assessment of implementation phases, barriers, and facilitators (Fig. 2 ) [ 34 ]. EPIS comprises 16 constructs over 4 domains (outer context, inner context, bridging factors, and innovation factors). We selected EPIS as our determinant framework as it includes clearly delineated implementation stages and allows for examination of change at multiple levels, across time, and through phases that build toward implementation. While EPIS was initially developed for implementation in public service, it has since been translated to healthcare, especially for complex multi-institutional healthcare interventions [ 34 , 35 , 36 ].
a Randomized Stepped Wedge design of the SCALED clinical trial. b Parallel, implementation evaluation guided by Explore, Preparation, Implementation and Sustain (EPIS) framework
Implementation evaluation across study sites
This trial will be conducted at 4 healthcare systems with 1–3 hospitals per system and is projected to occur over a 3 to 4-year period. The trial uses a randomized stepped-wedge design to scale an interoperable CDS system for the Berne-Norwood TBI CPG. Figure 1 a provides a schematic for the trial design. The order of health systems and sites will be randomly determined. This study will include a heterogeneous number of hospitals by trauma verification status, electronic health record (EHR) platform, bed size, and setting (Table 1 ). Our target population is adult patients admitted with an acute TBI defined as International Classification of Disease 10 Clinical Modification (ICD-10-CM): S06.1 – S06.9 or S06.A. Patients who die within 24 h of hospital admission and patients documented as “comfort cares” during the first 72 h of hospitalization will be excluded, as they would have a limited opportunity to receive adherence with the Berne-Norwood criteria. Additionally, patients with a pre-existing VTE or inferior vena cava (IVC) filter at the time of admission, and patients with a mechanical heart valve or ventricular assist device will be excluded from final analysis.
This study will also include up to 3 control sites (Fig. 1 a), a feature not typically included with historic stepped-wedge trial designs, which will strengthen our ability to understand external influences on the study findings. These control sites, which do not receive the CDS intervention and do not have any planned initiatives around guideline implementation, will allow the study to assess baseline adherence and variation in clinical practice over the study period.
TBI diagnosis upon admission will activate an interoperable CDS system leveraging the Stanson Health (Charlotte, NC) CDS platform [ 37 ], which is being expanded to include interoperable offerings for TBI VTE prophylaxis. This system provides a knowledge representation framework to faithfully express the intent of the Berne-Norwood prevention criteria computationally (Table 2 ). The interoperable FHIR data standard will be used for bi-directional data transfer between each site’s EHR and the CDS platform. Workflow integration includes a combination of both passive and interruptive provider and trauma system leader information and “nudges”. Table 2 represents the Standards-based, Machine-readable, Adaptive, Requirements-based, and Testable (SMART) L2 layer [ 38 ] of the Berne-Norwood criteria.
We will complete a rapid cycle CDS evaluation to optimize CDS workflow integration by conducting a user-driven simulation and expert-driven heuristic usability optimization as we have previously done [ 39 ]. For rapid cycle CDS evaluation, multidisciplinary trauma end-user “teams” will complete up to 3 scenarios designed to represent various extremes in TBI VTE prevention decision making. Simulation usability testing will be overseen by usability experts, who will catalogue usability issues that arise during simulation. Via consensus ranking, the development and planning teams will rank usability issues from 0 (cosmetic) to 5 (usability catastrophe). Using 10 predefined heuristics for usability design [ 40 ], we will conduct a heuristic evaluation of the CDS, then catalogue and rank usability issues. These results will inform CDS application design, optimized for TBI workflow integration.
Following CDS development, our healthcare system relies on a time-tested approach for the implementation and scaling of user-centered CDS: this approach is called the Scaling AcceptabLE cDs (SCALED) Strategy [ 41 ]. This framework integrates multiple evidence-based implementation strategies (Table 3 ).
The primary implementation outcome is patient-level adherence with the CPG: Specifically, did the patient received guideline-concordant care? Adherence will be measured as an all-or-none measure (binary endpoint at the encounter/patient-level). Thus, if a patient is low-risk for TBI progression, by 24 h they should have risk-specific VTE prevention ordered; if they receive this after 24 h, or if they receive the intermediate risk VTE prevention regimen, this would be deemed non-adherent. The primary effectiveness outcome is VTE (binary endpoint at the patient-encounter level). Safety outcomes evaluated include: TBI progression, in-hospital mortality, and bleeding events. A secondary hypothesis is that as the trial scales to additional sites, iterative implementations will be more efficient (reduced implementation time) and more effective (improved adoption). Secondary hypotheses will be evaluated using the RE 2 -AIM framework [ 42 , 43 ] and are displayed in Table 4 .
Data sources used in this trial include the Stanson Health CDS eCaseReport and site trauma registry. The eCaseReport is a living registry of all patients, and their associated clinical trial data elements, that were eligible for the CDS. All sites also maintain a trauma registry adhering to the National Trauma Data Standards [ 44 ], a requirement for ACS trauma center verification. This dataset is manually annotated by trained clinical abstractors. Data will be sent to the biostatistical team at 6-month intervals. Control and pre-implementation sites will provide their trauma registry in addition to supplemental standards-based EHR extraction of clinical trial data elements or manual abstraction. A data dictionary has been created for the study and will be made available on the trial webpage.
Survey instruments will be prepared using Likert-type scales. Outcomes will be calculated based on scoring guides for the following validated scales: Program Sustainability Assessment Tool (PSAT) [ 45 ], Clinical Sustainability Assessment Tool (CSAT) [ 46 ], Implementation Leadership Scale (ILS) [ 47 ], and Evidenced-based Practice Attitude Scale-36 (EBPAS-36) [ 48 ]. Two scales do not have scoring rubrics: the Organizational Readiness for Change Questionnaire [ 49 , 50 ] and the Normalization Measure Development (NoMAD) Questionnaire [ 51 , 52 , 53 ]. Since both of these scales group questions into constructs, they will be analyzed by generating mean Likert scores and standard deviations per construct, and a mean across constructs, at each of the four implementation phases [ 54 ].
To deeply investigate barriers and facilitators of successful implementation, semi-structured qualitative interviews of key personnel (clinical leadership and end-users, IT leadership and staff) will be conducted at each of the 4 implementation phases. Studies suggest saturation of new ideas occurs after approximately 12 interviews [ 55 ]. Additional samples will be added as needed if thematic saturation is not achieved. Following informed consent, interviews will be performed by a trained qualitative research assistant, audio recorded, and transcribed verbatim. An interview guide, informed by the EPIS framework, was developed to collect key informant experiences with CDS implementation with a focus on inner and outer context factors [ 56 ]. A hybrid approach, primarily deductive and secondarily inductive, approach will be applied. All interviews will be independently double-coded and coding discrepancies will be resolved through discussion. A descriptive thematic analysis approach [ 57 ] will be used to characterize the codes into themes and sub-themes representing the barriers and facilitators to implementation success.
Results for all instruments will be primarily stratified according to site implementation success at each study phase. Additional stratifications may include respondent role, discipline, and hospital system. Bar charts displaying mean survey domains with integrative quotations from the qualitative analysis will be used to facilitate data visualization and understanding of key themes representing barriers and facilitators to successful CDSS implementation.
Mixed-effects logistic regression models will be fit to test whether or not CDS implementation changes the likelihood of a VTE event during TBI admission (effectiveness outcome) and the likelihood that the clinical guideline was followed (implementation outcome). The models for these outcomes include fixed-effects for month (when available, to account for secular trends) and an indicator variable for whether the center had the CDS integrated in the EHR. The primary test statistic will be a Wald test of the coefficient for this treatment indicator. We will include random center-specific intercepts to account for correlation within center. Assuming there are 9 sites enrolled with an average of 400 TBI admissions per year and the typical site has between 20%-40% adherence to the clinical guidelines, we will have > 80.0% and > 99.9% power to detect a 5 and 10 percentage point increase in the adherence. Similarly, assuming the typical site has between a VTE event rate of 5–6%, we will have > 80.0% power to detect a 40%-50% reduction in VTE consistent with our published data [ 11 ].
This study is overseen by the University of Minnesota Surgical Clinical Trials Office and by an independent Data Safety Monitoring Board (DSMB). Even though this intervention is deploying a TBI clinical guideline that is currently considered best practice, we believe the addition of a DSMB will improve trial safety, data quality, and trial integrity [ 58 ]. DSMB membership will be independent from the study investigators and will consist of 3 members including: 1 trauma surgeon, 1 informaticist, and 1 statistician. Annual reports including data from all sites, including control sites, will be shared with the DSMB to assure timely monitoring of safety and data quality. The trial will not be stopped early in the event of CDS efficacy because a critical secondary outcome focuses on studying implementation and effectiveness over time.
Given the potential for a changing evidence-base, it is possible that best practice VTE prevention guidance may change during the study period or afterwards. A critical element in improving adherence with PCOR evidence is updating guidance based on this evidence – in this study, this requires ensuring that the CDS system remains current.
We will pilot a model for producing and maintaining TBI VTE prophylaxis 'Living Guidance and CDS' to ensure that the CDS remains current (Fig. 3 ). The University of Minnesota Evidence-based Practice Center (EPC) Evidence Generation team will conduct and maintain a “living” systematic review. Systematic review data will be uploaded to the AHRQ’s Systematic Review Data Repository (SRDR). “Living” implies that every 6 months the EPC team will evaluate and synthesize new evidence related to TBI VTE prophylaxis, update the existing systematic review and deliver it to a multi-stakeholder Guideline Committee. The Guideline Committee will then use the GRADE (Grading of Recommendations, Assessment, Development and Evaluations) evidence-to-decision (EtD) framework to develop VTE prophylaxis guidelines for patients with TBI [ 59 , 60 , 61 ]. A computational representation of these guidelines will be updated and maintained within the CDS platform by Stanson Health, the CDS Vendor.
Pilot process for “Living Guideline”
The ultimate goal of this study is to spread successful CDS tools and strategies to broadly improve TBI VTE-related care processes and outcomes. The research outlined above will surface sharable insights about what information needs to be presented to which people in what formats through what channels at what times to reliably deliver guideline-based care – i.e., specific instantiations of the “CDS 5 Rights Framework” applied to this target [ 62 ]. We will use Health Service Blueprint tools to describe our recommended implementation approaches; these tools are being applied in an increasing number of public and private care delivery organizations as a structured approach to ‘get the CDS 5 Right right’ for various improvement targets. We will further adapt and apply Health Service Blueprint foundations supported by VA and AHRQ [ 63 ] to capture VTE care transformation guidance in Health Service Blueprint tooling [ 64 ]. Presenting recommended CDS-enabled workflow, information flow – as well as and related implementation considerations and broader healthcare ecosystem implications – in this structured format will help organizations beyond the initial study participants put study results into action efficiently and effectively.
In this paper, we present the protocol for the SCALED trial, a stepped-wedge cluster randomized trial of a CDS intervention to improve adherence with VTE prevention best practices for patients with TBI. As a hybrid type 2 trial, this study will evaluate both implementation and effectiveness outcomes. In addition to investigating effectiveness, we will also be able to provide insight into the implementation challenges for deploying interoperable CDS across heterogenous health systems. In our pilot study [ 9 ], while patients who received guideline-concordant care had significantly improved outcomes, we noted that not all patients receive guideline concordant care following implementation. Additionally, best strategies for scaling interoperable CDS systems are poorly studied. Thus, this study represents one of the earliest implementation evaluations of scaling interoperable CDS systems across heterogeneous health systems.
This study has several strengths. First, it will rigorously test implementation of a CPG for VTE prevention across 9 U.S. trauma centers using a multi-faceted CDS platform supporting both passive and interruptive decision support. Second, it will rigorously investigate scalable and interoperable CDS strategies to deploy CPGs. Third, this study leverages a centralized eCaseReport generated by the CDS system, a solution which can drive data collection for future pragmatic trials. Importantly, this study takes place at trauma centers which are geographically distinct, utilize different EHR vendors, include both ACS-verified level 1 through level 3 trauma centers, and include rural, community, and university-based trauma centers. In addition to helping spread recommended care transformation strategies beyond additional study sites, documenting these approaches in Health Service Blueprint tools will also support creation of learning communities for sharing, implementing, and enhancing these strategies.
This study also has limitations. First, we are only investigating 4 trauma systems which already have fairly advanced informatics divisions and experience implementing interoperable CDS systems. Thus, these findings may not be broadly applicable to health systems with less informatics experience and expertise. Second, we are only investigating implementation across two EHR vendors: Epic and Cerner, thus these findings may not be applicable to health systems with different EHR vendors such as Meditech or Allscripts. However, the Health Service Blueprint implementation strategy representations should still enable users of other systems to glean valuable insights about components of the transformation approach less dependent on specific EHRs used.
In summary, this study will implement and scale a CDS-enabled care transformation approach across a diverse collaborative CDS community, serving as an important demonstration of this critical healthcare challenge. We will integrate lessons learned for a planned national scaling in collaboration with U.S. trauma societies. Finally, we will pilot an approach for the “Living Guideline” and use that to maintain evidenced-based decision logic within CDS platforms.
Following trial completion data will be made available upon request through the University of Minnesota Data Repository.
Heit JA. Venous thromboembolism: disease burden, outcomes and risk factors. J Thromb Haemost. 2005;3(8):1611–7.
Article CAS PubMed Google Scholar
Yorkgitis BK, Berndtson AE, Cross A, Kennedy R, Kochuba MP, Tignanelli C, Tominaga GT, Jacobs DG, Marx WH, Ashley DW, Ley EJ, Napolitano L, Costantini TW. American Association for the Surgery of Trauma/American College of Surgeons-Committee on Trauma Clinical Protocol for inpatient venous thromboembolism prophylaxis after trauma. J Trauma Acute Care Surg. 2022;92(3):597–604.
Article PubMed Google Scholar
Nicholson M, Chan N, Bhagirath V, Ginsberg J. Prevention of Venous Thromboembolism in 2020 and Beyond. J Clin Med. 2020;9(8):2467.
Article CAS PubMed PubMed Central Google Scholar
Geerts WH, Code KI, Jay RM, Chen E, Szalai JP. A prospective study of venous thromboembolism after major trauma. N Engl J Med. 1994;331(24):1601–6.
Nathens AB, Cryer HG, Fildes J. The American College of Surgeons Trauma Quality Improvement Program. Surg Clin North Am. 2012;92(2):441–54, x−xi.
Ingraham NE, Lotfi-Emran S, Thielen BK, Techar K, Morris RS, Holtan SG, Dudley RA, Tignanelli CJ. Immunomodulation in COVID-19. Lancet Respir Med. 2020;8(6):544–6.
Phelan HA, Eastman AL, Madden CJ, Aldy K, Berne JD, Norwood SH, Scott WW, Bernstein IH, Pruitt J, Butler G, Rogers L, Minei JP. TBI risk stratification at presentation: a prospective study of the incidence and timing of radiographic worsening in the Parkland Protocol. J Trauma Acute Care Surg. 2012;73(2 Suppl 1):S122–7.
Pastorek RA, Cripps MW, Bernstein IH, Scott WW, Madden CJ, Rickert KL, Wolf SE, Phelan HA. The Parkland Protocol’s modified Berne-Norwood criteria predict two tiers of risk for traumatic brain injury progression. J Neurotrauma. 2014;31(20):1737–43.
Article PubMed PubMed Central Google Scholar
Tignanelli CJ, Gipson J, Nguyen A, Martinez R, Yang S, Reicks PL, Sybrant C, Roach R, Thorson M, West MA. Implementation of a Prophylactic Anticoagulation Guideline for Patients with Traumatic Brain Injury. Jt Comm J Qual Patient Saf. 2020;46(4):185–91.
PubMed Google Scholar
Jacobs BN, Cain-Nielsen AH, Jakubus JL, Mikhail JN, Fath JJ, Regenbogen SE, Hemmila MR. Unfractionated heparin versus low-molecular-weight heparin for venous thromboembolism prophylaxis in trauma. J Trauma Acute Care Surg. 2017;83(1):151–8.
Tignanelli CJ, Silverman GM, Lindemann EA, Trembley AL, Gipson JC, Beilman G, Lyng JW, Finzel R, McEwan R, Knoll BC, Pakhomov S, Melton GB. Natural language processing of prehospital emergency medical services trauma records allows for automated characterization of treatment appropriateness. J Trauma Acute Care Surg. 2020;88(5):607–14.
Kim J, Gearhart MM, Zurick A, Zuccarello M, James L, Luchette FA. Preliminary report on the safety of heparin for deep venous thrombosis prophylaxis after severe head injury. J Trauma. 2002;53(1):38–42; discussion 3.
Cothren CC, Smith WR, Moore EE, Morgan SJ. Utility of once-daily dose of low-molecular-weight heparin to prevent venous thromboembolism in multisystem trauma patients. World J Surg. 2007;31(1):98–104.
Norwood SH, Berne JD, Rowe SA, Villarreal DH, Ledlie JT. Early venous thromboembolism prophylaxis with enoxaparin in patients with blunt traumatic brain injury. J Trauma. 2008;65(5):1021–6; discussion 6-7.
CAS PubMed Google Scholar
Scudday T, Brasel K, Webb T, Codner P, Somberg L, Weigelt J, Herrmann D, Peppard W. Safety and efficacy of prophylactic anticoagulation in patients with traumatic brain injury. J Am Coll Surg. 2011;213(1):148–53; discussion 53-4.
Byrne JP, Mason SA, Gomez D, Hoeft C, Subacius H, Xiong W, Neal M, Pirouzmand F, Nathens AB. Timing of Pharmacologic Venous Thromboembolism Prophylaxis in Severe Traumatic Brain Injury: A Propensity-Matched Cohort Study. J Am Coll Surg. 2016;223(4):621-31e5.
Lau R, Stevenson F, Ong BN, Dziedzic K, Eldridge S, Everitt H, Kennedy A, Kontopantelis E, Little P, Qureshi N, Rogers A, Treweek S, Peacock R, Murray E. Addressing the evidence to practice gap for complex interventions in primary care: a systematic review of reviews protocol. BMJ Open. 2014;4(6): e005548.
Tignanelli CJ, Vander Kolk WE, Mikhail JN, Delano MJ, Hemmila MR. Noncompliance with American College of Surgeons Committee on Trauma recommended criteria for full trauma team activation is associated with undertriage deaths. J Trauma Acute Care Surg. 2018;84(2):287–94.
Robbins AJ, Ingraham NE, Sheka AC, Pendleton KM, Morris R, Rix A, Vakayil V, Chipman JG, Charles A, Tignanelli CJ. Discordant Cardiopulmonary Resuscitation and Code Status at Death. J Pain Symptom Manage. 2021;61(4):770–780.e1.
Tignanelli CJ, Watarai B, Fan Y, Petersen A, Hemmila M, Napolitano L, Jarosek S, Charles A. Racial Disparities at Mixed-Race and Minority Hospitals: Treatment of African American Males With High-Grade Splenic Injuries. Am Surg. 2020;86(5):441–9.
Tignanelli CJ, Rix A, Napolitano LM, Hemmila MR, Ma S, Kummerfeld E. Association Between Adherence to Evidence-Based Practices for Treatment of Patients With Traumatic Rib Fractures and Mortality Rates Among US Trauma Centers. JAMA Netw Open. 2020;3(3): e201316.
Oliphant BW, Tignanelli CJ, Napolitano LM, Goulet JA, Hemmila MR. American College of Surgeons Committee on Trauma verification level affects trauma center management of pelvic ring injuries and patient mortality. J Trauma Acute Care Surg. 2019;86(1):1–10.
Tignanelli CJ, Wiktor AJ, Vatsaas CJ, Sachdev G, Heung M, Park PK, Raghavendran K, Napolitano LM. Outcomes of Acute Kidney Injury in Patients With Severe ARDS Due to Influenza A(H1N1) pdm09 Virus. Am J Crit Care. 2018;27(1):67–73.
Khairat S, Marc D, Crosby W, Al SA. Reasons For Physicians Not Adopting Clinical Decision Support Systems: Critical Analysis. JMIR Med Inform. 2018;6(2): e24.
Jones EK, Ninkovic I, Bahr M, Dodge S, Doering M, Martin D, Ottosen J, Allen T, Melton GB, Tignanelli CJ. A novel, evidence-based, comprehensive clinical decision support system improves outcomes for patients with traumatic rib fractures. J Trauma Acute Care Surg. 2023;95(2):161–71.
Marcos M, Maldonado JA, Martinez-Salvador B, Bosca D, Robles M. Interoperability of clinical decision-support systems and electronic health records using archetypes: a case study in clinical trial eligibility. J Biomed Inform. 2013;46(4):676–89.
FHIR Clinical Guidelines. http://build.fhir.org/ig/HL7/cqf-recommendations/ . Accessed 14 Sep 2021.
Mandel JC, Kreda DA, Mandl KD, Kohane IS, Ramoni RB. SMART on FHIR: a standards-based, interoperable apps platform for electronic health records. J Am Med Inform Assoc. 2016;23(5):899–908.
Goldberg HS, Paterno MD, Rocha BH, Schaeffer M, Wright A, Erickson JL, Middleton B. A highly scalable, interoperable clinical decision support service. J Am Med Inform Assoc. 2014;21(e1):e55-62.
Marcial LH, Blumenfeld B, Harle C, Jing X, Keller MS, Lee V, Lin Z, Dover A, Midboe AM, Al-Showk S, Bradley V, Breen J, Fadden M, Lomotan E, Marco-Ruiz L, Mohamed R, O’Connor P, Rosendale D, Solomon H, Kawamoto K. Barriers, Facilitators, and Potential Solutions to Advancing Interoperable Clinical Decision Support: Multi-Stakeholder Consensus Recommendations for the Opioid Use Case. AMIA Annu Symp Proc. 2019;2019:637–46.
Lomotan EA, Meadows G, Michaels M, Michel JJ, Miller K. To Share is Human! Advancing Evidence into Practice through a National Repository of Interoperable Clinical Decision Support. Appl Clin Inform. 2020;11(1):112–21.
Dolin RH, Boxwala A, Shalaby J. A Pharmacogenomics Clinical Decision Support Service Based on FHIR and CDS Hooks. Methods Inf Med. 2018;57(S 02):e115–23.
Dorr DA, D’Autremont C, Pizzimenti C, Weiskopf N, Rope R, Kassakian S, Richardson JE, McClure R, Eisenberg F. Assessing Data Adequacy for High Blood Pressure Clinical Decision Support: A Quantitative Analysis. Appl Clin Inform. 2021;12(4):710–20.
Moullin JC, Dickson KS, Stadnick NA, Rabin B, Aarons GA. Systematic review of the Exploration, Preparation, Implementation, Sustainment (EPIS) framework. Implement Sci. 2019;14(1):1.
Becan JE, Bartkowski JP, Knight DK, Wiley TRA, DiClemente R, Ducharme L, Welsh WN, Bowser D, McCollister K, Hiller M, Spaulding AC, Flynn PM, Swartzendruber A, Dickson MF, Fisher JH, Aarons GA. A model for rigorously applying the Exploration, Preparation, Implementation, Sustainment (EPIS) framework in the design and measurement of a large scale collaborative multi-site study. Health Justice. 2018;6(1):9.
Idalski Carcone A, Coyle K, Gurung S, Cain D, Dilones RE, Jadwin-Cakmak L, Parsons JT, Naar S. Implementation Science Research Examining the Integration of Evidence-Based Practices Into HIV Prevention and Clinical Care: Protocol for a Mixed-Methods Study Using the Exploration, Preparation, Implementation, and Sustainment (EPIS) Model. JMIR Res Protoc. 2019;8(5): e11202.
Jackson JM, Witek MA, Hupert ML, Brady C, Pullagurla S, Kamande J, Aufforth RD, Tignanelli CJ, Torphy RJ, Yeh JJ, Soper SA. UV activation of polymeric high aspect ratio microstructures: ramifications in antibody surface loading for circulating tumor cell selection. Lab Chip. 2014;14(1):106–17.
Mazzag B, Tignanelli CJ, Smith GD. The effect of residual Ca2+ on the stochastic gating of Ca2+-regulated Ca2+ channel models. J Theor Biol. 2005;235(1):121–50.
Jones EK, Hultman G, Schmoke K, Ninkovic I, Dodge S, Bahr M, Melton GB, Marquard J, Tignanelli CJ. Combined Expert and User-Driven Usability Assessment of Trauma Decision Support Systems Improves User-Centered Design. Surgery. 2022;172(5):1537–48.
Jakob N. Enhancing the explanatory power of usability heuristics. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '94). New York: Association for Computing Machinery; 1994. p. 152–8. https://doi.org/10.1145/191666.191729 .
Shah S, Switzer S, Shippee ND, Wogensen P, Kosednar K, Jones E, Pestka DL, Badlani S, Butler M, Wagner B, White K, Rhein J, Benson B, Reding M, Usher M, Melton GB, Tignanelli CJ. Implementation of an Anticoagulation Practice Guideline for COVID-19 via a Clinical Decision Support System in a Large Academic Health System and Its Evaluation: Observational Study. JMIR Med Inform. 2021;9(11): e30743.
Ingraham NE, Jones EK, King S, Dries J, Phillips M, Loftus T, Evans HL, Melton GB, Tignanelli CJ. Re-Aiming Equity Evaluation in Clinical Decision Support: A Scoping Review of Equity Assessments in Surgical Decision Support Systems. Ann Surg. 2023;277(3):359–64.
Holtrop JS, Estabrooks PA, Gaglio B, Harden SM, Kessler RS, King DK, Kwan BM, Ory MG, Rabin BA, Shelton RC, Glasgow RE. Understanding and applying the RE-AIM framework: Clarifications and resources. J Clin Transl Sci. 2021;5(1): e126.
https://www.facs.org/-/media/files/quality-programs/trauma/ntdb/ntds/data-dictionaries/ntds_data_dictionary_2022.ashx . Accessed 14 Sep 2021. ACoSNTDSDDA.
https://www.cdc.gov/pcd/issues/2014/13_0184.htm . Accessed 1/3/2021.
Malone S, Prewitt K, Hackett R, Lin JC, McKay V, Walsh-Bailey C, Luke DA. The Clinical Sustainability Assessment Tool: measuring organizational capacity to promote sustainability in healthcare. Implement Sci Commun. 2021;2(1):77.
Aarons GA, Ehrhart MG, Farahnak LR. The Implementation Leadership Scale (ILS): development of a brief measure of unit level implementation leadership. Implement Sci. 2014;9(1):45.
Rye M, Torres EM, Friborg O, Skre I, Aarons GA. The Evidence-based Practice Attitude Scale-36 (EBPAS-36): a brief and pragmatic measure of attitudes to evidence-based practice validated in US and Norwegian samples. Implement Sci. 2017;12(1):44.
Holt DT, Armenakis AA, Feild HS, Harris SG. Readiness for Organizational Change. J Appl Behav Sci. 2007;43(2):232–55.
Article Google Scholar
Weiner BJ. A theory of organizational readiness for change. Implement Sci. 2009;4:67.
Goodridge D, Rana M, Harrison EL, Rotter T, Dobson R, Groot G, Udod S, Lloyd J. Assessing the implementation processes of a large-scale, multi-year quality improvement initiative: survey of health care providers. BMC Health Serv Res. 2018;18(1):237.
Vis C, Ruwaard J, Finch T, Rapley T, de Beurs D, van Stel H, van Lettow B, Mol M, Kleiboer A, Riper H, Smit J. Toward an Objective Assessment of Implementation Processes for Innovations in Health Care: Psychometric Evaluation of the Normalization Measure Development (NoMAD) Questionnaire Among Mental Health Care Professionals. J Med Internet Res. 2019;21(2): e12376.
NoMAD. https://www.implementall.eu/17-nomad.html . Accessed 1/2/2021.
Ng F, McGrath BA, Seth R, et al. Measuring multidisciplinary staff engagement in a national tracheostomy quality improvement project using the NoMAD instrument. Br J Anesth. 2019;123(4):e506.
Guest G, Bunce A, Johnson L. How Many Interviews Are Enough?: An Experiment with Data Saturation and Variability. Field Methods. 2006;18:59–82.
Beidas RS, Stewart RE, Adams DR, Fernandez T, Lustbader S, Powell BJ, Aarons GA, Hoagwood KE, Evans AC, Hurford MO, Rubin R, Hadley T, Mandell DS, Barg FK. A Multi-Level Examination of Stakeholder Perspectives of Implementation of Evidence-Based Practices in a Large Urban Publicly-Funded Mental Health System. Adm Policy Ment Health. 2016;43(6):893–908.
Braun V, Clarke V. Thematic analysis. In Cooper H, Camic PM, Long DL, Panter AT, Rindskopf D, Sher KJ, editors. APA handbooks in psychology®. APA handbook of research methods in psychology, vol. 2. Research designs: Quantitative, qualitative, neuropsychological, and biological. American Psychological Association; 2012. p. 57–71.
Fiscella K, Sanders M, Holder T, Carroll JK, Luque A, Cassells A, Johnson BA, Williams SK, Tobin JN. The role of data and safety monitoring boards in implementation trials: When are they justified? J Clin Transl Sci. 2020;4(3):229–32.
Alonso-Coello P, Schunemann HJ, Moberg J, Brignardello-Petersen R, Akl EA, Davoli M, Treweek S, Mustafa RA, Rada G, Rosenbaum S, Morelli A, Guyatt GH, Oxman AF, Group GW. GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction. BMJ. 2016;353:i2016.
Rosenbaum SE, Moberg J, Glenton C, Schunemann HJ, Lewin S, Akl E, Mustafa RA, Morelli A, Vogel JP, Alonso-Coello P, Rada G, Vasquez J, Parmelli E, Gulmezoglu AM, Flottorp SA, Oxman AD. Developing Evidence to Decision Frameworks and an Interactive Evidence to Decision Tool for Making and Using Decisions and Recommendations in Health Care. Glob Chall. 2018;2(9):1700081.
Alonso-Coello P, Oxman AD, Moberg J, Brignardello-Petersen R, Akl EA, Davoli M, Treweek S, Mustafa RA, Vandvik PO, Meerpohl J, Guyatt GH, Schunemann HJ, Group GW. GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 2: Clinical practice guidelines. BMJ. 2016;353:i2089.
Osheroff JA. CDS and and the CDS & LHS 5 Rights. CDS/PI Collaborative: Getting Better Faster Together.
ACTS Project Team. Patient Journey and Service Blueprint How Tos. AHRQ evidence-based Care Transformation Support (ACTS) Home. [Online] October 2021. https://cmext.ahrq.gov/confluence/display/PUB/Patient+Journey+and+Service+Blueprint+How+Tos .
CDS Approach for Optimizing VTE Prophylaxis (VTEP) Society of Hospital Medicine (SHM) Recommendations1 Version 2; March, 2013. [online] https://www.healthit.gov/sites/default/files/cds/Detailed%20Inpatient%20CDS-QI%20Worksheet%20-%20VTE%20Example%20-%20Recommendations.xlsx .
Download references
This research was supported by the Agency for Healthcare Research and Quality (AHRQ), grant R18HS028583, the University of Minnesota Center for Learning Health System Sciences – a partnership between the University of Minnesota Medical School and the School of Public Health. The authors have no other conflicts of interest.
Authors and affiliations.
Department of Surgery, University of Minnesota, 420 Delaware St SE, MMC 195, Minneapolis, MN, 55455, USA
Christopher J. Tignanelli, Vincent Peta, Nicholas Lemke & Genevieve B. Melton
Institute for Health Informatics, University of Minnesota, Minneapolis, MN, USA
Christopher J. Tignanelli, Rubina Rizvi & Genevieve B. Melton
Center for Learning Health Systems Science, University of Minnesota, Minneapolis, MN, USA
Christopher J. Tignanelli, Rubina Rizvi, Mary Butler & Genevieve B. Melton
Center for Quality Outcomes, Discovery and Evaluation, University of Minnesota, Minneapolis, MN, USA
Christopher J. Tignanelli
Department of Medicine, Mayo Clinic, Scottsdale, AZ, USA
Surbhi Shah
Division of Biostatistics and Health Data Science, University of Minnesota, Minneapolis, MN, USA
David Vock, Lianne Siegel & Carlos Serrano
Department of Surgery, Johns Hopkins University, Baltimore, MD, USA
Elliott Haut
M Health Fairview, Minneapolis, MN, USA
Sean Switzer
School of Nursing, University of Minnesota, Minneapolis, MN, USA
Christie L. Martin
Department of Surgery, Indiana University School of Medicine, Indianapolis, IN, USA
Peter C. Jenkins
Center for Biomedical Informatics, Regenstrief Institute, Indianapolis, IN, USA
Thankam Thyvalikakath
Indiana University School of Dentistry, Indianapolis, IN, USA
TMIT Consulting, LLC, Naples, FL, USA
Jerome A. Osheroff
Department of Surgery, Geisinger Health, Danville, PA, USA
Denise Torres
Department of Biomedical Informatics, Geisinger Health, Danville, PA, USA
David Vawdrey
Department of Surgery, UC Davis School of Medicine, Sacramento, CA, USA
Rachael A. Callcut
School of Publish Health, University of Minnesota, Minneapolis, MN, USA
Mary Butler
You can also search for this author in PubMed Google Scholar
CT conceived and jointly designed the study protocol and helped write and critically revise this protocol paper, SS conceived and jointly designed the study protocol and helped write and critically revise this protocol paper, DV jointly designed the study protocol and helped write and critically revise this protocol paper, LS jointly designed the study protocol and helped write and critically revise this protocol paper, CS jointly designed the study protocol and helped write and critically revise this protocol paper, EH jointly designed the study protocol and helped write and critically revise this protocol paper, SS jointly designed the study protocol and helped write and critically revise this protocol paper, CM jointly designed the study protocol and helped write and critically revise this protocol paper, RR jointly designed the study protocol and helped write and critically revise this protocol paper, VP jointly designed the study protocol and helped write and critically revise this protocol paper, PJ jointly designed the study protocol and helped write and critically revise this protocol paper, NL jointly designed the study protocol and helped write and critically revise this protocol paper, TT jointly designed the study protocol and helped write and critically revise this protocol paper, JO jointly designed the study protocol and helped write and critically revise this protocol paper, DT jointly designed the study protocol and helped write and critically revise this protocol paper, DV jointly designed the study protocol and helped write and critically revise this protocol paper, RC jointly designed the study protocol and helped write and critically revise this protocol paper, MB jointly designed the study protocol and helped write and critically revise this protocol paper, GM conceived and jointly designed the study protocol and helped write and critically revise this protocol paper.
Correspondence to Christopher J. Tignanelli .
Ethics approval and consent to participate.
This study protocol was given the determination of “Exempt” as secondary research for which consent is not required. The Mixed Methods investigation was given the determination of “Not Human Research” as a quality improvement activity.
Not applicable.
The authors have no conflicts of interest to report.
Publisher’s note.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .
Reprints and permissions
Cite this article.
Tignanelli, C.J., Shah, S., Vock, D. et al. A pragmatic, stepped-wedge, hybrid type II trial of interoperable clinical decision support to improve venous thromboembolism prophylaxis for patients with traumatic brain injury. Implementation Sci 19 , 57 (2024). https://doi.org/10.1186/s13012-024-01386-4
Download citation
Received : 09 June 2024
Accepted : 14 July 2024
Published : 05 August 2024
DOI : https://doi.org/10.1186/s13012-024-01386-4
Anyone you share the following link with will be able to read this content:
Sorry, a shareable link is not currently available for this article.
Provided by the Springer Nature SharedIt content-sharing initiative
ISSN: 1748-5908
Grab your spot at the free arXiv Accessibility Forum
Help | Advanced Search
Title: rag foundry: a framework for enhancing llms for retrieval augmented generation.
Abstract: Implementing Retrieval-Augmented Generation (RAG) systems is inherently complex, requiring deep understanding of data, use cases, and intricate design decisions. Additionally, evaluating these systems presents significant challenges, necessitating assessment of both retrieval accuracy and generative quality through a multi-faceted approach. We introduce RAG Foundry, an open-source framework for augmenting large language models for RAG use cases. RAG Foundry integrates data creation, training, inference and evaluation into a single workflow, facilitating the creation of data-augmented datasets for training and evaluating large language models in RAG settings. This integration enables rapid prototyping and experimentation with various RAG techniques, allowing users to easily generate datasets and train RAG models using internal or specialized knowledge sources. We demonstrate the framework effectiveness by augmenting and fine-tuning Llama-3 and Phi-3 models with diverse RAG configurations, showcasing consistent improvements across three knowledge-intensive datasets. Code is released as open-source in this https URL .
Comments: | 10 pages |
Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Information Retrieval (cs.IR); Machine Learning (cs.LG) |
Cite as: | [cs.CL] |
(or [cs.CL] for this version) | |
Focus to learn more arXiv-issued DOI via DataCite |
Access paper:.
Code, data and media associated with this article, recommenders and search tools.
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .
Oslo Metropolitan University is Norway's third largest university with almost 22,000 students and over 2,500 employees. We have campuses in central Oslo and at Romerike. OsloMet educates students and conducts research that contributes to the sustainability of the Norwegian welfare state and the metropolitan region.
The Faculty of Technology, Art and Design (TKD) offers higher education and research and development (R&D) activities within technical subjects, arts and design. The Faculty has approximately 4.000 students and 400 staff members and is situated at Pilestredet Campus in downtown Oslo and at Kjeller Campus in Viken.
The Department of Built Environment offers bachelors and master’s degree programs in engineering, in addition to research and development activities. The department has approximately 60 staff members and 850 students.
The Department of Built Environment has a vacant PhD Fellowship position. This PhD project focuses on innovation strategies for cities through the lens of inclusive, safe and economically vital cities for the well-being of people. Until now, the multi-dimensional well-being indicators for cities are underresearched. Thus, this PhD project aims for developing a novel mixed-method innovation framework that allows practitioners a tool at hand that allows planning and designing resilient future cities, develop and test scenarios for informed-decision making and to shape policies through a data-policy interaction approach.
The idea of well-being has gained attention and become more important for people all over the world living in cities, towns, and rural areas. Well-being includes the physical, social and emotional state of a person. Inclusive, accessible, safe, economically vital cities with high quality environments are adding to a healthy, well-balanced lifestyle allowing for people to be physically active, socially interact and gain emotional wellness in a safe city. The notion of well-being is connected to the concepts of ‘Happy Cities’, ‘Healthy Cities’, and ‘Trauma-Informed Cities’. Thus, the idea of well-being in cities is a multi-dimensionional concept allowing to create future resilient cities.
This PhD study researches the interplay of socio-spatial, economic and aesthetic indicators creating an urban analytical design framework across scales enabling shaping policies to create safe, vital cities for the well-being of people.
The position is for a period of three years. The goal must be to complete the PhD program/degree within the decided time frame.
The following grade requirements are a condition for employment in the position:
Admission to the doctoral program in Engineering Science at the Faculty of Technology, Art and Design within three months of employment is a prerequisite for the position. If you already have a doctorate in a related field, you will not qualify for the position.
In assessing the applicants, emphasis will be placed on the department's overall needs and the applicant's potential for research within the field.
General criteria for employment in academic positions are covered by the Regulations on the University and College Ac t.
Personal suitability will be emphasised.
It is important to OsloMet to reflect the population of our region, and all qualified candidates are welcome to apply. We make active endeavours to further develop OsloMet as an inclusive workplace and to make adaptations to the workplace where required. You are also welcome to apply for a position with us if you have had periods where you have not been in employment, education or training.
Practical information about relocation to Oslo and living in Norway .
To be considered for the position, you must upload the following documents by the application deadline:
The following language tests are approved documentation: TOEFL, IELTS, Cambridge Certificate in Advanced English (CAE) or Cambridge Certificate of Proficiency in English (CPE). In these tests, you must have achieved at least the following scores:
We only process applications sent via our electronic recruitment system and all documents must be uploaded for your application to be processed. The documents must be in English or a Scandinavian language. Translations must be authorized. You must present originals at any interview. OsloMet checks documents, so that you as a candidate will get a real evaluation and fair competition.
The engagement is to be made in accordance with the regulations in force concerning State Employees and Civil Servants, and the acts relating to Control of the Export of Strategic Goods, Services and Technology. A background check may be conducted to verify information in submitted CVs and available documents. Background checks are not conducted without the consent of the applicant and relevant applicants will receive further information about this.
If you would like more information about the position, feel free to contact:
The position is paid according to the pay scale for Norwegian state employees, position code 1017 PhD fellow, NOK 532 200 per year.
If you have documents that cannot be uploaded electronically, please contact [email protected] .
If you would like to apply for the position, you must do so electronically through our recruitment system.
Deadline for applications: 04.09.2024
Ref.: 24/19730
OsloMet is a Charter & Code certified institution by the EU Commission holding the right to use the logo HR Excellence in Research (HRS4R). OsloMet is a member of the EURAXESS network supporting a positive work environment for researchers in motion.
OsloMet has signed The Declaration on Research Assessment (DORA) . DORA recognizes the need to improve the ways in which the outputs of scholarly research are evaluated.
The engagement is to be made in accordance with the regulations in force concerning State Employees and Civil Servants, and the acts relating to Control of the Export of Strategic Goods, Services and Technology. Candidates who are seen to be in conflict with the criteria in the latter law will not be considered for the position.
Visit OsloMet at LinkedIn , Facebook , Instagram
Products and Services
Master your goals. innovate. we'll tackle threats..
Get powerful security across all your networks, cloud, endpoints, and email to protect everything that matters, from anywhere.
More connected users and devices creates more complexity. Cisco Security Cloud makes security easier for IT and safer for everyone anywhere security meets the network.
Protect your organization across a multicloud environment, while simplifying security operations, improving scalability, and driving data-informed outcomes, powered by Cisco Talos.
Create a seamless experience that frustrates attackers, not users, by granting access from any device, anywhere, and adding more proactive security controls.
Improve ROI by consolidating vendors, reducing complexity and integrating your security.
Unified, end-to-end protection maximizes value, minimizes risk, and closes security gaps everywhere to defend against evolving threats. Protect access, apps, and innovation across your network to secure your future.
Better visibility and actionable insights across networks, clouds, endpoints, and email allows users to respond confidently to the most sophisticated threats at machine scale.
Cisco hypershield.
A new groundbreaking security architecture that makes hyperscaler technology accessible to enterprises of all sizes and delivers AI-native security for modern data centers and cloud.
A converged cybersecurity solution, grounded in zero trust, that radically reduces risk and delights both end users and IT staff by safely connecting anything to anywhere.
Detect the most sophisticated threats sooner across all vectors and prioritize by impact for faster responses.
Gain multidirectional protection across clouds to stop inbound attacks, data exfiltration, and lateral movement.
Secure applications and enable frictionless access with strong MFA and more. Establish user and device trust, gain visibility into devices, and enable secure access to all apps.
Simplify highly secure network access control with software-defined access and automation.
Get secure access to any application, on any device, from anywhere. Defend against threats targeting users and deliver seamless access for hybrid work.
Secure your apps and data with a powerful, flexible framework for a hybrid and multicloud world.
Secure your business by investigating, prioritizing, and resolving incidents through unified defense and contextual insights from data-backed, AI-powered security.
Global partnerships fight to end child exploitation together.
"Marriott has long championed human rights and human trafficking awareness. Combating CSAM is an important extension of that work. The IWF provided the level of rigor we needed in a URL list, and Cisco's security technology provided the means to easily apply it."
Abbe Horswill, Director, Human Rights and Social Impact
Company: Marriott International
"From securing stadiums, broadcasts, and fans to protecting the largest live sporting event in America, the right tools and the right team are key in making sure things run smoothly, avoiding disruptions to the game, and safeguarding the data and devices that make mission-critical gameday operations possible."
Cisco Security Enterprise Agreement
Experience security software buying flexibility with one easy-to-manage agreement.
Services for security
Get more from your investments and enable constant vigilance to protect your organization.
Sharpen your security insights
Set your vision to a more secure future with Cisco Cybersecurity Viewpoints. With specialized content from podcasts to industry news, you'll walk away with a deeper understanding of the trends, research, and topics in our rapidly changing world.
IMAGES
COMMENTS
Step 2: Choose a type of research design. Within both qualitative and quantitative approaches, there are several types of research design to choose from. Each type provides a framework for the overall shape of your research. Types of quantitative research designs. Quantitative designs can be split into four main types.
Step 2: Choose a type of research design. Within both qualitative and quantitative approaches, there are several types of research design to choose from. Each type provides a framework for the overall shape of your research. Types of quantitative research designs. Quantitative designs can be split into four main types.
The purpose of research design is to plan and structure a research study in a way that enables the researcher to achieve the desired research goals with accuracy, validity, and reliability. Research design is the blueprint or the framework for conducting a study that outlines the methods, procedures, techniques, and tools for data collection ...
Research design refers to the overall plan, structure or strategy that guides a research project, from its conception to the final analysis of data. Research designs for quantitative studies include descriptive, correlational, experimental and quasi-experimenta l designs. Research designs for qualitative studies include phenomenological ...
Research design methods refer to the systematic approaches and techniques used to plan, structure, and conduct a research study. The choice of research design method depends on the research questions, objectives, and the nature of the study. Here are some key research design methods commonly used in various fields: 1.
framework is a generative source of thinking, planning, conscious action, and reflection throughout the research process. A conceptual framework makes the case for why a study is significant and relevant and for how the study design (including data collection and analysis methods) appropri - ately and rigorously answers the research questions.
Research Design Framework. "I use terms like 'canvas' and 'design' because research requires both analytical and creative knowledge, skills, and abilities. There is no one best way to conduct research, and the answer to ALL research methods questions is, 'it depends.'". ( Latham, 2022 ). While this framework provides structure ...
A research design is the plan or framework used to conduct a research study. It involves outlining the overall approach and methods that will be used to collect and analyze data in order to answer research questions or test hypotheses.
for validity and reliability. Design is basically concerned with the aims, uses, purposes, intentions and plans within the. pr actical constraint of location, time, money and the researcher's ...
2. Research design. The research design is intended to provide an appropriate framework for a study. A very significant decision in research design process is the choice to be made regarding research approach since it determines how relevant information for a study will be obtained; however, the research design process involves many interrelated decisions [].
The first kind, "Research into design" studies the design product post hoc and the MIR framework suits the interdisciplinary study of such a product. In contrast, "Research for design" generates knowledge that feeds into the noun and the verb 'design', which means it precedes the design (ing).
Other studies have presented a research logic model or flowchart of the research design as a conceptual framework. These constructions can be quite valuable in helping readers understand the data-collection and analysis process. However, a model depicting the study design does not serve the same role as a conceptual framework.
Research design is the framework of research methods and techniques chosen by a researcher to conduct a study. The design allows researchers to sharpen the research methods suitable for the subject matter and set up their studies for success. Creating a research topic explains the type of research (experimental,survey research,correlational ...
If your design is poor, the results of the research also will not be promising.[ 2 ] Research design is defined as a framework of methods and techniques chosen by a researcher to combine various ...
Research Design A research design is the 'procedures for collecting, analyzing, interpreting and reporting data in research studies' (Creswell & Plano Clark 2007, p.58). ... the SCP framework ...
The Importance of Research Frameworks. Researchers may draw on several elements to frame their research. Generally, a framework is regarded as "a set of ideas that you use when you are forming your decisions and judgements" 13 or "a system of rules, ideas, or beliefs that is used to plan or decide something." 14 Research frameworks may consist of a single formal theory or part thereof ...
A research framework provides an underlying structure or model to support our collective research efforts. Up until now, we've referenced, referred to and occasionally approached research as more of an amalgamated set of activities. But as we know, research comes in many different shapes and sizes, is variable in scope, and can be used to ...
Research design and research methods are two essential components of any research study. While research design provides the overall plan and structure, research methods are the practical tools used to collect and analyze data. Both have distinct attributes that contribute to the reliability, validity, and generalizability of research findings.
Definition: Methodological framework is a set of procedures, methods, and tools that guide the research process in a systematic and structured manner. It provides a structure for conducting research, collecting and analyzing data, and drawing conclusions. The framework outlines the steps to be taken in a research project, including the research ...
For Durrheim (2004:29), research design is a strategic framework for action that serves as a bridge between research questions and the execution, or implementation of the research strategy. 4.2.3 RESEARCH METHODOLOGY. Schwardt (2007:195) defines research methodology as a theory of how an inquiry should
A Design Research Framework. Oct 25, 2022. Written By Erika Hall. Design Research Process Model (PDF) |. Alternative Style for Color Perception/B&W Printing (PDF) Recent discussions have been swirling around the phrase " democratization of research " concerning who should participate in what kind of research in design (product/service ...
Theoretical Framework. Definition: Theoretical framework refers to a set of concepts, theories, ideas, and assumptions that serve as a foundation for understanding a particular phenomenon or problem.It provides a conceptual framework that helps researchers to design and conduct their research, as well as to analyze and interpret their findings.. In research, a theoretical framework explains ...
3.1. Introduction. Chapter 3Research framework and Design3.1. IntroductionResearch m. thodology is the indispensable part of any research work. This guides the researcher about the flow of research and provides the. ramework through which the research is to be carried out. This chapter expounds the research paradigm, research approach, research ...
Design Thinking approaches to problem resolution: Systems Thinking methodologies arose from the consideration of social systems. The stakeholders are the designers. Design Thinking methodologies arose from the consideration of products and artifacts. The problems are ultimately resolved by people identified as a designer by trade.
Study aims and implementation framework. This trial consists of a stepped wedge hybrid effectiveness-implementation trial to scale the CDS system across 4 trauma systems and in parallel evaluate implementation strategy guided by the Exploration, Preparation, Implementation, and Sustainment (EPIS) implementation framework (Fig. 1a) [].We anticipate variability in CDS adoption across sites ...
ConspectusFlexible metal-organic frameworks (MOFs), also known as soft porous crystals, exhibit dynamic behaviors in response to external physical and chemical stimuli such as light, heat, electric or magnetic field, or the presence of particular matters, on the premise of maintaining their crystalline state. The reversible structural transformation of flexible MOFs, a unique characteristic ...
Implementing Retrieval-Augmented Generation (RAG) systems is inherently complex, requiring deep understanding of data, use cases, and intricate design decisions. Additionally, evaluating these systems presents significant challenges, necessitating assessment of both retrieval accuracy and generative quality through a multi-faceted approach. We introduce RAG Foundry, an open-source framework ...
A Computation-guided Design of Highly Defined and Dense Bimetallic Active Sites on a Two-dimensional Conductive Metal-organic Framework for Efficient H2O2 Electrosynthesis. Ji Liang, Corresponding Author. Ji Liang ... Institute of Metal Research Chinese Academy of Sciences, Shenyang National Laboratory for Materials Science, Institute of Metal ...
The Faculty of Technology, Art and Design (TKD) offers higher education and research and development (R&D) activities within technical subjects, arts and design. The Faculty has approximately 4.000 students and 400 staff members and is situated at Pilestredet Campus in downtown Oslo and at Kjeller Campus in Viken.
"From securing stadiums, broadcasts, and fans to protecting the largest live sporting event in America, the right tools and the right team are key in making sure things run smoothly, avoiding disruptions to the game, and safeguarding the data and devices that make mission-critical gameday operations possible."