Pragmatic Free Trial Meta
Pragmatic Free Trail Meta is an open data platform that facilitates research into pragmatic trials. It collects and distributes cleaned trial data, ratings and evaluations using PRECIS-2. This allows for a variety of meta-epidemiological studies to evaluate the effects of treatment across trials with different levels of pragmatism.
Background
Pragmatic studies provide real-world evidence that can be used to make clinical decisions. However, 프라그마틱 체험 of the term "pragmatic" is not consistent and its definition and evaluation requires clarification. Pragmatic trials must be designed to inform clinical practice and policy decisions, rather than to prove a physiological or clinical hypothesis. A pragmatic trial should aim to be as similar to real-world clinical practice as possible, including in its selection of participants, setting and design as well as the implementation of the intervention, determination and analysis of outcomes and primary analyses. This is a key distinction from explanatory trials (as described by Schwartz and Lellouch1) which are intended to provide a more complete confirmation of an idea.
The trials that are truly pragmatic must avoid attempting to blind participants or the clinicians, as this may cause bias in estimates of the effects of treatment. Pragmatic trials will also recruit patients from different healthcare settings to ensure that the results can be generalized to the real world.
Additionally, clinical trials should be focused on outcomes that matter to patients, such as the quality of life and functional recovery. This is particularly relevant in trials that require the use of invasive procedures or could have dangerous adverse consequences. The CRASH trial29, for instance, focused on functional outcomes to evaluate a two-page case report with an electronic system for monitoring of hospitalized patients with chronic heart failure. In addition, the catheter trial28 focused on symptomatic catheter-associated urinary tract infections as the primary outcome.
In addition to these features pragmatic trials should also reduce the requirements for data collection and trial procedures to cut down on costs and time commitments. Finally pragmatic trials should strive to make their findings as applicable to real-world clinical practice as possible by ensuring that their primary analysis is based on the intention-to-treat method (as described in CONSORT extensions for pragmatic trials).
Many RCTs which do not meet the criteria for pragmatism, but have features that are contrary to pragmatism, have been published in journals of varying types and incorrectly labeled as pragmatic. This can result in misleading claims of pragmaticity, and the use of the term should be standardized. The development of a PRECIS-2 tool that provides a standardized objective evaluation of pragmatic aspects is a good start.
Methods
In a pragmatic study, the aim is to inform policy or clinical decisions by demonstrating how an intervention would be implemented into routine care. Explanatory trials test hypotheses concerning the cause-effect relationship within idealised environments. Therefore, pragmatic trials might be less reliable than explanatory trials and might be more susceptible to bias in their design, conduct and analysis. Despite these limitations, pragmatic trials can contribute valuable information to decision-making in healthcare.
The PRECIS-2 tool evaluates the degree of pragmatism in an RCT by assessing it on 9 domains, ranging from 1 (very explicative) to 5 (very pragmatic). In this study, the domains of recruitment, organisation as well as flexibility in delivery flexible adherence and follow-up scored high. However, the main outcome and method of missing data were scored below the practical limit. This indicates that a trial can be designed with effective practical features, but without harming the quality of the trial.
It is difficult to determine the degree of pragmatism within a specific trial since pragmatism doesn't possess a specific attribute. Certain aspects of a research study can be more pragmatic than others. The pragmatism of a trial can be affected by changes to the protocol or the logistics during the trial. In addition, 36% of the 89 pragmatic trials identified by Koppenaal et al were placebo-controlled or conducted before licensing and most were single-center. They are not in line with the norm, and can only be called pragmatic if their sponsors accept that such trials aren't blinded.
A common feature of pragmatic studies is that researchers attempt to make their findings more meaningful by studying subgroups within the trial sample. This can lead to imbalanced analyses and lower statistical power. This increases the chance of missing or misdetecting differences in the primary outcomes. This was a problem during the meta-analysis of pragmatic trials as secondary outcomes were not corrected for covariates that differed at the baseline.
Additionally practical trials can be a challenge in the gathering and interpretation of safety data. This is because adverse events are usually self-reported and are prone to delays in reporting, inaccuracies or coding errors. It is important to improve the quality and accuracy of outcomes in these trials.
Results
Although the definition of pragmatism may not require that all clinical trials are 100% pragmatic, there are benefits when incorporating pragmatic components into trials. These include:
By including routine patients, the trial results can be more quickly translated into clinical practice. However, pragmatic trials have disadvantages. For instance, the right type of heterogeneity can help a study to generalize its results to many different settings and patients. However the wrong kind of heterogeneity may reduce the assay's sensitiveness and consequently reduce the power of a trial to detect small treatment effects.
A variety of studies have attempted to categorize pragmatic trials using various definitions and scoring systems. Schwartz and Lellouch1 have developed a framework that can discern between explanation-based studies that confirm the physiological hypothesis or clinical hypothesis and pragmatic studies that help inform the choice for appropriate therapies in real world clinical practice. Their framework comprised nine domains, each scoring on a scale of 1 to 5 with 1 indicating more explanatory and 5 indicating more practical. The domains were recruitment setting, setting, intervention delivery and follow-up, as well as flexible adherence and primary analysis.
The original PRECIS tool3 was an adapted version of the PRECIS tool3 that was based on the same scale and domains. Koppenaal et al10 developed an adaptation of this assessment, dubbed the Pragmascope that was simpler to use for systematic reviews. They found that pragmatic reviews scored higher on average across all domains, however they scored lower in the primary analysis domain.
This difference in primary analysis domains could be due to the way in which most pragmatic trials analyse data. Some explanatory trials, however don't. The overall score for pragmatic systematic reviews was lower when the domains of organization, flexible delivery, and following-up were combined.
It is important to remember that a pragmatic trial doesn't necessarily mean a low quality trial, and in fact there is a growing number of clinical trials (as defined by MEDLINE search, however this is not sensitive nor specific) that employ the term "pragmatic" in their title or abstract. These terms may signal a greater awareness of pragmatism within abstracts and titles, however it's unclear if this is reflected in the content.
Conclusions
As appreciation for the value of real-world evidence grows widespread, pragmatic trials have gained popularity in research. They are randomized studies that compare real-world alternatives to experimental treatments in development. They include patient populations closer to those treated in regular medical care. This approach can help overcome the limitations of observational research, such as the limitations of relying on volunteers and the lack of availability and the variability of coding in national registry systems.
Pragmatic trials also have advantages, including the ability to use existing data sources, and a greater probability of detecting meaningful differences than traditional trials. However, these tests could have some limitations that limit their reliability and generalizability. For example the participation rates in certain trials could be lower than expected due to the healthy-volunteer influence and financial incentives or competition for participants from other research studies (e.g. industry trials). A lot of pragmatic trials are restricted by the necessity to recruit participants quickly. Additionally some pragmatic trials lack controls to ensure that the observed differences are not due to biases in trial conduct.
The authors of the Pragmatic Free Trial Meta identified RCTs published up to 2022 that self-described as pragmatic. The PRECIS-2 tool was used to assess pragmatism. It covers domains such as eligibility criteria, recruitment flexibility, adherence to intervention, and follow-up. They found that 14 trials scored highly pragmatic or pragmatic (i.e. scoring 5 or above) in at least one of these domains.
Trials with high pragmatism scores tend to have more criteria for eligibility than traditional RCTs. They also include patients from a variety of hospitals. The authors argue that these characteristics can help make pragmatic trials more meaningful and useful for daily practice, but they do not guarantee that a trial using a pragmatic approach is free of bias. Furthermore, the pragmatism of trials is not a fixed attribute and a pragmatic trial that doesn't possess all the characteristics of an explanatory trial can produce reliable and relevant results.