Pragmatic Free Trial Meta
Pragmatic Free Trail Meta is an open data platform that allows research into pragmatic trials. It shares clean trial data and ratings using PRECIS-2, allowing for multiple and diverse meta-epidemiological studies that evaluate the effect of treatment on trials that employ different levels of pragmatism, as well as other design features.
Background
Pragmatic trials provide evidence from the real world that can be used to make clinical decisions. However, the usage of the term "pragmatic" is not consistent and its definition and evaluation requires clarification. The purpose of pragmatic trials is to guide the practice of clinical medicine and policy decisions, not to verify a physiological hypothesis or clinical hypothesis. A pragmatic trial should aim to be as close as it is to the real-world clinical practice, including recruitment of participants, setting up, implementation and delivery of interventions, determining and analysis results, as well as primary analyses. This is a major difference between explanation-based trials, as described by Schwartz and Lellouch1 which are designed to confirm a hypothesis in a more thorough manner.
Truely pragmatic trials should not conceal participants or the clinicians. This can result in an overestimation of the effect of treatment. Pragmatic trials should also seek to recruit patients from a wide range of health care settings to ensure that the results are generalizable to the real world.
Finally, pragmatic trials should focus on outcomes that are important to patients, like quality of life or functional recovery. This is especially important in trials that require the use of invasive procedures or could have serious adverse effects. The CRASH trial29 compared a 2-page report with an electronic monitoring system for hospitalized patients with chronic heart failure. The catheter trial28, however utilized symptomatic catheter-related urinary tract infection as its primary outcome.
In addition to these characteristics the pragmatic trial should also reduce the trial procedures and data collection requirements to reduce costs. Additionally pragmatic trials should try to make their findings as relevant to actual clinical practice as possible by making sure that their primary method of analysis is based on the intention-to-treat method (as described in CONSORT extensions for pragmatic trials).
Despite these guidelines however, a large number of RCTs with features that challenge the notion of pragmatism were incorrectly labeled pragmatic and published in journals of all types. This could lead to misleading claims of pragmatism, and the use of the term should be standardized. The creation of a PRECIS-2 tool that provides an objective, standardized assessment of pragmatic features is a first step.
Methods
In a pragmatic trial the goal is to inform clinical or policy decisions by showing how an intervention could be implemented into routine care. This is different from explanatory trials that test hypotheses about the causal-effect relationship in idealized situations. In this way, pragmatic trials could have less internal validity than studies that explain and are more susceptible to biases in their design, analysis, and conduct. Despite these limitations, pragmatic trials may provide valuable information to decisions in the context of healthcare.
The PRECIS-2 tool evaluates the degree of pragmatism within an RCT by scoring it across 9 domains ranging from 1 (very explanatory) to 5 (very pragmatic). In this study the domains of recruitment, organisation, flexibility in delivery, flexibility in adherence, and follow-up received high scores. However, the principal outcome and method of missing data were scored below the practical limit. 프라그마틱슬롯 suggests that it is possible to design a trial using good pragmatic features without harming the quality of the outcomes.
However, it's difficult to assess how pragmatic a particular trial is since pragmatism is not a binary attribute; some aspects of a study can be more pragmatic than others. A trial's pragmatism could be affected by modifications to the protocol or logistics during the trial. Koppenaal and colleagues found that 36% of the 89 pragmatic studies were placebo-controlled or conducted prior to the licensing. They also found that the majority were single-center. This means that they are not quite as typical and are only pragmatic in the event that their sponsors are supportive of the absence of blinding in these trials.
Furthermore, a common feature of pragmatic trials is that researchers attempt to make their findings more valuable by studying subgroups of the sample. This can result in unbalanced analyses that have lower statistical power. This increases the possibility of omitting or ignoring differences in the primary outcomes. In the case of the pragmatic trials that were included in this meta-analysis this was a serious issue since the secondary outcomes weren't adjusted for differences in the baseline covariates.
In addition, pragmatic trials can also be a challenge in the collection and interpretation of safety data. This is due to the fact that adverse events are generally reported by the participants themselves and are susceptible to reporting delays, inaccuracies, or coding variations. It is essential to improve the quality and accuracy of the outcomes in these trials.
Results
While the definition of pragmatism does not require that all trials be 100 100% pragmatic, there are benefits of including pragmatic elements in clinical trials. These include:
Incorporating routine patients, the trial results can be more quickly translated into clinical practice. However, pragmatic studies can also have drawbacks. For example, the right type of heterogeneity can help the trial to apply its findings to a variety of settings and patients. However, the wrong type of heterogeneity can reduce assay sensitivity and therefore decrease the ability of a trial to detect minor treatment effects.
A number of studies have attempted to categorize pragmatic trials, using various definitions and scoring systems. Schwartz and Lellouch1 created an approach to distinguish between research studies that prove a physiological or clinical hypothesis, and pragmatic trials that aid in the selection of appropriate treatments in real-world clinical practice. The framework consisted of nine domains evaluated on a scale of 1-5 which indicated that 1 was more lucid while 5 being more pragmatic. The domains were recruitment setting, setting, intervention delivery, flexible adherence, follow-up and primary analysis.
The initial PRECIS tool3 had similar domains and scales from 1 to 5. Koppenaal et al10 created an adaptation to this assessment dubbed the Pragmascope which was more user-friendly to use in systematic reviews. They found that pragmatic reviews scored higher across all domains, however they scored lower in the primary analysis domain.
This difference in primary analysis domains could be due to the way in which most pragmatic trials analyse data. Some explanatory trials, however, do not. The overall score for pragmatic systematic reviews was lower when the domains of organization, flexible delivery, and following-up were combined.
It is important to remember that a study that is pragmatic does not mean that a trial is of poor quality. In fact, there are an increasing number of clinical trials that employ the term "pragmatic" either in their abstract or title (as defined by MEDLINE however it is not precise nor sensitive). These terms could indicate a greater understanding of pragmatism in abstracts and titles, however it isn't clear if this is reflected in the content.
Conclusions
As appreciation for the value of real-world evidence becomes increasingly widespread the pragmatic trial has gained momentum in research. They are clinical trials randomized that evaluate real-world alternatives to care instead of experimental treatments under development. They have patient populations which are more closely resembling the patients who receive routine care, they use comparators that are used in routine practice (e.g., existing drugs), and they rely on participant self-report of outcomes. This approach can overcome the limitations of observational research, for example, the biases that are associated with the reliance on volunteers and the limited availability and coding variations in national registries.
Other advantages of pragmatic trials are the possibility of using existing data sources, and a greater probability of detecting significant changes than traditional trials. However, they may still have limitations which undermine their validity and generalizability. For example the rates of participation in some trials may be lower than expected due to the healthy-volunteer effect as well as incentives to pay or compete for participants from other research studies (e.g., industry trials). Practical trials are often restricted by the need to recruit participants quickly. Additionally, some pragmatic trials don't have controls to ensure that the observed differences are not due to biases in the conduct of trials.
The authors of the Pragmatic Free Trial Meta identified 48 RCTs self-labeled as pragmatist and published until 2022. They assessed pragmatism using the PRECIS-2 tool, which consists of the eligibility criteria for domains, recruitment, flexibility in adherence to interventions, and follow-up. They found that 14 of these trials scored pragmatic or highly pragmatic (i.e. scoring 5 or higher) in any one or more of these domains, and that the majority were single-center.
Trials that have a high pragmatism score tend to have broader eligibility criteria than traditional RCTs which have very specific criteria that aren't likely to be present in clinical practice, and they comprise patients from a wide variety of hospitals. These characteristics, according to the authors, could make pragmatic trials more useful and relevant to everyday practice. However they do not ensure that a study is free of bias. The pragmatism characteristic is not a definite characteristic; a pragmatic test that does not possess all the characteristics of an explanation study could still yield reliable and beneficial results.
