top of page
Gradient_edited.jpg

The flexibility trials

 

a review

by Danica Pietrzak

 

After reviewing the pros and cons of Randomized Control Trials, Observational Studies, Regression Point Displacement and Interrupted Time Series, one of the most important qualities of each study type was flexibility. With how varied research is, an adaptable method is a prerequisite for a successful study. The ideal chosen method would adapt to anything set within the parameters of a research area. From these concerns arise this review of each method’s flexibility under a vast net of conditions.

​

First, the Randomized Control Trials (RCT)s. RCTs, the lauded “favorite” or “golden boy” of our time, has indeed earned these titles in some manner. The act of randomization to prevent unintentional bias shields many trials from failure and poor statistics by attempting to separate the research from emotion-based decisions. While effective when done properly, this method also limits RCTs from having a hand in research regarding long-term effects and ethically grey areas. With the sheer amount of money and planning required for a RCT, long-term results while maintaining the standard of randomization do not appear to be feasible at this point in time. Also, with issues that appear more ethically striated, applying a method that could intentionally harm participants raises many concerns which would affect the support and success of the paper. 

​

From this, another method, the regression point displacement (RPD), blossomed. The RPD is a quasi-experimental design that compares the pre- and post-test data of multiple control groups and one test group. It contains similar ethical problems to that of the RCT, particularly that the treated group may be purposely exposed to a possibly harmful material or treatment. This hinders the general flexibility of the RPD, but RPDs are less expensive than the RCT. This means an RPD may be more practicable for longer-term testing and therefore is more flexible with size or time range as cost is not as much of a hindrance as it is with the RCT.

​

Looking past the RCT and RPD and onto observational studies introduces a new area of concerns. Observational studies are far more flexible, allowing for real-life observation of the effects of a study. There are several methodological concerns, particularly this trial’s increased susceptibility to overt or hidden bias. However, observational studies can apply to a myriad of situations which a RCT cannot, meaning they are more adaptable to various circumstances. They do not have the ethical concerns, size restrictions, or required randomization needed in RCTs and similar methods.

​

Interrupted Time series was created in part for treatments that have no viable control group. This by itself significantly limits the flexibility of this design. The ITS is a very useful tool when needed and can apply to many situations in which traditional methods would struggle with, but most certainly not one-size fits all goal flexibility-wise as there are available control groups for many areas of research. 

 

SOURCES

​

Meldrum, Marcia L. "A brief history of the randomized controlled trial: From oranges and lemons to the gold standard." Hematology/oncology clinics of North America 14, no. 4 (2000): 745-760.

​

Grossman, Jason, and Fiona J. Mackenzie. "The randomized controlled trial: gold standard, or merely standard?." Perspectives in biology and medicine 48, no. 4 (2005): 516-534.

 

​

​

​

​

Randomized Control Trials
 

First used: 1923, Sir Austin Bradford
Purpose: Randomizes a group of people in half, give one side the treatment and leave the other side alone. This means that the chances for other random medical occurrences happen equally across the two groups.
Limits: Time, size, reporting, cost, ethics    

Overall Rating: 8
Flexibility Rating: 6.5
 

Notes: 

  • Randomized Control Trials have played a huge social role in AIDS activism.

  • I have some questions about research ethics: if there is something thought to harm a human, it is often unethical for this trial to take place (ex: smoking), and an observational study would have to take place instead? 

  • With large followup times or larger groups of subjects, RCTs often become impractical. The common fallback is observational studies with stringently controlled application of trial material.

  • There is a significant issue with method reporting for RCTs. Published results seem worse than they actually are due to paper presentation. The issue with standardization of trial reporting (CONSTORT) is the assumption that if not written, it was not done, but methodology may be more extensive than what can fit into a paper.

  • The less likely a result is to occur, the larger the trial will have to be to accomodate for statistical significance, making a financially feasible RCT nearly impossible.

  • RCTs reduce allocation bias.

  • However, RCTs are still vulnerable to bias regardless of randomization, due to selection of research via funding, etc.

​

SOURCES

​

  • Stolberg, Harald O., Geoffrey Norman, and Isabelle Trop. "Randomized controlled trials." American Journal of Roentgenology 183, no. 6 (2004): 1539-1544.

  • Frakt, Austin B. "An observational study goes where randomized clinical trials have not." Jama 313, no. 11 (2015): 1091-1092.

  • Soares, Heloisa P., Stephanie Daniels, Ambuj Kumar, Mike Clarke, Charles Scott, Suzanne Swann, and Benjamin Djulbegovic. "Bad reporting does not mean bad methods for randomised trials: observational study of randomised controlled trials performed by the Radiation Therapy Oncology Group." Bmj 328, no. 7430 (2004): 22-24.

  • Rosenbaum, Paul R. "Observational study." Encyclopedia of statistics in behavioral science (2005).

​

​

​

​

​

Observational study

​

Overall Rating: 7Flexibility Rating: 9
Purpose: Find (ideally) causality, if not correlation, in a situation where controlled trials are not feasible (time, money, ethics, etc.).
Limits:  bias, inability to find causality

 

Overall Rating: 7

Flexibility Rating: 9


Notes: 

  • Observational studies can imply correlation but often lack the clarity RCTs have to prove causality between a stimulus and response.

  • There is a lack of randomization, meaning that different practitioners may take differing actions due to personal bias, skewing the trial. This is not a concern seen in RCTs.

  • However, observational studies are significantly better for mortality studies, as long-term effects are difficult to determine with an RCT

 

SOURCES

  • Frakt, Austin B. "An observational study goes where randomized clinical trials have not." Jama 313, no. 11 (2015): 1091-1092.

​
 

​

​

 

 

Regression Point Displacement


First used: William Trochim and Donald Campbell 1996
Purpose: use (possibly multiple) control groups to prove a treatment’s viability through pre and post-test data 
There is one single treated group score, and multiple untreated groups for a cumulative untreated base.
RPD is commonly used when people have data collected on multiple groups and introduces treatment to one of them. The viability of the treatment is determined by its deviation from the untreated groups’ pre and post-test data.
Limits: There would be similar limits to a Randomized Control Trial regarding exposing a test group to a substance/treatment if it could be harmful. Viewed primarily as an analysis strategy rather than design 
See link for example: https://conjointly.com/kb/regression-point-displacement-analysis/

 

Overall Rating: 6.5

Flexibility Rating: 7.5


Notes: 

  • This method is highly flexible and allows for a large population base (this is in contrast with the typical RCT that struggles with larger groups of subjects due to methodological or monetary reasons).

  • Concerns with previous usage as well as current statistical problems

  • Can use it as a means of adjustment. The RPDD is characterized by four major features: 1) the use of a single treated unit instead of many; 2) the use of aggregate-level data instead of individual-level; 3) the absence of any need to assure (or attempt to achieve by statistical adjustment) pre-treatment equality between treated and control groups; and 4) the avoidance of "regression artifacts" or under adjustment due to "errors in variables" by employing the observed regression line, rather than using it as a means of adjustment. 

 

 

SOURCES

  • Trochim, William MK, and Donald T. Campbell. "The regression point displacement design for evaluating community-based pilot programs and demonstration projects." Unpublished manuscript. Retrieved from http://www. socialresearchmethods. net/research/RPD/RPD. pdf (1996).

​

​

​

​

​

​

Interrupted Time series
 

First used: ??
Purpose: compares data after implementation to that before [before to…?], commonly used where the intervention is very common and no effective control group can be found (pre and post-test data is compared to determine causality)
 

Overall Rating: 7
Flexibility Rating: 5

Limits: 
There is a very specific criteria for research done with this method. If there was an effective control group, this method would be far less of an option. This is a concern as the trial cannot adjust to as many research areas.
Notes: 

 

SOURCES

  • Clark, David, Emmeline Edwards, Peter Murray, and Helene Langevin. "Implementation Science Methodologies for Complementary and Integrative Health Research." The Journal of Alternative and Complementary Medicine 27, no. S1 (2021): S-7.

  • McDowall, David, Richard McCleary, and Bradley J. Bartos. Interrupted time series analysis. Oxford University Press, 2019.

​

​

​

bottom of page