Skip to content

Advertisement

Open Access

Introducing the CONSORT extension to pilot trials: enhancing the design, conduct and reporting of pilot or feasibility trials

  • Luciana P. F. Abbade1Email author,
  • Joelcio F. Abbade2 and
  • Lehana Thabane3, 4
Journal of Venomous Animals and Toxins including Tropical Diseases201824:4

https://doi.org/10.1186/s40409-018-0142-2

Received: 3 January 2018

Accepted: 18 January 2018

Published: 2 February 2018

Abstract

Abstract

This editorial provides a brief overview of the importance of pilot or feasibility trials or studies, the challenges with current practices in their conduct and reporting, an introduction to the Consolidated Standards of Reporting Trials (CONSORT) extension to pilot trials aimed at improving their reporting, along with some key resources on aspects related to pilot and feasibility studies.

What are pilot studies and why they are important?

A randomized controlled trial (RCT) is considered the best type of design for studying causal links between interventional exposure and outcomes in all clinical areas including treatment of tropical diseases and procedures related to toxinology. Designing and conducting RCTs is a complex undertaking, and it is frequently time-consuming and expensive. Therefore, conducting a feasibility or pilot study/trial prior to the main RCT is often highly recommended so as to enhance the likelihood of success of the main study, and thereby saving time and costs [1]. Pilot or feasibility study can be defined as a small-scale investigation designed to test the feasibility of methods and procedures for later use in the design of a large scale study [2]. Thus, by definition, the term ‘pilot’ or ‘feasibility’ implies an intention for further definitive work in the future [3].

It is important to emphasize that the primary objectives and outcomes of pilot or feasibility study and the main study are different. The pilot or feasibility trial aims to address questions about prevailing uncertainty regarding the processes or aspects of the main study, such as recruitment, randomization, retention, assessment procedures, validity of tools, new methods and implementation of an intervention. On the other hand, a definitive or main trial has a primary objective of determining the efficacy or effectiveness of a new intervention [4, 5].

Another important distinction between a pilot trial and a main RCT is that in the main study we typically use a statistically significant result based on the minimal clinically important difference (MCID) on the primary outcome of interest as the criterion for declaring success for the trial. In contrast, in a pilot trial, researchers need to establish analogous criteria for determining success of feasibility to guide decisions about moving on to the large-scale trial. Such criteria are typically based on the primary feasibility objectives and outcomes [1].

What are the current challenges with pilot studies?

Although there is an increasing number of pilot/feasibility studies in the literature, there is substantive evidence that the reporting and conduct is very suboptimal [1, 3, 6, 7]. There is, for example: too much emphasis on hypothesis testing; no clear feasibility objectives/outcomes; inadequate descriptions of analytic plans; and no explicit mention that they were actually done to inform the design of future studies [7, 8]. Some reports of pilot trials show that the study is called “pilot” because of the small sample size, or because it was conducted in single center and without the clear intention of advancing to a main study [1]. Most journals have no editorial policies on pilot studies – making it harder for reviewers and editors to make optimal and consistent editorial decisions on manuscripts reporting such studies [3, 9].

What is the CONSORT extension to pilot trials?

The CONSORT was first published in 1996 and it was last updated in 2010. It is a guideline designed to improve the transparency and quality of the reporting of RCTs, and it was based on the “standard” two-group parallel design [10]. However, there are several variations to the standard trial methodology, and nowadays there are some CONSORT extensions, for example to cluster trials, non-inferiority and equivalence trials and to pilot and feasibility studies. These extensions of main CONSORT were done to help improve the reporting of these trials.

The CONSORT 2010 statement extension to randomized pilot and feasibility trials that are conducted in advance of a future definitive RCT was published in 2016 [11]. The publication of this extension was motivated by the growing number of studies described as feasibility or pilot and by research that identified deficiencies in their reports and conduct.

As mentioned before, pilot/feasibility trials differ from other forms of RCTs in their objectives – their focus is on assessing feasibility rather than effectiveness or efficacy. Therefore, there are some key differences between the main CONSORT statement [10] and CONSORT statement extension to pilot trials [11]:
  • First, an adapted CONSORT extension to abstracts of pilot trials checklist is provided with illustration of how to use the checklist.

  • Second, the items of standard CONSORT and those of the CONSORT extension to pilot trials are put side-by-side to make it easier for the researcher to identify items that:
    • remain unchanged (items 4a; 4b; 5; 7b; 8a; 9; 10; 11a; 11b; 13b; 14a; 15; 19; 25);

    • have been modified (items 1a; 1b; 2a; 2b; 3a; 3b; 6a; 6b; 7a; 8b; 12a; 13a; 14b; 16; 17a; 18; 20; 21; 22; 23; 24);

    • were added (items 4c; 6c; 19a; 22a; 26); and

    • were deleted (12b; 17b) – because they were deemed not relevant or applicable for pilot or feasibility trials.

  • Third, the guideline also provides some explanation and elaboration for each item – to explain why each included item is important with examples of best practices, and why each excluded item is not relevant to pilot trials. Finally, it recommends an adapted flow diagram of progress of the study with the number of participants that were screened, enrollment, intervention allocation, follow up, and assessment and reporting of each feasibility outcome of the study.

The latest versions on the CONSORT statement of RCT and its extensions are available at www.consort-statement.org and the Equator Network website www.equator-network.org.

Additional resources for improving the conduct and reporting of pilot trials

Table 1 provides a list of some useful resources that researchers can explore for more information about definitions, challenges with reporting, sample size and reporting pilot studies.
Table 1

Some key references about different topics related to pilot or feasibility trials

Main topic

Key references

Definitions

[4, 8, 9]

Challenges with reporting

[1, 3, 6, 7, 1214]

Sample size estimation

[5, 15]

Reporting

[11, 16, 17]

Conclusions

Pilot or feasibility studies provide a good opportunity to assess feasibility of large full-scale evaluation studies and they can also enhance the success probability of the main study. For enhancing the transparency and quality of pilot/feasibility trials, they should be scrutinized the same way as the main trials: they should be registered, and they should be reviewed for ethics approval like all research studies. All researchers have the ethical, scientific and economic responsibility and obligation to ensure that their findings about feasibility are widely disseminated in a timely manner. Adoption, adherence and enforcement of the CONSORT extension by journals would certainly make it easier for all stakeholders including authors, journal editors, manuscript reviewers, and researchers to enhance their reporting. Many journals have endorsed the CONSORT statement and its extensions, and recommend adherence to these guidelines in their instructions to authors.

Abbreviations

MCID: 

minimal clinically important difference

RCT: 

randomized controlled trial

Declarations

Acknowledgments

Not applicable.

Funding

Not applicable.

Authors’ contributions

All authors read and approved the final manuscript.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors’ Affiliations

(1)
Department of Dermatology and Radiotherapy, Botucatu Medical School, São Paulo State University (UNESP – Univ Estadual Paulista), Botucatu, Brazil
(2)
Department of Obstetric and Gynecology, Botucatu Medical School, São Paulo State University (UNESP – Univ Estadual Paulista), Botucatu, Brazil
(3)
Biostatistics Unit, Father Sean O’Sullivan Research Centre, St Joseph’s Healthcare, Hamilton, Canada
(4)
Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Canada

References

  1. Thabane L, Ma J, Chu R, Cheng J, Ismaila A, Rios LP, et al. A tutorial on pilot studies: the what, why and how. BMC Med Res Methodol. 2010;10:1.View ArticlePubMedPubMed CentralGoogle Scholar
  2. Everitt BS. Medical statistics from a to Z: a guide for clinicians and medical students. 2nd ed. Cambridge: Cambridge University Press; 2007.Google Scholar
  3. Arain M, Campbell MJ, Cooper CL, Lancaster GA. What is a pilot or feasibility study? A review of current practice and editorial policy. BMC Med Res Methodol. 2010;10:67.View ArticlePubMedPubMed CentralGoogle Scholar
  4. Leon AC, Davis LL, Kraemer HC. The role and interpretation of pilot studies in clinical research. J Psychiatr Res. 2011;45(5):626–9.View ArticlePubMedGoogle Scholar
  5. Cocks K, Torgerson DJ. Sample size calculations for pilot randomized trials: a confidence interval approach. J Clin Epidemiol. 2013;66(2):197–201.View ArticlePubMedGoogle Scholar
  6. Shanyinde M, Pickering RM, Weatherall M. Questions asked and answered in pilot and feasibility randomized controlled trials. BMC Med Res Methodol. 2011;11:117.View ArticlePubMedPubMed CentralGoogle Scholar
  7. Lancaster GA, Dodd S, Williamson PR. Design and analysis of pilot studies: recommendations for good practice. J Eval Clin Pract. 2004;10(2):307–12.View ArticlePubMedGoogle Scholar
  8. Whitehead AL, Sully BG, Campbell MJ. Pilot and feasibility studies: is there a difference from each other and from a randomised controlled trial? Contemp Clin Trials. 2014;38(1):130–3.View ArticlePubMedGoogle Scholar
  9. Loscalzo J. Pilot trials in clinical research: of what value are they? Circulation. 2009;119(13):1694–6.View ArticlePubMedGoogle Scholar
  10. Schulz KF, Altman DG, Moher D, CONSORT Group. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. J Pharmacol Pharmacother. 2010;1(2):100–7.View ArticlePubMedPubMed CentralGoogle Scholar
  11. Eldridge SM, Chan CL, Campbell MJ, Bond CM, Hopewell S, Thabane L, et al. CONSORT 2010 statement: extension to randomised pilot and feasibility trials. Pilot Feasibility Stud. 2016;2:64. eCollection 2016View ArticlePubMedPubMed CentralGoogle Scholar
  12. Kaur N, Figueiredo S, Bouchard V, Moriello C, Mayo N. Where have all the pilot studies gone? A follow-up on 30 years of pilot studies in clinical rehabilitation. Clin Rehabil. 2017;31(9):1238–48.View ArticlePubMedPubMed CentralGoogle Scholar
  13. Kistin C, Silverstein M. Pilot studies: a critical but potentially misused component of interventional research. JAMA. 2015;314(15):1561–2.View ArticlePubMedPubMed CentralGoogle Scholar
  14. Kraemer HC, Mintz J, Noda A, Tinklenberg J, Yesavage JA. Caution regarding the use of pilot studies to guide power calculations for study proposals. Arch Gen Psychiatry. 2006;63(5):484–9.View ArticlePubMedGoogle Scholar
  15. Eldridge SM, Costelloe CE, Kahan BC, Lancaster GA, Kerry SM. How big should the pilot study for my cluster randomised trial be? Stat Methods Med Res. 2016;25(3):1039–56.View ArticlePubMedGoogle Scholar
  16. Craig P. A new CONSORT extension should improve the reporting of randomized pilot and feasibility trials. J Clin Epidemiol. 2017;84:30–2.View ArticlePubMedGoogle Scholar
  17. Eldridge SM, Lancaster GA, Campbell MJ, Thabane L, Hopewell S, Coleman CL, et al. Defining feasibility and pilot studies in preparation for randomised controlled trials: development of a conceptual framework. PLoS One. 2016;11(3):e0150205.View ArticlePubMedPubMed CentralGoogle Scholar

Copyright

© The Author(s). 2018

Advertisement