One of the most common challenges of randomized controlled trials (RCTs), both published and unpublished, is related to problems with recruitment. Investigators' enthusiasm for ambitious recruitment in a trial often dissipates quickly with the realization that ambitious recruitment is often misguided. This common error has been dubbed "Lasagna's Law"1 and Muench's Third Law.2 Both laws point to the same principle: investigators greatly overestimate the pool of available patients who meet the inclusion criteria.3
Insufficient or untimely patient recruitment into RCTs has serious consequences. The length of the trial may need to be extended, leading to increased resource use and costs. Lengthy trials delay the availability of potentially beneficial treatments to the public.4 The integrity and validity of the study also rely on an adequate sample size. If the sample size is not achieved, there is an increased chance of committing a type II error (e.g., you are more likely to find no difference between treatments when one actually exists). The trial may have to be abandoned, and the results may not be publishable.
The recruitment rate is influenced by both patient and investigator factors. A recent systematic review by Abraham and colleagues5 identified reasons why eligible patients may not want to participate in real or hypothetical surgical RCTs. Surgeons were also asked why they did not want to enroll eligible patients into real or hypothetical surgical trials. The top reasons for patient nonentry were that the patient had a preference for a certain therapy, he or she did not understand the trial (trial too complex), the patient did not want to be randomly assigned to a treatment and he or she feared a negative outcome or receiving a treatment that he or she felt was inferior. Investigators had similar reasons for not entering eligible patients, including difficulty following the study protocol (trial too complex) and completing the follow-up requirements, preference for a certain therapy and difficulties obtaining informed consent from patients. Understanding and addressing potential patient and investigator concerns is important when developing a recruitment strategy.
In this article, we discuss the common issues encountered in recruiting patients for surgical trials. It is intended for anyone conducting surgical trials, including medical students, residents, and junior and senior researchers. By the end of this article, readers will be able to develop strategies to avoid some of the common pitfalls in recruitment and, if these difficulties occur, to rectify them.
STUDY PROTOCOL PHASE
It is at this stage of the trial that the issue of recruitment needs to be considered and addressed carefully. This section highlights the key elements in protocol development that directly affect patient recruitment in surgical trials.
Type of trial (explanatory v. effectiveness)
Early in protocol development, investigators need to decide if their trial will be an explanatory (efficacy) or a management (effectiveness, pragmatic) trial.6 We will use the following 2 hypothetical questions to explain these concepts.
Question 1: In an academic setting and under the care of a plastic surgeon, is endoscopic carpal tunnel release (ECTR) superior to open carpal tunnel release (OCTR) in alleviating pain and nocturnal paresthesia among highly compliant patients who have confirmed carpal tunnel syndrome (CTS) and are free of diabetes and rheumatoid arthritis?
Question 2: In various settings (academic or community) and under the care of a surgeon (i.e., plastic surgeon, neurosurgeon, orthopedic surgeon), is ECTR superior to OCTR in alleviating pain and nocturnal paresthesia among patients with confirmed CTS?
Question 1 can be considered an "explanatory" or "efficacy" question, because it attempts to answer the question of whether ECTR is superior to OCTR in a highly selected population of CTS patients (patients without diabetes or rheumatoid arthritis) and under controlled conditions (in an academic setting under the care of a plastic surgeon). Even if ECTR is found to be superior to OCTR in relieving pain and nocturnal paresthesia, it is possible that surgeons may choose not to adopt ECTR. Surgeons may argue that many of their patients have diabetes or rheumatoid arthritis and are seen in various settings by different surgeons, making the results of this study inapplicable to their patients. The strict inclusion criteria in question 1 may make the trial's results irrelevant in certain settings and centres. Explanatory or efficacy questions are best used in the introduction of new surgical techniques, when we would like to know if this new technique really works under ideal conditions.
Question 2 may be viewed as a "pragmatic," "management" or "effectiveness" question because it has broad inclusion criteria. For example, the patient population includes all types of CTS patients (i.e., with diabetes, rheumatoid arthritis and other comorbidities). Also, the patients are under the care of different surgeons in various settings (i.e., academic and community). If the results show that ECTR is superior to OCTR in alleviating pain and nocturnal paresthesia, it is likely that the surgical community will adopt ECTR because the results are generalizable to the overall patient population with confirmed CTS.
It is imperative that investigators think a priori about the design of the trial they want to undertake because it will almost invariably have an effect on recruitment. It is easier to recruit patients for pragmatic trials than for explanatory trials. Practically speaking, surgical trials are difficult to classify as exclusively "explanatory" or "effectiveness" trials. They usually lie on the spectrum covering these 2 extremes.
Sample size
Integral to a trial's success is a priori analysis of power or calculation of sample size. These calculations are done to ensure that the study has sufficient power to statistically detect a difference between the groups if a difference exists. However, in clinical research, detecting a statistically significant difference between groups may not be clinically relevant. For example, consider 2 competing treatments for tibia shaft fractures. Treatment A was found to have significantly higher risk of nonunion than treatment B. Would this finding convince you to adopt treatment A? You would likely want to know how large the difference was between nonunion with treatment A and B. It is important in surgical research to ensure that trials are powered to detect the minimum clinically important difference (MCID) between interventions that would likely lead to a change in practice. The MCID should be determined from previous literature or a pilot study before beginning the trial.7 The MCID is one of the components of power analysis that will have a large effect on sample size and patient recruitment
Another important consideration when calculating sample size is to account for the number of patients who will invariably drop out of the study or be lost to follow-up. It is recommended that the trial's sample size be increased to account for loss to follow-up. Inflation of the sample size may range from 10% to 40%, depending on the circumstances. An estimation of drop-outs can be obtained from the literature, if available.
Recruitment strategies
A detailed recruitment strategy should be developed and tailored specifically to the research question. There are a number of recruitment strategies:
1. All patients are recruited all at once and start the trial simultaneously.
2. Patients enter the trial in a "batch" mode.
3. Patients are recruited continuously until the desired sample size is achieved.
4. Patients are recruited until a fixed date is reached.
When choosing a recruitment strategy, one must consider the study population and the appropriateness and cost of the recruitment method before beginning the trial. There are many different recruitment methods, including media (i.e., television, radio and newspaper), physician referrals, press releases, fliers, random mailings, cold calls and internet. Chin Feman and colleagues8 recently investigated the cost-effectiveness of recruitment methods used in a trial for irritable bowel syndrome (IBS). The authors required 289 IBS patients for their study with an original recruitment budget of $5000. They anticipated a pool of 15 000 potential IBS patients from physician referrals and a pool of 180 000 potential IBS patients in the community. Different recruitment methods were used. In total, 2149 patients were screened and 289 were enrolled. The actual amount spent on recruitment was $75 056. Recruitment was anticipated to last 4 months but actually lasted 26 months. The cost per enrolment was $584, $522, $390, $224, $92 and $12 for audiovisual media advertisements, transit advertisements, print media advertisements, fliers, internet and referrals, respectively. Referral and flyer recruitment methods were the most effective, yielding the highest number of enrolments.
Recruitment in surgical trials has been described as "unpredictable,"9 with less than 50% of eligible patients being recruited.10 This is likely because surgical research spans a broad range of patient populations, differing in the onset, complexity and severity of their conditions. For example, consider a study designed to compare 2 surgical techniques for carpal tunnel release. Carpal tunnel release is the most common procedure performed by plastic surgeons, providing a large pool of potential research participants. In general, these patients are in good health with few comorbidities. A variety of recruitment methods could be used for this population. Now consider a study designed to compare 2 surgical techniques for treating subtrochanteric hip fractures. Recruitment in this trial would be less predictable, because enrolment depends on the occurrence of hip fractures. Recruitment in trauma trials is done consecutively until the desired sample size is reached because recruitment relies solely on physician referrals. 11
Feasibility issues
Patients are unlikely to enter studies that they find difficult to understand and that require multiple follow-ups. Likewise, investigators do not want to participate in studies that are overly complex and require them to spend excessive hours on paperwork. Therefore, when designing the trial, special considerations must be made for the length and complexity of the trial from the patient and investigator perspectives. Foremost, to be able to recruit patients, it is imperative that the trial itself is ethical and can pass the requirements of the institutional research ethics board at each site. Equally important, the investigators need to clearly articulate the relevance of the study to the appropriate stakeholders: the surgical patients who will submit themselves to the rigors of the trial and the surgeons who will contribute patients to the study and, in the process, upset their usual routine and suffer some financial loss. The relevance of the study needs to emphasize the possible benefits to present and future patients, to the health care system and to society in general.
It is important at this stage to think of likely sources of patients. Will the patients be the principal investigator's patients, will they be from a colleagues' practice or will they be from distant sites? Chances are that one surgeon alone will not have enough patients and will need to explore collaboration with other surgeons.12 The plausibility and feasibility of the study should be discussed with collaborating surgeons. Initiating a pilot study or a screening study among the investigators well before the trial begins will give some indication of the accrual rate and identify the potential barriers. The objectives of the pilot study should be to assess the feasibility of the study, identify siteand investigator-specific problems, determine the adherence of the investigators and patients to the study protocol, obtain an estimate of patient follow-up and drop outs and collect preliminary data for sample size calculation of the full trial.
The investigators of BMP-2 Evaluation in Surgery for Tibial Trauma (BESTT) trial identified 5 key criteria for evaluating feasibility of an investigational site:13
1. A standard of medical care that is compatible with the protocol.
2. An infrastructure available to support proper study conduct, including willing, experienced, skilled study personnel and the appropriate facilities to complete all study evaluations.
3. Sufficient documented patient volume: typically, a site qualification survey that addresses the site's ability to recruit must be sent to the potential site. In addition, a pilot study may be conducted at the potential site to further assess patient volume.
4. The site investigators must have the ability and willingness to comply with all study procedures.
5. The site must have central study procedures.
Using the above criteria in the BESTT trial, 400 potential investigational sites across 14 countries were screened, 80 sites were visited to formally assess site feasibility, and 60 sites were selected. Reasons for the exclusion of sites included a lack of infrastructure, inadequate patient volume and unwillingness of clinicians.
STUDY CONDUCT PHASE
Once the trial is initiated, patients are screened for eligibility. Patients must meet the inclusion and exclusion criteria before informed consent can be obtained. The surgeon is often the patient's initial point of contact about the study. Therefore, it is of paramount importance that the surgeon is knowledgeable and enthusiastic about the trial. Csimma and Swiontkowski13 reported challenges in recruitment in the BESST trial. Two important lessons were learned: first, of the 60 centres initially recruited to the study, 80% of the recruitment occurred at 26 centres, and second, the most important factor associated with recruitment was an enthusiastic lead investigator at the site. The surgeon should not obtain informed consent from the patients him- or herself to ensure that patients do not feel coerced to participate; when possible, the study coordinator should meet with patients privately to obtain informed consent in an unhurried fashion. It is imperative that the study coordinator be friendly and approachable and have a thorough understanding of the study to be able to answer patients' questions. The rapport developed in this relationship can help to minimize the dropout rate.
It is important to have a trajectory graph of recruitment. The principal investigator should review recruitment at participating sites regularly to address problems or praise top recruiting sites. If the recruitment rate deviates from the expected trajectory, the graph will allow the investigator to pinpoint periods of slow recruitment. Problems with recruitment can be attributed to the protocol or trial, staff or site, surgeon or patient.
Protocol- or trial-specific recruitment issues
If recruitment is consistently slow, the problem may be related to the strict inclusion and exclusion criteria, lack of sufficient budget for sites or investigator apathy. The inclusion and exclusion criteria should be reviewed and may need to be changed to facilitate recruitment. Ethical issues can arise during the trial because of new findings about the treatment or intervention. Unexpected costs may come up that drain funding earlier than anticipated. As well, difficulties in the randomization process may be detected that make it necessary to revisit the protocol. In randomized controlled trials, a centrally located (telephone or Internet) randomization process is the best method to ensure concealed patient allocation (i.e., minimize the risk of selection bias). However, if the system fails, patients may be excluded from the trial. To minimize the possibility of excluding patients, a back-up strategy, such as keeping study envelopes at a central site for manual randomization, should be implemented.
Here are some tips for addressing protocol- or trialrelated recruitment issues:
1. Revisit the protocol for complexity and confusion in writing.
2. Re-evaluate the inclusion and exclusion criteria.
3. Increase the duration of recruitment by a fixed length of time or leave it open.
4. Increase the budget for recruiting activities.
5. Simplify the informed consent process.
Staff- and site-specific recruitment issues
There also may be staff- and site-specific issues. The study coordinator can visit the surgeon's office or clinic and find out first-hand what the problems are and try to resolve them. It may be necessary to have a general meeting with all the participating site coordinators if the problems are occurring at all sites. Frequent reminders, clarifications and encouragement are important strategies for maintaining the recruitment tempo. If repeated problems arise, the principle investigator or steering committee members (e.g., in a multicentre trial) should step in and correct the problem or ask the participating surgeon or site to terminate their participation in the trial; there is no point in putting more money into a sinking arrangement. Such drastic measures are often used as a last resort for consistently noncompliant and noncommitted sites.
Here are some tips for addressing site- or staff-related recruitment issues:
1. Recruit new investigators and new sites.
2. Replace marginally performing sites.
3. Obtain information about why recruitment is low.
4. Provide feedback.
5. Hold investigator meetings to address issues.
6. Motivate investigators by frequent communication.
7. Have regular investigator meetings to resolve the issues.
Surgeon-related recruitment issues
Problems related to surgeons need to be explored carefully. They may be because of inadequate staffing in a busy office or clinic where the trial takes a secondary role to routine office visits. If this is the case, support may be provided to these offices to ensure that recruitment takes place. If the staff are impolite to patients, this needs to be addressed, or the surgeon must be alerted. Space and time need to be provided for patients to complete the necessary questionnaires. If this is not possible, then these questionnaires can be mailed to them. If the problem is too many visits, consideration should be given to reducing the visits to only the most critical ones. It is possible that the surgeon may find the study too divergent from his or her routine care. In this case, the study procedures and paperwork should be streamlined to better fit with the surgeon's routine care. Another issue is the surgeon's beliefs and motivation for agreeing to participate in the trial. For example, despite having a strong preference for one form of treatment, a surgeon may agree to participate in a study as a favour to a colleague or because research is a requirement at his or her institution or hospital. This problem is difficult to rectify once the trial is underway. One option would be to ask the surgeon to terminate his or her participation.
Tips for dealing with surgeon-specific recruitment issues include:
1. Increase scientific or professional incentives for the investigator.
2. Use quotas to measure progress (multiple investigators, multicentre trials). For example, in an ongoing randomized controlled trial comparing 2 surgical techniques for breast reduction, the first author (A.T.) required that each coinvestigator contribute a minimum of 20 patients with complete follow-up (1 year) to be considered as a coauthor of any publications.
Patient-related recruitment issues
There are also patient-related problems. The trial may be inconvenient for patients because of an inability to take time off work to attend follow-up visits, difficult or expensive parking, or problems obtaining a babysitter. The trial coordinator should recognize these issues and relax the appointment visits to accommodate the patient's burden or even offer to pay for parking fees. Monetary incentives such as cash or entry into a prize draw have been found to sig - nifi cantly improve patient recruitment in survey studies.14 However, there are ethical concerns when using monetary incentives. Large financial incentives have been thought to interfere with the process of informed consent by altering patients' decision-making process,15 and patients may overlook the risks. If monetary incentives are found to be ethically appropriate, the incentives should be given at study completion to prevent or reduce early withdrawal from the trial. Paying patients before participation may encourage withdrawal from the study. Determining the appropriate payment model relies on funding availability.
Here are some tips for minimizing patient-related recruitment issues:
1. Determine factors that encourage patients to enter studies.
2. Provide written information to patients.
3. Educate patients about the trial before asking for their consent. Ensure that the patients are well informed about the trial.
4. Take the time to answer their questions.
STUDY FOLLOW-UP PERIOD
To avoid difficulties with loss to follow-up, it is important to exclude patients from the trial who are unlikely to comply with the requirements of the trial. Examples include individuals who may be unable to complete the outcome assessments, those who intend to move to another city, those who have no fixed address and those who profess genuine uncertainty as to whether they should participate in the study.16
Patient follow-up needs to be carefully monitored and documented to identify any patterns. Patients are unlikely to comply with follow-up if they find it too burdensome. The study should provide follow-up visits that coincide with routine visits to the office or clinic and facilitate patients' preferences. During these visits, the study coordinator should be present to answer any questions or concerns that participants may have. The office staff or study coordinator should provide reminders about subsequent visits and attempt to minimize the time patients spend in the clinic. If patients are lost, then the clinical centre should be notified, and a great effort should be made to locate the participant. To facilitate this, contact information for a next of kin should be obtained at enrolment. If the problem is the burden of visits, then the frequency of the visits should be renegotiated.
CONCLUSION
Surgeons planning to conduct clinical research need to consider the issues of patient recruitment ahead of time and plan different strategies to minimize and avoid these potential difficulties at different stages of their study. We have compiled a list of key considerations to help surgeons and surgical researchers overcome common pitfalls of patient recruitment to help them complete their study in a timely manner and reduce unnecessary costs (Box 1).
Competing interests: No funding was received for the preparation of this paper. Dr. Bhandari was funded, in part, by a Canada Research Chair, McMaster University, Hamilton, Ont.
[Sidebar]
Box 1. Tips to avoid or minimize recruitment issues at different stages of surgical research studies
Study protocol phase
* Achieve an adequate sample size
* Know the patient population and the likely sources of patients
* Simplify the study protocol
Study conduct phase
* Re-evaluate the inclusion and exclusion criteria if recruitment is low
* Identify sites with consistently low recruitment and address the sitespecific problems. Add new investigators and sites if necessary
* Set recruitment quotas and provide incentives to maintain investigator interest
* Spend adequate time with patients and answer any questions they have about the study
Study follow-up period
* Exclude patients who are unlikely to comply with the required follow-up
* Schedule follow-up visits to coincide with routine visits to the office or clinic and facilitate patients' preferences
* Make every effort to locate lost patients
[Reference]
References
1. Lasagna L. Problems in publication of clinical trial methodology. Clin Pharmacol Ther 1979;25:751-3.
2. Bearman JE, Loewenson RB, Gullen WH. Muench's postulates, laws and corollaries, or biometrician's views on clinical studies (Biometric Note 4). Bethesda (MD): Office of Biometry and Epidemiology, National Eye Institute, National Institutes of Health; 1974.
3. Spilker B, Cramer J. Patient recruitment in clinical trials. New York (NY): Raven Press Ltd.; 1992.
4. Watson JM, Torgerson DJ. Increasing recruitment to randomised trials: a review of randomised controlled trials. BMC Med Res Methodol 2006;6:34.
5. Abraham NS, Young JM, Solomon MJ. A systematic review of reasons for nonentry of eligible patients into surgical randomized controlled trials. Surgery 2006;139:469-83.
6. Schwartz D, Lellouch J. Explanatory and pragmatic attitudes in therapeutical trials. J Chronic Dis 1967;20:637-48.
7. Cadeddu M, Farrokhyar F, Thoma A, et al. Users' guide to the surgical literature: how to assess power and sample size. Laparoscopic versus open appendectomy. Can J Surg 2008;51:476-82.
8. Chin Feman SP, Nguyen LT, Quilty MT, et al. Effectiveness of recruitment in clinical trials: an analysis of methods used in a trial for irritable bowel syndrome patients. Contemp Clin Trials 2008;29:241-51.
9. Jack WJ, Chetty U, Rodger A. Recruitment to a prospective breast conservation trial: Why are so few patients randomised? BMJ 1990;301:83-5.
10. Allen PJ, Stojadinovic A, Shriver CD, et al. Contributions from surgeons to clinical trials and research on the management of soft tissue sarcoma. Ann Surg Oncol 1998;5:437-41.
11. Simunovic N, Devereaux PJ, Bhandari M. Design considerations for randomised trials in orthopaedic fracture surgery. Injury 2008;39:696-704.
12. Thoma A. Challenges in creating a good randomized controlled trial in hand surgery. Clin Plast Surg 2005;32:563-73.
13. Csimma C, Swiontkowski MF. Large clinical trials in musculoskeletal trauma: Are they possible? Lessons learned from the international study of the use of rhBMP-2 in open tibial fractures. J Bone Joint Surg Am 2005;87:218-22.
14. Martinson BC, Lazovich D, Lando HA, et al. Effectiveness of monetary incentives for recruiting adolescents to an intervention trial to reduce smoking. Prev Med 2000;31:706-13.
15. Resnick DB. Increasing the amount of payment to research subjects. J Med Ethics 2008;34:e14.
16. Sprague S, Leece P, Bhandari M, et al. Limiting loss to follow-up in a multicenter randomized trial in orthopedic surgery. Control Clin Trials 2003;24:719-25.
[Author Affiliation]
Achilleas Thoma, MD, MSc*[dagger][double dagger]
Forough Farrokhyar, MPhil, PhD[dagger][double dagger]�
Leslie McKnight, MSc�
Mohit Bhandari, MD, MSc[double dagger]�
From the *Division of Plastic and Reconstructive Surgery, Department of Surgery, St. Joseph's Healthcare, the [dagger]Surgical Outcomes Research Centre, the Departments of [double dagger]Clinical Epidemiology and Biostatistics and �Surgery, and the �Division of Orthopaedic Surgery, McMaster University, Hamilton, Ont.
Accepted for publication
Jan. 27, 2009
Correspondence to:
Dr. A. Thoma
101-206 James St. S.
Hamilton ON L8P 3A9
fax 905 523-0229
athoma@mcmaster.ca
