In response to the need for effective clinical research training programs in hematology, the American Society of Hematology (ASH) created the Clinical Research Training Institute (CRTI) in 2003 and added a formal mentorship component (in which each participant is matched with a CRTI faculty member) in 2011. Though CRTI has conducted several types of evaluations since its inception, the focus of these evaluations has been on clinical training, not program outcomes.
Allison A. King, MD, MPH, PhD, of the Washington University School of Medicine in St. Louis, Missouri, and authors conducted a single, cross-sectional study of previous CRTI participants to determine if CRTI effectively trained – and retained – young doctors in hematology research.
The investigators collected and analyzed responses via self-reported surveys for more than 100 previous CRTI participants. The majority (over 90%) of respondents remained in academic hematology and were still involved in hematology research, highlighting the successes of the CRTI program.
“This formal mentorship program in clinical research is associated with a high retention of academic hematologists who are involved in research,” Dr. King told ASH Clinical News. “Our data further support the value of mentoring in training and retaining talented junior hematologists.”
The one-year CRTI program begins with a week-long workshop offered to approximately 20 participants and led by approximately 20 established clinical researchers, as well as representatives from funding agencies such as the National Heart, Lung, and Blood Institute and the National Cancer Institute. The workshop includes presentations of each participant’s research project, lectures, and small-group sessions. After the workshop, small groups reconvene twice a year – at the ASH annual meeting in December and at the ASH headquarters in Washington, DC, in May.
Dr. King and authors conducted three sets of surveys among previous CRTI participants:
- Cross-sectional survey: This assessment was conducted in 2014 to capture the number of grants, publications, and research involvement a participant had within the previous year. Sixteen questions were focused on demographic characteristics and the participant’s perceived influence of CRTI participation on his or her career.
- Pre- and post-summer workshop analysis: Participants reported their confidence in research immediately following the summer workshop. (Only results from the 2014 to 2015 survey were included in this analysis because the questions were consistent, while the questions from the 2009 and 2015 surveys were not consistent.)
- Evaluation of mentorship program: Participants described the perceived outcome of mentor interaction in academic productivity or career opportunities. (Results from the 2012 and 2013 surveys were evaluated because this survey was not distributed in 2014 or 2015.)
Between 2005 and 2012, the cross-sectional survey was distributed to 160 participants; 115 responded (72% response rate), but six responses did not include demographic data. Of the remaining 109 respondents, 42 percent had a focus on benign hematology, 65 percent treated adults, 48 percent were male, and 69 percent were white. At the time of program inclusion, 79 percent were fellows and 57 percent participated in 2009 or later.
After participants were given a series of statements about their experiences with CRTI, they were asked if they strongly disagreed, disagreed, agreed, or strongly agreed with the statements. Most respondents (63%) strongly agreed that CRTI facilitated their career development as an independent researcher and, compared with the other 37 percent of respondents, those who strongly agreed were more likely to have had the following: at least one grant in the previous year (55% vs. 33%; p=0.026), more published articles (median = 3 vs. 2; p=0.017), and a greater percent effort in research (50% vs. 30%; p<0.0002).
Dr. King and researchers found that male respondents reported a higher median number of published articles (median = 3 vs. 2; p=0.0092) and a higher median percent effort in research (50% vs. 40%; p=0.0029) compared with females. “This gender disparity is important to emphasize in program planning,” the authors wrote. “With 50 percent of faculty being female, many of whom are working mothers, we should encourage discussions about approaches that have been helpful to academic success from more senior female faculty.”
Responses on the pre- and post-surveys, which were distributed to 41 participants and returned by 38 (93% response rate), echoed the responses on the cross-sectional survey. After participating in the program, doctors were more likely to express feeling confident in their skills as researchers, presenters, and team leaders. For example, comparing the pre- and post-workshop responses, more doctors reported that they “strongly agreed” with the following statements:
- I am able to develop a sound, scientific hypothesis (0 vs. 85; p=0.0005).
- I feel I have a comprehension of the principles of clinical research design and execution (11 vs. 71; p<0.001).
- I am confident in my knowledge to identify the regulatory requirements of clinical research (9 vs. 49; p=0.0005).
- I feel confident in my current strategies for pursuing and developing a successful career in clinical research (6 vs. 60; p<0.0001).
- I feel confident in my skills to formulate, develop, and sustain a multidisciplinary clinical research team (9 vs. 54; p=0.0003).
- I am confident in my skills in preparing research results and presenting the work to varied audiences (9 vs. 69; p<0.0001).
The mentorship survey was distributed to 40 participants; 38 responded (95% response rate). About three-quarters (n=28; 74%) reported that “mentorship resulted in increased knowledge or skills in conduct of research.” Other benefits attributable to CRTI mentorship, including promotions and publications, are presented in TABLE.
Limitations of the study include the use of non-standardized assessments, as well as the potential for responder bias. “Respondents may have been biased to report positive feelings or gratitude for a program that was provided at no cost to them,” Dr. King said. “Given the duration of time between CRTI participation and the 2014 cross-sectional survey, we anticipate that most of the ‘honeymoon period’ positivity would have subsided.” The study also lacked a control group, and CRTI respondents may have been more destined to be successful in the field, as they were selected as the most promising applicants for the program.
Based on their findings, Dr. King and authors made several recommendations for future CRTI programs, including:
- continuing to emphasize interactions of CRTI participants (for evaluation and for “celebrations of success”) through social media and newsletters
- increasing the participation of underrepresented minorities
- standardizing the language and timing of CRTI evaluations (at intervals of 3 years after attendance at the summer didactic session)
“This evaluation supports the continued efforts to mentor and train junior academic hematologists in a structured approach,” the authors concluded, adding, however, that “it is difficult to know how much CRTI directly influenced these positive outcomes.”
King AA, Vesely SK, Elwood J, et al. The American Society of Hematology Clinical Research Training Institute is associated with high retention in academic hematology. Blood. 2016 October 26. [Epub ahead of print]
|TABLE. Benefits Observed During a One-Year Mentorship Program (n=38)|
|Mentor interaction resulted in publication.||6
|Mentor interaction resulted in a presentation.||7
|Mentor interaction resulted in a poster.||4
|Mentor interaction led to me learning new teaching method or approach.||4
|Mentor resulted in increased knowledge or skills in conduct of research.||28
|Mentor resulted in increasing new clinical knowledge.||9
|Mentor facilitated job change or promotion.||6
|Mentor facilitated involvement in a clinical trials cooperative group.||6