Research by Design

Jennifer Kesselheim, MD, MEd, MBE
Associate Fellowship Program Director for Education and Senior Physician at Dana-Farber Cancer Institute, Assistant Professor of Pediatrics at Harvard Medical School, Co-Director of ASH Medical Educators Institute

Educating young scientists to design successful research projects often comes down to teaching them the right questions to ask.

A high-quality educational research project begins with a well-articulated study question. The ideal research question should check the following boxes:

Is it new? The answer to your research question should provide new information. Focus on novelty, rather than recapitulating work that already has been done.

Is it important? Ask yourself whether your colleagues will find the answer to your research question interesting. If they won’t find it meaningful or if the answers won’t affect their practice, it’s not important enough to pursue.

Is it feasible? A question can be new and important, but if it can’t provide any answers, it’s pointless. Feasibility is essential, and the resources for answering the question need to be considered upfront. Do you have the funding? Will it take too much time? Do you have access to the appropriate number of participants to create meaningful results? Frequently, the question needs to be modified so that it is feasible to answer with the available resources.

Does it answer “why?” The best research explains the “why” of a finding, as opposed to the “who, what, when, and where” of the results. Often, a researcher will propose a project that, were it done, would lead people to say, “Okay, these results are interesting,” but the conversation stops there. The researcher needs to take those interesting results and extend the conversation to figure out where the answers lead – whether to a change in practice or an even more important research question.

Defining and Refining Goals and Expectations

After completing the first – and most difficult – task of a research project, the question needs to be turned into a statement of what will be tested and what outcomes are expected.

The best hypotheses are detailed, accounting for potential developments and different facets of the research question as the research project develops.

First, define the study population. Does your hypothesis address the defining characteristics of the study group? Who will be included and excluded? If the eligibility criteria leave you with a very small study sample, which sampling scheme will you use to collect data? Describing the study population also helps you answer questions about the study’s feasibility and refine your expectations.

“For young investigators, one of the major obstacles to performing a successful research project is also one of the greatest advantages: Enthusiasm.”

Jennifer Kesselheim, MD, MEd, MBE

Next, you need to think about the comparison group with respect to your hypothesis. Are you going to analyze every participant in the same way, or will you divide participants into groups to compare them with one another? If the latter, how will you divide them? What are the confounders? How can you equalize those across comparative groups?

The last step is to think about your outcomes. How will you measure them? Are you going to identify outcome measures that have already been used in published medical literature that are relevant to your study question, or do you need to design new outcome measures? What constellation of outcome measures will you need to create compelling data to answer your study question?

One Step at a Time

For young investigators, one of the major obstacles to performing a successful research project is also one of the greatest advantages: Enthusiasm. Many are so excited about the research and eager to get going that, unfortunately, they often overlook some components of a well-designed research project. Before embarking on a research project, it is worth taking the time to define and refine your goals. So, slow down, wait for the right conditions, and make sure the relevant considerations have been made.

When I began my research career in medical education, I was eager to start my research into ethics and humanism training among residents and fellows. I dove in without fully considering the feasibility and focus of the project and eventually found myself in the middle of a very complicated, not particularly well-defined research project.

Luckily, my mentor at the time stepped in and taught me not to put all of my eggs in one basket. I realized that my research project was trying to cover every aspect of ethics and humanism training.

Just as young researchers might want to hit the ground running with a research project before taking the time to fully flesh out their research questions, they may want to answer every question at once. That enthusiasm is wonderful, but, as I learned, it needs to be reined in a bit.

So, with my mentor’s advice, I took a step back and broke the central question into several smaller, more focused questions that could be answered in several more manageable projects. Doing this also helped me see my interests as a trajectory of projects down a particular line of inquiry.

For instance, to measure and potentially improve ethics and humanism education among pediatric residents and fellows, my colleagues and I performed a needs-assessment survey to determine where they needed more support in their training. We then developed a curriculum that was responsive to their perceived needs. Our next step was to initiate a pilot study in which 10 training programs administered the curriculum.

We needed an appropriate instrument to evaluate the outcomes of the curriculum, so we developed a new measure to assess fellows’ and residents’ skills related to humanistic medicine. Now we are recruiting programs to participate in the next version of the study, in which these programs will be randomly assigned to administer the new curriculum or not, and our new assessment instrument will test whether fellows and residents exposed to our curriculum actually have improved skills.

We have been performing the research for about five years and each project has built upon the previous and informed the next, leading to a stronger research hypothesis and project. Also, each one of the projects has become an opportunity for publication or to demonstrate leadership. Rather than design one big project that may never happen, it is better to march along with several manageable projects as you find your niche.

The difference between someone who has been trained in conducting research and someone who is brand new to research design is the ability to recognize and navigate the possible pitfalls of a study’s design. I was fortunate to have a mentor who helped me at every step by offering insight from his experiences – both the successes and the missteps. Much of that comes from training, but experience is even more important; the more you do it, the better you get.