MOC Machinations

The changing landscape of MOC Knowledge Check-Ins and how the American Society of Hematology continues pursuing a pragmatic approach to the MOC process. 

The past five years have been turbulent for the American Board of Internal Medicine (ABIM), its Maintenance of Certification (MOC) program, and the health-care practitioners who participate in its certification activities.

In 1990, ABIM eliminated its “board-certified-for-life” status (although physicians certified in 1989 or earlier were grandfathered in). Next, it launched a series of changes to its every-10-year recertification program that ultimately morphed to include mandatory participation in periodic, ongoing MOC activities. Then, in 2014, ABIM announced that it also would start reporting on the MOC status of those who held lifetime certificates.1 This decision, which capped off 24 years of changes, enraged many diplomates.

In response to frustrations voiced by the medical community, ABIM and ABIM Foundation President and Chief Executive Officer Richard J. Baron, MD, issued an apology: “ABIM clearly got it wrong. We launched programs that weren’t ready and we didn’t deliver an MOC program that physicians found meaningful. We want to change that.”2

The next phase of ABIM’s MOC program was set into motion as the organization recommitted to providing valuable MOC activities and decreasing the burden on physicians’ time and resources. ABIM began to consider providing alternatives to the every-10-year, high-stakes secure examination (known as “MOC Part 3”); like shorter, lower-stakes, but more frequent assessments of knowledge.2

Open-book tests, shorter examinations, and more user-friendly assessments are welcome changes, but for almost two decades, medical specialty societies like the American Society of Hematology (ASH) have questioned the underlying premise of MOC programs, citing a lack of evidence that mandatory summative assessments (or an assessment used to evaluate learning at the end of a defined period of time) are practice-changing or beneficial to patients.

“In general, the concept of ensuring that physicians who care for patients continually educate themselves and learn in a changing world of medicine is a good one,” said Joseph Mikhael, MD, MEd, chief medical officer of the International Myeloma Foundation and professor at City of Hope Cancer Center in Duarte, California. “The challenge has been that the models by which this is conducted do not reflect the way in which we learn in medicine,” said Dr. Mikhael, who also serves as a councillor on ASH’s executive committee.

ASH Clinical News recently spoke with representatives from ASH and ABIM to gain insight into the new MOC requirements for hematologists and to determine whether a more user-friendly MOC program will translate to better patient outcomes.

What’s Changed

The most substantial change to ABIM’s MOC program happened earlier this year: Starting in 2018, the organization launched its “Knowledge Check-In” assessment options. These exams can be taken at home, work, or a testing center; will last about two to three hours; will provide results and feedback more quickly than the traditional exams; and are open book using the UpToDate clinical decision support resource.3 Knowledge Check-Ins are now available for the specialties of internal medicine and nephrology, and the tentative rollout schedule anticipates expansion to all specialties by 2020 (see TABLE 1).

“The two-year assessments that ABIM is offering are breadth of field, across the whole discipline,” Dr. Baron explained to ASH Clinical News. “We want to offer people an opportunity to take a less terrifying assessment that they could do at home or in the office, but would still be a valid, credible assessment.”

This two-year assessment will be available for the hematology specialty starting in spring 2019. Two test dates will be available in the spring (April 23 or 27) and two in the fall (October 15 or 19). See TABLE 2 for more information about upcoming testing dates.

ABIM also is exploring ways to assess subsets of knowledge relevant to physicians’ practice – a common request from MOC participants.3 For instance, as part of its “principled but pragmatic” approach to improving MOC for hematologists, ASH is in the process of negotiating with ABIM to develop “focused” Knowledge Check-Ins targeting specific subsets of hematology practice, such as general hematology, nonmalignant hematology, and malignant hematology. More information about these proposed programs will be announced later in 2018.

While the program gets off the ground, ABIM has decided to make the first year that a Knowledge Check-In is offered a “no-consequence” year, meaning physicians who do not pass will be able to retake the exam in the next two years without a change to certification status – a policy for which ASH strongly advocated.

A recent announcement from the American Society of Clinical Oncology (ASCO), however, added a layer of confusion by introducing yet another format into the mix: In spring 2018, ASCO and ABIM announced a partnership to create shorter, two-year assessments that will replace ABIM’s two-year Knowledge Check-In for Medical Oncology, which was originally planned to roll out in 2020.4 The organizations are co-developing content for the assessments in hopes of creating a test that “keeps pace with rapidly evolving cancer science, research, and oncology practice,” according to an ASCO press release announcing the new pathways. Oncologists will not need an ASCO membership to take the shorter, two-year assessment pathway.

Similarly, the American College of Physicians and the American College of Cardiology (ACC) have partnered with ABIM to develop collaborative pathways through which physicians can maintain board certification. For instance, in the field of cardiology, the ACC would provide clinicians with learning material and assessments modeled after its lifelong learning self-assessment program.5

Much of the recent MOC evolution has been in response to feedback from medical societies representing diplomates. “ABIM sent inquires to diplomates, asking them to choose what they believe are the most important and most commonly seen topics in practice,” said Marc S. Zumberg, MD, professor and section chief of non-malignant hematology at the University of Florida Health. “For rare topics, they could then grade their importance.”

Based on feedback to these surveys, which ASH encouraged its membership to participate in, the blueprints for exams have been updated to be more relevant to physicians’ day-to-day practice, explained Dr. Zumberg, who is a member of the ABIM Hematology Board and former chair of ASH’s MOC Working Group. “In fact, in the hematology MOC, the pass rate has gone up from 79 percent in 2015, to 90 percent in 2016, to 95 percent in 2017,” he said.6 “Either we are doing a better job of testing what is relevant to hematologists or we are educating hematologists better.”

A Learning Process

ASH favors a system that offers periodic, low-stakes, formative assessments that facilitate lifelong learning by helping identify knowledge gaps and providing ongoing feedback. As part of this goal, the society is engaged in open and candid discussions with ABIM regarding the future of the hematology Knowledge Check-Ins that will be rolled out next year, particularly in making them less cumbersome and daunting.

“The idea of forcing people to take exams is not in concordance with adult learning theory,” said Alan Lichtin, MD, a hematologist at Cleveland Clinic and chair of ASH’s MOC Working Group.

Adult learning theory, or andragogy, postulates that adults learn and assimilate new information through self-directed learning and from internal rather than external factors.7These theories, to a degree, run counter to the idea of mandatory, periodic, summative assessments. But, according to Dr. Baron, ABIM is committed to this idea: “Our view is that having summative assessments over the course of a career to assess individual skills and knowledge is important. We think it adds value.”

To support this idea, Dr. Baron pointed to recently published research that found that maintaining certification was positively associated with physician performance scores on Healthcare Effectiveness Data and Information Set (HEDIS) measures.8

The researchers compared annual HEDIS scores between more than 1,000 internists who were initially certified in 1991 but who did (n=786) or did not (n=474) maintain certification 20 years later. Those who were consistently meeting MOC requirements were slightly more likely to have better scores on a set of performance measures for mammography screening, diabetes, and cardiac disease care. However, the analysis was potentially confounded by several factors, like unobserved patient, physician, and practice characteristics and the inability to determine the clinical significance of observed differences in HEDIS score.8

Other studies have found that ABIM certification is associated with a lower likelihood of state medical licensure disciplinary actions, greater adherence to diabetes management guidelines, and $5 billion lower annual costs for Medicare.9-11

However, in Dr. Lichtin’s view, there is not much evidence that MOC aids in learning, changes patient outcomes, or provides value. He also identified another confounding factor: “The literature is mostly from people involved with ABIM. We are submitting ourselves to a testing process that is not proven to demonstrate that people taking the test are truly learning.”

This is particularly concerning for the specialty of hematology, which focuses on the care of patients with rare diseases. “Patients do not seek hematologists out or go see them because they have maintained certification, but for their expertise,” Dr. Lichtin said.

In response, Dr. Baron argued that if physicians are very specialized and engaging only in self-directed learning, those activities might not result in much learning at all. There would be no assessment to see what, if anything, had been learned.

“If there are no consequences to an assessment, people will not focus on it and they won’t gain benefit,” Dr. Baron said. “Physicians wouldn’t be able to say, ‘Hey, here are some areas I know very well and here are some that I may not know as well.’”

Where Do Things Stand?

In the years since ABIM’s initial MOC shake-up, several state legislatures have attempted to uncouple MOC from hospital insurance contracts. States are considering or have passed legislation that would protect physicians who do not fulfill ABIM or other board MOC requirements by prohibiting health plans, hospitals, or licensing agencies from requiring the certification.

For example, Oklahoma recently passed a law that prohibits discrimination based on whether a physician has maintained board certification.12 Effective in November 2016, the legislation prohibited the Oklahoma Allopathic Medical and Surgical Licensure and Supervision Act from mandating a physician to secure MOC as a condition of licensure, reimbursement, employment, or admitting privileges at a hospital in the study.

Dr. Baron noted that the certification has never been required for licensure in any state, though ABIM does require licensure to achieve certification.

“That is a common misconception,” he said. “About 80 percent of licensed physicians are board-certified and 20 percent are not. Individual health-care institutions make the decision about who gets to do what in their hospital or health system. They are responsible for making those decisions, with patient welfare in mind.”

Dr. Lichtin said he is distressed that the process has entered the political sphere. “I like the idea that doctors can police themselves,” he said. “I don’t think society is going to be better served living in a state where the state legislature has determined the certification process.”

But, he acknowledged that he, and all hematologists, have to work within the existing MOC environment. “If you are up for recertification, take the 10-year exam or the every-two-year exam to prove that you can maintain your certification,” Dr. Lichtin said. “That is what we have to do because that is the landscape right now.”

The MOC landscape might shift again in the near future, and ASH will continue to work with the ABIM Hematology Board (of whom the majority of members are also members of ASH) as it explores the feasibility of developing focused modules for the Hematology Knowledge Check-Ins.

“The MOC process is a way to assure society that doctors are up to date, but it has been the subject of much anxiety and controversy,” Dr. Lichtin said. “I hope there comes a time when hematologists and ABIM can reach a mutual understanding of how to prove that we are continuously learning.”—By Leah Lawrence

References

  1. American Board of Internal Medicine. Assessment 2020 Task Force findings to focus discussion of changes to ABIM certification and maintenance of certification. Accessed June 22, 2018, from http://assessment2020.abim.org/final-report.
  2. American Board of Internal Medicine. ABIM announces immediate changes to MOC program. Accessed June 22, 2018, from www.abim.org/news/abim-announces-immediate-changes-to-moc-program.aspx.
  3. American Board of Internal Medicine. MOC assessments in 2018. Accessed June 22, 2018, from https://www.abim.org/maintenance-of-certification/moc-faq/moc-assessments.aspx.
  4. Hudis CA. Changes to MOC: new pathways for assessment. ASCO Connection. Accessed June 8, 2018, from https://connection.asco.org/node/83021.
  5. American College of Cardiology. Internal medicine organizations to explore new options for physicians maintaining board certification. Accessed June 29, 2018, from http://www.acc.org/latest-in-cardiology/articles/2017/09/07/14/52/internal-medicine-organizations-to-explore-new-options-for-physicians-maintaining-board-certification.
  6. American Board of Internal Medicine. First-time taker pass rates – maintenance of certification. Maintenance of certification exam pass rates 2008-2017. Accessed June 8, 2018, from https://www.abim.org/~/media/ABIM%20Public/Files/pdf/statistics-data/maintenance-of-certification-pass-rates.pdf.
  7. Teaching Excellence in Adult Literacy (TEAL). TEAL center fact sheet no. 11: adult learning theories. Accessed June 9, 2018, from https://lincs.ed.gov/sites/default/files/11_%20TEAL_Adult_Learning_Theory.pdf.
  8. Gray B, Vandergrift J, Landon B, et al. Associations between American Board of Internal Medicine Maintenance of Certification status and performance on a set of healthcare effectiveness data and information set process measures. Ann Intern Med. 2018 June 12. [Epub ahead of print]
  9. McDonald FS, Duhigg LM, Arnold GK, et al. The American Board of Internal Medicine Maintenance of Certification examination and state medical board disciplinary actions: a population cohort study. J Gen Intern Med. 2018 March 7. [Epub ahead of print]
  10. Lipner RS, Young A, Chaudhry HJ, et al. Specialty certification status, performance ratings, and disciplinary actions of internal medicine residents. Acad Med. 2016;91:376-81.
  11. Gray BM, Vandergrift JL, Johnston MM, et al. Association between imposition of a Maintenance of Certification requirement and ambulatory care-sensitive hospitalizations and health care costs. JAMA. 2014;312:2348-57.
  12. Oklahoma Text Search and Retrieval System. Enrolled Senate Bill No. 1148. Accessed June 22, 2018, from http://webserver1.lsb.state.ok.us/cf_pdf/2015-16%20ENR/SB/SB1148%20ENR.PDF.

The ASH Self-Assessment Program

The American Society of Hematology Self-Assessment Program (ASH-SAP) encompasses both adult and pediatric hematology, and both malignant and non-malignant disorders.

The most recent edition includes 23 updated chapters, newly rendered illustrations, and nearly 300 questions. It is designed for self-directed learning and review with case studies, key points, graphics, animation, audio, and glossaries.

To learn more about ASH-SAP and to take advantage of the new online-only features, including animation and audio, as added enhancements to the Sixth Edition by visiting hematology.org/ashsap.

Exploring Alternative Pathways

To address medical societies’ and health-care providers’ concerns with its MOC program, ABIM could look to alternative models being explored by other certifying boards. Despite having access to the same data about physician self-assessment that ABIM does, these organizations reached markedly different conclusions about the need for summative assessments or punitive measures. All of these include periodic, low-stakes, formative assessments that ASH advocates.

The American Board of Anesthesiology (ABA) was a pioneer in creating alternative pathways: In 2014, it launched its MOC in Anesthesiology Minute (MOCA Minute) program, an interactive platform designed to replace MOC Part 3.11 MOCA Minute was originally designed as a tool to help ABA diplomates prepare for certification exams, but eventually replaced part of the MOCA examination. The assessment allows diplomates to assess their knowledge continuously, fill knowledge gaps, and demonstrate proficiency by answering 30 questions per quarter.

MOCA Minute questions can be answered over time and, in fact, diplomates cannot answer more than 30 questions per calendar quarter. After each question, the diplomate receives the rationale for the correct answer and links for more information. If the MOCA Minute performance standard is not acquired in the year certification expires, a proctored exam is required to retain certification.

Similarly, the American Board of Pediatrics (ABP) has developed a Maintenance of Certification Assessment for Pediatrics (MOCA-Peds), an online, non-proctored assessment platform.12 Each quarter, ABP diplomates receive 20 timed, multiple-choice questions that they can answer from their personal computers. Once the questions are complete, the respondent receives immediate feedback on correct and incorrect responses, as well as rationale and references for the correct responses. If diplomates do not pass the MOCA-Peds by the end of the fourth year of ABP’s five-year MOC cycle, they must sit for a proctored exam.

Earlier this year, around the same time that ASCO and ABIM announced their new collaborative pathways, the American Board of Medical Specialties voted to make the American Board of Obstetrics and Gynecology (ABOG) pilot literature-based assessment program a permanent option for its diplomates.3 This option measures physicians’ knowledge through the ABOG’s Lifelong Learning and Self-Assessment (LLSA) component, which requires participants to read a total of 180 selected articles and answer 720 corresponding questions over a six-year period.

If physicians perform well on the LLSA component through years one through six of a 10-year MOC cycle, they can receive credit for meeting the MOC external assessment requirement. While diplomates must also continue to meet other MOC standards, the LLSA program will help physicians “realize savings by not paying the external assessment fees … and not losing a full day away from their practices and patients in order to fulfill the external assessment requirement,” according to an ABOG press release.3

These are all examples of formative evaluation, where the intent is to monitor learning and provide ongoing feedback, said Alan Lichtin, MD, a hematologist at Cleveland Clinic and chair of ASH’s MOC Working Group.

Joseph Mikhael, MD, MEd, chief medical officer of the International Myeloma Foundation and professor at City of Hope Cancer Center in Duarte, California, also cited the Canadian MOC program (through which he maintains certification) as an example of formative learning. The program requires participants to complete a certain number of activities to earn credits in each five-year cycle. These activities reflect the type of work the physician does, Dr. Mikhael said, and can be earned in a variety of manners including attending CME lectures, searching the literature, and participating in learning activities.

“There is a component of trust in the Canadian program, although people can be audited,” he said. “Because it is not exam-based, it more accurately reflects what I do on a regular basis and is much more clinically relevant.”

References

  1. The American Board of Anesthesiology. Part 3: MOCA Minute. Accessed June 9, 2018, from http://www.theaba.org/MOCA/MOCA-Minute.
  2. The American Board of Pediatrics. MOCA-Peds. Accessed June 22, 2018, from https://www.abp.org/mocapeds.
  3. American Board of Obstetrics and Gynecology. From pilot to permanent: ABOG’s program offering an innovative pathway integrating lifelong learning and self-assessment and external assessment is approved. Accessed June 29, 2018, from https://www.prnewswire.com/news-releases/from-pilot-to-permanent-abogs-program-offering-an-innovative-pathway-integrating-lifelong-learning-and-self-assessment-and-external-assessment-is-approved-300648728.html.