Maintenance of Credibility

Two years after changing its MOC requirements, the ABIM is still working to address concerns about the program.

The American Board of Internal Medicine’s (ABIM) Maintenance of Certification (MOC) program was back in the news this fall when it released the final report of its Assessment 2020 Task Force, “A Vision for Certification in Internal Medicine in 2020.”1

The report evaluated ABIM’s current MOC program and provided a list of recommendations for future iterations of the certification and recertification processes, including a list of competencies physicians should possess and the best way to assess them.

The task force responsible for the report was convened in 2013 and comprises ABIM leadership and experts in assessment, education, health care, and consumer advocacy. Their goal, as stated in the report, was to “develop a vision for the future of assessment in internal medicine and associated subspecialties,” making several key recommendations designed to increase the value of MOC to physicians while decreasing the burden on their time and resources.1

Notably, before the most recent controversy about ABIM’s updates to its MOC program started, the organization had already assembled its Assessment 2020 Task Force as part of an ongoing effort by ABIM to find ways to evolve and improve, according to Harlan Krumholz, MD, chair of the Assessment 2020 Task Force and professor of medicine at Yale School of Medicine in New Haven, Connecticut. “An important facet of the development was the effort to include many voices and perspectives. We had a public website, sought public input, studied public comments,” Dr. Krumholz said. “We also were committed to learning as much as possible about the evolving science of assessment – and determining what was working best in many other fields that are engaged in these activities.”

However, this newest announcement has only further complicated the matter, exacerbating the sense of uncertainty and frustration many hematologists have with the MOC program, according to Marc J. Kahn, MD, MBA, Peterman-Prosser professor of medicine in the Section of Hematology/Medical Oncology at Tulane University School of Medicine in New Orleans, Louisiana, and chair of the ASH Maintenance of Certification Working Group, whose comments in the article reflect his own views.

“People are confused about which requirements remain and which do not,” Dr. Kahn said.

ASH Clinical News recently spoke with Dr. Kahn, Dr. Krumholz, and other physicians involved with ABIM about the results of the Assessment 2020 Task Force, the current status of MOC, and what the future may hold.

A History of Change

The first big change to ABIM’s certification program came 25 years ago when it did away with its “board-certified for life” status, which allowed physicians certified in or before 1990 to maintain certification without completing any additional requirements over their career.

“In 1990, the ABIM decided that lifetime certifications didn’t make sense,” Dr. Kahn explained, given the evolving nature of medicine and the pace of new scientific discovery. “They had an obligation to make sure people were keeping up.”

After 1990, diplomates wanting to maintain their certification had to complete a three-part program: maintain licensure (Part 1), complete knowledge-based self-assessments (Part 2), and sit for secure exams every 10 years (Part 3).

More recently, the ABIM announced that, as of January 1, 2014, all of its diplomates, including those considered “grandfathered-in” were required to actively participate in MOC activities to be listed as “ABIM certified, meeting MOC requirements” on the publicly available directory of physicians on the ABIM website.

With the revised program, ABIM ramped up certification requirements: Diplomates had to complete at least one MOC activity every two years, earn 100 MOC points – including 20 in medical knowledge – every five years, and pass the MOC exam for certification every 10 years. The 2014 changes also expanded Part 4 of its exam, requiring diplomates to earn MOC points in the areas of Patient Voice and Patient Safety, in addition to the Practice Assessment component.

“The announcement about Part 4 requirements is when the medical community went berserk,” Dr. Kahn said. “Physicians were being asked to do things that they already do every day at their own institutions. Many saw this as nothing more than busy work.”

In the wake of these announcements, several online petitions were launched calling for ABIM to recall its new MOC requirements, with tens of thousands of physicians signing in agreement. Many large professional medical societies, including ASH, also voiced concern about the new requirements.

In February 2015, Richard J. Baron, MD, ABIM president and chief executive officer, issued an apology to the organization’s diplomates, stating: “ABIM clearly got it wrong. We launched programs that weren’t ready and we didn’t deliver an MOC program that physicians found meaningful. We want to change that.”2

In response to the wave of backlash it experienced from ASH and other groups, ABIM took the following actions:

  • Suspending Part 4 of the MOC program
  • Modifying the public reporting language of diplomate status
  • Freezing MOC enrollment fees at the 2014 rates

The organization also promised new ways for internists to demonstrate self-assessment of medical knowledge and an update of its internal medicine MOC exam to more accurately reflect physicians’ real-world practice.

According to a statement from ASH President David A. Williams, MD, this action by the ABIM marked “a critical step toward addressing concerns voiced by ASH.”3

“The ABIM heard the concerns and frustrations voiced by the medical community, and we are working in good faith to address them,” Michael E. Williams, MD, told ASH Clinical News. Dr. Williams is chief of the ABIM Hematology Board and Byrd S. Leavell professor of medicine and professor of pathology at University of Virginia School of Medicine in Charlottesville, Virginia. “ABIM is seeking to make this process a more valuable one, while maintaining the credibility and value of the credential for our members.”

The Task at Hand

What changes does the Assessment 2020 Task Force recommend for the future? Their final report included three key items:1

  • Replacing the 10-year MOC exam with assessments that are more frequent and less burdensome
  • Focusing assessments on cognitive and technical skills
  • Exploring the need for certification in specialized areas, without the requirement to maintain underlying certifications, while being transparent about specialization to the public

According to Dr. Krumholz, the Assessment 2020 report contains “clear, crisp” recommendations that could bring assessment into a new era, transforming it into an effective, efficient system of evaluation that has a low burden on physicians and a high value for society.

“It leverages modern learning and assessment methods and reduces time spent in activities that may not directly improve performance,” Dr. Krumholz said. “The report is intended to stimulate public dialogue about how best to advance our profession’s methods of assessment.”

He was quick to clarify that the report is advisory, rather than binding. “The intent was for there to be much independent review and comment once the report was released.”

From Recommendation to Action

“The fact that ABIM wants to re-examine the mandated 10-year secure exam and recognize the importance of sub-specialization is encouraging, but the devil is in the details,” Dr. Kahn said. Right now, however, those details are missing. “Conceptually, it is all hard to argue with, but without details it is hard to put together what it will all mean.”

The prospect of replacing the 10-year secure exam is the most exciting aspect of the Task Force’s report, according to Dr. Kahn, because it is viewed as burdensome by many physicians.
“I write secure exams, and I like multiple-choice tests, but I acknowledge that they are not the best way to teach adults,” Dr. Kahn said. “The main problem with the secure exam is that if someone gets a question wrong, there is no feedback.”

The last thing that patients or third-party payers want is physicians trying to “guess” the right answer to a clinical question without any outside help, he said. This form of assessment does not represent real-world practice. In reality, if a physician does not know the answer to something, he or she can do some research or ask colleagues for help – learning throughout the process.

Dr. M. Williams agreed that many physicians find the 10-year exam taxing.

“Preparing for the exam is a very high-pressure process,” he said. “Physicians worry that failing the exam will have repercussions for privileges at hospitals and will likely require taking the exam over again.”

The stress of potential failure and loss of certification is a very real one for physicians sub-specializing in hematology. Both Drs. Kahn and M. Williams pointed out that the pass rate for the hematology exam is somewhat lower than that seen with other specialties.

The pass rate on the first attempt at the hematology recertification exam in the past four years has ranged between 80 and 87 percent. For other specialties, though, the pass rate typically ranges between 85 and 94 percent.4

“The pass rates are essentially telling us that one out of every five or six practicing hematologists does not have the knowledge base that a group of experts deemed necessary in order to be board-certified in hematology,” Dr. Kahn said. “That is too high. ABIM is using an expert panel, rather than using practitioners as a validity panel.”

The large portion of hematologists who practice in very specialized areas (such as coagulation or a specific hematologic malignancy) are also frustrated by the breadth of the knowledge tested by the 10-year exam, Dr. M. Williams said.

“Having to prepare for a general exam in areas that they do not practice in their day-to-day clinical effort is seen as a waste of their time,” he said. “It is also not reflective of a sub-specialist’s great expertise within a narrow focus.”

And, as Dr. Kahn pointed out, that is just one specialty area. Many hematologists are seeking to maintain certification in multiple specialties: internal medicine, hematology, and occasionally oncology. That means preparing – and paying for – three separate exams.

“In practice, people become very focused, which is a good thing,” Dr. Kahn said. “ABIM needs to understand that narrow specialization helps the field; it doesn’t hurt it.”

Examination Alternatives

What are the alternatives to the standard secure exam procedure? One proposal would involve physicians having to complete three modules within a five-year period, rather than a massive, once-every-10-year exam, Dr. M. Williams said.

“For example, a hematologist specializing in hemostasis and thrombosis would complete two modules in that area and perhaps one module in general hematology,” Dr. M. Williams explained. “It is important for sub-specialists board-certified in hematology to maintain some knowledge and awareness of the field in general and to keep abreast of the many advances in the discipline.”

Another alternative testing model that has gotten some attention recently is the American Board of Anesthesiology’s (ABA’s) Maintenance of Certification in Anesthesiology (MOCA) program, MOCA Minute. MOCA Minute was originally designed as a tool for helping ABA diplomates prepare for certification exams, but will now actually replace part of the MOCA examination.5

In this program, ABA diplomates are given 120 questions per calendar year (a maximum of 30 questions per quarter), as opposed to a 10-year exam. Once a question is accessed in the Web-based application, ABA diplomates have one minute to answer the question, and, whether or not the question is answered correctly, the test-taker can view the rationale explaining the correct response and links to supporting information. In addition, questions will occasionally cover similar material in order to gauge knowledge retention over time.

“It’s hard to tell if people would view getting a question every week as more or less burdensome over the long haul,” Dr. M. Williams said. “Many hematologists have commented that they are fine taking an exam once every 10 years, while others would rather do something more frequently. It’s a difference of opinion that we have to take into account when designing an MOC program for thousands of physicians.”

Maintenance of Advocacy

Although the recommendations coming from the Assessment 2020 Task Force seem to be a move in the right direction, some physicians and professional societies, including ASH (see SIDEBAR), continue to have reservations about the MOC program.

One criticism that ABIM addressed in its 2015 apology for the 2014 changes: the high cost of the program’s fees and exams. In 2014, the program fee for certification in internal medicine was $1,940 for 10 years, or an annual fee of $194. This fee covered one internal medicine recertification exam, access to ABIM’s self-assessment products, and the cost of CME credit claimed from the use of these modules.6 Physicians seeking sub-specialty recertification are charged with a $2,560 fee ($256 per year); if more than one recertification was required, physicians pay the full fee for the most expensive certification and half the fee for any subsequent certifications.

These fees, however, do not account for the cost of any other activities required to accrue MOC points.

Broken down into annual fees, these costs seem reasonable, but, according to a recent cost-analysis published in Annals of Internal Medicine, the actual costs of participating in ABIM’s MOC program may be much higher than previously estimated – in both time and money.7 Alexander T. Sandhu, MD, from the University of California, San Francisco, and colleagues calculated testing costs and time costs associated with the 2015 version of the MOC program and found that hematologist-oncologists would incur an average of $40,495 in MOC costs over 10 years, with the cost of time spent satisfying requirements accounting for 90 percent of MOC costs. For comparison, general internists are estimated to pay $16,725 over 10 years.
“The ABIM MOC program will generate considerable costs, predominantly due to demands on physician time,” Dr. Sandhu and colleagues concluded. “A rigorous evaluation of its effect on clinical and economic outcomes is warranted to balance potential gains in health-care quality and efficiency against the high costs identified in this study.”7

“Another problem is that ABIM has really little data that what they do makes a difference,” said Dr. Kahn. “It costs a lot of money, and they don’t have data showing that it improves patient care or clinical outcomes.”

A 2012 literature review supported this conclusion: The available studies did not demonstrate that board-certified physicians provided substantially better care. The authors did note, though, that this doesn’t necessarily mean that ABIM MOC participation has no impact on patient outcomes or quality of care; rather, any potential impact is likely offset by “the effect of external influences on patient care, [and] the fact that physicians completing residency training may have similar overall performance regardless of any particular standardized test result.”8

Finally, there is also concern that the MOC program fails to account for the variety of ways that physicians and physician-scientists accrue knowledge and stay well-informed about updates in their field.

“If I write published articles or chapters in textbooks, the only way to do that is by staying up to date,” Dr. Kahn offered as an example, “yet this has never been recognized as an acceptable pathway to show that I am keeping current.”

Dr. M. Williams agreed that ABIM’s MOC program should consider incorporating these types of educational activities. “There are many individuals, either in academic or community practice, who are deeply involved with ongoing clinical research, testing new drugs or diagnostics, publishing that work, or helping develop guidelines for patient care,” Dr. M. Williams said. “Those sorts of activities need to be recognized and accepted as part of the MOC process.”

Another proposal, as suggested by Paul S. Teirstein, MD, in a commentary published in The New England Journal of Medicine, is adopting Continuing Medical Education (CME) to replace ABIM MOC.9

However, this approach lacks the rigorous assessment of ABIM MOC – a crucial part of physician MOC, according to Dr. M. Williams. “Most people who really think about this topic know that just attending and documenting CME is not adequate,” he said in response to the suggestion. “There needs to be a valid and defensible form of assessment that confirms that a physician is keeping up in the field and can therefore maintain his or her credentialing for ABIM.”

A Plan for 2020 and Beyond

“I believe most people would agree that MOC is important and want physicians to keep their knowledge base current,” Dr. Kahn said. “ABIM certification has always had an implied authority. It has been around a long time, but, right now, they may be in jeopardy of losing that monopoly.”

In the wake of criticism of the ABIM’s MOC program, certain groups have established their own re-certification programs to better serve the needs of their constituency.

For instance, the National Board of Physicians and Surgeons, a grassroots initiative to establish an alternate certification program, of which Dr. Teirstein is president, offers an option for two-year certification. The $169-fee includes all of a physician’s specialties. Certification criteria include previous certification by an American Board of Medical Specialties member board, a valid medical license, at least 50 hours of Accreditation Council for Continuing Medical Education–accredited CME in the last two years, and active hospital privileges (for certain specialties). According to its website, the program is working to gain acceptance by hospitals and payers.10

“This program is only recertifying people who are already initially certified,” Dr. Kahn explained. “They are up and running and do not charge a lot of money, but this certification has not yet been widely accepted at hospitals.”

According to Dr. Kahn, if the ABIM wants to maintain its standing in the MOC realm, it will have to continue to actively solicit input from specialty societies and include the opinions of people who are non-academic practitioners. Anything less than this type of real and ongoing collaboration will be problematic, he said.—By Leah Lawrence 


References

  1. American Board of Internal Medicine, Assessment 2020 Task Force. A vision for certification in internal medicine in 2020. Accessed November 3, 2015 from http://assessment2020.abim.org/final-report.
  2. American Board of Internal Medicine. ABIM announces immediate changes to MOC program. Accessed November 3, 2015 from www.abim.org/news/abim-announces-immediate-changes-to-moc-program.aspx.
  3. American Society of Hematology. Statement from ASH president David A. Williams, MD, on changes to ABIM Maintenance of Certification requirements. Accessed November 3, 2015 from www.hematology.org/Newsroom/Press-Releases/2014/3667.aspx.
  4. American Board of Internal Medicine. First-time taker pass rates – Maintenance of Certification. Accessed November 9, 2015 from www.abim.org/pdf/pass-rates/moc.pdf.
  5. The American Board of Anesthesiology. MOCA Minute. www.theaba.org/MOCA/MOCA-Minute. Accessed November 3, 2015.
  6. American Board of Internal Medicine. Enroll, cost, and policies: cost (paid MOC program fee 2014 or after). Accessed November 3, 2015 from www.abim.org/maintenance-of-certification/policies.aspx#enrollcost.
  7. Sandhu AT, Dudley A, Kazi DS, et al. A cost analysis of the American Board of Internal Medicine’s Maintenance-of-Certification program. Ann Intern Med. 2015;163:401-8.
  8. Buscemi D, Wang H, Phy M, Nugent K. Maintenance of certification in internal medicine: participation rates and patient outcomes. J Community Hosp Intern Med Perspect. 2012;2:10.3402.
  9. Teirstein PS. Boarded to death – why maintenance of certification is bad for doctors and patients. N Engl J Med. 2015;372:106-108.
  10. National Board of Physicians and Surgeons. Apply for certification. Accessed November 3, 2015 from https://nbpas.org/apply-for-certification.

ASH Calls for Continued Changes to Maintenance of Certification

In early November, American Society of Hematology President David A. Williams, MD, released a statement expressing disappointment with the American Board of Internal Medicine (ABIM) Assessment 2020 Task Force report and calling for a movement by the ABIM away from “a policing function” and instead urging a “focus on continuing professional development.”

The statement referenced a letter to ABIM President Richard Baron, MD, with a series of comments on the recently released report:

  • MOC should be customizable instead of using a “one-size-fits-all” approach. Specifically, ASH strongly urges ABIM to recognize the special contributions to clinical care that physician-scientists provide and develop a more realistic approach to the re-certification process for this group of practitioners.
  • As there is a significant lack of high-quality, systematic, external research into MOC, ABIM should fund extramural research to inform the evidence base for decision-making in these processes.
  • Hematologists uniformly note that the secure, closed-book examination required of diplomates every 10 years is not meaningful given the diversification of career paths in the subspecialty and the wasted time spent memorizing facts. ASH applauds the Assessment 2020 Task Force for recognizing these failures.
  • ABIM will further diminish its credibility with the community if it continues to apply a unilateral approach to the development of a new vision for MOC. Consensus building requires hard work, multilateral discussions, and shared visions and agendas. The Society looks forward to a time when ABIM fully embraces such an approach

The response letter that ASH sent to ABIM is not the first interaction that the Society has had with the organization. In fact, the Society has been engaged in critiquing the ABIM regarding its MOC program and recommending revisions to the program since 2001. Building on this history, ASH is leading an informal coalition of other medical specialty societies that represent internists to discuss MOC and coordinate regarding fixes to the program.

In addition, this past summer the Society organized a meeting of the ASH MOC Working Group, Dr. Baron, Dr. M. Williams, and ASH and ABIM staff. The following points were discussed: why ABIM continues to see the MOC program as a way to police the quality of internists; the insular nature of ABIM governance and the need for increased transparency and accountability; the failure of the secure, closed-book examination to be meaningful to diplomates; the perceived lack of value in the program and the perception that MOC is meant to generate net revenues for ABIM; and the lack of high-quality, systematic, external research into ABIM MOC as well as continuing medical education.

ASH representatives and members of the ABIM Hematology Specialty Board will be meeting during the ASH annual meeting to continue their conversations about modernizing MOC.

Ongoing advocacy will continue in 2016 to advance the Society’s positions.

SHARE