Evaluating fellows: Procedural derm, dermatopathology programs lacking

Key Points

National report - When it comes to performing required semiannual evaluations of fellows, both dermatopathology fellowship programs and procedural dermatology fellowship programs fall short, according to a recent study.

Some dermatologists criticize the study's design, however. Furthermore, critics and supporters of the study agree that the American Board of Medical Specialties (ABMS) fellowship evaluation process does not translate directly to smaller subspecialty fellowship programs, which could partly explain the study's findings.

To evaluate whether procedural dermatology and dermatopathology training programs meet ACGME fellow evaluation requirements and determine how many use electronic evaluation systems, researchers mailed a six-page questionnaire to program directors of all procedural dermatology and dermatopathology fellowship programs accredited by the Accreditation Council for Graduate Medical Education (ACGME) for the 2006-07 academic year.

Overall, researchers found no statistically significant differences in fellow evaluation practices between these two subspecialties. Though nearly all respondents said evaluation is important for successful completion of fellowships (p = 0.27), 16 percent of procedural programs and 25 percent of dermatopathology programs said they do not evaluate fellows (p = 0.44; Freeman SR et al. Dermatol Surg. 2008;34:873-877).

Nearly half of programs that evaluate fellows use electronic tools such as Web sites or e-mail.

The survey's results surprised researchers, says Robert Dellavalle, M.D., Ph.D., M.S.P.H., chief of the Denver VA Medical Center's dermatology service and a study co-author.

"We thought that procedural dermatology fellowships might be better or worse at fellow evaluation than dermatopathology fellowships, due to the relative newness of the procedural dermatology fellowships," he explains.

However, Randall K. Roenigk, M.D., says the study is flawed, mainly because it is a questionnaire survey that relies on memory.

"In dermatopathology, a fellow cannot sit for the exam if an evaluation is not complete and on file with the American Board of Dermatology (ABD)," says Dr. Roenigk, who is ABD president.

Returning evaluations

More specifically, program directors must return annual competency-based evaluations to the ABD or American Board of Pathology in May of the dermatopathology fellowship year, says Antoinette F. Hood, M.D., ABD executive director. Application forms to take the dermatopathology examination require additional evaluation by program directors, she adds.

Additionally, Dr. Roenigk says a questionnaire focusing on a fraction of the accreditation process shortchanges the peer-reviewed ACGME evaluation process.

"It's not perfect," he says, "but it's the best process available in this country. Use of the six core competencies as part of the evaluation is expected by both the ACGME and ABD, but translating those competencies to small subspecialty fellowship programs has been a problem for everyone in medicine who trains subspecialists."

Dr. Dellavalle says evaluating educational programs is inherently difficult, and small-program evaluation is even more so. For starters, the one-to-one nature of dermatology fellowships makes anonymous evaluation all but impossible, he says.

"It is not surprising that some programs are not formally evaluating their fellows," says Alexander Miller, M.D., president of the American Society for Mohs Surgery. However, he says he's certain that, "as in all situations involving human interactions, informal evaluations and give-and-take happen constantly."

The study was essentially a benchmark to identify opportunities for improvement in fellow evaluation, "not a 'gotcha' situation," Dr. Dellavalle says.

But even if one discounts the 20 percent of surveyed programs that are not conducting evaluations, he says, "There's still the issue of whether (the rest) are conducting the evaluations on the six core competencies."

Of the programs that provided evaluation forms for researchers to critique, around half failed to address all six core competencies (patient care, interpersonal and communication skills, medical knowledge, professionalism, practice-based learning and improvement and systems-based practice).

"Having all fellowships evaluate on all six ACGME core competencies is a clear, achievable goal," he says.

To that end, the ACGME's Dermatology Residency Review Committee has begun incorporating streamlined common program requirements into one-year fellowship programs, says Julie Jacob, ACGME's manager of communications.

This move will reduce programs' administrative burden and help to standardize fellowship education, she says.

Disclosures: This study was supported by grants from the National Institutes of Health and the National Cancer Institute. Drs. Roenigk, Miller and Hood report no relevant financial interests.

For more information: http://www.uchsc.edu/