Introduction – Deborah E. Powell, M.D.
Dr. Powell welcomed participants to the second annual medical education retreat and noted the success of last year’s retreat. She emphasized the primary mission of the school, the education of medical students. Dr. Powell stated that she expects to receive a report this fall from the LCME on the April 1999 limited site visit. She noted the wonderful job that was done in constructing the summative competencies for KU graduates. The Dean stressed that the curriculum is a work in progress, evolutionary not revolutionary.
Dr. Powell described her open forums with the medical student classes and a student’s concern that the US News and World Report did not rank the medical school. She remarked to the student that she thought these rankings were a popularity contest. After reflection, she believes that if we are truly committed to an outstanding and innovative curriculum, we are positioned to do this. We have two campuses: Kansas City, a traditional program and Wichita, a community-based program. In addition, we have a fledgling Medical Education Network. The Dean challenged participants to develop an innovative, unique program to bring us into the national spotlight.
State of the Curriculum - David R. Calkins, M.D., M.P.P., and Allen Rawitch, Ph.D.
Dr. Calkins described the progress that has been made since last year’s retreat. Topics addressed last year were the Learning Environment, Teaching Methods in Years One and Two, Longitudinal Themes, Student Evaluation in Years One and Two, Student Evaluation in Years Three and Four and Clinical Teaching Sites. In regards to the Learning Environment, a committee was formed to discuss and plan Academic Societies. Regarding Teaching Methods in Years One and Two, the issues of increasing small group learning and exploring the use of problem-based learning (PBL) were raised. This past year, PBL groups were added to the Physical Diagnosis course in year two. Scheduling in years one and two was also addressed and faculty have strived to assure that students have enough time for self-directed learning. The Summative Competencies document addresses several of the topics discussed in the Longitudinal Themes group, including the doctor-patient relationship and professionalism. One issue from the Student Evaluation in Years One and Two session that is being addressed is coordination of exam scheduling in the first and second years. From Student Evaluation in Years Three and Four, clerkship directors are being encouraged to provide formative and summative feedback and there are plans to develop an exam to assess clinical skills at the end of the third year. The development of the Medical Education Network and a Rural Track were part of the discussion of the Clinical Teaching Sites group last year.
Dr. Rawitch discussed the activities of the Education Council over the past year. The Year One and Two Curriculum Oversight Committee, Year Three and Four Curriculum Oversight Committee and Introduction to Clinical Medicine Committee worked on their respective portions of the curriculum. The Objectives Working Group wrote the Summative Competency Document and the Appendices (undergoing refinement through faculty input). The Course Credit Hour Group is preparing a report on the allocation of credit hours and the Student Evaluation group is working on a review of the student evaluation system. Actions taken by the Council include: consideration and/or approval of new courses or changes to courses in the departments of Family Medicine, Surgery, Internal Medicine, Emergency Medicine, and Ob/Gyn; approval of the name change of ATMY and PHYS 840 to NEUS 840; approval of the Summative Competency Document and draft Appendices; and a reduction in the length of the Ambulatory Specialties clerkship from eight to four weeks and from a selective to an elective.
The Council also participated in the LCME limited site visit and pre-visit preparation.
Session 1 – Student Evaluation
Facilitators: Giulia Bonaminio, Ph.D., and Allen Rawitch, Ph.D.
What methodology are we currently using to evaluate students?
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Polling of the group showed that no course or clerkship used all or most of these methods and that no course or clerkship used only a single form of evaluation to assess students. In the majority of cases a written exam of some sort appeared to be part of the evaluation process in combination with one or more of the other methods listed above. It was generally agreed that available time and resources were factors in choosing evaluation methods.
What are the objectives of student evaluation?
There was a consensus that the reasons for student evaluation are:
What are the appropriate strategies which can be used to accomplish these objectives?
What resources (human, physical and financial) will be required for these strategies?
The resources that the group felt were needed to use these strategies effectively and which were not currently in place were:
There was a general consensus in the group that there are not currently enough resources identified for education and evaluation efforts to allow the full use of a number of the desired evaluation methodologies in some departments. These resources included both funds and faculty or staff time availability.
Is the same grading system appropriate for the pre-clinical and clinical years?
While the group did not have sufficient time to discuss this question, it was raised in the general session during the summary report of the group’s activities. It was pointed out that an ad hoc committee, commissioned jointly by the Education Council and the Academic Committee, is currently looking at the question of how we evaluate medical students and is charged with looking at the issue in the broad sense and to the future. It was re-affirmed that this committee and the faculty, as a whole, need to look objectively at what the best grading or evaluation system would be for the School of Medicine and whether the same system should apply to the pre-clinical and clinical years. The initial reporting with recommendations from the ad hoc committee will go to the Academic Committee and the Education Council for action.
Session 2 – Role of the Network Sites in Educational Programs
Facilitators: David Calkins, M.D., M.P.P., and Ken Kallail, Ph.D.
Four questions were addressed: (1) How have we used the six Medical Education Network sites to date? (2) What has been the experience of students who have taken third-year clerkships (or portions of clerkships) in Network sites? (3) Should we continue to offer third-year clerkships in Network sites? If so, how many third-year students should we train in these sites? How should we identify students who will train in Network sites? (4) What would be the role of Network sites in a Rural Track?
Dr. Calkins introduced the session by exhibiting a map of the six Medical Education Network regions and a list of the Medical Education Directors. Third-year medical students have been hosted in 5 of the 6 sites for all or part of a clerkship. Participating clerkships include: Pediatrics, Obstetrics/Gynecology, Family Practice, Ambulatory Medicine/Geriatrics, and Neuropsychiatry. Informal reports suggested that volunteer faculty and medical students have enjoyed the educational experiences. Some concern was expressed that faculty must ensure that students have enough time to study while in the Network sites.
Ob/Gyn students generally have done well during third-year clerkships in the Network sites. Dr. Gunter suggested that the students performed on exams similarly to those trained in KC. To support students in the Networks, Ob/Gyn will videotape some lectures and assign required readings.
Two pre-medical programs are supported by the Network sites and their Directors: the Summer Mentor Program and the Scholars in Primary Care Program. Directors recruit faculty and students for the program and participate in student selection and training. Discussion shifted to tracking Kansas students who leave Kansas for college, but want to attend KUSM. Tracking is difficult since there is no pre-medical student database. Students who relinquish their Kansas residency also face rigid residency requirements when they come back for medical school.
Dr. Gladden reported that 90% of the participants in the Rural Primary Care Research and Practice Program who have graduated from medical school are in primary care residencies and 90% remain in Kansas.
Session participants emphasized that communication must be excellent between the Network sites and clerkship directors. Many programs now exist that require volunteer faculty in the various regions. Communication between programs is improving, but needs to get even better. A database is planned that documents where students are placed from the various educational programs.
Housing and other resources are critical to placing students in the Network sites. McPherson is building a medical education facility attached to the hospital. It will have sleeping rooms, a library, and televideo capabilities.
The medical school is engaged in a planning process to define what the Network sites can be and what they will do. The plan must tie the Network sites to the AHECS, the Office of Rural Health Education and Services, and other programs, such as the Rural Preceptorship. To continue programs, resources need to be identified and used effectively, including those from the medical school, the volunteer faculty, local hospitals, and the community.
Several options have been discussed related to placing medical students in rural communities. Currently, medical students take a rural rotation in the fourth year and have other discrete opportunities in the first, third, and fourth years. A Rural Track would help to link rural programs and opportunities. A Rural Track might link the Scholars in Primary Care Program and Dr. Gladden’s summer rural experience to clinical rotations in the third and fourth years. Students could spend up to 24 weeks in the third year in a rural community by linking the Family Practice, Ambulatory Medicine/Geriatrics, Pediatrics, and Obstetrics/Gynecology clerkships in a Network site. The most innovative option is to develop a longitudinal Rural Track that covers most or all of the third and fourth years. Dr. Powell suggested that it may be necessary to identify a few interested students and pilot such a track.
Grading and evaluation is an issue when considering a Rural Track. Most clerkships have several faculty that provide input to the grade. Rural rotations usually have only one faculty member. A student evaluation system must be developed that is fair and equitable regardless of the track in which a student is placed. Clinical skills exams should assist with assessing students whose learning experiences are different.
Brown University School of Medicine’s Competency-Based Curriculum
Presenter: Stephen R. Smith, M.D., M.P.H., Associate Dean for Medical Education
Brown University School of Medicine
Dr. Smith described the development of the Brown University School of Medicine’s Competency-Based Curriculum, MD 2000, and its implementation. He began by describing the Flexnerian model: define fundamental knowledge, teach fundamentals, test for knowledge of fundamentals, and hope for the best. Dr. Smith met with faculty, administrators, and students and asked what characteristics they felt made a good physician. The planning for the competency-based curriculum was done in a "backward" fashion by defining the graduate first, designing measures and standards of performance, and developing learning experiences to help students meet these standards.
Dr. Smith discussed Brown University’s nine abilities: Effective Communication, Basic Clinical Skills, Using Basic Science in the Practice of Medicine, Diagnosis, Management, and Prevention, Lifelong Learning, Self-Awareness, Self-Care, and Personal Growth, Social and Community Contexts of Health Care, Moral Reasoning and Clinical Ethics and Problem Solving. Each course and department take ownership of at least one of the nine abilities. In order to set up a performance-based assessment, a list of tasks is developed dealing with issues relating to safety, economics, etc. Dr. Smith also referred to the Brown University School of Medicine’s competency attainment grid. The grid includes each of the nine abilities and the number of learning experiences a students needs to complete satisfactorily in order to fulfill the competency for that ability at that level (beginner, intermediate or advanced). Every student must achieve an intermediate level in all competencies and an advanced level in four abilities of their choice. Dr. Smith played a video, which showed a student using effective communication and life-long learning skills, describing anatomy in a clinical context based on library readings and citations, and presenting orally to peers and faculty in a comprehensive way. Students are required to demonstrate all 9 abilities or they do not graduate. Dr. Smith commented that students’ behavior is determined by evaluation.
A question was asked regarding how to measure moral reasoning and clinical ethics. Dr. Smith described an OSCE station where a preceptor tells a student to get consent from a patient with pancreatic cancer to receive a highly toxic drug that has a low response rate and no long-term survivors for this type of cancer. The standardized patient evaluates the student on 5 areas: side effects information, patient value system, plan with options, actual plan with steps and mutuality. The student also is evaluated on written work.
Dr. Smith described a future project at Brown where real patients of physicians will be trained to evaluate the attendings and residents. He also recommended asking residency directors to rate graduates’ performance.
Brown University students are graded pass, fail and honors. Competency certification is separate from grades. Brown will be implementing a 4th year OSCE at the request of clerkship directors. A question was asked regarding the class size at Brown. The class size is 68.
Session 3 – Evaluation of Clinical Skills
Facilitators: Joe Gladden, M.D., and Doug Woolley, M.D.
What is the purpose of a comprehensive, competency-based evaluation of clinical skills upon completion of the third year? How will we use the findings?
What should be the content of this exam? What domains of knowledge, skills and attitudes should we assess?
How should we evaluate student performance on this exam? What sort of summary of this evaluation should be provided to students?
How can we assure comparability of this exam on both campuses?
Who will be responsible for the development and administration of this exam?
What resources will be required?
What concerns will students have about this exam? How can we address these concerns?
Session 4 – Professional Development
Facilitators: Garold Minns, M.D., and Christopher Crenner, M.D., Ph.D.
What is professionalism?
The discussion developed in three parts. We talked first about existing definitions of professionalism and referred to the American Board of Internal Medicine (ABIM) document on professionalism. Next, we discussed the sources of present concern in medicine about professionalism. Finally, we reviewed some additions or additional emphasis we would make to the ABIM definitions.
The ABIM definitions were distributed. We reviewed the categories used by ABIM of altruism, accountability, excellence, duty, honor and integrity, and respect for others. The values expressed seemed laudatory and non-controversial.
Why talk about professionalism now? First, one discussant suggested, because a breach of ‘professionalism’ may prove more likely to derail a physician’s practice and career than a breach of clinical competence. In addition, we may now be seeing more potential for breaches of professionalism in our trainees. Another discussant suggested that the distinction of medicine as a sovereign profession might be more in jeopardy today. Changing commercial and organizational contexts for practice threaten the profession’s identity. In response, someone else noted that the sociologists’ definition of a profession includes the requirement for autonomous self-government among its members. Medicine may be experiencing new challenges to its professional autonomy and so to its professional identity. Another discussant noted that a physician’s ability to handle legitimate complexity and uncertainty in practice might be another professional skill that marks medicine as distinct from trades and businesses. We discussed further whether professionalism in our more recent generations of students might be in decline. We talked about the possible declining moral character among this "new generation" in general. Concern about sounding like our parents was raised tongue-in-cheek in this regard.
Discussants made several suggestions about additions or additional emphasis to the ABIM list. Sensitivity and compassion were raised as key, under-emphasized qualities in medical professionalism. Referring to the Brown competencies, we added commitment to life-long learning and self-awareness/self-growth to the list. We mentioned the role of service to society and community as a characteristic of professionalism. This remark connected back to our discussion of medicine distinguished as a profession rather than a trade, craft or commercial service.
How should we assess these qualities in applicants?
A near quorum from the Admissions Subcommittee assisted in a review of the status of assessment for character and service, among other elements, among our applicants. We identified letters of recommendation as helpful, but considered adding more explicit guidance to the writers to identify these qualities in applicants. We discussed the AMCAS form. It allows for significant discussion of character and service criteria. More important, perhaps, were the weighting received by these qualities in the final decisions. Another discussant raised the question of the need to screen more effectively for strong negative characteristics in a small number of applicants. We agreed that strengths in academics and particularly science should not blind us to weaknesses in other, more difficult-to-assess areas. We talked about individual, one-on-one interviewing as a possible method for screening out negative qualities that otherwise might not be visible in applicants. Another discussant raised concern about the variability and subjectivity of one-on-one interviewing for selecting applicants. We agreed that we get a high quality of students on the whole, though more attention in applicants to the capacity to transgress professionalism might be useful.
What aspects of the current academic/institutional environment strengthen or weaken these qualities in our graduates?
We identified many strengths including faculty development, potential for role-modeling, and special educational opportunities.
Among these strengths faculty development contributed by reviewing faculty to provide feedback on specific performance, by providing faculty credit towards promotion and tenure for advising/mentoring with students, and by encouraging peer evaluation of teaching for faculty. The new academic societies appeared to offer assistance to professionalism by providing greater opportunity for students to interact with faculty who would be excellent role models for students. The White Coat Ceremony helped to endorse values of professionalism publicly for the entering students. The forthcoming third-year ICM curriculum emphasizes professional values. One support for professionalism within this curriculum may be the critical incident reports which will allow students in small groups to discuss positive and negative experiences during third year.
We identified several weaknesses in the environment. Several discussants and especially the medical student representatives raised concerns about faculty behavior that might adversely affect medical student professionalism. In the pre-clinical years, discussants highlighted the importance of treating the students as adults, of acknowledging the students’ other outside commitments and of holding rigorously to the scheduled class times for starting and ending lectures and other meetings. Several examples of clinical faculty behaviors also emerged. It was thought to be especially crucial to provide good role models for interactions with patients in clinics and on wards. We discussed the possibility that individual lapses by clinical faculty in professional behavior might conceivably outweigh many other neutral or positive role models.
What programs might further development of professionalism?
We discussed further the academic societies, the third-year ICM curriculum and the White Coat Ceremony. We talked about a more detailed orientation to the third year at the conclusion of ICM in the second year. Perhaps there will be some kind of contract with the students entering the third year. The contract would spell out what is expected of students on the wards including issues of professional behavior. One discussant suggested that the contract would be mutual with the clinical faculty also endorsing proper behavior toward the student clerks.
Another discussant brought up the issue of peer evaluation among the students. Students might have better awareness of lapses of professional behavior in their peers. The difficulties of peer review were aired. Students returning from time away from their class might be at special risk to receive unfair reviews by their peers, for example.
The issue of OSCE’s to assess professionalism came up. One discussant raised the example of the OSCE station for ethical behavior toward a patient enrolling in a risky treatment protocol, which had been mentioned by the visiting commentator, Dr. Smith.
There is an HMO council on credentials locally. They are keenly interested in the problem of certifying professionalism and might wish to contribute to our projects in some way.
The idea of involving patient and community review mechanisms arose. Several discussants endorsed the importance of patient surveys and patient advocacy as tools for monitoring and addressing the professional behavior of physicians.
Another group important for fostering student professionalism was the graduate trainees. One discussant offered the new efforts to develop resident and intern teaching skills in the hospital as an example of a program that will encourage professionalism. Another discussant offered the example of a successful program in a medicine residency which offered advice on teaching and professionalism to residents under the rubric of a course on "medical leadership."
We did not address directly the difficult question of measuring professionalism.
©
The University of Kansas Medical Center