Learning from Medical School–Outcome Based Assessment: An “aha” moment

Ever since I got involved a few years ago with the Best Practices project, I started putting educational objectives on my syllabi.  These are the objectives I had on my Fall 2007 Family Law syllabus which incorporated a joint Medical Legal training project on Domestic Violence using standardized clients.  

 As a result of taking the class, I expect students to:

  • Develop familiarity with general principles and social policies of family law.
  • Develop familiarity with leading New Mexico family law cases and New Mexico statutes.
  • Prepare to take and pass the domestic relations section of the New Mexico Bar examination.
  • Develop familiarity with the processes for resolving family law disputes.
  • Develop pleading drafting skills in the area of family law.
  • Improve interviewing and counseling skills, specifically in the context of domestic violence.
  • Better understand medical perspectives on domestic violence.
  • Enhance research and writing skills in area of family law.  (If paper option chosen.)                             

As part of the preparation for the joint training, the  Medical/ Legal team  worked on developing the domestic violence problems for the pre-test.  The Medical School members of the team  (which included specialists with the Medical School’s Department of Assessment and Learning) asked me to develop criteria for evaluating performance.  An example was given:  “At the end of the training students should be able to list three reasons why women (or men) stay in abusive relationships”.  

I resisted.

“That is not how I teach. I usually ask the students to come up with such a list.  There are hundreds of reasons,” I said, “and it is powerful for students to listen to their peers describe them.  And they always come up with something new… that I would not have mentioned.  And, they begin to understand that there are reasons why women stay.”

“So it is not the specific reasons that you expect the students to learn…what do  you want them to learn (and then to demonstrate that they have learned)?”

“Well, that they understand that there are reasons people stay in abusive relationships. I hope it will help them become a little less judgmental, so that they can help them more effectively.”   

“So,  you want to change their attitude about women in abusive relationships?”

“Yes, I guess I do.”

“Well it doesn’t make sense to test their knowledge when you expect as a result of the training to change their attitude”?

“Well, I can only test their knowledge…and it is not fair to test them on their attitude”.

“Well, then how can you establish criteria for determining whether your teaching is having any effect on their learning?”

That is a good question.  The exams I give are traditional law school essay exams that require them to spot issues, analyze them and discuss them.  I try to model bar exam questions.   I try to make them complicated and maybe throw in a tricky issue or two or three so that when I grade them, they are easy to grade because their scores spread out.  I then rank their performance on the exam, make decisions about the grade to assign to a particular score and then I am finished    I realized that all I do on the exam is sort the students’ performance.  The exam tests the same skill tested by the LSAT and the same skill they learned and honed in the first year of law school.    Adding my teaching objectives to the syllabus simply tells the students my goals for them.  It is not a criterion referenced, outcome-based assessment that would tell me what the students actually learned in my class.  And, since we were doing a pre-test and post-test, it would be useful to figure out what I really wanted them to learn, and to design a valid way to measure what they learned.  It turns out that there are attitude measures.  And,   the professionals from the office of Assessment and Learning know how to access them and how to administer them.  I may not want to “grade” them on attitude, but it might be interesting to see if the course had any effect on their attitude, instead of assuming that it must.

Wow, this out-come based assessment is a lot more complicated than I thought!  More about my collaboration with the Medical School in my next post!

Advertisements

4 Responses

  1. Antoinette:

    I would be very interested in hearing from you about your experiences with medical education not only with respect to assessment, but also with respect to the motivational level and focus of medical students.

    Hilary Sale of Iowa has a very provocative work in progress in which she criticizes the Carnegie Foundation book “Educating Lawyers” for failing to take into consideration the motivations of law students for entering law school.

    http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1014114

    She compares the motivations of law students with the reasons many medical students entered their profession. Whereas many medical students have a deep desire to become doctors and consider their profession to be “a calling”, many students apply to law school with very uncertain motivations and levels of desire.

    Would you agree with this assessment? If so, what can we do about the differing motivations and levels of desire of law students?

    What lessons can we learn from medical education in this regard?

  2. Erik,

    That insight about the differences between students who choose to study to become doctors and students who choose to study law is very interesting. Certainly, the grueling nature of medical study and residency means that only the most dedicated would undertake it. Also, I think there is something about the nature of healing and saving lives that attracts individuals interested in serving their fellow human beings. In contrast, as you and Prof. Sales point out, not all students who go to law school intend to serve clients. I think that Carnegie and Best Practices make the important assumption that the purpose of law school education should be to train lawyers—that is those who intend to serve clients. While law school can be an excellent educational experience for an individual who does not intend to use his/her law degree. Legal educational pedagogy should not be designed for those who do not intend to use it, but rather should be designed to prepare those who do. And, to respond to your provocative post with another provocative idea…maybe we should take motivation into account in admissions…Antoinette

  3. Antoinette:

    I would agree that motivation should be taken into account in admissions. But that is a very difficult thing to do. I am a little cynical as to the capacity of a personal statement in an application to reflect an applicant’s true motivations for going to law school.

    Many medical schools require on campus interviews as part of their application process. It would be expensive both for applicants and for law schools (in terms of faculty and staff time), but law schools should consider this step.

    (As an aside, taking desire to be a lawyer in account in admissions would require law schools to say no to tuition revenue from students who are not sure about legal careers, are more interested in an extra credential, or seek an alternative route to another career like consulting. that might be a bitter pill for many schools to swallow).

    More provocatively, to what extent do you think law schools dampen the motivation of those who DO enter school with “a calling”? My sense is that many students often fail to see the connection between the rigorous, analytical (and often dry) work they do in the classroom and real world impact.

    Best Practices starts to address this concern. But, I think your collaboration with the medical community might provide a further vehicle to impress upon law students the real world impact they can have and the enormous stakes for clients of having a well-educated lawyer.

    Part of the reason medical students study so hard is that they realize that people’s lives and health will hang in the balance. Law students should feel the same sense of purpose and the same sense of urgency.

    Does exposing law students to medical education and the motivation of medical students have a positive impact on the drive of law students? Erik

  4. Erik,

    Despite my provocative question, I don’t want to exclude students who want to go to law school for reasons other than becoming a lawyer. (Look at Barack Obama after all!) However, I think the law school curriculum and pedagogy should be based on the assumption that it is preparing students to practice law. It will still be a great education and prepare them for leadership and many other possibilities.

    It might be interesting to ask students when they apply specifically why they are applying to law school in addition to asking for the open ended “personal statement”. It would get them to focus on the question and also send the message that law school is about preparing lawyers. I think we can still state that the law school curriculum will be great preparation for many career and leadership options.

    I think your point about dampening their motivation is a really important one. I know that I have seen students who felt reinvigorated about their “calling”, once they actually represented a client in the clinic, so I think the clinic can be a remedy. However, I also have seen students find that the client contact is not so comfortable for them, and perhaps they should choose a career path that is more in the books and the ideas than in client contact. That is also very important. And, maybe if we structure the classroom experience so that it seems more relevant, we won’t dampen their motivation so much. I think that was the strength of having the students interview a “standardized client” i.e. a trained actor as part of my Family Law class. It certainly made the information about domestic violence seem very relevant.

    I plan to post something about the interaction between the emergency medicine residents and the law students, but I don’t think we had enough interaction to evaluate whether and how the interaction affected law students’ motivation. It would be great if my students would blog about the experience from their perspective.

    Antoinette

Comments are closed.

%d bloggers like this: