Five Problems to Avoid in Writing Student Learning Outcomes

As law faculty across the country strive to improve student learning and meet ABA standards of accreditation through the assessment process, it is perhaps appropriate to stop and assess our efforts in that regard.  Here are five common problems that occur when first writing learning outcomes for a course:

1. Don’t focus on you – focus on the students
Student learning outcomes are designed to give students an idea of what they will be learning.  Avoid learning outcomes that describe what or how your will teach and instead focus on what the students will be able to know, do, or believe.

NOT: UMKC457  Trees as Thought
Student learning outcome:  In this course, I will be exploring the philosophical thought experiment “If a tree falls in a forest and no one is around to hear it, does it make a sound?”  I will explain my book “Trees as Focal Points for Reality” and refute critics of the proposals presented therein.

BETTER: UMKC457  Thought Experiments
Student learning outcome:  At the end of this course, students will be able to think critically and communicate effectively the metaphysical theories regarding the existence of that which cannot be perceived. Students will be able to describe how the theory of subjective idealism has impacted religious and scientific philosophy.  Through discussion and written reflection, students will demonstrate clarification of their individual values.

2. Avoid Vague Verbs
Probably one of the most common verbs found in student learning outcomes is “understand,” as in “students will understand [course content].” The problem with this as a learning outcome is that it is difficult to know what evidence would demonstrate that understanding.  A student learning outcome that uses more active and concrete verbs can unpack the type and degree of “understanding” that a professor expects.

NOT:   LAW8000  Family Law
Student learning outcome:  Students will understand the law regarding marriage regulation and the constitutional constraints on that regulation and the law of divorce, including child custody.

BETTER:  LAW 8000 Family Law
Student learning outcome: At the end of this course, students will be able to:
• identify the legal issues raised by a fact pattern involving a marriage regulation, make critical and effective arguments regarding the meaning of that regulation and its constitutional validity, and confidently predict the outcome of a challenge to that regulation
• identify relevant facts necessary to gather from a client seeking a divorce and child custody with property including real estate and pensions; draft a complete and legally effective petition for that divorce and custody action, including a parenting plan; and identify legal issues and make critical and effective arguments, applying the statutory and case law, to determine the divorce, property division, child custody and economic support in the case.
To read more about it, see Chapter Two. Understanding Understanding, of GRANT WIGGINS & JAY MCTIGHE, UNDERSTANDING BY DESIGN (2nd Ed. 2005).

3. Avoid “elementitis”
A student learning outcome should not merely summarize the syllabus or be a list of topics the course will cover.  Rather, the student learning outcomes should focus on thematic elements that tie these topics together or ways in which the students will be able to use this knowledge.  As David Perkins of the Harvard Graduate School of Education notes:
We educators always face the challenge of helping our students approach complex skills and ideas. So what to do? The two most familiar strategies are learning by elements and learning about. In the elements approach, we break down the topic or skill into elements and teach them separately, putting off the whole game until later — often much later….to have a little fun I call it ‘elementitis.’
DAVID PERKINS, MAKING LEARNING WHOLE: HOW SEVEN PRINCIPLES OF TEACHING CAN TRANSFORM EDUCATION (2010).  Avoid student learning outcomes that are plagues by “elementitis” and describe instead what it is students will be able to do with course coverage.

4. Don’t Always Expect Mastery
Student learning outcomes should indicate not only the content the students will learn but how well they will learn it.  We cannot aim for mastery of all aspects of the course.  Rather, learning outcomes in some courses are necessarily going to be at an introductory level (students will “recognize” or “describe” or “identify”) while other outcomes may be aimed at higher levels of mastery.  An effective tool to determine the proficiency level of your learning outcomes is Bloom’s Taxonomy of Educational Objectives, which provides a hierarchy of increasingly sophisticated learning outcomes.  To read more about it and see a list of verbs associated with differing levels of learning, see Rex Heer, A Model of Learning Objectives from Iowa State University Center for Excellence in Learning & Teaching (2012). To read an application of this model to law school, see Paul Callister, Time to Blossom: An Inquiry into Bloom’s Taxonomy as a Hierarchy and Means for Teaching Legal Research Skills 202:2 LAW LIBRARY JOURNAL 191 (2010-12).

5. Don’t Avoid Outcomes that May be Difficult to Measure
Student learning outcomes for a classroom rarely will focus entirely on the acquisition of knowledge.  At a minimum, most classes expect students to develop their cognitive and communication skills in using the knowledge base of the course.  Courses may also help students to clarify values, reconsider beliefs, appreciate new perspectives, or develop greater self-awareness.  Some faculty recognize that these skills and values are some of the most important benefits that students take away from the courses, but are reluctant to state these as learning outcomes because they are unable to “test” these outcomes.  However, any important skill or value can be assessed – even if there is a good deal of subjectivity involved in that assessment.  By stating these objectives as learning outcomes, faculty members can challenge themselves and their students to more clearly describe the dimensions of this learning.  Measurements of this learning may be through written reflections, observations of performance, or surveys of opinions.  These are perfectly valid assessment tools.

Unlearning as Learning Outcome

As the newly revised ABA accreditation standards 301 and 302 now require law schools to clearly articulate and publish their learning outcomes for their students, so individual faculty members must do likewise. Yet it is not uncommon to see these learning outcomes statements that read like the table of contents of the textbook used to teach the course. To truly be effective in driving learning and teaching, learning outcomes must be targeted, concrete, measurable and active (not “learning about” but “learning how to”).

How do we most effectively choose and articulate these learning outcomes? In MAKING LEARNING WHOLE: HOW SEVEN PRINCIPLES OF TEACHING CAN TRANSFORM EDUCATION 83-89 (2010)., educational specialist David Perkins emphasizes that learning is most effective if learners “work on the hard parts.” Similarly, the UNDERSTANDING BY DESIGN framework, originally developed by Grant Wiggins and Jay McTighe, emphasizes beginning the search for course goals by looking for the “Big Idea” in the course. These are the ideas or themes that can be used throughout a legal career and that require a lot of work to master.

One of the most effective ways to uncover these “big ideas’ or ‘hard parts” is to focus first on unlearning outcomes – that is, preventing and addressing predictable misunderstandings in the course. Thus, for example, much of the first year of law school is devoted to “unlearning” the positivist philosophy of students who believe the law is resolutely determinate. These fundamental misunderstandings are persistent, difficult to overcome and block learning of new ideas. Students construct knowledge by building on prior understandings. If those prior understandings are incomplete or incorrect, new learning will be flawed as well. As summarized by NATIONAL RESEARCH COUNCIL, COMMITTEE ON DEVELOPMENTS IN THE SCIENCE OF LEARNING, HOW PEOPLE LEARN: BRAIN, MIND, EXPERIENCE, AND SCHOOL: EXPANDED EDITION 11 (2000), “teachers need to pay attention to the incomplete understandings, the false beliefs, and the naive renditions of concepts that learners bring with them to a given subject.”

In her new book, Building a Better Teacher: How Teaching Works (and How to Teach It to Everyone) ( 2014), Elizabeth Green reviews the research concluding that effective teachers (as measured by student learning gains) are those who are able to identify the reasons that students misunderstand and help them to unlearn those misunderstandings.

Some of the most fundamental misconceptions that students bring to a subject from their own experience (or from bad course outlines passed around from prior semesters) must be discovered in the classroom. Brief classroom assessment devices such as “minute papers” or statements for the students to complete can easily generate a range of incorrect or incomplete understandings for any given topic.  The mission to discover student errors leads faculty to many of the best practices in teaching: regular interaction with students, frequent and meaningful feedback, and active learning strategies.

The power of an “unlearning” perspective on assessment improves student learning, but also quickly leads faculty to a deeper understanding of what assessment of student learning oucomes means.  Assessment is not an end-point, a box to be checked, reported and forgotten, but is an iterative process of discovery and experiment that drives students and faculty learning alike. Assessment tools (such as quizzes, socratic dialogue, essays, simulations, and reflections) might be used to unearth student misconceptions.  These misconceptions then become the basis for the learning outcomes around which one can build a course and assessments then can be used to determine the extent to which one is successfully dislodging misunderstanding and misconception and replace it with a solid framework mastery.

Shultz and Zedeck: Collaboration and Motivation in Orientation!

One-Ls at Albany Law, just like those at many other schools, are in the midst of Fall 2014 Orientation. Today, I participated as a  “faculty observer” in a collaborative skill building exercise organized by our Associate Dean Alicia Ouellette.  Imagine my delight to see copies of Schultz and Zedeck’s 26 lawyering effectiveness factors distributed at each table in the school gym!

Teams of 20-25 students, most of whom had either just met each other or not yet met, were tasked with:

  • Assembling a small children’s bike (to be donated to the Boys and Girls Club); the first team to both build the bike and have a team member ride the teeny-tiny bike around the orange cone course set in the gym would be declared winner. :)
  • Building the tallest pasta-marshmallow structure
  • Making sure every student on the team participated in the endeavor.

Faculty participants were assigned to observe what they saw happen during the group exercise, report their observations to their student team, and explore with the student teams questions such as:

  • what worked well?
  • what was challenging about  mandatory collaboration?
  • what might they have done differently to more effectively collaborate?
  • what might these exercises suggest about effective lawyering?

The students brought good humor to the task.  They brought a range of experiences, including a few with engineering backgrounds and/or “mom/dad” know-how, and a range of abilities. The fact that the bikes were to be REALLY used by local community members was a motivating factor.  In fact, students vocally expressed concern about the safety of the quickly assembled bikes noting,  “Remember, some kid is going to ride this!” and “It has to be safe.”

By the end of the assigned time period, everyone in my group had participated …. at least a bit. The debriefing was more effective than one might have predicted. One student on my team noted gender differences in approaches – a number of women were reading instructions for assembling the bike while a few of the males started to immediately put pieces of the bike together. This led to a discussion of THE CONFIDENCE GAP.  Another student noted the difference between working on a task when you know what the outcome should look like (the bike) and working on a concept without a uniform or agreed upon vision of what the outcome looks like (the highest pasta structure). Many students reflected on the significant importance of communication skills, particularly listening.

Other teams reflected on the challenge of being asked to accomplish a collective task when most members of the team felt inadequately prepared. With faculty guidance, that team explored when that might happen in law school or in practice.  Issues such as time management, resource management – one team ran out of tape – and problem solving techniques were also discussed. Students, encouraged by faculty suggestions, also pondered what kind of teams they might participate in their post-graduation future .

As I looked around the tables, I could not help but think of Richard Susskind’s book,  Tomorrows Lawyers.  These one-Ls will be entering a profession and a world in which working with others, problem solving, creative thinking, and clear communication will be even more critical for those in our profession than in times past.   As graduates, these students will be participating in teams and in collaborative enterprises that we faculty probably cannot now envision.  However, it is our job to facilitate their acquisition of the kinds of skills and capacities and attitudes that will best serve them in the uncertain but potentially exciting future.   Happy New Semester all! Happy Facilitating!

Orientation 20140813_142119

 

Orientation Pic 2 Orientation Pic 7 Orientation Pic 6 Orientation Pic 5 Orientation Pic 4 Orientation Pic 3 Orientation Pic 1

The Baby Has Finally Been Birthed!

Comprehensive revisions passed

The ABA House of Delegates passed the comprehensve revisions with “minimal  fuss” according to the ABA Journal linked  above.  One area, however, garnered  significant attention and also resulted in  an odd, though perhaps meaningless ,  procedural move.  The House voted  to send back to the Section on Legal Education for further consideration the comment to standard 305 which prohibits payment to students for credit-based courses.

What does this mean? Law schools which have not already done so must start identifying, articulating publicly and assessing student learning out outcomes, providing every student six  credits of clinic or clinic-like experiential courses and requiring students to take two credit hours worth of professional responsibility coursework.

Well, it’s a start……

Transferring Best Practices to a Domestic Violence Agency–i.e. the real world

On February 1, 2014 I left the ivory tower of a law school I had loved for 27 years to become the executive director of Enlace Comunitario, a non-profit agency focused on eliminating domestic violence in Latino immigrant communities through intervention services such as case management, counseling and legal services and prevention activities such as leadership development, education and outreach.   This was a big transition for me, but I am loving it!  And, my long time involvement with Best Practices for Legal Education has paid off in this context.  How, you might ask, are the skills transferable?  Well I will give some examples in my next few posts…but I will share immediately that I am working to create a teaching and learning culture at my agency.   Specifically, my goal is to build the capacity of folks in the agency so that when I step down one or several of the staff members will feel ready to take on the helm. And, of course, I will want staff members to step up to take their place. Already, we are training a counselor to become a counselor supervisor and we are training a former receptionist to become a case manager. I love seeing my staff take on the teaching role! And, they are good at it.

So…one of the foundational principles of best practices is to work to develop learning objectives for your students.  Well, it is not a stretch to work with staff members and develop learning objectives with them!  And, creating evaluations that fit the job duties and the learning objectives was fun:  Each job criteria or learning objective is evaluated as follows:  “in training”, “needs improvement” “good work” or “awesome, can teach this knowledge, skill or value”.   So far the staff has responded positively to the new evaluation process.  We will finish up this month!  I will let you know how it goes!

Five Tool Lawyers

Leading Northwest legal practitioner and technology entrepreneur Marty Smith has an interesting post on the Five Tool Lawyer over at Legal Refresh. Using the metaphor of the Five Tool Lawyer, Marty breaks apart the stages of problem solving, incorporating risk analysis in a way I found helpful. In my response Five Tool Lawyers and Legal Education, I critique aspects of the Five Tool Lawyer metaphor for compressing too much into the 1st [Use interviewing skills to gather client facts, goals and needs] and 5th tools[Counsel, document, negotiate and advocate on behalf of client]. But here’s why I thought the metaphor was compelling:

"Compelling, because [it] moves beyond issue spotting v. problem solving to articulate the stages of problem solving, targeting a spotlight on often overlooked aspects. . . . By focusing on risk, the metaphor highlights two often neglected stages of the lawyer’s work – “use judgment to assess actual risks” and “problem solve for best way to meet client’s needs with minimal risk.” At the same time, it implicitly places the legal problem in the larger context of the individual’s life, or the business’s health. And it underscores the fact that lawyers need to know how to assess the significance of legal risks within that larger context."

The definitive statement of a law professor’s REAL mission, or once again Harvard Law retires the chutzpah title

I cannot improve upon Scott Fruehwald’s presentation of this happy news in a recent email:

“Harvard is offering this course in the spring:

BECOMING A LAW PROFESSOR

“There are many elements that go into becoming a law professor, but at the core of the process of moving from law student to law professor is scholarship.  How do you choose your topic?  How do you write an article?  What will become your area of expertise?  What have others written about this subject area, and how do you break new ground?  How do you engage with fellow scholars in the midst of the writing process?

This reading group will focus on the generative scholarly process that is at the center of the life of the law professor.  [Emphasis added.]  Each week, a member of the faculty will present a working draft of her or his scholarship, and that piece will be discussed by the group.  Discussions will focus in part on the genesis of the research project being presented, in order to demonstrate how articles develop from the first spark of an idea to final publication.  Students will also explore substantive issues raised in the pieces, the better to become familiar with the latest work being done across a variety of subject areas.  Students will also develop their own research and scholarly agendas as the semester progresses.

Admission is by application via email to Susannah Barton Tobin at stobin@law.harvard.edu by November 8, 2013.  Please include a paragraph expressing your interest in the reading group and a CV and transcript.

As is the norm with reading groups, there will be no examination or paper requirement, and the class will be graded credit/fail.” (http://www.law.harvard.edu/academics/curriculum/catalog/index.html?term=Spring+2014&type=Clinic&type=Course&type=Reading+Group&type=Seminar&rows=10&year=2013-2014)

The course description says it all.

Happy New Year,  Scott Fruehwald  [If you are attending the AALS Conference, please come by the ABA Publishing table and look at my book, Think Like a Lawyer: Legal Reasoning for Law Students and Business Professionals (2013).]”

All right, I [VM] cannot resist adding one comment:  The best part of this deal is that the participating faculty actually get teaching credit for this!   Well, maybe no, teaching is probably not exactly a heavy load at HLS anyway; no, maybe the best part is that the selected students (or their parents/spouses, or the US government) are paying tuition for them to serve as extra unpaid research assistants.

Or perhaps the very best part of this course is the unabashed clarity of its focus exclusively on scholarship, without a shred of pretense, or allocation of even a single seminar to such irrelevancies  as teaching, advising, evaluating and assessing, mentoring, curriculum development, preparation for practice, or engaging (with and without students) in advocacy to improve and reform the law, our profession, or society — to mention only a few of the professorial roles one might name.   (Perhaps all these skills are innate for Harvard grads?)  This course was presumably approved by at least a bare majority of the Harvard faculty (assuming that they still bother to review course proposals) so I suppose it is an accurate  statement of that faculty’s priorities.

The truly sad part is that many of us, on Appointments Committees and as faculty members, will end up not merely voting to hire the graduates of this course, but trying everything in our power to lure them on to our faculties — including promises that as long as they churn out scholarship that is “cutting edge” (nothing as pedestrian as, say, realistic law reform proposals), no one will expect them to work as hard at learning how to teach effectively or fulfill any of the other aforementioned professorial functions.  And in the course of discussions about how to seduce these young superstars who wear the crimson “H” on to third-tier faculties (at least to start), no doubt someone will point out “Look, s/he took a course on law teaching at Harvard!  One with the evocative title of ‘Becoming A Law Professor’!  What could be a better indicator of serious commitment to teaching?”

Footnote:  Naturally, given the topic, I feel compelled to add at least one:  Of course, these remarks would be seriously misplaced if “Becoming A Professor” is one of several HLS courses or programs, some of which address the aspects of professorship that I am asserting Harvard neglects.  I can’t be sure — the hundreds of courses in the HLS catalog are just too daunting — but I did try searching the past three years for any course that included the terms “teach” “teaching” “education”  “educate” “professor” “school” and some others.  Only one item surfaced, a legal history course last spring entitled “American Legal Education,” taught by Visiting Professor Daniel Coquillette.  http://www.law.harvard.edu/academics/curriculum/catalog/index.html?type=Clinic&type=Course&type=Reading+Group&type=Seminar&rows=10&year=2012-2013  It does sound fascinating, a course that I would recommend to students and enjoy myself.  While certainly it would be of use to anyone who aspires to the law professoriate, given its scope and format it does not quite occupy a curricular niche comparable to the new scholar-centric offering.

Follow

Get every new post delivered to your Inbox.

Join 630 other followers

%d bloggers like this: