Unlearning as Learning Outcome

As the newly revised ABA accreditation standards 301 and 302 now require law schools to clearly articulate and publish their learning outcomes for their students, so individual faculty members must do likewise. Yet it is not uncommon to see these learning outcomes statements that read like the table of contents of the textbook used to teach the course. To truly be effective in driving learning and teaching, learning outcomes must be targeted, concrete, measurable and active (not “learning about” but “learning how to”).

How do we most effectively choose and articulate these learning outcomes? In MAKING LEARNING WHOLE: HOW SEVEN PRINCIPLES OF TEACHING CAN TRANSFORM EDUCATION 83-89 (2010)., educational specialist David Perkins emphasizes that learning is most effective if learners “work on the hard parts.” Similarly, the UNDERSTANDING BY DESIGN framework, originally developed by Grant Wiggins and Jay McTighe, emphasizes beginning the search for course goals by looking for the “Big Idea” in the course. These are the ideas or themes that can be used throughout a legal career and that require a lot of work to master.

One of the most effective ways to uncover these “big ideas’ or ‘hard parts” is to focus first on unlearning outcomes – that is, preventing and addressing predictable misunderstandings in the course. Thus, for example, much of the first year of law school is devoted to “unlearning” the positivist philosophy of students who believe the law is resolutely determinate. These fundamental misunderstandings are persistent, difficult to overcome and block learning of new ideas. Students construct knowledge by building on prior understandings. If those prior understandings are incomplete or incorrect, new learning will be flawed as well. As summarized by NATIONAL RESEARCH COUNCIL, COMMITTEE ON DEVELOPMENTS IN THE SCIENCE OF LEARNING, HOW PEOPLE LEARN: BRAIN, MIND, EXPERIENCE, AND SCHOOL: EXPANDED EDITION 11 (2000), “teachers need to pay attention to the incomplete understandings, the false beliefs, and the naive renditions of concepts that learners bring with them to a given subject.”

In her new book, Building a Better Teacher: How Teaching Works (and How to Teach It to Everyone) ( 2014), Elizabeth Green reviews the research concluding that effective teachers (as measured by student learning gains) are those who are able to identify the reasons that students misunderstand and help them to unlearn those misunderstandings.

Some of the most fundamental misconceptions that students bring to a subject from their own experience (or from bad course outlines passed around from prior semesters) must be discovered in the classroom. Brief classroom assessment devices such as “minute papers” or statements for the students to complete can easily generate a range of incorrect or incomplete understandings for any given topic.  The mission to discover student errors leads faculty to many of the best practices in teaching: regular interaction with students, frequent and meaningful feedback, and active learning strategies.

The power of an “unlearning” perspective on assessment improves student learning, but also quickly leads faculty to a deeper understanding of what assessment of student learning oucomes means.  Assessment is not an end-point, a box to be checked, reported and forgotten, but is an iterative process of discovery and experiment that drives students and faculty learning alike. Assessment tools (such as quizzes, socratic dialogue, essays, simulations, and reflections) might be used to unearth student misconceptions.  These misconceptions then become the basis for the learning outcomes around which one can build a course and assessments then can be used to determine the extent to which one is successfully dislodging misunderstanding and misconception and replace it with a solid framework mastery.

The Baby Has Finally Been Birthed!

Comprehensive revisions passed

The ABA House of Delegates passed the comprehensve revisions with “minimal  fuss” according to the ABA Journal linked  above.  One area, however, garnered  significant attention and also resulted in  an odd, though perhaps meaningless ,  procedural move.  The House voted  to send back to the Section on Legal Education for further consideration the comment to standard 305 which prohibits payment to students for credit-based courses.

What does this mean? Law schools which have not already done so must start identifying, articulating publicly and assessing student learning out outcomes, providing every student six  credits of clinic or clinic-like experiential courses and requiring students to take two credit hours worth of professional responsibility coursework.

Well, it’s a start……

The most fun grading ever, really

The words “fun” and “grading” rarely appear in the same sentence.  However, my large section civil procedure and evidence students’ extra credit projects really are fun to grade.

 The Assignment

Students get extra credit if they develop a creative way to explain one of the more complicated concepts we cover in class.  Students receive the following instructions:

You may work alone, or in a team of up to four people, to create a video, comic book, song, game, poster or some other creative way to explain any one of the following concepts: relation back of amendments; work product; expert discovery; summary judgment; claim and/or issue preclusion [in Evidence the list includes hearsay, character evidence, expert testimony and impeachment]. You may choose to focus on particular aspects of these concepts or the entire concept. 

The project will be worth up to 6 raw score points toward your final raw score total. [The most raw score points available in the class toward a final grade, excluding extra credit, is 100].

Points for the extra credit project will be allocated based upon: creativity; content chosen and explained [i.e. if you take a very simple portion of a rule and explain that, you will likely not get many points]; demonstrated understanding of the applicable rule[s]; communication of the rules to other students. Your project will be part of the review session in the final class.

The Projects

About two-thirds of the class normally turns in a project.  The grades usually are between 4 and 6 points, although I have given some projects a 1 or 2.

Some students developed projects based on television shows or movies. For example, in civil procedure, students developed this game show video on work product

The student actors all imitated various faculty members – complete with wigs, hand gestures, and hair flips. Another group of students in civil procedure did a spoof on the Back to the Future movies to explain relation back.

Other students developed a twitter feed on work product.  Some students used board games for inspiration.  A group of civil procedure students produced an elaborate game entitled “Battleship Preclusion

bp photo

The next year, many of those students took my Evidence class and created a new game: “Escape from the House of Hearsay

 bp photo1

Others developed projects using music and poetry.  For example, in Evidence, students developed a song to help explain hearsay, “The Hearsay Saga of Johnny and Sue”:

This stanza from a poem on character evidence made me laugh:

Hope you enjoyed this poem, I sure had fun.
I hope character evidence no longer makes you want to run
Despite this poem I’m sure we will still all cram
So that we do not fail this godforsaken Evidence exam.

These are just a small sampling of the wonderful creative projects.

It’s Not Just Fun and Games

The Best Practices suggestion that doctrinal faculty use multiple methods to assess student learning [chapter 7] prompted me to develop this creative extra credit assessment.

The assessment has multiple learning objectives.  First, the projects require students to learn the material because they cannot communicate creatively if they do not fully grasp the underlying doctrine.  Anecdotally, when polled via anonymous clickers about the assignment, most said working on the project was either very, or somewhat, useful to their learning.

This assessment also seeks to develop students’ abilities to communicate complex material beyond how they would do so in class or on an exam.  Using different mediums to communicate information is a useful skill.

Additionally, this project allows students to express themselves creatively.  Creativity and innovation are amongst the Shultz/Zedeck lawyering effectiveness factors.

Finally, assessing students on what largely end up being visual presentations provides an opportunity to assess the students’ grasp of the material in a format that may be used by tomorrow’s lawyers to communicate information.

The Impact on the Final Grade

Why make it extra credit rather than required?  Although I believe the projects have educational value, I make them extra credit because some students get anxious at the idea of having to engage creatively with the material.  Also, I want students to have some degree of autonomy about where they spend their time and energy.

I have been asked if this type of extra credit project “changes the curve”.   Underlying that question is the assumption that the way we traditionally grade has a validity that may be skewed by a project such as this one. I question that assumption.

This project measures students’ ability to understand, and communicate that understanding, in a different, but not less valid, way than a multiple choice or essay exam questions.  In some ways, giving extra credit for these projects is analogous to giving class participation credit.

Additionally, those who do not participate presumably can use the time students spent on the projects to study the doctrine.  Thus, the non-participants at least theoretically might have a leg up in terms of the material to be tested via a traditional final.

The Take Away

As we explore ways to prepare our students for practice in tomorrow’s world, we should consider alternative ways to assess knowledge and communication skills, and we should encourage creativity and outside the box thinking.  These projects do that.  And, they are fun to grade.

Evidence Based Experiential Learning?

Over on the Legal Whiteboard, Bill Henderson has an interesting post noting that despite the current call for more experiential education, we lack evidence to answer two key questions:

“(1) Among experiential teaching methods, which ones are the most effective at accelerating professional development? And (2) among these options, how much does each cost to operate? Quality and cost must be assessed simultaneously.”

Henderson is the principal researcher on Northeastern Law’s Outcomes Assessment Project (OAP) that is attempting to answer the question “Does Northeastern’s legal education model accelerate the development of law graduates who are ready to practice and to serve clients?” As Henderson notes, selection effectsmake these challenging questions to answer given Northeastern’s distinctive characteristics, including a progressive, public interest tradition, and a student body with high numbers of women and LGBT students decades before the rest of legal education.

If the OAP project shows that Northeastern’s legal education model does accelerate the development of its graduates, here’s an interesting follow-up question: Will that result be due to the co-op model specifically, or simply to the greater integration of exposure to practice into their students’ education than is typical. In other words, would a different version of a “marble cake” curriculum model have the same benefits?

The Ideal Law School Graduate? A ‘People Person’ Who Can Do Research

By: Jacob Gershman

You can be a sharp writer and a nimble researcher who is skilled at analyzing cases.

But for law school graduates entering the workforce, it’s the softer skills, like work ethic, collegiality and a sense of individual responsibility, that really impress legal employers, according to a new study.

University of Dayton School of Law researchers conducted focus with legal employers to find out what they expect from new law school graduates.

Dayton law professor Susan Wawrose said researchers had thought that the attorneys would focus mostly on the need for basic practical skills, like writing, analysis and research. But comments on soft skills — defined as “personal qualities, habits, attitudes and social graces that make someone a good employee” — tended to dominate the responses.

“The most surprising outcome of our research was the primary importance employers placed on the ‘intra- and interpersonal (socio-emotional)’—soft skills—needed for workplace success,” writes Ms. Wawrose, who authored a report on the study appearing in the Ohio Northern University Law Review.

The researchers interviewed 19 attorneys in the Dayton area who are “actual or potential employers” of graduates from the law school. Most were employed at law firms of varying size. Several others worked as in-house counsel, as an assistant federal public defender, or for legal aid.

The focus group participants said ideal job applicants have a strong work ethic, can work independently without excessive “hand holding,” and would bring a positive attitude to the workplace.

One attorney griped about new hires who “come in . . . [with] this expectation that we’ll sit down and kind of spoon feed them.” Others agreed that some attorneys fresh out of school think “they have a law school degree so they’re entitled to rise up and become partner.”

Other comments suggested that law schools put more of an emphasis on teaching research:

Employers, particularly those with more years in practice, rely on new attorneys to be research experts. The employers in our focus groups have high expectations when it comes to new hires’ research skills, i.e., “[t]hey should be able to adequately and effectively find everything that’s up to the minute.”

Being a research expert also means knowing how to scour books, not just websites, the paper said. “Statutes, treatises and encyclopedias, and desk books are the sources employers still use in paper form. For this reason, new attorneys may want to be familiar with these paper sources,” writes Ms. Wawrose.

The employers also observed that while some new hires are good at cranking out a “full-blown research memo,” the same ones stumble on shorter assignments:

The purpose and audience of the assignment are the key. “[T]hey need to be very cognizant of who their audience is.” Is the document for a client? And, which client? Is it the one who is “very busy” and “want[s] to know, ‘boom,’ ‘what’s the answer[?]’” Or, is it the client who is “all into the details” and will feel “nervous if you don’t give them all the specifics.”

http://blogs.wsj.com/law/2013/11/25/the-ideal-law-school-graduate-a-people-person-who-can-do-research/

Congratulations UNM and Editors of the proposed new Best Practices Book!

This weekend, the University of New Mexico hosted a workshop BEST PRACTICES IN LEGAL EDUCATION: The Walls Are Coming Down” in which draft chapters of a new “Best Practices” book were reviewed and discussed.  The proposal to create a second book focused on best practices in legal education is the brainchild of Professor Antoinette Sedillo Lopez, ably assisted by Professors Deborah Maranville, , Carolyn Kaas and Lisa Bliss. The symposium workshop brought together law professors from throughout the country interested in how legal education and the world of law schools has changed since the publication of the 2007 book Best Practices in Legal Education. Facilitated by Professors Beryl Blaustone and Alex Scherr, the conference explored how many law professors fluidly move from former silos of clinical, legal writing, lawyering, librarian, doctrinal, theory, or skills concentrations to pioneer a new kind of curriculum, better prepare students for the profession, explore the limits and usefulness of technology, and deepen the understanding and learning of law students through self-improving assessment processes.

Fully cognizant of the pressures on legal educators, the fact that not all in legal education welcome the need to change, and the moral imperative to address the concerns of debt-ridden unemployed law students, the authors, editors, advisory board members and readers reviewed challenges, cross-cutting themes and areas of promise. They engaged in innovative thinking about how to move legal education forward for the good of the profession, society and the students who desire to be lawyers of tomorrow. The keynote speaker for the Friday night dinner and author of the first book, Professor Roy Stuckey, directed the participants’ attention to what legal education should look like in 2027. At the same time, he reminded us that those seeking to improve legal education today stand on the shoulders of folks such as the honorable Rosalie Wahl and former ABA president Bob MacCrate who paved the way for the changes we have seen in the last 40 years. He recalled their joint mission to prepare “agents for justice in our communities.”

Every law graduate needs to understand fully that civic professional role of the lawyer. And every admittee to the bar has a sworn duty to improve our system of and access to justice. Returning to those principles can help prioritize our cost-cutting and can position us to move forward in the best interests of our students, our institutions and the society our profession is pledged to serve.

A rose by any other name: Evaluation and Assessment at Cross Purposes

A barrier to developing, improving, or sharing our assessment practices is the confusion surrounding the vocabulary of assessment.  Whenever it occurs or by whatever method, assessment is simply the process of discovering what and how well students have learned and then using that information to improve. One can quickly become mired in a sea of words that feel like jargon, with assumptions that confuse and distort the real meaning of this otherwise familiar practice of all good instructors. Part of the problem is that the language is not our own and so, by its very adoption, reinforces the impression that assessment is an intrusion into our classrooms.  Because these confusions are so destructive to the ability of an institution to move forward with assessment, we must either work to make this vocabulary our own or develop a different vocabulary for the same ideas.

Even though discovering what students have learned in order to improve teaching is a natural part of a good teacher’s practice, law schools are having difficulty in knowing exactly what this talk of assessment means. Faculty frequently mistake outcomes assessment for something more complex, unusual, or even sinister. “Assessment” becomes confused with “evaluation” (as in program or teacher evaluation) or “standardized testing,” and, before long, we are thinking of K-12 school district funding decisions based a “No Child Left Behind” external control of education.

There is a fundamental difference between assessing student learning for the purposes of program or teacher evaluation and assessing student learning for the purposes of improving that learning.  If we are assessing for accountability, we collect data (e.g., pass rates) about students learning outcomes that we do not necessarily control (e.g. bar exams) so that we can report that data to external constituencies (e.g. accreditors). In contrast, if we are assessing for student learning, we observe evidence (e.g., essays, performances) of student learning outcomes that we have designed ourselves so that we can interpret and use that evidence to improve the learning of our students.  When accountability to those outside the learning process is the driving force behind assessment, the temptation may be to assess only those learning outcomes that we know students have mastered and avoid looking for places where learning could be significantly improved.  We might skew our teaching and curricula away from learning outcomes we truly care about to more closely match the learning outcomes we believe outsiders consider important.  Of course that already does happen to some degree.  The influence of ABA standards of accreditation and bar examinations on curricula is so obvious we may not even recognize the degree to which our faculty control of the program of legal education is directed by these learning outcomes and assessment methods.

It is against this backdrop of fear that some law teachers approach the topic of outcomes assessment.  However, resisting assessment out of a concern that others will rob law faculty of their freedom means giving up one of the most powerful tools to protect that freedom. If a faculty can clearly communicate the learning goals they have for their students, and can demonstrate how their program of legal education leads to more students accomplishing those learning goals at higher levels of mastery, that proof of learning can become powerful tool for demonstrating accountability: to the students, the academy, the bar, and the public. That is not to say that assessment for accountability will not be required or should not be undertaken with seriousness of purpose and honesty in method.  Assessment for improving student learning, however, should be just as important, if not more so, so that we can be accountable to ourselves and our students.

Follow

Get every new post delivered to your Inbox.

Join 513 other followers

%d bloggers like this: